Data Integration: SCADA and Live Pattern Matching

July 1, 2009

Supervisory Control and Data Acquisition (SCADA) has been used for the past 40 years to monitor and control utility networks and manufacturing processes, and provide real-time information on networks to allow timely reactive responses to be made to incidents.

Experienced operations staff can identify complex situations based on information provided by SCADA systems, and systems are in place elsewhere to ensure prompt reaction to these situations. Predicting the occurrence of complex situations would allow a more proactive approach to network management. There are now methods to predict such situations, based on the automatic recognition of patterns in data supplied from SCADA and other data systems.

This paper considers methods for using pattern recognition on live data to create a proactive network management methodology.

Water companies are constantly challenged to meet, or exceed, ever-increasing regulatory requirements with fewer and fewer staff.

Technology and technical solutions have improved greatly over the past few years, and these already help the water companies in their challenges.

Figure 1. Collecting network information effectively in real time requires devices that can operate at low-power and with wireless communications.
Figure 2. In order to meet the challenges of managing field work, water companies need to ensure the field force can receive work orders and view key information remotely using wireless communications technology.

Pattern matching is widely used in image and signal processing, as a means of automatically identifying actual or potential events or features.

The combination of such techniques with other utility systems could help to provide a comprehensive solution for proactive management of water networks.

Improvements in SCADA
The utility industry uses SCADA, also known as telemetry, as a fundamental component of their operations. Network management systems provide the water utility with critical information on the status of the network components and allow the network operators to effectively manage the treatment and distribution of clean and waste water. A typical wide-area SCADA system will cover a very large part of a water company’s operational network.

Originally based on meeting safety requirements (such as alerting a control room when a level is too high or low) SCADA systems are now also used to collect performance information on their network assets, such as the number of hours a pump or motor has been running for.

Water companies are becoming more proactive in the use of this performance data. For instance, with information on how many hours a pump or motor has run for, a maintenance manager can optimize maintenance based on how heavily equipment is used, which has an additional benefit that unscheduled outages are reduced.

The Increasing Challenges for Water Companies
Water companies have made huge steps to improve drinking water quality, reduce losses through leakage, and reduce incidents such as sewage spills. However, the regulatory bodies continue to press the companies concerned for improvements, and these demands present some significant challenges. These challenges all incorporate a number of themes:

  • To identify potential issues before they become real problems
  • To schedule remedial work in a timely manner in order to rectify underlying causes
  • To learn from potential and real situations so as to improve the ability to identify potential issues

For example, the objective of minimizing the number of incidents of sewer flooding and overall to reduce the number of properties at risk of sewer flooding, a water company must be able to:

  • Collect real-time sewer network information such as level and flow
  • Relate this real-time information to a network model or other heuristic information, in order to determine points in the network that require urgent remedial work
  • Coordinate field teams to rectify the work in the required timescale and identify that the work is complete and the issue is resolved
  • Have a high-level view of their achievement of regulatory compliance

The collection of real-time network data in a cost-effective manner may often require the use of battery-powered devices with wireless (usually GSM) communications, as otherwise it is either too expensive or impractical (for instance, to install mains power and communications links), or inadequate (for instance, logging data locally and collecting it manually will not provide true real-time status).

Most water companies will have a hydraulic model of their clean and wastewater networks. Real-time information can be applied to this model, in order to identify areas where there are potential issues. Traditionally this is an offline process, taking place periodically, with intervals anywhere between four hours and one month. In order to meet the challenges laid down by the water regulators, methods are required that can provide reliable information in real time or near real time.

Once potential problems are identified, the water company has another issue, to ensure that field teams are issued with appropriate work instructions and tasked with completing work in a timescale necessary to avoid an incident. Water companies already face a real challenge to ensure all maintenance and remedial work is planned and completed effectively. Work Management Systems, together with automatic Field Force Scheduling and Optimization Systems are widely available and commonly used by water companies to manage their field operations.

Finally, the task of monitoring regulatory compliance is a significant one, consuming large amounts of time and effort for a typical water company. The provision of information in a “management dashboard” style is becoming increasingly common. The aim is to provide the Key Performance Indicators (KPIs) that managers need to monitor in an easy-to-understand form so that the current situation is clearly understood and important decisions can be made quickly.

Pattern Matching to Identify Situations
The use of pattern matching is not new and has been successfully applied using signal and image processing to a very wide range of scenarios, such as to speech recognition (recognizing words based on patterns in speech signals), automated manufacturing (detecting faults in manufactured product by comparing an image of a good unit with those being made and highlighting differences), and submarine detection (detecting types of submarine from the specific type of noise given off by their different engines).

There are many methods and algorithms for pattern matching, but in principle they all follow the same approach: to identify particular situations (or syndromes) by analysis of data from one or more sources. Most pattern-matching methods apply the process of self-learning, so that for increasing data analysis the accuracy of the analysis increases.

The benefit of analyzing data is that it can be done automatically using computer processing. This greatly reduces the time taken to complete the analysis and also increases the accuracy of the results.

The potential disadvantage of an automated approach is that it is entirely dependent on the accuracy of the rules provided in the first place. However, if a self-learning solution is used then with sufficient data input then the analysis results will continually improve.

Proactive Network Management
To achieve the level of proactive network management necessary to comply with the increasing regulations from the regulatory bodies it is essential that water companies integrate their key systems together to provide an integrated view as shown in Figure 4.

In this approach, real-time network status is provided to the water company control room as normal. In addition, this data is combined with metrological data as inputs to a pattern-matching engine that will compare the current data to identified syndromes and raise alarms to the SCADA system if syndromes indicating problems or potential problems are detected.

Based on information from the SCADA system, the Control Room will raise work orders automatically or manually and these work orders will be scheduled and optimized accordingly before being issued to the field force. Information relating to KPIs will be made available to management users and this will show an up-to-date view of the status of regulatory compliance.

A Real-World Example
The following example has been modified to retain the anonymity of the client involved.

In this example, a process takes a number of inputs of raw materials and produces a number of discrete outputs of product. An incident occurred whereby one of the outputs of the process was damaged and a significant loss of product resulted. This was identified after it occurred and rectified.

Furthermore, by using the real data and identifying the resulting event, it would be possible to identify future circumstances that match the pattern and thus indicate another potential occurrence of the same event.

Summary An approach to the proactive management of networks has been considered. This approach integrates the existing utility systems such as SCADA and Work Management with information from other sources such as metrological data, and uses pattern-matching techniques to look for syndromes or potential events occurring in the network that require remedial work and automatically schedule field engineers to perform the work required before an incident occurs. With sufficient self-learning, the integrated system should improve its ability to identify potential problems, and water companies will be able to schedule remedial work in a timely manner, thus greatly reducing the number of unplanned incidents. 
About the Author

Steve Mustard

Steve Mustard is the president and CEO of au2mation.

Photo 39297166 © Mike2focus |
Photo 140820417 © Susanne Fritzsche |
Microplastics that were fragmented from larger plastics are called secondary microplastics; they are known as primary microplastics if they originate from small size produced industrial beads, care products or textile fibers.
Photo 43114609 © Joshua Gagnon |
Dreamstime Xxl 43114609