Management at United Water New Jersey understands that implementing a robust supervisory control and data acquisition (SCADA) system at a water treatment plant can provide more than centralized control of processes that ensures a reliable water supply for customers. United Water operates SCADA systems for water and backup and emergency power at its recently upgraded Haworth Water Treatment Plant in Haworth, NJ. The utility depends heavily upon SCADA when shedding load during demand response events that are reducing the plant’s energy costs.
The Haworth plant is one example of the growing realization among water utility managers that process control can do more than conserve water or ensure its delivery. The amount of power that some plants consume is not insignificant and process control can optimize operations as well as energy use. Utilities that supply their own off-the-grid backup power for plants also need process control for this mission-critical function.
The Haworth plant was constructed in the mid-1960s with a 60-million-gallon-per-day (mgd) capacity and was upgraded into a 200-mgd peaking facility in the late 1980s. The water distribution network also includes 13 wells, a 113-square-mile watershed, nearly 15,000 fire hydrants, and more than 2,000 miles of water mains. In spring 2009, the plant underwent a $100 million renovation that improved process efficiency and water quality and exceeds all current regulations. The renovation included high-rate dissolved air flotation (DAF) for sedimentation clarification, making the plant the largest in the nation to use DAF. The network serves about 350,000 billable customers and nearly 1 million people in Bergen, Hudson, and Sussex counties in northern New Jersey. Production usually peaks at about 188 mgd in midsummer.
PSE&G supplies the main power supply to the plant. PPL Electric Utilities operates a backup power system at the plant for United Water that consists of four, 2-MW Caterpillar natural gas-fired generators. Emergency power is supplied by Solar Turbines. United Water buys electricity on the next-day, hourly Locational Marginal Pricing (LMP) market. United Water bids and contracts for base level quantities of gas and electric and purchases the balance of electric on the LMP market. United Water runs the backup power system when gas prices are relatively low–especially during the summer months.
Water treatment and distribution, as well as Solar Turbines for emergency power, utilize a GE Proficy HMI/SCADA iFIX system that monitors plant processes. A Schneider Electric ClearSCADA system is used for remote site HMI. The systems control pumps, chemical feed controls, an ozone generation system, filter controls, and a residual handling system in the plant, and will control system components at remote sites such as wells, booster stations, distribution storage tanks, pressure-reducing valves, and regulators.
United Water is also in the process of improving radio communications at the remote sites and obtaining new licenses for backhaul frequencies. The plant, which serves as the hub of United Water’s northern New Jersey operations, is equipped with an operator control room staffed by three full-time employees. They monitor integrated data points from the plant and PPL Electric SCADA systems.
Process control of water production is nothing new to United Water New Jersey. When the plant capacity was increased, a Westinghouse distributed control system (DCS) was implemented. Another DCS from HSQ later replaced the original Westinghouse system. The GE SCADA system was implemented during the 2009 plant renovations.
PPL Electric has a dedicated SCADA system equipped with Modicon PLCs for the plant’s main power supply. According to Chris Brophy, the resident SCADA expert for United Water, this SCADA system pulls together 30-40 different electric data for water production such as voltage and frequency for the plant operators to monitor.
“Years ago, you basically had incoming power, and that’s what you knew,” recalls Brophy. “You didn’t know where it was going to be
View of Mid-Dakota Rural Water Systems’ water distribution network via its SCADA system
consumed in the plant, and you didn’t know what was efficient and what wasn’t efficient in the plant.”
The new equipment in the facility, including motor control systems and the DAF system, “has power monitoring so we can literally monitor—besides the gross numbers coming in—each process, so you can see which process is a little off, how they’re comparing to each other, and where energy is being consumed,” says Brophy. “You can’t manage it unless you have the information.”
Data are transmitted directly from the plant via fiber-optic/Ethernet and from field sites to the control room, according to Keith Kolkebeck, engineering systems manager for United Water. A reporting program from utility technology and management consulting provider EMA, Inc., compiles the data for analysis.
United Water New Jersey counts on its SCADA system during occasional load shedding that is deemed cost-effective when natural gas prices are favorable. Energy Curtailment Specialists manages a demand response program on the grid of PJM, a regional transmission organization that coordinates the movement of wholesale electricity in 13 states and the District of Columbia. United Water has committed to a 6.1-MW load reduction at the Haworth plant during demand response events.
“We get a two-hour notice of an emergency event, and it can run up to six hours,” says Brophy. “It can happen multiple times per year, and we have two hours to reduce load as much as we can. We’ve usually been performing about 100 to 150% of our commitment when required.” Kolkebeck added that some of the plant’s larger pumps consume significant energy and that the facility routinely pulls 12-14 MW.
“Our peak days generally coincide with the electric operators’ peak days,” says Brophy of the Haworth plant’s water production. “When it’s hot and humid and people are running air-conditioning, they’re also using water–the peak days coincide almost exactly.” Because the plant “pumps to demand,” using water stored in tanks with some pumps powered down is not an option, he adds.
Kolkebeck says that United Water uses the GE SCADA data to continually monitor and control processes in the plant. Process changes do not involve directly changing PLC codes, but, rather, changing parameters, setpoints, and control points.
“We do pump efficiency testing and testing on our systems to make sure they’re energy efficient, and if they’re not, we have a maintenance program in place to take corrective action,” he says.
Brophy says that since the plant upgrades took place a couple of years ago, it appears that United Water has reduced its kilowatt-hours by roughly 1/2% to 1% per million gallons of water. This might improve with greater monitoring capabilities built into the remote sites.
Intelligence-Enabled Demand Response
The more intelligent a water distribution network becomes, the more challenging it can become to shed power during a demand response event without losing water delivery reliability. This is where further automation of process controls can be helpful.
That is the stage of process control implementation at the Perris, CA-headquartered Eastern Municipal Water District (EMWD), one of the largest water purveyors in southern California and serving a population of about 755,000 across 542 square miles. The EMWD is a major consumer of electricity, at $10 million in annual electricity costs. With this in mind, the district enrolled in a demand response program managed by energy management consultant EnerNOC and committed to reduce electricity consumption by about 3 MW by shedding load at its main treatment and distribution facilities. For doing this, EMWD receives annual payments from EnerNOC totaling about $120,000.
Dan Howell, director of purchasing and contracts for EMWD, pointed out that working with a third-party demand response manager such as EnerNOC is advantageous. EMWD has participated in utility interruptible programs for the past 15 to 20 years, but the regulatory environment has changed recently, Howell reports.
“We’re located in the South Coast Air Quality Management District, which is probably one of the most heavily regulated air districts in the United States,” he says. “So we were challenged with operating generation assets in order to meet some of the interruptible provisions that the utility has under their programs. If you fail to interrupt when you’re on an uninterruptible tariff, you’re assessed large penalties and fines associated with every kilowatt-hour you use during that interruption period.
“Aggregators such as EnerNOC don’t have penalties per se for failure to interrupt. The economic incentive is typically less under the third-party aggregators as it would be under the utility programs, but the penalty doesn’t exist. In order to meet our delivery obligations, third-party aggregators such as EnerNOC became a good alternative.”
California, the nation’s most populous state, faces greater energy supply challenges than most. To deal with the challenge, Southern California Edison (SCE) manages programs such as a Base Interruptible Program (Schedule TOU-BIP). Customers that select this program are required to choose a Firm Service Level that reflects the amount of electricity the customer determines is necessary to meet their operational requirements during an interruptible event. They must also choose a participation option, which is the amount of time (15 or 30 minutes) the customer requires in order to respond to the event.
EMWD receives lower overall utility rates for participating in the program. The combined reduction efforts reduce the overall demand for electricity in California and potentially prevent power interruptions. Penalties are assessed to commercial customers that do not participate when an interruptible event occurs. Customers that have agreements with third-party aggregators, such as EnerNOC, are not assessed penalties but merely forgo payments received from the aggregators for non-participation in a given interruptible event. For many organizations—public and private water utilities, for example—determining how to shed load without compromising core operations poses a challenge.
Demand response, and partnering with a third-party aggregator, suits EMWD because it cannot participate at all of its facilities in every interruptible event, according to Howell. The utility has more than 250 accounts with SCE and, in addition to over 8 MW’s enrolled in SCE programs, can shed another 3 MW of load during interruptible events among its Hemet Water Filtration and Perris Water Filtration plants and five other facilities. Facilities with higher energy demand include its Hemet Water Filtration and Perris Water Filtration plants, which account for about half of EMWD’s load shedding capabilities under the third-party demand response events. None of the facilities that are contracted with EnerNOC have backup power systems.
“There are challenges in meeting air regulations for operating those standby emergency generation generators,” says Howell. “We believe that where we do have contracts with EnerNOC, that we have adequate water storage, and we have redundant facilities perhaps elsewhere that can help supplement during those events.”
EMWD operates separate SCADA systems for its water distribution system and for its four wastewater treatment plants; each of the latter has a dedicated system. None of the wastewater treatment plants is managed under the EnerNOC program. The water distribution network is equipped with a Telvent OASyS SCADA system and a Derceto energy management system (EMS).
The water distribution network has a relatively high level of automation, according to Howell. EMWD monitors tank levels and pressure throughout the system, for example, and optimizes network operation at any given time and level of water demand.
“Not many water utilities are using that level of sophistication,” says Howell. “Our EMS looks at the best way to operate in terms of both meeting our water delivery obligations, and from a financial perspective with regard to energy use. It’s a real-time pump scheduler, not a reporting system. It’s literally looking at projected water demands throughout our service area, and then looking at available assets to meet those demands and what it costs to run individual assets and then optimizing which should run first. It takes into consideration the utility rates, the time of day, the pumping capacities, the efficiency of one pump versus another—it’s selecting that and the speed at which to operate.”
The water SCADA system also has a high level of sophistication for demand response events and may soon become even more sophisticated. When informed of a demand response event, EMWD staff reviews SCADA information to both determine whether the utility can participate and, if so, how to operate the network in order to comply.
“What we’re now working on is the potential to automate that—we’re looking at working with the utility, Honeywell, and Derceto, and EnerNOC to automate the receipt of a demand response event from [SCE] and EnerNOC through our SCADA system in our Integrated Operations Center to allow the staff to make a yes/no decision to participate or not, and then resolve how the system will operate. So there’s a human interface there. That’s the sophistication that’s coming into play.”
The EnerNOC-managed demand response program benefits both EMWD and the state, Howell notes.
“[The refunds] go to directly offsetting our operating costs,” he says. “So to the extent that we reduce operating costs, it benefits our ratepayers. We are a municipal nonprofit water district, so any opportunity to reduce operating costs directly affects our ratepayers–this is one example of that. The benefit to the environment and the state is that [the program] defers the cost and the impacts of constructing additional power generation. It is a major undertaking in this state to construct power generation facilities.”
More water utilities in California would be able to participate in interruptible events if they can increase their process control sophistication, Howell concludes.
“Other utilities that may not have that level of sophistication may be reluctant to participate because they just don’t know what’s going on,” he says. “In this industry, we tend to err on the side of being overly conservative when it comes to being able to meet water supply, so without that good SCADA background, we just wouldn’t have the information in order to make these kinds of decisions.”
Monitoring a Vast Rural Territory
Across the often-vast distances of a rural water utility, a SCADA system really helps out the staff by identifying problems in the water distribution network from a computer screen, rather than onsite visual inspection. Scott Gross, operations manager at the Miller, SD-based Mid-Dakota Rural Water System (MDRWS), which serves about 30,000 rural customers in 16 communities in all or parts of 14 counties in a territory covering about 7,000 square miles, can attest to that. He is responsible for overseeing a network consisting of about 4,000 miles of distribution pipeline, 115 miles of mainline, and 16 storage tanks ranging in size from 100,000 to 2.5 mg for a total storage of 7.8 mg. The system keeps expanding, too: the capacity of a treatment plant located near Pierre, SD, was recently increased from 9 to 13.5 mg, and a 2-mg storage tank and 6,000-gallon-per-minute booster station was added as well.
Gross recalled that the expansive nature of the system meant that it took a while for a SCADA system to be fully implemented. The treatment plant went online in 1997, but it took until 2006 for a Micro-Comm SCADA system to cover the entire network. MDRWS uses a SCADAview CSX (Client/Server/X-platform) central-based (CTU) telemetry system, a cross-platform application that has versions for Windows, Mac OS X, and Linux. Gross and MDRWS primarily use the system to monitor tank levels and flow rates. One major reason why this system was chosen is the vast distances comprising MDRWS, according to Gross. The system uses telemetry stations that compile operating data by radio signals. The system is equipped for signal transmission across the vast spaces. Rolling terrain on the western half of the territory does not make signal transmission any easier, according to Harvey Aberle, whom Gross succeeded in fall 2011. As a result, the SCADA system uses antennas mounted on 14 water towers located throughout the territory.
The combination of the SCADA system and VFDs enables MDRWS to achieve significant energy savings, although MDRWS had not been able to quantify the savings as of fall 2011 with the ongoing expansion of the system.
“The number one savings is that we don’t have to have manpower go out to each station twice a week–we do that off of SCADA,” says Aberle. “Also, some of the biggest power savings result from the fact that everything runs on VFDs. We have control of VFDs, so we don’t have that rush of power when we start pumps up. We can also control the speed of the VFDs, so we know that if we have a high demand coming in for the day—say if it’s 100 degrees Fahrenheit, we will actually start pumps up from the central computer here in Miller. Or, if we know that it’s going to be a cold week, we turn the speed of the VFDs down, because we don’t need to move all of that extra water into the tanks and not use it. I know that, to start up a pump for the initial surge of power, with a VFD you save three times the demand charge versus a hard start on a regular pump.”
A SCADA system is a powerful tool, but human capital is still needed for maintenance.
“The biggest killer for MDRWS is ice storms,” says Aberle. “They ice up the antennas, and then we can’t communicate. The winters here are pretty tough, and we actually take a shotgun out and shoot the antennas clear so that we can keep communications going.”
Gross adds that the odds of damaging the antennas from 180 feet away on the ground are minimal.
“You can’t get guys up there—the ladders are too iced up,” he explains. “You’re shooting [small] number eight shot, so there’s a pretty small chance of breaking anything.”
A SCADA system is a necessity for monitoring such a spread-out water distribution network, Gross concludes.
“We couldn’t run this without SCADA,” he says. “We’d be running guys ragged without this, and it would probably double the workload—we only have 10 operators.”
An Illinois water utility was thought to be the victim of a foreign cyber attack. On November 8, the water utility’s pump system failed, and initial reports indicated that utility mangers feared the utility had been attacked by cyber criminals hacking into the utilities network via their unsecured SCADA system. Ultimately, the FBI and the Department of Homeland Security (DHS) determined that the utility had not been hacked (www.pcmag.com/article2/0,2817,2396835,00.asp).In a statement relating to the investigation, a DHS spokesman stated, “After detailed analysis, DHS and the FBI have found no evidence of a cyber intrusion into the SCADA system of the Curran-Gardner Public Water District in Springfi eld, Illinois.”
Although this particular SCADA hack was just a false alarm, cyber security experts warn that similar tactics could be used successfully in the future across a wide variety of SCADA systems; including those used for nuclear reactors and chemical plants.
Unfortunately, outdated SCADA systems litter much of the country’s industrial and commercial landscape. In an interview quoted in Daily Tech (www.dailytech.com/Cyber+Attack+on+Illinois+Water+Utility+Sparks+DHSFBI+Investigation/article23331.htm), Lani Kass, former senior cyber policy adviser to the US Joint Chiefs of Staff and the US Air Force, says, “Many [SCADA systems] are old and vulnerable. There are no fi nancial incentives for the utility owners to replace and secure these systems, and the costs would be high.”
When initial reports of the alleged cyber attacks surfaced, it was suggested that the perpetrators had accessed the utilities SCADA system using stolen credentials from the SCADA software company—a methodology that is not too far fetched according to Dave Marcus. Marcus, Director of Security for McAfee labs, warns that SCADA networks lack some of the security protocols common in standard computer networks, there’s no way to know whether or not our systems have not already been compromised.
In an interview in PCWorld (www.pcworld.com/businesscenter/article/244359/water_utility_hacked_are_critical_systems_at_risk.html), Marcus outlines some of the biggest concerns regarding SCADA systems and suggests some preliminary precautions that can be taken in response to this new security threat. He also recommends some of the following precautions:
• Include “cyber” in all risk management.
• Set up extensive penetration testing.
• Set up extensive counter-social engineering training.
• Put a SCADA-specifi c CERT plan and team in place.
• Network with law enforcement at all levels.
• Expect to get attacked and take appropriate counter measures.
The question remains, if SCADA networks represent significant targets for terrorists or other politically motivated attacks, are we doing enough to defend our systems? It’s diffi cult to determine whether or not a cyber attack has already taken place, but if SCADA networks are “easy targets,” perhaps it’s time to start implementation of other, additional, security measures.
