The history of Macomber Map®

Mike Legatt, Ph.D.

I’ve always been fascinated by the intersection of people and technology, and my interest in this intersection in the energy world was honed during the August 2003 blackout. As an amateur (ham) radio operator, I spent several hours in a New York EOC (emergency operations center), and watched what happens behind-the-scenes when the lights go out.

I consider one of my greatest blessings the fact that I have the opportunity to work on improving human-computer interaction at ERCOT, the independent system operator (ISO) responsible for managing the reliability and markets of the bulk electric system for most of the state of Texas. In 2007, I was hired to work on user interfaces as part of ERCOT’s transition from a Zonal to Nodal energy market, a shift that would bring significant leaps in computation, granularity of analysis, efficiency of energy markets, and ultimately increase the speed and complexity of data seen by ERCOT control room operators.

The work that I do as human factors engineer was by no means started by me; several years before I came to ERCOT, my colleagues Gary Macomber and Kate Horne started looking at the way people throughout ERCOT, but especially in the control room, saw and processed information, and the ways in which we could help better support their situation awareness and decision making.

In 2007, I had the tremendous honor of being hired to work at ERCOT, under Diran Obadina, my mentor in IT, and Gary Macomber, my human factors mentor. Both supported and taught me so much, and I have a tremendous admiration for them both. As part of my work, I began to meet weekly with our control room operators, shift engineers, and management. It quickly became clear quickly that we needed to have a “rich discussion”, not just a theoretical one, in which everyone was able to see the same thing and discuss how best to visualize the information. That’s when Macomber Map started.

For almost a year, Macomber Map was a demonstration tool. Every week, I’d meet with the operators, show the prior week’s enhancements, ask for feedback, and then spend the week modifying the code based on the feedback I received. Because our training schedule is set up in such a way that I would meet a different shift every week, it also meant that I had a wealth of different opinions through the process.

I think in many ways this was always the “secret sauce.” In many industries, there’s a trend for software to go “over the fence,” where software developers would write the user interfaces they think the end user needs, test and productionalize, then ship to the operator’s computer, at which point the end user would see it for the first time. It quickly became obvious that this way of doing things is terrifying to an operator, because their ability to do their job is dependent on some third party having enough of a good idea of what they need. It’s a testament to ERCOT’s operators that they were so adaptable, but when this collaborative way of working was done on this project, they embraced it fully. Ultimately, this created a relationship between us, and after a while we’d get phone calls and emails requesting additional features, most of which were easy to implement from our perspective, but saved them a lot of time or improved their efficiency significantly.

From the very beginning, the map connected to our network model (at ERCOT, we call the system NMMS), in order to determine roughly where the equipment was on the map and how it was interconnected. After several months, in order to be able to have a discussion about a real-time condition, I added in linkages to pull data from several areas of our EMS (energy management system), and the map became a functional prototype. Again, once that linkage was in place, I got a great deal of useful feedback from the operators on additional needed information, or the way in which that information needed to be aggregated to help them.

After some time, it became clear that a wide-area visualization system was needed, and there was an opportunity to link together several disconnected systems: visualizing the state of the grid, showing real-time and contingency violations, aggregating information, navigating between high- and low-level information, and viewing component history. Through the process of the training cycle, these components were added to the application.

ERCOT management then underwent the process of analyzing several different products, including productionalizing this one, in order to provide our operators, engineers and management with the needed situation awareness tools. As this process started, Gary became very ill. To our sadness, his illness was quite aggressive, and he died a few days later. When ERCOT management completed its analysis and chose to make the map we all developed together a product, we chose to name it in his memory, as the Macomber Map.

Once, many years ago, Gary and I were talking about the way in which the software industry and intellectual property areas are so competitive. He said to me, and I agreed, “It isn't that important who owns a product, as long as it’s used to help as many people as possible. Ultimately, having reliable electric power is so important, so anything that helps support that as much as possible is a good thing.” It was clear to us both, even back then, that the eventual path for this product would be the open source community.

Over the past several years since the Macomber Map first came online in our control rooms, enhancements still continue. Based on severe weather events, tremendous growth in intermittent renewables, and new types of reporting and compliance needs, new features continue to be added. This past year, as part of our training program for our market participants, the Macomber Map was updated to work with our simulator (to allow control of equipment on the simulator). A Macomber Map server was added, so that the large amount of data needed by the map becomes one to many, allowing more people to use it.

Since I started giving presentations on Macomber Map in 2009, I’ve occasionally received questions from people outside the energy industry. Could you use the map to visualize traffic on congested roads during an evacuation route? Could you use it to visualize the flow of natural gas along a pipeline? Could you show a weather map, visualize prices, view the internals of a plant, factory, etc.? The answer has always been yes and no. Yes, because with the codebase and the architecture around an integration layer, one should be able to associate data from a variety of sources to a particular element. No, because our time constraints are such that we can’t make it happen alone.

And, that brings us to where we are today. The ERCOT community that developed Macomber Map hopes that open sourcing the map will do several things: create an opportunity for people in the energy (and other fields) to create better situation awareness views, customizable to their needs, and to create a community around the product to support and grow it.

If you’re here reading this today, then you’re a part of that community. On behalf of myself and ERCOT, thank you.

Mike Legatt

Last edited Dec 31, 2013 at 7:58 PM by MichaelLegatt, version 5