• Santiago Peñate

Updated: Aug 5

What do you do when the power flow of a model does not converge? change parameters until it works somehow? not anymore! This article aims at explaining the most common issues and their solution when the power flow of an electrical grid model does not converge.


Introduction

Let's provide some base concepts to be sure we are speaking in the same terms.


What is the power flow study?

The power flow study, contrary to its name , aims at computing the nodal voltages that satisfy the nodal power balance condition. This means that the computed power injections using the aforementioned voltages are close to the specified nodal power injections (The given generation minus the given load) After the voltages are obtained, the branch flows are computed once. This study provides the steady state electrical magnitudes of a power system, so it is most useful in situations where we can assure that the power system is steady. An example of a situation when this is not true is a large disturbance such as short-circuit.


What does convergence mean?

To "converge" means that the nodal power balance condition has been satisfied up to a certain numerical threshold. A good numerical threshold is 0.000001, a threshold used in practice is 0.001 for speed, and sometimes because the model is not as good as it should be and the numerical methods get stuck at higher errors.


Issues

1. Nonsense data issue

The first issue I'd like to bring up is the presence of nonsense data. But how can you know what is nonsense? Some hints:

  1. Zero branch reactance. All branches should have some meaningful reactance value. In the case of DC branches, there should be some resistance value.

  2. Very low branch reactance values. Sometimes, exists the temptation to model switches and jumpers with very low reactance values like 0.0000001. This hurts the model condition number, thus its convergence properties. Those branches should be removed from the model with a topological process.

  3. Not in per-unit. Most power system models require the impedances to be provided in the per-unit. If not, the values will not make sense.

  4. Out of range values. Some parameters only admit values in a certain range. For instance, does it make sense to have a -0.05 p.u. resistance? no it doesn't. Ideally, the software should only allow values in range because the users are not always aware of what sensible ranges are.

  5. Reversed ranges. When specifying a range it is clear that the lower value has to be actually lower than the upper value. Sometimes the range is reversed by mistake.

  6. NaN values. This is an obvious one, but not to be underestimated. Not-a-Number is not a valid number anywhere, hence, no input field should ever be NaN. The same applies to empty numerical values.

There may be other parameters that might contain harmful data that is not immediately obvious. In those cases a histogram of the values is a very useful tool to find outliers. GridCal features automatic detection of nonsense values and provides histograms of the most influential magnitudes in its model inspection tool.


2. Active power imbalance issue

This one is very common, and easily overlooked. As we mentioned, the power flow study computes the steady state values of a circuit. For a circuit to be in steady state, there must be an exact balance between the load and the generation. In the numerical power flow study we need to leave a free generator, located at the "slack" node. This generator can provide any amount of power, but that does not mean that the grid can transport it anywhere. In transmission grids, the balance should typically be within a 3% error margin, less is even better. In distribution grids there might not be any balance at all if there are no generators. In those cases the slack will provide all the power, but unlike the transmission grids, the distribution grids are designed for that situation and the power transport from a single entry point is not an issue.


Solution: To solve the active power imbalance, the typical solution is to run an optimal power flow where the generation is dispatched according to their cost. This produces the generation values that satisfy the demand with good enough criteria to be able to satisfactorily solve the power flow problem. Alternatively, one could just scale the generation to match the load.


3. Voltage guess issue

This one is less obvious and it is solely related to the numerical methods used to solve the power flow; The gold standard algorithm to solve the power flow study is the Newton-Raphson algorithm. First reported to solve the power flow problem in 1967 by William. F. Tinney and Clifford E. Hart, it has quadratically convergence properties. However, the Newton-Raphson algorithm and most of the state of the art algorithms, rely on an initial guess of the solution to iterate until reaching the convergence.


We know that in the real world, only one solution exists for each loading state. But since we are dealing with a mathematical model, it has been proven that the solution space of the power flow equations contains more than one solution that is mathematically possible. So how do we know if we are getting to a meaningful solution? In my experience a meaningful solution is one such that the voltages are not in collapse state (given that we have no nonsense values and balanced power injections) these are voltages in the range (0.9 , 1.1) p.u.


Back to the issue at hand; Sometimes we need to run a sequence of power flow studies, and experience shows that if we use the last state solution as the initial point for the new state, the algorithm converges faster. This is a good practice if both loading states are similar like the voltage solution at 10AM might be a good initial point for the situation at 11AM. However, if the loading states are very different, using one of the voltage solutions as an initial guess for the other might lead to a garbage solution or to no solution at all.


Solution: Use the plain voltages profile. This is 1.0 p.u. for all PQ nodes, and the generator set point voltage for the PV nodes and the slack node. In the rare event that this does not lead to a solution, a voltage initial guess from a close situation might help. However experience shows that if this is the case there are other problems to consider.


4. Transformers virtual taps issue

If you connect a transformer end rated at 11kV to a bus-bar rated at 10 kV, it is because by design you expect to have 11 kV / 10 kV = 1.1 p.u. voltage at the bus bar. However if you connect a 132 kV transformer end to the same 10 kV bus-bar, the voltage would be 13.2 p.u. and that is probably not a good decision. In fact it is an error. This is a very common error that slips under the radar, because sometimes despite the wild virtual taps generated the numerical algorithm finds a solution.


Solution: Check that the difference between the transformer ends' voltage ratings and the buses voltage rating are not too different, say a 10% maximum. Most likely the transformer is connected backwards, so flip it.


5. Reactive power transport issue

Sometimes the reactive power is not generated close enough to where it is needed and the grid cannot transport it efficiently enough. In these cases, the numerical algorithm cannot find a solution and it is quite difficult to tell why.

The obvious solution is to provide more reactive power close to where it is needed. For instance, locating capacitor banks or generators with reactive power capability.

But believe it or not, there is an easier solution; Does your load characterize the reactive power properly? Sometimes not. Specially in models used in planning where the reactive power is derived from the active power using a simple rule. Ensure that there are loads that generate reactive power as well as loads that consume it, because that is what happens in real life.


6. Incompatible controls

What if you have a transformer controlling the voltage of a bus and a generator controlling the voltage of a bus nearby and it is almost impossible to transport the necessary active and reactive power to satisfy that control situation? Then the algorithm will not find a solution, or the convergence properties deteriorate greatly. To avoid this we need to:

  • Avoid concurrent controls. This is, avoid having more than one device controlling the same magnitude.

  • Avoid nearby controls that control the same magnitude to different values.

  • In general, avoid having too many controls. The power flow algorithm is an optimization function and sometimes it can stagnate or fail to converge with too many controls.

7. Singular Jacobian matrix

Sometimes you may be notified by the power flow program that the Jacobian matrix is singular, hence the power flow is not solvable. What does this mean?

The Jacobian matrix is a matrix composed by the derivatives of the power flow equations with respect to the voltage module and the voltage angle. This matrix is used to solve a linear system to compute the voltage magnitude and angle increments in the Newton-Raphson (and Newton-Raphson like) algorithms. This is done at each iteration to determine the next step, so if the linear system is not solvable, there is no next step to be found.

In practice, this means that the power system is in voltage collapse, but how? This happens if the load exceeds what the grid can transport by the declared impedances. So here there are only two things to consider:

  • Is the load way too high? If so, consider grid reinforcements. But most likely, the issue is with the next item.

  • Are the impedances correct? This one relates to the rest of this article where bad data, wrong transformer connections and the likes may be playing a part in the Jacobian matrix singularity.


Final thoughts

The more hands that touch a model (of any kind) the greater the risk of bugs. That sentence could probably be set in stone, but calls the question, can we do any better? I believe we can if we use expert rules like the ones described before in conjunction with computer software that oversees the models creation. This is not a new idea and in computer programming languages, such tools are called linters. So, can we create a linter for power systems? Yes we can, and we definitely should.







176 views0 comments

Updated: Jul 12


Summary

  • The integrated framework is 1300% more efficient than the business as usual.

  • 2 million euros in productivity savings can be expected in a 5 year period.

  • The technical implementation is expected to take about a year.

  • Once removing the technical barriers, incremental developments are possible.

  • Resistance to cultural change is the main drawback.


Introduction

We are immersed in the great transition from fossil fuels to cleaner energy sources. To enable this transition, we need to perform an astounding amount of calculations, but the methods and tools that are commonly used are sorely lacking. The tools’ interoperability issues have become the bottleneck in the decision making processes related to energy infrastructure. There is too much at stake to continue with the business as usual.


The modeling situation

Investment decisions in the electrical sector have always been hard to make because the effects of adding or removing equipment in an electrical system are not linear. For instance, if we see that installing a 3MW transformer near a city produces a benefit, we cannot ensure that installing two of them will produce twice the benefit. It may be worse, it may be better. This is due to the different network effects that one or two transformers produce depending on their location and utilization once installed. The network effects are counterintuitive to the human brain, hence the need for specialized software.


In the past, energy sources were predictable (coal, nuclear, hydro, oil and gas), so the grid investment studies were simpler; The study of the peak demand and valley demand were sufficient to assess the performance of an asset. This is no longer a valid approach. In the last 20 years, a significant part of energy produced has evolved from being stable and polluting, to being variable and less environmentally damaging. Suddenly, the evaluation of a couple of scenarios could not determine whether a given investment was going to be sufficient or not due to the variability introduced by the solar radiation and the wind. To this uncertainty, we must add climate change and the instability of the fossil fuels supply.


Like every great challenge, there is more than one aspect to it. If we break down the exact issues becoming bottlenecks to policy and investments decision making, on one side, we have the excruciating difficulty to collaborate when creating models of the infrastructure, and on the other side, the model evaluation and debugging times that span from hours to weeks depending on the model accuracy and size. Sadly we must add that the mainstream software programs are deliberately not interoperable, making it very hard to build lean processes that alleviate the workload.


Naturally, these issues produce delays, unsatisfactory results and rotation in the engineering teams. This situation has been going on for at least 15 years and there are few efforts at the moment moving in the right direction.


Reinventing the wheel

If we look at the described situation with fresh eyes, we cannot help but wonder, why is everything so old fashioned? Why is there no “google docs” for models? Why is there no “Git” for models? Why do I have to keep sending excel files over email? (well, maybe just dropping them into a shared folder) Our conclusion is that there has been a lack of innovation because the dominant software is very hard to replace, because to do so requires reinventing the wheel; reinventing power flow, reinventing optimal dispatch, reinventing the file formats, etc.


Solving the collaboration issue; The field is dominated by closed standards and formats. Mainly those matching the mainstream software manufacturers’ views. What about a simple and open electrical modeling format? While working at the planning department of REE, one of the main things we did was to design a simple yet complete Json file format for exchanging data among all applications. Later this led to the creation of a collaboration and tracking system that allowed users to create models with full tractability. CIM/CGMES, despite being a kind of standard format, was out of the question due to its gratuitous complexity. Simplicity is the ultimate form of sophistication, or so we believe.


Solving the interoperability issue; The base situation is usually that all the modeling is done with a closed source program that becomes the centerpiece. This constraints innovation and forces users to come up with maybe too creative solutions to hack this. We discovered that the best strategy was to have such programs as side pieces of an open modeling system that uses the open and simple format mentioned before, so the hacking is not needed at all. This allows users to continue using their preferred software while allowing competition and innovation to improve the data processing pipeline. For instance, now we can run hundreds of simulations in the cloud with a custom developed calculation engine, while still being able to run one of them in the desktop with the commercial dominant software.


Solving the computation capacity issue; The commercial software today was designed in the 1970’s and it has been made abundantly clear that they’re not going to change the design to new computational paradigms. Fine, we have just detailed how to get out of the closed ecosystem by having our simple and open modeling format. This allowed us to build a competitor piece of software to run on the cloud at scale unlike no manufacturer product today. More calculation capacity equals greater ability to run more scenarios to better understand the energy transition. A simple and open file format and computer programs developed around it made us free to solve the collaboration, interoperability and computation capacity issues.


The cost of business as usual

The typical planning business as usual workflow involves the use of traditional power systems (grid modeling) software and the use of a market modeling software to simulate the market effects of the electrical infrastructure. The process involves quite some file preparation and model adaptation. The diagram looks like this:


Evaluating a single decision costs at a employee internal rate of 40€/h:

Step

Time

Cost

Design changes on the grid modeling software

*8 h

320 €

Prepare the input data coherently for the market model

168 h

6.720 €

Adjust the market model results to the grid model software

40 h

1.600 €

Prepare a report

8 h

320 €

Total

224 h

8.960 €

* These times may vary.


We can do better

The business as usual workflow has plenty of steps in which a person has to intervene to adapt the data for the next step. That is a source of friction where mistakes can happen and people get frustrated.


If we observe the modeling workflow from a processes point of view, we immediately find that we have many “sources of truth” that evolve over time with no tractability. We also observe that we are forced into that because the grid modeling software and the market modeling software are incompatible. What if everything was compatible? Then we arrive to a much more lean process where the employees only need to intervene where they add value:


In this process, the data resides in a single place, where the modeling software loads and saves the different models coherently and tractably. Then the modeling software can be provided with custom work routines such as Electrical plus market simulations that produce coherent results, which are processed into automatic reports. The comparable cost scheme improves radically:

Step

Time

Cost

Design changes on the grid modeling software

*8 h

320 €

Prepare a report

*8 h

320 €

Total

16 h

640 €


1300% cost improvement


Now let’s say that we run 50 of these processes per year. With the business as usual approach that is 448k€ per year only on work labor, not counting the ad-hoc software maintenance or the process variations that may make that pipeline even slower. With the improved workflow, the cost of performing the same 50 runs is 32k€ per year. That makes up over 2 million € in productivity savings in five years.


Timeline, side effects and drawbacks

We have outlayed how to make the decision making process leaner and we have calculated the economic benefit. How do we implement the new data architecture? The experience in the planning department of REE is that everyone in the department was aligned with the need for change. After presenting the solution, everyone concluded that the need for a central source of truth was imperative. However, changes take time. In our case there has been a 3 year changing period that includes the research and development to produce the software designs and implementations. There is still a fourth year to go. This long period is due to the fact that we have been changing the workflow while the old workflow was still being used. It would be reasonable to expect that similar change would take about a year in an organization with a clear implementation project, where the software was not an unknown variable like it has been for us.


We have streamlined our decision making process, but far more importantly, we have removed most of the technical barriers for innovation. Now, if we need to add a field to the data model, we just do it. We don’t need to beg the manufacturer anymore. The same goes for the development of new functionality; if we need a new specialized type of simulation, we can develop it in our framework incrementally. The benefit of being able to do something that was impossible before is infinite.


The change has been quite radical. This does not come free from resistance; not everyone shares the same vision of the solution and many are still emotionally attached to the closed manufacturer's solutions. Mostly because of the personal edge gained from being experts in certain manufacturer software. Naturally, if those software and workflows lose importance, those people feel that they get relegated as well. The reality is that those people become more capable than before to provide their best insight, but it is difficult to recognize it at first. Certainly the resistance to cultural change is the most important drawback.


Summary

  • The integrated framework is 1300% more efficient than the business as usual.

  • 2 million euros in productivity savings can be expected in a 5 year period.

  • The technical implementation is expected to take about a year.

  • Once removing the technical barriers, incremental developments are possible.

  • Resistance to cultural change is the main drawback.


187 views0 comments