The use of Artificial Intelligence and Reinforcement Learning for industrial optimization has its roots in the successes of control theory. Early advances in Reinforcement Learning were inspired by behavioral psychology where good behavior is reinforced with a reward and bad behavior with a punishment. The environment with which the agent is interacting is usually formulated as Markov Decision Process and solution is found by solving Bellman equations.

The latest breakthroughs in Reinforcement Learning allow us to deploy agents that don’t need to know anything about the environment in order to find the optimal behavior strategy.

An agent can learn by experiencing the environment through actions and rewards. Algorithms such as Proximal Policy Optimization developed by Open AI researchers allow agents to efficiently operate in the environments where action space is continuous: for example when setting voltage level.

However, learning directly by interacting with environments is not always practical. This is why we developed an approach for training the agents on virtual environments by using the data from control systems like SCADA or any other Industrial IoT gateways if those are available. The goal of learning is to understand the complex patterns of the systems and find actions that maximize set rewards. Rewards, of course, depend on goal that is to be achieved. One may choose to maximize utility, safety or revenue. Of course, it’s possible to combine multiple goals.

Artificial Intelligence and Energy Assets

Finding the optimal operating strategy of energy assets is an every challenging problem that only becomes harder as the usage patterns become more volatile.

For example, in Power Utilities, the rise of renewables and distributed energy sources increases the number of actions and their interdependencies available to an operator. Improving environmental performance without increasing costs or endangering reliability requires a help of an artificial intelligence capable of consuming and analyzing the great amount of data available today. In Oil & Gas, the operators must come up with new operating models capable of accounting for short and mid term market dynamics while keeping their operations safe, environmentally friendly and profitable.

The state of art Artificial Intelligence and Reinforcement Learning will revolutionize the energy sector by enabling of superhuman performance in the modern challenges. Expert rule-based systems and abstract models can no longer be sufficient and the leaders of the industry must adopt intelligent systems capable of controlling nonlinear, non-stationary hidden environments in order to keep the operations optimal.

Comparing Volitant AI to GE Advanced Controls OpFlex, Schneider Electric Foxboro Evo Process Automation System, Siemens SIMATIC Automation Tool and others

Today many operators use rules or abstract models to dispatch assets. Solutions like GE’s Advanced Controls OpFlex, Schneider Electric’s Foxboro Evo Process Automation System, Siemens SIMATIC Automation Tool and other allow trained teams to create customized models that work for a specific power plant. Energy systems are difficult to model due to their parameters dependence on the time-varying signals. Computer simulation of the behavior of complex systems and components, which requires the solution of many equations and extensive use of closure relations, has become very important in modern design. Such models require the full knowledge of the expert to be digitized and converted into steps that can be used in decision making. However, such solutions are often constrained and are not able to guarantee optimal operation in energy systems. Volitant AI’s reinforcement learners, on the other hand, are able to learn directly from their experience of the energy systems. They can efficiently deal with nonlinearity, non-stationarity and hidden environments in the energy systems.