Abstract

This study introduces an adaptive intelligent agent-based flog leaping optimizer to tackle the economic load dispatch problem in power systems, specifically addressing valve-point effects. Unlike conventional non-traditional algorithms, it offers a more dynamic and deterministic problem-solving strategy, characterized by its simplicity, usability, convergence efficiency, solution quality, and robustness. To enhance the performance of the shuffled frog leaping algorithm (SFLA), which may suffer from slow exploration in later iterations and susceptibility to local optima, this paper proposes the fusion of Adaptive multi-agent-based evolutionary reinforcement learning with the leaping algorithm. This hybrid approach capitalizes on the complementary strengths of both algorithms. By leveraging this synergy, this method demonstrates superior performance, achieving optimal results with reduced global and local iterations and it also limits the stochastic approach. The proposed hybrid methodology and its variations are rigorously evaluated using two distinct test systems, including 13 and 40 thermal unit systems with incremental fuel cost functions considering valve-point effects. The experimental results demonstrate the efficacy and promise of the proposed approach, outperforming several benchmark techniques commonly used in the field.

1. Introduction

The economic load dispatch (ELD) problem, which is a non-convex non-linear constraint optimization problem, always encourages researchers to develop new techniques that reduce not only the cost of power generation but also increase the reliability and decrease the greenhouse effect by reducing thermal power production. So, it is very much needed to explore more next-generation modern optimization techniques to cater to the growing number of ELD complexity and optimize the load dispatch problem.

For solving load dispatch problems, several traditional mathematical, computational, and intelligent techniques have been developed so far. Conventional methods such as lambda iteration, gradient-based method ¹, and homogeneous linear programming (LP) ² are used to solve the ELD problem by altering the fuel cost curve in a monotonically increasing function or piecewise linear function. These approaches disregard the portions of the incremental cost curve that are not monotonically or continuously increasing. Because of valve point loadings, ramp rate limits, etc. modern power generation units’ input-output characteristics are intrinsically non-linear. So, the fuel cost curve is estimated according to the necessity but the use of such estimation may lead to a massive loss of revenue over time in the conventional optimization solution. Dynamic programming, proposed in ³, is a method to solve discontinuous and non-linear ELD problems but increasing system size increases simulation time in this method. Different non-traditional optimization techniques based on AI have been effectively used to solve ELD problems. These methods are evolutionary programming ^{4, 5}, particle swarm optimization ⁶, tabu search ⁷, differential evolution ⁸, biography-based optimization ⁹, genetic algorithm ^{10, 11}, artificial neural network ¹², intelligent water drop algorithm ¹³, etc. Some studies have been done for ELD problems with valve point effect such as novel niche quantum genetic algorithm ¹⁴, hybrid quantum mechanics inspired particle swarm optimization ¹⁵, combining of chaotic differential evolution and quadratic programming ¹⁶, biogeography-based optimization ¹⁷, hybrid solution methodology integrating particle swarm optimization (PSO) algorithm ^{18, 19, 20}, enhanced bee swarm optimization method ²¹, enhanced adaptive particle swarm optimization (EAPSO) algorithm ²², the sequential quadratic programming (SQP) method ²³ and artificial bee colony algorithm ²⁴. Every approach has its limits. Neural networks can become computationally expensive due to their iterative nature, leading to longer processing times. Genetic algorithms suffer from premature convergence, which can degrade their performance and limit their search capabilities. Particle Swarm Optimization (PSO) algorithms can progress slowly, especially when adjusting velocity step sizes becomes challenging, potentially hindering fine-grained search. In the case of multi-modal functions, PSO may struggle to reach the global optimum. Yet, regulating the control parameters in hybrid methods poses challenges, and enhancements often involve incorporating mutation operators. DE has been found to yield a better and faster solution, satisfying all the constraints, both for uni-modal and multi-modal systems by using its different crossover strategies. However, with an increase in system complexity and size, the DE method is unable to map its entire array of unknown variables together in an efficient way. A multi-objective-teaching–learning-based optimization algorithm ²⁵ and a modified teaching–learning algorithm ²⁶ are employed to solve the dynamic economic emission dispatch problem where problem formulation is much more complicated. In ²⁷ a self-adaptive modified firefly algorithm is suggested to solve the ELD problem. While this algorithm is well-suited for parameter tuning, it faces challenges related to getting stuck in local optima. Researchers have introduced a meta-heuristic algorithm known as the Shuffled Frog Leaping Algorithm (SFLA) ^{28, 29}. It aims to model and mimic the behavior of frogs searching for food laid on stones randomly located in a pond. It links the advantages of the social behavior-based particle swarm optimization (PSO) algorithm and the gene-based memetic algorithm (MA). The SFLA technique is used in unit commitment ³⁰. The basic Shuffled Frog Leaping Algorithm (SFLA) is enhanced by incorporating GA crossover during local iterations, resulting in the Modified Shuffled Frog Leaping Algorithm (MSFLA)to solve ELD problems with non-convex characteristics ³¹.

Many other metaheuristic optimization techniques like the Backtracking Search Algorithm (BSA) ³², Evolved bat algorithm ³³, Dragonfly Algorithm ³⁴, Grey Wolf Optimizer ³⁵, and Artificial Bee Colony Optimization ³⁶ employed for ELD problems. All these methods are challenging with problems like low convergence rates and sticking to local optima. Another population-based Sine Cosine Algorithm (SCA) has been proposed in ³⁷ where multiple initial random populations are generated and moved outward or toward the best solution. The look-ahead economic dispatch was evaluated using a sensitivity matrix under the influence of data corruption to make a fast evaluation ³⁸. A cooperative reinforcement learning (RL) algorithm was proposed to achieve distributed economic dispatch in a microgrid with energy storage systems ³⁹. A dynamic economic dispatch problem for integrated energy systems was solved using an improved DDPG algorithm ⁴⁰. A deep RL method incorporating with deep deterministic policy gradient method is proposed to solve the ELD problem ⁴¹.

In recent years researchers are paying more attention to how optimization problems can be integrated using machine learning techniques. The review paper ⁴² evaluates different techniques on how data-driven decision-making improves rule-based optimization. ⁴³ shows how the agent-based deep RL algorithm worked to mimic the decision process of real system operators by learning from past decisions and rewards in real-life power system applications. After careful consideration of all the research work, it is observed that the evaluation method for RL in power system economic dispatch remains under-explored, especially relating to its exceptional ability to adapt to various scenarios.

The paper introduces a novel hybrid algorithm, termed intelligent multi-agent evolutionary Reinforcement Learning for multi-objective ELD optimization problems. This problem involves optimizing the operation of power plants subject to various constraints, including valve-point effects that cause non-smooth and non-convex characteristics in the cost function.

The proposed method represents an innovative fusion of the Shuffled Frog Leaping Algorithm (SFLA) with reinforcement learning techniques, aimed at enhancing the optimization process for complex problems. SFLA draws inspiration from the social behavior of frogs, utilizing a population-based approach to search for optimal solutions. However, to mitigate stochastic behavior and improve convergence efficiency, we introduce an adaptive intelligent agent into the algorithm. This agent guides the frogs toward the best positions, thereby enhancing the exploration and exploitation capabilities of the algorithm.

Reinforcement learning is integrated into SFLA to further augment its performance. By iteratively applying reinforcement learning techniques, the algorithm dynamically adjusts its exploration-exploitation balance, thereby reducing the range of fitness values over iterations. This iterative refinement aids in converging towards the global optimum solution. The amalgamation of reinforcement learning with SFLA enables a synergistic combination of local and global search techniques, fostering a more robust and efficient optimization process compared to algorithms reliant solely on global search strategies.

To validate the effectiveness of the proposed method, extensive experimentation is conducted on various power system models, including those comprising 13 and 40 generating units. Through these experiments, the performance of the proposed method is thoroughly evaluated and compared against existing literature. The results demonstrate the superior accuracy, efficiency, and convergence properties of the proposed method in solving the Economic Load Dispatch (ELD) problem within power systems.

In summary, this paper introduces a promising hybrid algorithm that leverages the strengths of both SFLA and reinforcement learning to address complex optimization challenges in power systems. By offering improvements in accuracy, efficiency, and convergence compared to traditional optimization methods, the proposed approach holds significant potential for advancing the state-of-the-art in power system optimization.

2. Problem Formulation

A. Objective Function

The goal of the Economic Load Dispatch (ELD) problem, which exhibits non-convex characteristics, is to minimize the generation cost (). This cost primarily includes the fuel expenses incurred by thermal power plants, all while adhering to the operating constraints of a power system—constraints that span both linear and non-linear The fuel cost function is represented in equation (1) where respective cost coefficients and is the generated power of unit i.

The quadratic fuel cost function (shown in Figure 1) can be formulated as

Unit of cost/hr (1)

In this study, the valve-point effects are also taken into account as a complementary component of the objective function. Therefore, the cost function is described as the superposition of sinusoidal functions and quadratic functions. The cost function curve is shown in Figure 2.

The dotted line in Figure 3 shows the cost function variation considering the valve-point effects. The cost function is derived from the ripple curve to enhance the accuracy of our modeling. This curve contains higher-order nonlinearity and discontinuity due to the valve point effect and to incorporate such effects, sinusoidal functions are added with fuel cost function. Therefore, instead of using equation (1), the modified cost function is used, which is

(2)

or

(3)

Where, and are constants of the valve point effect of generators. Hence, the total fuel cost must be minimized according to equation (4).

(4)

B. Constraints

The objective of the ELD problem is subjected to the following equality and inequality constraints.

Equality Constraints

The total generated power from all units must equal the sum of the power demanded by the load and the total transmission loss. Mathematically, this constraint can be expressed as follows:

(5)

Where must be less than tolerance, as according to the power balance criterion, generated power (P_G) should be equal to total load demand (P_D) plus total line losses (P_L)?

The system transmission network loss is computed by Kron's loss formula, which represents loss as a function of the output level of the system-generating units. The matrix formulation is shown in (6).

(6)

In a system comprising N power plants, the loss coefficients are defined as follows:

(7)

To calculate transmission losses, apart from Kron's formula other methods such as Newton's approaches using polar coordinates and rectangular coordinates and several nonlinear techniques are also being used. Nowadays transmission losses are simultaneously determined in conjunction with optimal power flow calculation. In this paper, loss coefficients are calculated using the optimal power flow solution along with the load flow method.

Inequality Constraints

The power output of each unit (i) must be greater than or equal to the minimum power permitted and also be less than or equal to the maximum power permitted on that specified unit. Thus, the inequality constraint is expressed as:

(8)

Complications of the ELD problem with and without valve point effect considering both equality and inequality constraints, have been overcome by introducing a new adaptive intelligent agent-based flog leaping optimizer for the ELD problem.

3. Proposed Work-flow of Adaptive Intelligent Agent-based Flog Leaping Optimizer (AdaFLO) for ELD Problem

This optimizer is inspired by the frog leaping algorithm but Each frog iteratively adjusts its position guided by the intelligent agent. The frog's movement is influenced by the agent's guidance, aiming to improve the solution. The learning rate may dynamically adjust based on the performance of the frogs and the agent's observations.

The adaptive agent is the intelligent agent that evolves based on the frogs' behavior and exploration of the solution space. It dynamically adjusts its strategy to guide the frogs towards promising regions.

There are mainly two stages in the optimizer. First is the creation of the agent which will guide the frog to jump in the proper direction instead of random places along the search place. Hence, while creating the agent, we will be doing the exploration of the solution space. The second part of the algorithm is how the agent will help the frogs (individual solutions) to jump to the proper location within the solution space which is nothing but the exploitation of the specific solution spaces.

To remove the local optima, we will be effectively using the learning rate to generate the agent accordingly.

The optimizer algorithm is defined as follows:

1. generate a population of frogs randomly adhering to all the constraints

2. sort the population and find the best one

3. generate agent based on the best frog

a. for all the variables, take a constant, and for each interval, calculate the fitness value keeping all other variables fixed.

b. Calculate and sort the fitness values generated for all the combinations.

c. Take the order of the variables in ascending as well as descending order of the fitness values.

d. Observe and find the position of each variable lying under the solution space based on the fitness values. These positions corresponding to each variable are the weightage for each attribute that impacts the fitness value.

4. From the entire population, fetch the best one, worst one and randomly pick one.

5. Improve the position values for all these 3 frogs based on the agent recommendations.

6. Take all three newly positioned frogs into the population and find the best one.

7. Based on the learning rate, generate the agent (step 3) again and use the recommendations accordingly.

8. Go to step 4 and continue until certain global iterations are met.

9. Fetch the best fitness frog which is the optimal solution.

4. Simulation Result and Analysis

This section employs two examples to illustrate the effectiveness of the proposed intelligent multi-agent evolutionary Reinforcement Learning featuring SFLA concerning the quality of the solution obtained.

Case I: Generator number 13.

Case II: Generator number 40.

In all the cases transmission losses are considered as standard value and valve point effects are considered.

The program is developed in Python.

Case Study I:

The test system consisted of 13 units taking account of valve point loading. The accurate cost model using (equation 3), considers the valve point effect; the sinusoidal function is included in the quadratic cost function. In this case, the load demand expected to be determined was PD =1800MW.

The results obtained for case study I with solution quality and comparison with other methods are shown in Table 1 which shows that the AdaFLO succeeded in finding the best solution for the tested method compared to other benchmark methods. The proposed method is capable of producing a higher-quality solution than most evolutionary methods.

The best generation value of each unit is tabulated in Table 1. It can be inferred from the results that the solution obtained from the proposed AdaFLO method is better than other techniques. The mean cost achieved by the proposed method is found to be the least. This implies that, on average, the quality of solutions obtained by the method is better than other methods. Moreover, the average execution time for this algorithm suggests that the process is capable of solving at a very high speed.

Figure 3 shows how the variables got optimized by the reinforcement agent and finally converged into the final fitness value over the generations.

Table 2 reveals that FEP performed worst in terms of generation cost and MFEP took maximum computational time. AdaFLO possessed higher solution time compared to ICA-PSO, DEC-SQP, and BBO, as it runs global as well as local iterations simultaneously to get better solutions. It has better global search capability compared to other benchmark methods mentioned in Table 3. In this test case, it emerged as the most favorable method when considering convergence rate, solution time, and the likelihood of achieving improved solutions. The best result obtained for solution vector Pi, where i=1, 2 … 13 with the proposed approach with a minimum cost of 17238.43$/hr is given in Table 2.

Case Study II

Test case II consisted of 40 generating units, with modifications to incorporate the valve-point loading. The load demand is 10500 MW. Table 2 shows the best solution time and minimum cost achieved by the proposed method, compared with other benchmark methods. MFEP was the slowest among the twelve. The minimum generation cost obtained by AdaFLO is 121063.55 $/hr is the least among the remaining cases. It has been observed that the improved mechanism of AdaFLO owns stronger global search ability, higher precision, faster velocity of convergence, and better robustness.

Comparative study

Solution quality

From the above analysis, it is clear that the minimum costs achieved by AdaFLO are the lowest in all the case studies. Those are best and less than reported in other benchmark methods. It can be said that the performance of the AdaFLO method is consistent for large convex-type systems. For both convex and non-convex ELD problems, the method proves its ability to reach global minima in a consistent manner and its better convergence characteristic.

Computational efficiency

The AdaFLO approach is also efficient as far as computational time is concerned. The time requirement is significantly less and either comparable or better than other mentioned methods. In both the case study, it possesses less time compared to CEP, FEP, MFEP, IFEP, and QPSO. Only BBO, DEC-SQP.So as a whole, it can be said that the AdaFLO method is more computationally efficient than the previously mentioned methods.

5. Conclusion

In this paper, we introduce a novel approach that combines the Shuffled Frog Leaping Algorithm (SFLA) with Adaptive Intelligent Agent-Based Learning (AdaFLO) to tackle constraint economic dispatch problems, taking into account the valve point non-linearities of generators. We estimate an optimal range for global iteration, local iteration, and population size for AdaFLO to effectively solve various test cases. By conducting case studies on 13 and 40-unit test systems with valve-point effects and load flow constraints, we demonstrate the feasibility and effectiveness of the AdaFLO method. Our results, compared with other evolutionary techniques, highlight AdaFLO's capability to obtain optimal solutions for fuel cost functions of non-smooth and non-differentiable test systems.

The proposed hybrid method leverages the reinforcement learning operation and the Shuffled Frog Leaping Algorithm, offering a robust solution approach, even in scenarios demanding higher processing time. Employing a divide-and-conquer strategy in implementing this algorithm enhances its efficiency. Consequently, it emerges as a leading non-linear programming technique for addressing constrained optimization challenges, with its effectiveness being contingent upon the selection of the initial point.

Future research avenues could extend the application of this method to various domains within power system optimization, such as optimal power flow, voltage control, optimal capacitor placement, feeder balancing, and beyond.

References