The objective of this project is to develop the Particle Swarm Optimization (PSO) algorithm from scratch in Python. It aims to implement the core principles of PSO, including the initialization of particles, velocity updates, and position adjustments, while allowing particles to converge towards optimal solutions based on the balance between exploration and exploitation.
In addition to developing the PSO algorithm from scratch, I also used the pyswarms package to test the algorithm on a few selected objective functions which allowed to assess the algorithm's performance.
This project was developed using a Notebook. Therefore if you're looking forward to test it out yourself, keep in mind to either use a Anaconda Distribution or a 3rd party software that helps you inspect and execute it.
Therefore, for more informations regarding the Virtual Environment used in Anaconda, consider checking the DEPENDENCIES.md file.
Particle Swarm Optimization (PSO) is an optimization technique inspired by the social behavior of birds and fish. It involves particles (potential solutions) moving through a search space influenced by both their own best positions and the group's best solution.
Key elements include particle velocity, inertia, and social and cognitive acceleration factors, which help balance exploration and exploitation of the search space.
PSO requires few hyperparameters which makes it versatile and suitable for various tasks. Adaptive PSO variations adjust parameters dynamically to improve optimization performance.
Given the characteristics of the algorithm, I have chosen a set of objective functions to test its performance on.
Objective Functions assess each particle's position and provide a fitness value, allowing the algorithm to iteratively refine solutions and approach the global optimum.
These functions are designed to represent a variety of optimization challenges, providing a comprehensive evaluation of how well the algorithm adapts to different types of problem spaces.
|
Objective Functions Selected
|
||
|---|---|---|
|
Name
|
Formula
|
3D Graph
|
|
Sphere
|
f(x) = ∑di=1 xi2
|
|
|
Rastrigin
|
f(x) = 10d + ∑di=1 [xi2 - 10 cos(2πxi)]
|
|
|
Rosenbrock
|
f(x) = ∑d-1i=1 [100(xi+1 - xi2)2 + (xi - 1)2]
|
|
Here are the results obtained by using the pyswarms package to minimize the previously selected objective functions.
|
PSO Results
|
|||
|---|---|---|---|
|
Objective Function
|
Cost History
|
2D Particle Search
|
3D Particle Search
|
|
Sphere
|
|
|
|
|
Ratrigin
|
|
|
|
|
Rosenbrock
|
|
|
|
These results demonstrate how effectively the algorithm minimizes these functions, highlighting its versatility and performance across diverse problem landscapes.
README.md by Gonçalo Esteves