Python Tools for Model Fitting and General Optimization

Optimization Algorithms

• Solid - A Python framework for gradient-free optimization
  • Include algorithms like Genetic Algorithm, Stimulated Annealing, Particle Swarm, Harmony Search, and Stochastic hill Climb
• GAFT - A Genetic Algorithm Framework in Python
  • 有一系列中文blog介绍背后设计和算法，非常适合学习遗传算法
• pyswarms - A research toolkit for particle swarm optimization in Python
  • Wikipedia introduction of PSO: Particle Swarm Optimization
  • Document includes a very nice introducation of the PSO algorithm
• The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) Algorihm
  • A stochastic derivative-free numerical optimization algorithm for difficult (non-convex, ill-conditioned, multi-modal, rugged, noisy) optimization problems in continuous search spaces.
  • pycma - A Python implementation of CMA-ES
• proxmin - Proximal Minimization in Python
  • Based on the paper by Fred Moolekamp & Peter Melchoir (2018)
  • Includes algorithms like Alternating Direction Method of Multipliers (ADMM) and Block-Simultaneous Direction Method of Multipliers (bSDMM)
  • Used as the backend of multi-band deblending code scarlet
• pagmo2 - A C++ / Python platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model
  • A a scientific library for massively parallel optimization used by ESA.
  • Document can be found here
  • Includes a long list of global and local optimization algorithms
• symfit - Symbolic Fitting; fitting as it should be
  • Pythonic way to do fitting.
• zfit - scalable pythonic fitting
  • The zfit package is a model manipulation and fitting library based on TensorFlow and optimised for simple and direct manipulation of probability density functions. Its main focus is on scalability, parallelisation and user friendly experience.

Bayesian Optimization

• BayesianOptimization - Pure Python implementation of bayesian global optimization with gaussian processes
  • Bayesian optimization works by constructing a posterior distribution of functions (gaussian process) that best describes the function you want to optimize.
• gaussbock - Fast parallel-iterative cosmological parameter estimation with Bayesian nonparametrics
• vbmc - Variational Bayesian Monte Carlo (VBMC) algorithm for posterior and model inference
  • VBMC is a novel approximate inference method designed to fit and evaluate computational models with a limited budget of likelihood evaluations (e.g., for computationally expensive models).
  • A Python implementation is planned.