Model Tuning and HPO

Weights and biases is a platform for tracking and visualizing machine learning experiments as well as team collaboration

Comet is a platform for managing the machine learning lifecycle. Comet users are able to track, compare, explain and reproduce machine learning experiments.

Optuna is an open-source hyperparameter optimization framework for automating hyperparameter search. Optimizations are for an objective function in different trials. It supports popular machine learning frameworks such as TensorFlow, PyTorch, and Scikit-learn

auto-sklearn is an automated machine learning toolkit built on top of Sklearn. It performs hyperparameter tuning as well as algorithm selection. It works by taking advantage of Bayesian optimization, meta-learning, and ensemble construction.

HyperOpt is a Python package that provides distributed asynchronous hyperparameter optimization.

SigOpt is a cloud-based optimization platform. It’s a model development tool that makes it easy to track runs, visualize training, and scale hyperparameter optimization for any type of model built with any library on any infrastructure. SigOpt’s REST API and client libraries (Python, R, Java) integrate into any existing ML workflow. It’s a robust tool with many functional features

Kera Tuner is a library for distributable hyperparameter tuning in Keras. It enables the tuning of various parameters in TensorFlow models.

TPOT (Tree-based Pipeline Optimization Tool) is a Python Automated Machine Learning software (AutoML) that optimizes machine learning pipelines using genetic programming. TPOT automates even the most tedious parts of machine learning by intelligently exploring thousands of possible pipelines to find the best one for your data. It’s built on top of scikit-learn.

Talos is a tool that fully automates hyperparameter tuning and model evaluation. It exposes Keras functionality entirely and there is no new syntax or templates to learn. It’s a good tool if you want to remain in complete control of Keras models in an easy and automated way, Talos allows you to quickly configure, perform, and evaluate hyperparameter optimization experiments that yield state-of-the-art results across a wide range of prediction tasks.

Scikit-Optimize is a simple and efficient library that implements several methods for sequential model-based optimization. The library is built on top of NumPy, SciPy, and Scikit-Learn.

RayTune is a product of Ray (others include RLlib, RaySGD, and RayServe) – a simple, universal API that lets you build distributed applications. Tune is an open-source framework that allows for hyperparameter optimization to quickly increase your model performance. Tune leverages a variety of optimization algorithms, reducing the cost of tuning by aggressively terminating bad hyperparameter evaluations, intelligently choosing better parameters to evaluate, or even changing the hyperparameters during training to optimize hyperparameter schedules

Katib is a an open-source Kubernetes-native project for automated machine learning (AutoML). Katib supports hyperparameter tuning, early stopping, and neural architecture search (NAS). It helps to tune your hyperparameters by testing a range of values for each one. You define what your objective is and set up, and experiment so Katib can try different hyperparameter values.