Experiment Tracking

Spell is an MLOps platform for training and deploying machine learning models in a convenient way. It has tools necessary for model training, hyperparameter search, and experiment management within an intuitive UI to give you more visibility into your ML models and better deliver results.

Replicate is a version control system for machine learning models. It’s an open-source platform that helps to quickly ship ML models to production. It also helps to run and re-train them. Replicate is made of two tools: Keepsake – a Python library that uploads files and metadata to Amazon S3 or Google Cloud Storage. Cog – lets you define models in a standard format, store them in a central place, and run them anywhere.

Datmo is a command-line interface (CLI) and collaborative web platform that helps to track and share work across teams when building algorithms. It offers an end-to-end workflow solution that includes production-level orchestration. Datmo equips your existing workflow with model versioning, environment setup, and experiment reproducibility.

ClearML (formerly Trains) is a complete, open source ML / DL experimentation and MLOps solution. ClearML eliminates the time-consuming and error-prone tasks associated with development, version tracking, and the full ML lifecycle for automation and scaling. The tool comprises the ClearML Python Package, ClearML Hosted Service (or your own self-hosted ClearML Server), and the MLOps ClearML Agent that make a unified solution

Kubeflow is an open-source cloud-native machine learning platform for orchestrating complicated machine learning workflows on containerized environments using Kubernetes.

ModelDB is an open source machine learning model versioning, metadata, and experiment management.

Determined AI is an open-source deep learning training platform for training models with built-in hyperparameter tuning and distributed training.

Guild AI is an open-source experiment tracking toolkit that supports common machine learning frameworks such as PyTorch and TensorFlow.

Deepkit is a real-time open-source machine learning tool and training suite. It has all the tools necessary for experiment execution, tracking, and debugging. It enables smooth team collaboration and experimentation with ML models on different levels.

Comet is a platform for managing the machine learning lifecycle. Comet users are able to track, compare, explain and reproduce machine learning experiments.

Weights and biases is a platform for tracking and visualizing machine learning experiments as well as team collaboration

MLflow is an open-source platform for managing the machine learning lifecycle, including experimentation, reproducibility, deployment, and a central model registry

Neptune is an experiment tracking hub bringing organization and collaboration to data science projects. Neptune records your entire experimentation process – exploratory notebooks, model training runs, code, hyperparameters, metrics, data versions, results, exploration visualizations, and more. Everything is stored and backed-up in an organized knowledge repository, ready to be accessed, analyzed, shared, and discussed with your team. No matter what type of problems you are working on, Neptune fits them all, from evaluating credit risk to finding the nuclei in divergent images.

Amazon SageMaker is a cloud machine-learning platform that enables developers to create, train, and deploy machine-learning models in the cloud. It also enables developers to deploy ML models on embedded systems and edge-devices.

TensorFlow is an open-source machine learning library that supports multiple programming languages including Python and Javascript. TensorFlow provides various tools for building and bringing deep learning models to production. The library became more popular in its second version(TensorFlow 2.x) because of integrating Keras as its high-level API. Keras is an open-source library that provides a simple to use Python interface for neural networks.