- Type
- MLOps and LLMOps platform
- Founded
- 2017
- Headquarters
- New York, USA
- Key use
- Experiment tracking, model registry, LLM evaluation
- Notable users
- Netflix, NVIDIA, DoorDash
- Related
- Weights and Biases, MLflow, Neptune.ai, LangSmith
- Type
- MLOps and LLMOps platform
- Founded
- 2017
- Headquarters
- New York, USA
- Key use
- Experiment tracking, model registry, LLM evaluation
- Notable users
- Netflix, NVIDIA, DoorDash
- Related
- Weights and Biases, MLflow, Neptune.ai, LangSmith
Comet ML is a machine learning operations (MLOps) platform that provides experiment tracking, model versioning, dataset management, production monitoring, and large language model evaluation capabilities. Founded in 2017, Comet ML targets enterprise AI and data science teams who require reproducibility, collaboration, and governance features across the full model development lifecycle. It is used by AI teams at organisations including Netflix, NVIDIA, and DoorDash.
Background
As machine learning moved from research projects to production systems, teams encountered systematic challenges in managing the proliferation of experiments. A data scientist might train hundreds of model variants while tuning hyperparameters, each with a different configuration, dataset version, and set of metrics. Without dedicated tooling, tracking which configuration produced the best model and reproducing results was difficult. Comet ML was founded to address this problem, positioning itself as the system of record for ML experiment metadata.
The platform has since expanded beyond experiment tracking to cover the full model lifecycle, including production monitoring and, from 2023 onwards, evaluation tooling for LLM-based applications.
Core Features
Experiment Tracking
Comet ML automatically captures metrics, hyperparameters, environment information, source code, and git commit hashes when a developer instruments their training script with the Comet SDK. This instrumentation requires only a few additional lines of code and is compatible with major frameworks including PyTorch, TensorFlow, Keras, scikit-learn, and Hugging Face Transformers.
Experiments are displayed in a web-based dashboard where users can compare runs across any logged parameter or metric, view training curves, inspect model artefacts, and share results with collaborators. Custom visualisations including confusion matrices, ROC curves, histograms, and embedding projections can be logged directly from training code.
Model Registry
Comet ML's model registry provides version control for trained models, enabling teams to promote model versions through stages such as staging, production, and archived. Each registry entry links back to the experiment that produced it, providing full lineage from training run to deployed model. This lineage is increasingly expected by financial and healthcare sector auditors who require documentation of how production models were developed and validated.
Production Monitoring
The platform includes tools for monitoring deployed models over time, detecting statistical drift in input data distributions and output distributions that may indicate model degradation. Alerts can be configured to notify teams when model performance metrics fall below specified thresholds.
LLM Evaluation
From 2023, Comet ML expanded its offering to include LLM evaluation and observability under the Opik product line. Opik provides open-source tooling for tracing LLM application calls, scoring outputs using automated LLM-as-judge evaluations, and managing prompts. This positions Comet ML as a competitor to dedicated LLMOps tools such as LangSmith and Langfuse, with the advantage of integration with its existing experiment tracking infrastructure.
Positioning and Market
Comet ML is frequently compared with Weights and Biases and MLflow. Weights and Biases has a strong position in the research and academic community, emphasising rich visualisation and collaboration features. MLflow, an open-source project originally created by Databricks, is widely deployed in enterprise settings where Databricks or Apache Spark are already in use. Comet ML differentiates itself with a stronger emphasis on enterprise access controls, role-based permissions, on-premise deployment options, and integrations with enterprise data and security infrastructure.
The platform supports single sign-on through SAML providers, role-based access control at the project and workspace level, and on-premise or private cloud deployment for organisations with strict data residency requirements.
Malaysian data science and AI teams at large enterprises, including banks such as Maybank and CIMB, telecommunications providers such as Maxis and TM, and energy companies such as Petronas, use experiment tracking and model governance tools as part of their MLOps maturity programmes. Comet ML's enterprise features — role-based access control, model lineage, and on-premise deployment — align with the governance requirements that these regulated industries face under Bank Negara Malaysia guidelines and the Malaysia AI Governance Framework.
The Securities Commission Malaysia's guidance on AI in capital markets and BNM's risk management expectations for financial institutions both implicitly require model governance capabilities that experiment tracking platforms like Comet ML provide. Being able to demonstrate that a production credit scoring or fraud detection model is traceable to a specific training run, dataset version, and evaluation result is increasingly part of what Malaysian financial regulators expect.
Malaysian universities and MDEC-affiliated institutions offering data science and AI programmes include MLOps as a curriculum component. Comet ML, alongside Weights and Biases and MLflow, appears in advanced data science courses at Universiti Malaya, Universiti Teknologi Malaysia, and in professional training programmes funded by HRD Corp. Practical familiarity with experiment tracking platforms is becoming a baseline expectation for AI engineering roles in the Malaysian job market.
The open-source Opik component of Comet ML's LLM evaluation offering is relevant to Malaysian AI startups that need LLMOps tooling without enterprise pricing. Its compatibility with self-hosted deployment means that teams building applications on local or regional cloud infrastructure can maintain full data sovereignty while still benefiting from systematic LLM evaluation capabilities.
See Also