mlflow
MLflow ML lifecycle management. Use for ML experiment tracking.
Install
npx skillscat add g1joshi/agent-skills/mlflow Install via the SkillsCat registry.
We need to produce a 2-3 sentence plain-text summary, objective, factual, no marketing language, no superlatives, no calls to action. At most 60 words. Should explain what skill does, problem it solves, when to use. We need to avoid bullet points, headings, markdown. Just plain sentences. Let's craft: "The MLflow skill enables tracking of machine learning experiments, logging parameters, metrics, and artifacts, and supports GenAI tracing and model registry functions.
MLflow
MLflow is the standard for tracking experiments. v3.0 (2025) pivots to GenAI, adding LLM Tracing, Prompt Management, and "LLM-as-a-Judge".
When to Use
- Experiment Tracking: Logging hyperparameters (
lr=0.01) and metrics (accuracy=0.98). - GenAI Tracing: Visualizing the full chain of a RAG application.
- Model Registry: Versioning models (
my-model/v3) for deployment.
Core Concepts
Tracking URI
Where logs are stored (local ./mlruns or remote http://mlflow-server).
Autologging
mlflow.autolog() automatically captures params from Scikit-learn, PyTorch, etc.
LLM Tracing
OpenTelemetry-based tracing to debug prompt chains.
Best Practices (2025)
Do:
- Use
mlflow.evaluate(): To run "LLM-as-a-Judge" metrics on your RAG pipeline. - Use Prompt Engineering UI: MLflow 3.0 has a UI to iterate on prompts.
Don't:
- Don't use it for data storage: Log artifacts (models), not datasets. Log metadata about datasets instead.
References
Categories
Install
npx skillscat add g1joshi/agent-skills/mlflow Install via the SkillsCat registry.