Experiment tracking becomes necessary long before a full MLOps platform does. Teams usually notice the need when metrics live in chat messages, hyperparameters live in notebooks, and nobody can explain why model version final_v7_really_final outperformed the others.

MLflow is a good fit precisely because it can start small. The goal is not to install a platform empire. The goal is to capture runs, parameters, metrics, and artifacts with low operational friction.

MLflow observability screenshot from the official site

Image source: MLflow.

The Low-Overhead Architecture

For small and mid-sized teams, the clean setup is usually:

  • MLflow tracking server
  • lightweight backend store for metadata
  • S3 on AWS or GCS on GCP for artifacts

The point is to separate experiment metadata from large binary artifacts. That keeps the server simple while letting cloud object storage handle scale.

One way to think about tracking value is:

$$ Value_{tracking} \propto \frac{Runs\ comparable}{Setup\ friction} $$

If setup friction is too high, teams stop logging. If logging stops, the system is useless regardless of feature depth.

Minimal Local-to-Cloud Setup

Start with the smallest viable command surface:

pip install mlflow boto3 google-cloud-storage
mlflow server \
  --host 0.0.0.0 \
  --port 5000 \
  --backend-store-uri sqlite:///mlflow.db \
  --artifacts-destination s3://my-mlflow-artifacts

On GCP, swap the artifact destination to a GCS bucket:

mlflow server \
  --host 0.0.0.0 \
  --port 5000 \
  --backend-store-uri sqlite:///mlflow.db \
  --artifacts-destination gs://my-mlflow-artifacts

This is enough for many teams to move from chaos to traceability.

What the Training Code Should Look Like

Keep client instrumentation minimal so people actually use it.

import mlflow

mlflow.set_tracking_uri("http://mlflow.internal:5000")
mlflow.set_experiment("fraud-baseline")

with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.001)
    mlflow.log_param("batch_size", 64)
    mlflow.log_metric("val_auc", 0.912)
    mlflow.log_artifact("reports/confusion_matrix.png")

That is the right starting point. Do not begin with custom plugins, multiple registries, or cross-region replication unless the team is already blocked by scale.

AWS vs GCP Decision

The choice is usually operational, not architectural:

  • choose AWS if the team already standardizes on IAM roles, S3, and VPC-hosted services
  • choose GCP if the team already runs workloads around GCS, service accounts, and managed Postgres options

MLflow itself does not force strong vendor lock-in. That is one of its main advantages.

When to Upgrade the Setup

The minimal design stops being enough when:

  • concurrent users overload the local metadata store
  • artifact volume makes cleanup and retention a real problem
  • you need stronger access control or managed backups

At that point, replace SQLite with a managed relational backend and keep the rest of the interface stable.

Practical Advice

The best experiment tracking system is rarely the most elaborate one. It is the one that your team will actually use every day without opening an infrastructure ticket.

Start with one server, one bucket, and disciplined logging. Complexity can be added later. Recovering missing lineage is much harder.

References