Files

Upload and manage experiment artifacts — checkpoints, configs, results, and arbitrary blobs. Files are automatically checksummed, organized by prefix, and addressable by path.

For image uploads see /images. For frame buffers and ring-buffer patterns see /buffering. For time-series media tracks see /tracks.

API at a Glance

python

from ml_dash import Experiment

with Experiment(prefix="alice/project/my-experiment").run as exp:

    # Upload a file from disk
    exp.files("checkpoints").upload("./model.pt")
    exp.files("checkpoints").upload("./model.pt", to="best.pt")

    # Save Python objects directly
    exp.files("configs").save_json({"lr": 1e-3}, to="config.json")
    exp.files("configs").save_text("yaml: content\n", to="view.yaml")
    exp.files("data").save_blob(b"\x00\x01", to="data.bin")
    exp.files("checkpoints").save_torch(model, to="model.pt")
    exp.files("data").save_pkl(obj, to="data.pkl")
    exp.files("plots").save_fig(fig, to="loss.png")
    exp.files("videos").save_video(frames, to="rollout.mp4", fps=30)

save_* methods accept the same metadata kwargs as upload() (see below) and return a result dict with id, filename, path, sizeBytes, and checksum.

The unified save() method dispatches based on content type: strings that are file paths go through upload(), bytes through save_blob(), dicts/lists through save_json(), and numpy arrays through save_image().

Save with Metadata

Attach a description, tags, and arbitrary metadata to any saved file:

python

result = exp.files("models").save_torch(
    model,
    to="best_model.pt",
    description="Best model from epoch 50",
    tags=["checkpoint", "best"],
    metadata={"epoch": 50, "val_accuracy": 0.95},
)

Param	Type	Description
`to`	`str`	Destination filename (relative to prefix)
`description`	`str`	Human-readable description
`tags`	`list[str]`	Tags for filtering and search
`metadata`	`dict`	Custom JSON-serializable metadata

Paths and Organization

The argument to exp.files(...) is a logical prefix. Use it to group related artifacts:

python

exp.files("models").upload("model.pth")
exp.files("models/checkpoints").upload("best.pth")
exp.files("config").save_json(config, to="config.json")
exp.files("results").upload("results.csv")

Direct Style (Alternative)

You can also skip the prefix call and embed the full path in to=:

python

exp.files.upload("./model.pt", to="models/checkpoints/best.pt")
exp.files.save_json({"k": "v"}, to="configs/config.json")
exp.files.save_text("content", to="notes/view.yaml")

Listing, Downloading, Deleting

python

# List
all_files   = exp.files().list()
model_files = exp.files("models").list()
pngs        = exp.files("images").list("*.png")
configs     = exp.files().list("**/*.json")

# Download
exp.files("model.pt").download()                       # current dir
exp.files("model.pt").download(to="./local.pt")        # custom dest
paths = exp.files("images").download("*.png", to="./out")

# Delete
exp.files("some.txt").delete()
exp.files("images").delete("*.png")

Downloads automatically verify SHA256 checksums against the recorded value.

Updating Metadata

Update the description, tags, or metadata of an existing file by its file_id:

python

exp.files(
    file_id="abc123",
    description="Updated description",
    tags=["new", "tags"],
    metadata={"updated": True},
).update()

Deduplication

Each file is stored under a unique snowflake ID, so saving the same logical filename multiple times preserves all versions rather than overwriting. The path layout on disk is:

files/{prefix}/{snowflake_id}/{filename}

In remote mode, files land at s3://bucket/files/{namespace}/{project}/{experiment}/{prefix}/{file_id}/filename with metadata (path, size, SHA256, tags, description) in MongoDB. Maximum file size: 100 GB.

End-to-End Example

python

import torch
from ml_dash import Experiment

with Experiment(prefix="alice/cv/resnet").run as exp:
    exp.params.set(model="resnet50", epochs=100)

    best_acc = 0.0
    for epoch in range(100):
        loss, acc = train_one_epoch(model, loader)
        exp.metrics.log(epoch=epoch, train=dict(loss=loss, accuracy=acc))

        if (epoch + 1) % 10 == 0:
            exp.files("checkpoints").save_torch(
                model.state_dict(),
                to=f"epoch_{epoch + 1}.pt",
                tags=["checkpoint"],
                metadata={"epoch": epoch + 1, "accuracy": acc},
            )

        if acc > best_acc:
            best_acc = acc
            exp.files("models").save_torch(
                model.state_dict(),
                to="best.pt",
                description=f"Best model (acc={best_acc:.4f})",
                tags=["best"],
                metadata={"epoch": epoch + 1, "accuracy": best_acc},
            )