Comparing Experiments

ML-Dash provides two ways to compare experiments: Live Compare (instant, no configuration) and Compare View (persistent, .dashrc-driven).

Live Compare

Live Compare is an auto-generated comparison view that requires no configuration file. It appears when you check two or more experiments in the experiment list.

How to Use

1. In the middle experiment list, check the checkbox of two or more experiments.
2. The Live Compare tab appears in the right panel automatically.
3. Charts are generated instantly — one chart per shared metric.

Two Comparison Modes

Live Compare behaves differently depending on whether the selected experiments share the same parent folder.

Same Parent Folder — Individual Comparison

When all selected experiments are in the same folder:

• Each experiment becomes one line on each chart.
• A metric is included if it exists in at least 2 of the selected experiments.

my-project/
├── run-lr0.001/   ✓ selected
├── run-lr0.01/    ✓ selected
└── run-lr0.1/     ✓ selected

Different Parent Folders — Group Comparison

When selected experiments come from different folders:

• Each folder group becomes one aggregated line (mean of all experiments in that group).
• A metric is included only if every experiment in every group has logged it.

model-a/
├── seed-1/   ✓ selected  ─┐ → one line: "model-a"
└── seed-2/   ✓ selected  ─┘

model-b/
├── seed-1/   ✓ selected  ─┐ → one line: "model-b"
└── seed-2/   ✓ selected  ─┘

Saving as .dashrc

To persist a Live Compare configuration for future use, click Save as .dashrc in the Live Compare tab. This creates a .dashrc file in the selected folder, preserving the exact series, colors, and layout.

Compare View

Compare View is a persistent, fully configurable comparison powered by a .dashrc file placed in a folder (not inside an experiment).

How It Appears

• Select any folder in the file tree that contains a .dashrc file.
• The Compare tab activates automatically (higher priority than the README tab).
• The view shows a chart grid defined by the .dashrc.

Editing the Configuration

1. Click the Edit button in the Compare tab.
2. A split view opens: YAML editor on the right, live preview on the left.
3. Changes parse after 300ms of inactivity; charts re-fetch after 1500ms.

Compare .dashrc Configuration

Minimal Example

charts:
  - ctype: line
    xKey: default
    yKey: train.loss
    series:
      - prefix: alice/my-project/experiment-a
        label: Experiment A
      - prefix: alice/my-project/experiment-b
        label: Experiment B

series Field Reference

Each item in the series array corresponds to one line on the chart and one API request.

Choosing Between prefix and experimentIds

Use prefix when the experiment path is stable and human-readable — it makes the .dashrc portable and easy to read:

series:
  - prefix: alice/my-project/baseline
    label: Baseline

Use experimentIds when you need to pin exact experiments, for example to average several seeds into one line. (A .dashrc exported from Live Compare pins single runs with experimentId, and groups with prefix.)

series:
  - experimentIds:
      - exp_abc123
      - exp_def456
    label: Seed Group A

What Compare .dashrc Ignores

• The metrics section — header metric filters are only meaningful for single-experiment views.
• Charts with ctype: image or ctype: video — media grids are not shown in Compare View.

Full Example

charts:
  - ctype: line
    title: Training Loss
    xKey: default
    yKey: train.loss
    xLabel: Steps
    yLabel: Loss
    bins: 500
    series:
      - prefix: alice/rl-project/sac-baseline
        label: SAC (baseline)
        color: "#5470c6"
      - prefix: alice/rl-project/sac-rff
        label: SAC + RFF
        color: "#91cc75"
      - prefix: alice/rl-project/td3-baseline
        label: TD3 (baseline)
        color: "#ee6666"
        dash: 4 4

  - ctype: line
    title: Eval Reward
    xKey: default
    yKey: eval.episode_reward
    xLabel: Steps
    yLabel: Reward
    bins: 500
    series:
      - prefix: alice/rl-project/sac-baseline
        label: SAC (baseline)
        color: "#5470c6"
      - prefix: alice/rl-project/sac-rff
        label: SAC + RFF
        color: "#91cc75"
      - prefix: alice/rl-project/td3-baseline
        label: TD3 (baseline)
        color: "#ee6666"
        dash: 4 4

Live Compare vs Compare View

Next: See the .dashrc Reference for a complete field listing.