# ML-Dash API Reference

Complete API reference for the ML-Dash Python SDK. For tutorials and workflow examples, see [/parameters](/parameters.md), [/metrics](/metrics.md), [/files](/files.md), and [/complete-examples](/complete-examples.md).

## Table of Contents

- [Experiment](#experiment)
  - [Constructor](#constructor)
  - [Properties](#properties)
  - [RunManager (`exp.run`)](#runmanager)
- [Parameters (`exp.params`)](#parameters)
- [Logging (`exp.log`, `exp.logs`)](#logging)
- [Metrics (`exp.metrics`)](#metrics)
- [Files (`exp.files`)](#files)
- [Auto-Start (`dxp`)](#auto-start)

---

## Experiment

The `Experiment` class is the main entry point for ML-Dash. It represents a single machine learning experiment run.

```python
from ml_dash import Experiment
```

### Constructor

```python
Experiment(
    prefix: Optional[str] = None,
    *,
    readme: Optional[str] = None,
    tags: Optional[List[str]] = None,
    bindrs: Optional[List[str]] = None,
    metadata: Optional[Dict[str, Any]] = None,
    dash_url: Optional[Union[str, bool]] = None,
    dash_root: Optional[str] = ".dash",
)
```

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `prefix` | `str` | `None` | Universal key in format `owner/project/path/name` |
| `readme` | `str` | `None` | Human-readable experiment readme/description |
| `tags` | `List[str]` | `None` | Tags for categorization and search |
| `bindrs` | `List[str]` | `None` | Binders for advanced organization |
| `metadata` | `Dict[str, Any]` | `None` | Additional structured metadata |
| `dash_url` | `str \| bool` | `None` | Remote API URL. `None` = local-only (no remote); `True` = use default remote (`https://api.dash.ml`); string = custom URL. Token auto-loaded from `~/.dash/token.enc` |
| `dash_root` | `str` | `".dash"` | Local storage root path. Set to `None` for remote-only mode |

**Prefix format:** `owner/project/path.../name` — first segment is owner, second is project, remaining segments form the folder path, last segment becomes the experiment name.

**Example:**

```python
# Local mode
exp = Experiment(prefix="alice/my-project/exp-001", dash_root=".dash")

# Remote mode
exp = Experiment(
    prefix="alice/my-project/exp-001",
    dash_url="https://custom-server.com",
)
```

### Properties

| Property | Type | Description |
|----------|------|-------------|
| `experiment.name` | `str` | Experiment name |
| `experiment.project` | `str` | Project name |
| `experiment.readme` | `str` | Experiment readme/description |
| `experiment.tags` | `List[str]` | Experiment tags |
| `experiment.bindrs` | `List[str]` | Experiment bindrs |
| `experiment.folder` | `str` | Folder path |
| `experiment.id` | `str` | Experiment ID (remote mode only, after start) |
| `experiment.data` | `dict` | Full experiment data (remote mode only, after start) |

### RunManager

`exp.run` returns a `RunManager` supporting three usage patterns: context manager, decorator, or manual control.

#### Context Manager

```python
with Experiment(prefix="alice/proj/exp").run as exp:
    exp.log("Training started")
    # Auto-completes on success, auto-fails on exception
```

#### Decorator

```python
exp = Experiment(prefix="alice/proj/exp")

@exp.run
def train(experiment):
    experiment.log("Training...")
    return "done"

result = train()
```

#### Manual Control

```python
exp.run.start()
try:
    exp.log("Training...")
    exp.run.complete()
except Exception:
    exp.run.fail()
```

#### Methods

| Method | Description | Status Set |
|--------|-------------|------------|
| `run.start()` | Start the experiment | `RUNNING` |
| `run.complete()` | Mark as successfully completed | `COMPLETED` |
| `run.fail()` | Mark as failed | `FAILED` |
| `run.cancel()` | Mark as cancelled | `CANCELLED` |

---

## Parameters

Access via the `exp.params` property. See [/parameters](/parameters.md) for usage patterns.

### `params.set(**kwargs)`

Set or merge experiment parameters. Supports nested dicts (auto-flattened to dot notation for storage).

**Returns:** `ParametersBuilder`

```python
exp.params.set(learning_rate=0.001, batch_size=32)
exp.params.set(model={"architecture": "resnet50", "layers": 50})
# Stored as: {"learning_rate": 0.001, "batch_size": 32,
#             "model.architecture": "resnet50", "model.layers": 50}
```

### `params.get(flatten=True)`

Retrieve parameters.

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `flatten` | `bool` | `True` | If `True`, return dot-notation flat dict; if `False`, return hierarchical dict |

**Returns:** `dict`

```python
flat = exp.params.get()
# {"learning_rate": 0.001, "model.architecture": "resnet50"}

nested = exp.params.get(flatten=False)
# {"learning_rate": 0.001, "model": {"architecture": "resnet50"}}
```

---

## Logging

Log messages with severity levels and optional metadata. See [/logging](/logging.md) for patterns.

### `exp.log(message, level="info", metadata=None, **kwargs)`

Log a message.

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `message` | `str` | required | Log message |
| `level` | `str` | `"info"` | Log level: `debug`, `info`, `warn`, `error`, `fatal` |
| `metadata` | `dict` | `None` | Structured metadata |
| `**kwargs` | any | — | Metadata as keyword arguments |

```python
exp.log("Training started")
exp.log("Warning: Low GPU memory", level="warn")
exp.log("Epoch completed", level="info", epoch=1, loss=0.5)
```

### Fluent API: `exp.logs`

Convenience methods for each level.

| Method | Equivalent |
|--------|------------|
| `exp.logs.debug(msg, **meta)` | `exp.log(msg, level="debug", **meta)` |
| `exp.logs.info(msg, **meta)` | `exp.log(msg, level="info", **meta)` |
| `exp.logs.warn(msg, **meta)` | `exp.log(msg, level="warn", **meta)` |
| `exp.logs.error(msg, **meta)` | `exp.log(msg, level="error", **meta)` |
| `exp.logs.fatal(msg, **meta)` | `exp.log(msg, level="fatal", **meta)` |

```python
exp.logs.info("Training started", epoch=1)
exp.logs.error("Failed to load data", error_code=500)
```

### Log Levels

| Level | Description |
|-------|-------------|
| `debug` | Detailed diagnostic information |
| `info` | General informational messages (default) |
| `warn` | Warning messages for potential issues |
| `error` | Error messages for failures |
| `fatal` | Fatal errors causing termination |

---

## Metrics

Track time-series metrics. Access via `exp.metrics`. See [/metrics](/metrics.md) for patterns.

### `exp.metrics(prefix)`

Create/get a `MetricBuilder` for the given namespace prefix.

**Parameters:** `prefix: str`

**Returns:** `MetricBuilder`

```python
exp.metrics("train").log(loss=0.5, accuracy=0.85)
```

### `metrics.log(**data)`

Log a metric data point. When called on a prefixed builder, the prefix is applied. When called on the top-level `metrics`, supports grouped fields like `train=dict(...)`, `eval=dict(...)`.

**Returns:** `MetricBuilder`

```python
exp.metrics("train").log(loss=0.5, accuracy=0.85)
exp.metrics.log(
    epoch=epoch,
    train=dict(loss=train_loss, accuracy=train_acc),
    eval=dict(loss=val_loss, accuracy=val_acc),
)
```

### `metrics("prefix").buffer(**data)`

Buffer per-batch values in memory (not yet written). Use with `log_summary()` to aggregate.

**Returns:** `None`

```python
for batch in dataloader:
    loss = train_step(batch)
    exp.metrics("train").buffer(loss=loss)
```

### `metrics.buffer.log_summary(*aggregations)`

Compute and log summary statistics from buffered values.

**Parameters:** `*aggregations: str` — names of aggregations. Defaults to `"mean"`. Supported: `mean`, `std`, `min`, `max`, `count`, percentile codes like `p50`, `p90`, `p95`, `p99`.

**Returns:** `None`

```python
exp.metrics.buffer.log_summary()                          # mean only
exp.metrics.buffer.log_summary("mean", "std", "min", "max", "count")
exp.metrics.buffer.log_summary("p50", "p90", "p95", "p99")
```

### `metrics.flush()`

Flush pending metric writes to disk/remote.

**Returns:** `None`

```python
exp.metrics.log(epoch=epoch).flush()
```

### `metrics("prefix").read(start_index=0, limit=100)`

Read logged data points for the given metric.

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `start_index` | `int` | `0` | First index to read |
| `limit` | `int` | `100` | Maximum points to return |

**Returns:** `dict` — `{"data": [...], "startIndex", "endIndex", "total", "hasMore"}`

```python
data = exp.metrics("train_loss").read(start_index=0, limit=100)
```

### `metrics("prefix").stats()`

Get summary statistics for a metric.

**Returns:** `dict` — `{"count", "firstValue", "lastValue", ...}`

```python
stats = exp.metrics("train_loss").stats()
```

### Method Summary

| Method | Returns | Description |
|--------|---------|-------------|
| `metrics(prefix)` | `MetricBuilder` | Get builder with prefix |
| `metrics.log(**data)` | `MetricBuilder` | Log data point |
| `metrics("p").log(**data)` | `MetricBuilder` | Log with prefix |
| `metrics("p").buffer(**data)` | `None` | Buffer for summary |
| `metrics.buffer.log_summary(*aggs)` | `None` | Log aggregated summary |
| `metrics.flush()` | `None` | Flush pending writes |
| `metrics("p").read(start_index, limit)` | `dict` | Read data points |
| `metrics("p").stats()` | `dict` | Get statistics |

---

## Files

Upload, download, and manage experiment files. Access via `exp.files`. See [/files](/files.md) for patterns.

### `exp.files(**kwargs)`

Create a `FileBuilder`.

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `path` | `str` | Logical path/prefix (positional, e.g. `"models"`, `"configs"`) |
| `description` | `str` | File description |
| `tags` | `List[str]` | File tags |
| `bindrs` | `List[str]` | File bindrs |
| `metadata` | `dict` | File metadata |
| `file_id` | `str` | File ID (for download/update/delete) |
| `dest_path` | `str` | Destination path (for download) |

```python
builder = exp.files("models")
builder = exp.files(file_id="abc123")
```

### `save(local_path, **kwargs)`

Upload a file from local disk.

**Returns:** `dict`

```python
exp.files("models").save("./model.pt")
exp.files("models").save(
    "./model.pt",
    description="Best checkpoint",
    tags=["best"],
    metadata={"epoch": 50},
    bindrs=["v1", "production"],
)
```

### `save_json(content, to)`

Serialize a Python object as JSON and upload.

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `content` | `Any` | JSON-serializable object |
| `to` | `str` | Target filename |

**Returns:** `dict`

```python
exp.files("configs").save_json({"lr": 0.001}, to="config.json")
```

### `save_torch(model, to)`

Save a PyTorch model (or state dict) via `torch.save`.

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `model` | `Any` | `nn.Module` or state dict |
| `to` | `str` | Target filename |

**Returns:** `dict`

```python
exp.files("models").save_torch(model, to="model.pt")
exp.files("models").save_torch(model.state_dict(), to="model.pth")
```

### `save_pkl(content, to)`

Serialize a Python object via `pickle` and upload.

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `content` | `Any` | Picklable object |
| `to` | `str` | Target filename |

**Returns:** `dict`

```python
exp.files("data").save_pkl({"results": [1, 2, 3]}, to="data.pkl")
```

### `save_fig(fig=None, to, **kwargs)`

Save a matplotlib figure.

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `fig` | `Optional[Any]` | `None` | Figure (uses current via `plt.gcf()` if `None`) |
| `to` | `str` | required | Target filename |
| `**kwargs` | any | — | Passed to `fig.savefig` (e.g. `dpi`, `bbox_inches`) |

**Returns:** `dict`

```python
exp.files("plots").save_fig(to="plot.png")
exp.files("plots").save_fig(fig=fig, to="plot.pdf", dpi=150, bbox_inches="tight")
```

### `save_video(frames, to, fps=20, **kwargs)`

Encode a list/array of frames as MP4 or GIF.

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `frames` | `Union[List, Any]` | required | Frames (grayscale or RGB ndarray, or list of frames) |
| `to` | `str` | required | Target filename (`.mp4`, `.gif`) |
| `fps` | `int` | `20` | Frames per second |
| `codec` | `str` | — | Codec name (e.g. `"libx264"`) |
| `quality` | `int` | — | Quality setting |

**Returns:** `dict`

```python
exp.files("videos").save_video(frames, to="output.mp4", fps=30)
exp.files("videos").save_video(frames, to="hq.mp4", codec="libx264", quality=8)
```

### `list()`

List files for the current builder filters.

**Returns:** `List[dict]`

```python
exp.files().list()                       # all files
exp.files("models").list()               # by prefix
exp.files(tags=["checkpoint"]).list()    # by tags
```

### `download(dest_path=None)`

Download a file (requires `file_id`).

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `dest_path` | `Optional[str]` | `None` | Target local path; if `None`, uses original filename |

**Returns:** `str` — local path of the downloaded file

```python
path = exp.files(file_id="123").download()
path = exp.files(file_id="123").download(dest_path="./model.pt")
```

### `update()`

Update file metadata (requires `file_id`).

**Returns:** `dict`

```python
exp.files(
    file_id="123",
    description="Updated",
    tags=["new"],
    metadata={"version": "2.0"},
).update()
```

### `delete()`

Soft-delete a file (requires `file_id`).

**Returns:** `dict`

```python
exp.files(file_id="123").delete()
```

### Method Summary

| Method | Parameters | Returns | Description |
|--------|------------|---------|-------------|
| `files(**kwargs)` | see above | `FileBuilder` | Create builder |
| `save(local_path, **kwargs)` | `str`, kwargs | `dict` | Upload file |
| `save_json(content, to)` | `Any`, `str` | `dict` | Save JSON |
| `save_torch(model, to)` | `Any`, `str` | `dict` | Save PyTorch model |
| `save_pkl(content, to)` | `Any`, `str` | `dict` | Save pickle |
| `save_fig(fig, to, **kwargs)` | `Optional[Any]`, `str`, kwargs | `dict` | Save matplotlib figure |
| `save_video(frames, to, fps, **kwargs)` | `Union[List, Any]`, `str`, `int`, kwargs | `dict` | Save video |
| `list()` | — | `List[dict]` | List files |
| `download(dest_path)` | `Optional[str]` | `str` | Download |
| `update()` | — | `dict` | Update metadata |
| `delete()` | — | `dict` | Soft delete |

---

## Auto-Start

The `ml_dash.auto_start` module exposes `dxp`, a pre-configured, auto-started `Experiment` singleton for quick prototyping and notebooks.

```python
from ml_dash.auto_start import dxp
```

### Behavior

- **Pre-configured:** name `"dxp"`, project `"scratch"`, local storage at `.dash`
- **Auto-started:** ready to use on import; no `.run.start()` needed
- **Auto-completed:** closed on Python exit via `atexit`
- **Full API:** supports all `Experiment` methods

### Fixed Configuration

| Property | Value | Mutable |
|----------|-------|---------|
| Name | `"dxp"` | No |
| Project | `"scratch"` | No |
| Storage Mode | Local (`.dash`) | No |
| Local Path | `".dash"` | No |
| Parameters | empty initially | Yes |
| Tags | empty | No |
| Readme | `None` | No |

### Usage

```python
from ml_dash.auto_start import dxp

dxp.params.set(lr=0.001, batch_size=32)
dxp.metrics("train").log(loss=0.5, step=0)
dxp.files("models").save("model.pt")
# Auto-completed on Python exit
```

### Manual Lifecycle

The standard `run` methods still work:

```python
dxp.run.complete()   # close manually
dxp.run.start()      # reopen
dxp.run.fail()       # mark failed
```

### Regular Experiment vs. `dxp`

| Feature | `Experiment` | `dxp` |
|---------|--------------|-------|
| Import | `from ml_dash import Experiment` | `from ml_dash.auto_start import dxp` |
| Configuration | User-defined | Fixed |
| Lifecycle | Manual / context manager / decorator | Auto-started, auto-completed |
| Storage | Local or remote | Local only |
| Instances | Many | Single global |
| Use case | Production / multi-run | Prototyping / notebooks |