Benchmark Metrics
Memory is only half the story: a change that saves 2 KB of flash can cost cycles. Benchmark metrics let your CI attach any named number to a commit (a cycle count, a boot time, a CoreMark score) so MemBrowse can:
- Plot it under the memory chart on the Target Dashboard, on the same time axis, so a memory change and a performance change line up
- Compare it between two builds in the Benchmarks tab of Build Comparison
- Show it to AI agents through the MCP server
How Metrics Work
- Metrics belong to a commit of an existing target: upload the commit's memory report as usual, then its metrics. The target must already exist; a metrics upload never creates targets or commits. Metrics may arrive before or after the memory report of their commit, and appear once both are in.
- Each metric has a name and a value (any finite number), and optionally a unit such as
cycles,msoriter/s. The unit belongs to the metric, not to each value: set it once and later uploads keep it. - Uploads merge. Uploading a metric again replaces its value; metrics an upload leaves out are untouched. Separate CI jobs can therefore each post their own metrics for the same commit, and a retried job is harmless.
- Metrics do not count against your upload limits.
Upload From GitHub Actions
Add a step after your benchmark has run, with the same project API key you use for memory reports:
- name: Upload benchmark metrics
uses: membrowse/membrowse-action/metrics-action@v1
with:
target_name: stm32f4
api_key: ${{ secrets.MEMBROWSE_API_KEY }}
metrics: |
Boot time=${{ steps.bench.outputs.boot_ms }} ms
CoreMark=${{ steps.bench.outputs.coremark }}
metrics takes one metric per line, as NAME=VALUE or NAME=VALUE UNIT. Blank lines and lines starting with # are ignored.
If your benchmark harness writes a JSON file, pass it instead:
metrics_file: bench.json
The file is an array of {"name", "value", "unit"} entries (unit is optional), the format github-action-benchmark uses for custom tools:
[
{"name": "Message Passing", "unit": "cycles", "value": 412},
{"name": "Synchronization", "value": 389.5}
]
When both are given, metrics overrides the same names from the file.
| Input | Description |
|---|---|
target_name | Target the metrics belong to. It must already have a memory report. Required. |
api_key | Project API key. Required, except on pull requests of open source projects, where an empty key uploads tokenless (see below). |
metrics | Metrics, one per line. |
metrics_file | Path to a JSON metrics file. |
commit | Full SHA the metrics were measured at. Defaults to the event's commit: the pull request head for pull_request events. |
api_url | API base URL, for on-prem deployments. Defaults to https://api.membrowse.com. |
Fork Pull Requests
Metrics from a pull request work like tokenless memory report uploads: on a fork or Dependabot PR the MEMBROWSE_API_KEY secret is empty, and the action uploads the metrics without authentication. MemBrowse checks with GitHub that the commit belongs to the pull request. This only works for projects marked as open source; pushes always need the API key.
Because a pull request's commits can end up on a branch once it is merged, metrics uploaded this way are kept apart from your CI's own:
- They cannot set a metric's unit.
- They cannot change a commit whose metrics were uploaded with the API key.
- If metrics are later uploaded for the same commit with the API key, those replace the pull request's metrics.
Upload From Any CI or Locally
With the CLI:
membrowse metrics -m "Boot time" "$BOOT_MS" ms -m CoreMark "$SCORE" \
--target-name stm32f4 --api-key "$MEMBROWSE_API_KEY"
membrowse metrics --file bench.json \
--target-name stm32f4 --api-key "$MEMBROWSE_API_KEY"
Metrics can also be given as NAME=VALUE or "NAME=VALUE UNIT" arguments. They are attached to the current HEAD; pass --commit-sha for another commit, or --github to take the commit from the GitHub Actions event. Run membrowse metrics --help for every option.
Viewing Metrics
- Target Dashboard: check a metric above the memory chart to plot it in a second chart directly under it. See Benchmark metrics on the Target Dashboard.
- Build Comparison: the Benchmarks tab compares each metric between the two builds. See Benchmarks Tab.
- Removing a metric: metrics are deleted only in Project Settings, by a project owner or admin, which removes the metric from every commit of the target. The project API key can upload metrics but not delete them.