MULTIVOX in mirdata — a Track API for a circle of singers
What mirdata is for, why MULTIVOX got a loader, and how SingerPosition + Track turn a multimodal take into callable objects.
What mirdata is
mirdata is a Python library of common loaders for Music Information Retrieval datasets.
Repo: mir-dataset-loaders/mirdata
Docs: mirdata.readthedocs.io
Paper: Bittner, Fuentes, et al. — mirdata: Software for Reproducible Usage of Datasets, ISMIR 2019 (PDF, Zenodo).
The paper’s problem is familiar if you have ever reused someone else’s MIR experiment: dataset folders drift, annotation formats disagree, and every project rewrites its own loader. Results stop being comparable because “the data” was not the same object twice.
mirdata’s answer is a shared interface:
- download (or instruct download) into a known layout
- validate presence + checksums via a dataset index
- load audio / annotations / metadata into common shapes (annotation side aligned with mir_eval)
- expose work as
Dataset→Track(and multitrack where needed)
import mirdata
ds = mirdata.initialize("orchset")
ds.download()
ds.validate()
track = ds.choice_track()When it is useful
Use mirdata when you want the canonical dataset version behind a stable Python API — training loops, eval scripts, notebooks — without maintaining private path scrapers.
Use it when reproducibility matters: same index, same checksums, same load functions across machines and papers.
Skip writing a loader (or skip mirdata) when the corpus is throwaway, closed, or already wrapped by something your stack already standardizes on. mirdata shines for shared, cited MIR corpora.
Why a MULTIVOX loader
MULTIVOX is a multimodal, spatial audio–visual set of a-cappella performances: circle layouts (6 or 16), ORTF far-field, per-singer near-field, 360° video, singer metadata. Dataset: Zenodo record.
Without a loader, “using the dataset” means spending time on the CSV metadata: learning its columns, parsing rows by hand, then wiring those fields to the right audio/video paths — which stems exist per session, how circle indices map to angles, which takes lack audio_360. That work usually dies in someone’s notebook.
I wrote the core loader so MULTIVOX shows up like every other mirdata dataset: initialize("multivox"), index-backed paths, track-level access. The point was not a demo script. It was to put a multimodal layout — many streams, optional stems, circle geometry — behind the same programmatic surface researchers already use for GuitarSet, Orchset, and the rest. Shipped as PR #696 (merged 2026-02-06).
The take: API layer, not path soup
A mirdata loader is an API over the underlying files. You stop thinking in raw paths and start thinking in objects.
What you get in practice:
| Layer | Role |
|---|---|
| Index (JSON) | Registry: track id → file paths + checksums |
Dataset | download / validate / iterate / track(id) |
Track | one performance as a unit of work |
SingerPosition | one seat in the circle — metadata + helpers (can be empty) |
@property accessors | load audio/video on demand into arrays (and optional None where the corpus is sparse) |
That is DTO-shaped without needing a separate DTO library: the track and position classes are the transfer objects your training code calls.
SingerPosition is the MULTIVOX-specific piece. It is not “a row from the CSV” and not only a singer id. It models one seat in the circle around the far-field mics / 360° camera: facing direction, height, gender, role, whether the seat is empty, and helpers to reach that seat’s near-field audio. Angular position (degrees around the circle) is computed from circle size + seat index, so callers do not re-derive the geometry. Non-circle participants (e.g. instructor) exist as positions with spatial fields left unset.
import mirdata
dataset = mirdata.initialize("multivox")
track = dataset.track("C1_07052025_S1_MAMAINES_P6_S6")
pos = track.singer_positions["S1"]
pos.gender # 'M' / 'F' / None
pos.facing_direction # 'C' | 'O' | 'L' | 'R' | None
pos.angular_position # degrees, from circle size + seat index
pos.is_empty_position()
if pos.has_near_field(track):
audio, sr = pos.near_field_audio(track)Angles are even steps around the circle, with 0° in front of the far-field mics / 360° camera. The loader does not store a degree column — it derives it:
# inside SingerPosition.angular_position
angle_increment = 360.0 / locations_at_circle # 60° if N=6, 22.5° if N=16
return index_at_circle * angle_increment # S1 → 0°, S2 → 60°, …If the seat is not on the circle (instructor), index_at_circle / locations_at_circle are unset and angular_position is None.
At track level, far-field mixture and the full seat map look like this:
mixture, sr = track.audio_ortf_stereo
for sid, pos in track.singer_positions.items():
print(sid, pos.angular_position, pos.facing_direction)Consumers write against method names and types. The filesystem layout stays an implementation detail behind the index.
OOP choices that paid off
MULTIVOX is a good stress test for mirdata’s object model: many streams per take, optional fields, geometry that should be computed once and queried often.
What I leaned on:
Inheritance. Track and Dataset subclass mirdata’s core types so MULTIVOX behaves like every other loader (same download/validate/track loop).
Composition. A track owns a map of SingerPositions (defined above), not a pile of parallel lists of genders / facings / heights.
Computed domain fields. pos.angular_position (formula above) — geometry stays on the type, not re-derived in every notebook.
Lazy accessors. Heavy audio uses @property (not cached arrays for far-field stems): load when read, fail loudly with FileNotFoundError when a required path is missing, return None when audio_360 was never captured. The API documents optionality in the return type and the docstring.
Self-documentation at the call site. Names like audio_ortf_stereo, singer_angular_positions, has_near_field are the docs for the common path. Longer explanation lives next to the property for the edge cases (session without audio360, instructor seat, empty positions in a 16-singer layout).
None of this is exotic architecture. It is making the dataset’s real structure visible in types so downstream code stays boring.
Local bring-up notes (kept from the lab scrap)
Working with uv + .venv. Optional deps for the full suite. pytest-xdist to mirror CI parallelism. --black is not a pytest flag in this repo — format outside pytest.
For tests/test_full_dataset.py it helped to keep a local copy under ~/mir_datasets instead of re-downloading every run. Full integration against the real tree catches index / path / optional-stem issues that the tiny fixtures miss. I also wrote a short throwaway script that used the loader like a real caller — initialize → track(...) → hit the accessors — to validate the API before relying only on pytest.
Review nits that mattered on my side: far-field ORTF accessors as @property; required stems raise; optional audio_360 may be None.
Note
PR #696 was collaborative. My commit landed the MULTIVOX module, index script, and audio/metadata tests. yujin-kimmm added video loading in review — not only for MULTIVOX, but the first video load path in mirdata (load_video / track.video) — plus tests, CI deps (OpenCV first, then moviepy), formatting, and a try/except so users without the video stack fail clearly. See the commit list.
Open source is taking hits from unread generated code — some projects are narrowing or closing contribution because of it. I used coding agents on this change; I also read the code before it went up, and the review thread was attentive. Not every project gets that. Agents do not replace diligence. They make reading and review more necessary, not less.