MermaidSeg

computer vision
open source
conservation
MLOps
data engineering
Core contributor (250+ commits) to an open-source coral segmentation repo using DINOv3.

Role: core contributor, ML and data engineering
Code: github.com/data-mermaid/mermaid-segmentation
Stack: PyTorch · DINOv3 · LoRA (PEFT) · Hugging Face Transformers · AWS SageMaker · S3 · MLflow · Docker · DuckDB / ibis · Parquet · Gradio / ZeroGPU · uv · GitHub Actions · pytest

The problem

MERMAID is an open-source coral-reef monitoring platform from the Wildlife Conservation Society (WCS). Divers photograph reef transects, and every image then has to be labelled by hand, point by point. That is the bottleneck. A segmentation model that can label benthic cover (hard coral, algae, sand and so on) turns hours of annotation into minutes and makes large-scale monitoring practical. The open problems are data (labels are scattered across sources with inconsistent taxonomies) and training that can be reproduced and run in the cloud.

What I did

Models

  • Fine-tuned a DINOv3 vision transformer for benthic segmentation, using LoRA adapters on the attention q/v projections and frozen-backbone linear-probe baselines.
  • Added focal, taxonomic and dual-morphology losses, with hierarchy-aware metrics. These suit the long-tailed, hierarchical label space of reef organisms.

Data

  • Built the CoralNet ETL pipeline and web scraper on SageMaker (permitted scraping). It tracks 477K images across 1,053 sources and 24M+ point annotations in versioned Parquet on S3, with audit and reconciliation reports that flag incomplete sources.
  • Wrote the label-mapping layer that reconciles 3,400+ CoralNet labels into the MERMAID benthic hierarchy (77 training classes). It resolves 85%+ of annotations. I also fixed a bug where numeric CoralNet IDs failed to resolve against name-keyed mappings, which had silently mapped almost nothing.
  • Wrote a coverage audit that ranks unmapped labels by annotation count, so the MERMAID team knows which mappings to add first. One unmapped label held 1.7M annotations.
  • Made data loading faster and more memory-efficient (O(1) annotation lookup, a numpy-native index).

MLOps and reproducibility

  • Built the MLflow tracking integration. Each run logs metrics, per-class IoU, checkpoints and artifacts, so results can be reproduced and shared.
  • Designed YAML-first experiment configs with offline schema validation, plus a launcher, so researchers and R&D staff can start cloud training jobs without touching the code.
  • Added MLflow-based checkpoint resume for Managed Spot training, and wrote runbooks and diagnostics for silent training failures.

Open-source delivery

  • Helped ship the public demo (see MERMAID demo) on Hugging Face Spaces with ZeroGPU.
  • Maintained the CI, dependency pinning, test suite and docs that let outside contributors work in the repo.

Result

  • Replaced ad-hoc notebooks with one reproducible training setup: experiments run from a config file and are tracked in one place.
  • Built the data foundation for coral segmentation: 24M+ annotations across 1,000+ sources, with 76% of training annotations landing in the 77-class target set.
  • Cut training cost and failures through spot-instance resume, data-loader fixes and early validation of configs.
  • Became the most active contributor in the final stretch: monthly commits rose from 6 to 116 between February and July, with about 208K lines added.
  • Made the code usable beyond the core team, from the public demo to documented launch runbooks.