← All changelogs

WP6 Digital Twins — What shipped this month

Changelog · 8 May → 8 June 2026 · branch main · 59 commits · 7 merged PRs · 3 contributors
Blue — blueberry farm · wp6-blue.spohf.fontysvenlo.dev Red — tomato greenhouse · wp6-red.spohf.fontysvenlo.dev Grey — public platform demo · wp6-grey.spohf.fontysvenlo.dev Shared — twin-agnostic platform

Headlines

🔗 The Open live buttons (↗) deep-link into the deployed dashboards. Blue and Red require a login on first open; Grey is a public demo (no login).

🪲 Manual data sources for the Blue twin

The blueberry farm receives data by hand — trap counts and annual plant measurements — that no automated pipeline collects. This month it became first-class data.

A shared, reusable manual-upload capability

SharedRedBlue

Red's "Sijia" Excel-ingest flow was extracted into a twin-agnostic shared/manual_ingest/ capability: content-addressed file storage, transactional all-or-nothing apply, a shared manual_uploads audit trail, file pruning, and the preview/apply/history UI. Red now runs on it unchanged; Blue gained the whole pipeline it never had.

Onboarding a new manual source is now "write one parser + descriptor and wire it into a twin" — everything else is shared. Blue's insect feed was the second consumer; long_data the third.

Open live — Blue /status (admin) Upload cards + coverage split live here

Insect-trap CSV upload — Blue's first manual source

Blue

Admins upload a trap-counts CSV (total insects + Drosophila suzukii, a primary blueberry pest), see a validation preview (parsed rows, devices, date range, regressions vs. existing data, skipped rows), then apply atomically. Counts then plot on the dashboard like any other sensor, and "last insect upload" freshness shows on the home page.

  • CSV parser + ManualSource descriptor + metadata + test fixture
  • wp6-blue-ingest-insects CLI — validates & ingests locally or against the remote/prod TSDB (earliest end-to-end check, before any UI)
  • Blue switched to OIDC auth so uploads can be admin-gated like Red
  • Helm: dedicated blue manual-uploads PVC + volume mount

Yearly long_data plant measurements — second manual source

Blue

Annual lab/field measurements (shoot length, Brix, firmness, berry weights, per-plant yield, phenology scores) arrive once a year as Long_Data <year>.xlsx — 2024 (English labels) and 2025 (Dutch labels, finer treatment codes) exist today. Modelled as one device per treatment, harmonising the differing yearly vocabularies, and ingestible per-year without disturbing prior years.

  • wp6-blue-ingest-long-data CLI as the primary path
  • Three small generic extensions to the shared capability (sample preservation, per-year replace) rather than a copy of the flow

📏 Multi-height sensor views for the Red twin

Light in the greenhouse varies sharply with height (the natural-light sensor above the lamps vs. total-light below). A new section makes that vertical structure visible.

Multi Height section hub

Red

A landing hub under /multi_height gathers height-based views as cards — new views slot in by adding one entry. Cumulative DLI charting and a date parameter landed alongside it.

Simple Greenhouse View & Wire Sensor Trends

Red
  • Simple Greenhouse View — latest PAR and Daily Light Integral mapped onto the greenhouse layout at each sensor height (SVG layout).
  • Wire Sensor Trends — PAR, temperature, humidity and CO₂ over time from the multi-height wire, one line per height.

Data model: multi-wire, per-height devices

Red

The underlying model switched to wire_sensors — each height on the wire is its own device, supporting multiple wires. Directional reference images of the physical twin were added to the docs.

⚠︎ Note for the meeting: the wire sensors are physically deployed outside for testing (~June 2026), so current values are not representative — risk-threshold tuning is intentionally deferred.

Commits
2dbc43c feat(multi-height): switch to wire_sensors, multi-wire per-height devices
f131630 feat(multi-height): add section hub, cumulative DLI chart, and date param
c580f37 docs: add red twin directional images

📊 Charts & visualization

Faster charts that scale to long date ranges, and richer reading of the data.

Server-side aggregation + min/max range bands

SharedBlueRed
  • Server-side chart aggregation via TimescaleDB time_bucket, with a count-weighted merge on the client — long ranges render without shipping every raw point.
  • Min/max range band drawn on line charts, with the band's min & max surfaced in the hover tooltip.
  • Unique line colours via a sorted-index palette (no more clashing series).

Home & dashboard UI refresh

BlueGrey

Explore was consolidated into 3 source-aware tabs; a hero card and template card were added to the home page; PAR dashboard, dark-theme fixes and stronger, de-duplicated colours landed. These generic platform capabilities are best seen on the public Grey demo (no login needed).

🟢 Status & coverage

The /status pages now distinguish where data comes from.

Coverage split by manual vs. sensor source

BlueRed

Both twins' status pages now split coverage into manual and sensor sources, so a manual upload's freshness reads separately from live sensor feeds. Unmeasured weeks were softened from solid to translucent grey for clarity.

⚙️ Platform & architecture (under the hood)

Less visible, but it's what keeps the twins fast and the codebase maintainable.

TimescaleDB performance & Red parity

SharedRed
  • Background continuous-aggregate refresh policy + mode-aware in-band refresh; scoped incremental cagg refresh.
  • Postgres tuning and resource bumps for analytical reads.
  • Red gained TSDB-cached aggregates, reaching status-data parity with Blue.

Clean layering & CI

Shared
  • Twin-agnostic OpenMeteo client moved to shared/weather.py; Blue owns its readings DDL while db/schema stays twin-agnostic — upholding the rule that shared/ must not know red/blue specifics.
  • E2E tests run in a single CI check with a dev-auth mode so the Blue app starts without OIDC secrets.