Notes for sunstream.tv

Work in progress with Jo Caimo, who came up with the idea. These are technical notes on how to do the realtime prediction.

Let’s predict the current state of the sun 8min in advance, in realtime!

Challenge

Predict the AIA 304 Å channel of the sun 8 mins in advance based on recent SDO data, then stream video of it to clients

Every 15min a new image is available online, so we need both interpolation plus forecasting

Main idea: Use a reversible neural codec to map image space to smaller latent space. Do online GP in the latent space. Add some noise and transform back to image space.

Goal: smooth 25-30 fps space weather TV

• VAE options
  • ceVAE (old but documented, 1 λ, 128‑latent) – PyTorch → ONNX → Web GPU
  • SDO–FM (2024, 9 λ, 500× compression) – ready HuggingFace weights
• Online GPs
  • MO‑IHGP – steady‑state multi‑output, C++ repo
  • Factorial‑SDE GP – variational, streaming friendly
• Datasets
  • SDO‑ML v2 (512²)
  • Heliocloud (native 4 k²)

Estimated costs

tl;dr one modest GPU keeps the lights on, < €0.35 h all‑in.
Browsers choke on real‑time decoding, Kalman is peanuts

* on‑demand, 24 × 7;‑30 % if reserved.

Cheapest sane path: rent one GEX44, pin decoder on GPU, Kalman on CPU thread, stream AVIF frames. Coffee budget level.

Go to next post ▤ or ▥ previous post in media series.