Xue dataset dataset.ringsaturn.me/xue/
The data root of Xue, a static-site
weather viewer: live forecast runs, radar and satellite windows and three
point products, published as Zarr v3 stores and plain JSON and described by
a static STAC 1.1.0 catalog. Everything
here is served as-is from an object store; there is no API.
Start here
- catalog.json
- The root Catalog: one
child Collection per live source and
point product, one for the historical showcase cases. Each Collection's
item link is the newest run or issue at a path that never
changes; every link is relative.
- docs/stac.md
- The catalog's contract: the layout, what each Catalog, Collection and
Item carries, and how every document is derived from a run's manifest.
- docs/zarr-profile.md
- The Zarr v3 profile of a store (
<bundle>.zarr/): one
uint8 array per variable, one shard, six-frame time chunks,
CF scaling attributes, so xarray.open_zarr reads it.
- docs/format.md
- The quantization, codebooks and temporal residual coding behind the
stores (the
.xue container they are derived from).
- Point products
- docs/sounding.md
(radiosonde soundings),
docs/airport.md
(METAR / TAF) and
docs/tc.md
(tropical cyclone tracks): each issue is an
index.json plus
one NDJSON file read by byte range.
- README
- The project, the sources and their cadence, publishing, and the data
licences.
Reading it
With pystac and xarray:
import pystac, xarray as xr
catalog = pystac.Catalog.from_file("https://dataset.ringsaturn.me/xue/catalog.json")
run = next(catalog.get_child("gfs").get_items()) # the live Item, gfs/item.json
ds = xr.open_zarr(run.assets["tmp2m"].get_absolute_href())
run.properties["cube:variables"] lists what else the run
carries and in which unit. Only the newest run per source is kept, so hold
an Item no longer than the source's cadence; a link into a run directory
(gfs.<run>/) stops resolving when the run is pruned.
Collections
The live column is read from each Collection with JavaScript.
Layout
latest.json, latest-<source>.json
- The live pointer per source, the only mutable object the viewer
reads (
no-cache); its STAC face is
<source>/collection.json.
<source>.<run>/
- One immutable run:
manifest.json, one
<bundle>.zarr/ store per variable with its reduced
tiers, posters, and item.json. A rolling window (radar,
satellite) adds a round directory, <HHMM>/.
<product>.<issue>/
- One issue of a point product:
index.json, the NDJSON file
it indexes by byte offset, and item.json.
showcase/, showcase.json
- Historical cases, kept indefinitely, with their own Collection.
Licences
The data belongs to its producers; each Collection names them under
providers and its terms under license, as
listed here. Xue's own code is MIT OR Apache-2.0.
- NOAA GFS 0.25°
- NOAA open data dissemination
- ECMWF IFS open data 0.25°
- CC-BY-4.0
- ECMWF AIFS Single open data 0.25°
- CC-BY-4.0
- ECMWF IFS HRES 0.1° (via Open-Meteo)
- CC-BY-4.0
- NOAA GFS surface fluxes
- NOAA open data dissemination
- NOAA HRRR 3 km
- NOAA open data dissemination
- NOAA GEFS-Aerosols 0.25°
- NOAA open data dissemination
- NCEP CFSv2 seasonal
- NOAA open data dissemination
- CMA radar mosaic
- the producer's own terms; see the Collection's providers
- NOAA MRMS radar mosaic
- NOAA open data dissemination
- JMA precipitation nowcast (hrpns)
- 気象庁ホームページ利用規約 (compatible with CC BY 4.0)
- Himawari-9 AHI full disk
- NOAA Himawari on AWS: terms of use (attribution required, no endorsement)
- GOES-19 ABI full disk
- NOAA GOES on AWS: terms of use (attribution requested, no endorsement)
- GOES-18 ABI full disk
- NOAA GOES on AWS: terms of use (attribution requested, no endorsement)
- Meteosat-12 FCI full disk, hourly
- CC-BY-4.0
- Radiosonde soundings (WIS2)
- WMO Unified Data Policy (Resolution 1, Cg-Ext(2021)): core data, free and unrestricted, attribution of the original source requested
- Airport observations and forecasts (METAR / TAF)
- NWS disclaimer: a work of the US government, in the public domain
- Tropical cyclone tracks
- NWS disclaimer: the US agencies' products are works of the US government, in the public domain; Creative Commons Attribution 4.0, for the ECMWF open data tracks