Weather Data Reproducibility

Overview

Create a compact provenance manifest that identifies the exact weather inputs, subsets, software, transformations, and output artifacts used by a run. Verify the manifest before claiming that a result can be replayed or reproduced.

This skill records and checks lineage. It does not discover or download data by itself; pair it with the appropriate model, observation, radar, or satellite fetching skill.

When to Use This Skill

• A map, sounding, time series, dataset, or benchmark must be auditable.
• Weather inputs came from mutable URLs, mirrors, byte-range subsets, or caches.
• A result must distinguish model initialization, forecast lead, and valid time.
• Temporary GRIB2, NetCDF, radar, satellite, or observation files will be deleted after processing.
• Two outputs differ and their inputs or processing environment must be compared.
• A publication or release needs machine-readable data and software provenance.

Do not claim bit-for-bit reproducibility merely because a source URL and run time were logged.

Reproducibility Contract

Decide which claim the workflow supports:

• Traceable: The source and processing history can be audited.
• Replayable: The same identified inputs can still be retrieved and the recorded procedure can be rerun.
• Bitwise reproducible: The rerun is expected to produce identical bytes in a pinned environment.
• Scientifically reproducible: Equivalent inputs and methods produce the same scientific conclusion within a stated tolerance.

Record the intended claim and its tolerance. Mutable remote objects, floating software versions, lossy images, parallel reductions, and platform-dependent code can make the stronger claims impossible.

Minimum Manifest

Use a versioned JSON document. Keep request intent separate from the source that was actually resolved.

{
  "schema_version": 1,
  "created_utc": "2026-09-18T09:00:00Z",
  "claim": {"level": "replayable", "tolerance": null},
  "request": {
    "dataset": "hrrr",
    "product": "prs",
    "requested_valid_time": "2026-09-18T12:00:00Z",
    "variables": ["temperature", "relative_humidity"]
  },
  "resolved": {
    "initialization": "2026-09-18T06:00:00Z",
    "forecast_hour": 6,
    "valid_time": "2026-09-18T12:00:00Z",
    "provider": "aws"
  },
  "inputs": [
    {
      "role": "pressure_fields",
      "uri": "s3://bucket/exact-object-key",
      "object_identity": {
        "etag": "opaque-identity",
        "last_modified": "2026-09-18T07:01:02Z",
        "size_bytes": 987654321
      },
      "selection": {
        "inventory_uri": "s3://bucket/exact-object-key.idx",
        "inventory_sha256": "HEX_DIGEST",
        "byte_ranges": [[1000, 1999], [5000, 6999]]
      },
      "materialized_sha256": "HEX_DIGEST"
    }
  ],
  "processing": {
    "software": {"application": "1.2.3", "python": "3.13.7"},
    "parameters": {"point": [35.22, -97.44], "nearest_method": "model-aware"},
    "transformations": ["decode GRIB2", "convert K to degC"]
  },
  "artifacts": [
    {
      "path": "sounding.png",
      "media_type": "image/png",
      "size_bytes": 123456,
      "sha256": "HEX_DIGEST"
    }
  ]
}

Use null for an intentionally absent value and omit fields whose meaning is unknown. Never fill a required-looking field with a guess.

Identify Every Input

For each remote object or API response, record as available:

• provider, dataset, endpoint, bucket, and exact key or immutable URL;
• version ID, generation number, release, or dataset revision;
• ETag as opaque object identity, Last-Modified, and content length;
• provider-supplied checksum and checksum algorithm;
• a locally computed SHA-256 for downloaded or materialized bytes;
• response media type and content encoding;
• retrieval time and successful fallback provider;
• license or attribution identifier when required.

An S3 ETag is not always an MD5 digest, particularly for multipart or encrypted objects. Store it for identity and compute a cryptographic content hash when the actual bytes must be verified.

Record the Selection

The source object alone is insufficient when only part of it was used. Record:

• GRIB2 inventory URI and inventory content hash;
• exact inventory rows, search expression, and inclusive byte ranges;
• Zarr group, array names, chunk keys, and coordinate slices;
• API query parameters and a hash or retained copy of the raw response;

…