SPLap 2.1.1 · engineering notes

Data sources, providers and attribution

SPLap 2.1 ("Motorsport Telemetry Lab") replays and analyses real Formula 1 sessions. This document says where the data comes from, what each adapter actually implements, what is observed versus derived versus modeled, and what the licensing and credential rules are. Field-level detail lives in docs/DATA_SCHEMA.md; the import walkthrough is docs/IMPORTING_A_RACE.md.

1. Provider adapter architecture

OpenF1 REST ----> openf1.py / importer.js --\
FastF1 (opt.) --> providers_extra.py --------+--> canonical events (splap.canonical/1)
Jolpica -------> providers_extra.py ---------/        |
                                          track.derive -> geometry (splap.track/1)
                                          frames.build -> replay frames (splap.replay.frames/1)
                                                       |
                              bundle.write (offline) / ingest.py (index) / fixture (lookups)

The canonical schema is the only contract the app depends on. Adapters are thin: they rename provider fields, attach provenance and a stable source_record_id, and count what they found into the capability object. Nothing downstream reads a provider payload.

1.1 OpenF1 (primary)

• Endpoint base https://api.openf1.org/v1 (openf1.py, importer.js). Plain REST, JSON, CORS-enabled, no API key, so the browser importer can call it directly from Splunk Web.
• Endpoints used per session: sessions, meetings, drivers, laps, stints, pit, race_control, weather, position, intervals, location, car_data; overtakes, session_result and team_radio are optional (failures are swallowed). session_result is fetched but not normalised into any event.
• location.x/y/z are decimetres in an arbitrary venue frame; SPLap keeps them as-is and stores unit_m = 0.1 in the geometry. location and car_data arrive at roughly 3.7 Hz per car; the capability object records telemetry_hz / location_hz as that nominal constant.
• Rate limit: both clients space requests at 2.5 req/s (rate_per_s=2.5, ratePerS: 2.5) and retry 429/5xx with exponential back-off (6 attempts). The Python client caches every response under .cache/openf1 so re-runs are free.
• Windowing (--window iso_start iso_end): location, car_data and intervals are filtered with date> / date<. Laps, stints, pits, race control, weather, overtakes and radio are always fetched for the whole session (they are small). Two deliberate exceptions:
• position is fetched unwindowed. Position rows are sparse change events; a 13-minute window could contain no row for a car that has held P5 for twenty laps, which would leave it without a position at the first frame.
• intervals are fetched from 120 s before the window start so every car has a gap/interval value when the first frame is built.
• Telemetry can be restricted to selected drivers (drivers= / --profile selected) or skipped (--no-telemetry), in which case location/car_data are unavailable in the capability object.

1.2 FastF1 (optional, Python only)

providers_extra.fastf1_available() reports whether the fastf1 package imports; fastf1_fetch(year, gp, session_name) loads a session through FastF1 (which reads F1 live timing) and produces canonical lap, location, car_data and weather events plus a driver list and capability object. What it does not do today: it emits no session, meeting, driver, position, interval, pit, stint or race_control events; t1 is fixed at t0 + 3 h and date_end is null; lap events lack the team fields; brake_state is 0/1 rather than OpenF1's 0/100. It has not been used to build any shipped asset and there is no CLI subcommand wired to it. Treat it as a proof that a second telemetry provider fits the schema, not as a supported path.

1.3 Jolpica (calendar and results metadata)

jolpica_calendar(season) and jolpica_results(season, round) call https://api.jolpi.ca/ergast/f1 (the community continuation of Ergast) and return plain dicts labelled source_provider: "jolpica", provenance: "observed". They exist to label meetings and cross-check results; nothing in the shipped app or the Monza bundle depends on them.

1.4 TUMFTM racetrack database (reference only)

The TUM racetrack database (centerlines and widths for many circuits, published under an open licence) is a candidate fallback geometry when a session has no usable location data. It is not used anywhere in the code today: there is no adapter, no bundled file and no reference in the build. The shipped Monza geometry (splap/appserver/static/sessions/monza_2025/track.json, bundles/monza_2025_window/track.geojson) is derived entirely from OpenF1 location samples of the session itself (track.derive; reference lap and sample counts are recorded under reference). If TUMFTM were adopted it would enter as a provenance: "observed" centerline in a different coordinate frame that must be registered to the venue frame; width_m would then become observed instead of the current modeled 12 m constant.

1.5 Bundled sample sessions

• monza_2025 – 2025 Italian Grand Prix race, 13.5-minute window 14:00:00–14:13:30 UTC, 20 cars, 1620 frames at 500 ms, import_id imp_17f4a807fa72146b. Shipped as lookups plus static assets and as the offline bundle bundles/monza_2025_window.
• Demo Race 07 – synthetic, original fixture (splap_demo_* lookups, tools/gen_demo.py), labelled SYNTHETIC TEST FIXTURE in the UI. Contains no real-world data and is the rights-free fallback.

2. Capability object and "unavailable"

Every session carries capabilities (see DATA_SCHEMA.md section 4). Each channel is {available, count, provenance}. The UI reads it before querying:

• car_data.available=false shows TELEMETRY UNAVAILABLE on the pit wall and disables the speed/throttle/gear overlays instead of drawing zeros.
• A frame whose cov column is 0 has no position; the car is not drawn.
• drs.available=false renders DRS as UNAVAILABLE, distinct from CLOSED.
• aero is always unavailable; no provider exposes it. Panels that would need it stay labelled.
• Tyre compound missing for a car shows ? with a tooltip, not a default compound.

"Unavailable" therefore means "the provider did not give it for this session/window", not "zero". The same word is used as a provenance value on geometry parts (pit_lane.provenance, sectors.provenance, drs_provenance) when derivation had nothing to work with.

3. Observed, derived, modeled in the shipped Monza session

4. Licensing and attribution

The attribution text shipped with the data is bundles/monza_2025_window/ATTRIBUTION.md; the same text is embedded in the static session.json and shown in the UI footer. Summary:

• OpenF1 is an unofficial, community-run API. Its data is provided free for non-commercial use; the underlying timing data remains the property of its original providers. Check openf1.org for current terms before any commercial distribution.
• Jolpica is used only for metadata when at all; it is likewise a community API.
• No official artwork. The build contains no team or series logos, sponsor graphics, car renders, driver photographs (headshot_url is deliberately null), circuit maps or official typography. Circuit geometry is derived from position data, not copied from any publication.
• Team palettes are curated approximations (team_palettes.json) layered on the provider's single team colour. They are not official brand colours.
• Names and results (driver names, acronyms, numbers, team names, lap times, positions) are facts of public record. Rights holders (Formula One Licensing BV, the FIA, teams) nonetheless assert rights over timing data and marks and may object to commercial redistribution.
• Splunkbase risk. Bundling a real F1 session (the Monza lookups and static assets, and the offline bundle) in a public Splunkbase listing carries rights risk under both the OpenF1 non-commercial condition and rights-holder positions. This is a decision for the app owner, not something the code resolves. The safe configuration is to ship only the synthetic Demo Race 07 fixture and let users import real sessions themselves with the importer; the code supports that by keeping the demo as the labelled, rights-free fallback and by making imports self-service.
• SPLap is not affiliated with, endorsed by or licensed by Formula One, the FIA, any team, driver, circuit, Splunk LLC, OpenF1 or Jolpica. Formula 1 and related marks are trademarks of their owners.

5. Credentials policy

• OpenF1 and Jolpica need no keys; nothing is stored for them.
• Splunk credentials, HEC tokens and session tokens never appear in browser JavaScript, source control, static assets, bundles or log lines.
• Browser importer (importer.js): writes go through the user's own authenticated Splunk Web session, via the splunkd/__raw proxy to /services/receivers/stream, with credentials: 'same-origin' and the CSRF form key in X-Splunk-Form-Key. It checks for the edit_tcp (or admin_all_objects) capability by reading authentication/current-context rather than probing with a write. No token is created, requested or persisted; the only browser-side state is a resumable checkpoint (import.<import_id>) in localStorage.
• CLI (tools/splap_import.py ingest): credentials come only from --user/--password, --token, --hec-token or the environment variables SPLUNK_PASSWORD, SPLUNK_TOKEN, SPLUNK_HEC_TOKEN. They are held in memory for the run and are not written to the bundle, cache or status events. TLS verification is off by default (--verify to enable), which is appropriate for a local dev instance only.
• splap:import_status events record phase, counts and (browser) a truncated user agent; never credentials or hostnames beyond the index name.

6. Offline bundle

bundle.write produces a directory (optionally zipped) that splap_import.py ingest --bundle can load with no network access:

Ingest adds the circuit event, expands frames into replay_frame events and stamps import_id on every row. See docs/DATA_SCHEMA.md for field definitions and docs/IMPORTING_A_RACE.md for the step-by-step import procedure.