SPLap 2.1.1 · engineering notes

SPLap canonical data schema (splap.canonical/1)

This document describes the event schema that every SPLap data path produces: the Python CLI (tools/splapdata/*, tools/splap_import.py), the browser importer (splap/appserver/static/js/importer.js), offline bundles (bundles/*) and the packaged lookups (splap/lookups/splap_monza_2025_*.csv.gz). It is written from the code; where the code and this document disagree, the code wins and this document should be fixed.

Source of truth for names and values: tools/splapdata/canonical.py.

1. Design goals

1.1 import_id

basis = json.dumps({"p": provider, "s": str(session_key), "profile": profile, "w": window or None}, sort_keys=True)
import_id = "imp_" + hashlib.sha256(basis.encode()).hexdigest()[:16]

• provider: "openf1" (or "fastf1").
• session_key: provider session key, always stringified ("9912").
• profile: one of quick, engineering, selected, full (CLI --profile, default full).
• window: [iso_start, iso_end] list or null. The window strings are used exactly as passed (the shipped Monza window is ["2025-09-07T14:00:00", "2025-09-07T14:13:30"]).

Because sort_keys=True orders the keys p, profile, s, w and Python's default separators are ", " and ": ", the exact hashed string for the shipped session is:

{"p": "openf1", "profile": "full", "s": "9912", "w": ["2025-09-07T14:00:00", "2025-09-07T14:13:30"]}

which gives imp_17f4a807fa72146b (see splap/appserver/static/data/sessions.json and the bundle manifest.json). importer.js (I.importId) rebuilds this string byte-for-byte, including the Python spacing, so a browser import and a CLI import of the same session+profile+window collide on purpose. Note: I.sha256 falls back to a non-SHA hash when window.crypto.subtle is unavailable (non-secure context, i.e. plain http://); in that case browser and CLI ids differ.

1.2 Time handling

Provider timestamps such as date_start / date_end on session and meeting events are passed through in the provider's own format (2025-09-07T13:00:00+00:00), not re-serialised.

2. Common fields (every event)

Produced by base_event() in canonical.py. Adapters add per-type fields on top.

Driver-scoped events (lap, stint, pit, position, interval, overtake, team_radio) also carry the drv() block: driver_number, driver_code, driver_name, team_id, team_name, team_primary_color. location, car_data and race_control carry only driver_number.

3. Sourcetypes

All values are observed unless stated. "Origin" is the OpenF1 endpoint and raw field (openf1.normalise).

splap:meeting (one per import)

Timestamp = session date_start.

splap:session (one per import)

splap:driver (one per driver)

splap:circuit (one per import, provenance=derived)

Built at ingest (ingest_bundle, I.run), not by the provider adapter. Timestamp = session date_start.

splap:location (~3.7 Hz per car, decimetres)

splap:car_data (~3.7 Hz per car)

splap:lap

Timestamp = date_start of the lap; laps without date_start are dropped.

splap:position

Fetched for the whole session regardless of window (positions are sparse change events).

splap:interval

Fetched from 2 minutes before the window start so every car has a value at the first frame.

splap:pit

splap:stint

Timestamp = date_start of lap lap_start for that driver, else session start.

splap:weather

splap:race_control

splap:overtake

splap:team_radio

splap:replay_frame (provenance=derived, 500 ms step)

One event per car per frame, only where coverage is true. Built at ingest from the compact frame payload (section 6). Fields: driver_number, x, y, dist_m, heading, speed_kph, gear, throttle_pct, brake_state, drs, position, gap_to_leader, interval, lap_number, compound (single letter), tyre_age, pit (0/1), pit_progress (0–1 or null). x/y are decimetres; heading radians; dist_m metres along the derived centerline. Not present on this sourcetype: schema_version, circuit_key, session_type, source_record_id.

splap:import_status

Written by the ingest layer at start and complete. Fields: import_id, phase, detail, counts (Python), browser (browser importer, truncated user agent), plus session identifiers. timestamp_utc is wall-clock time of the import, not race time.

4. Capability object

capability_object(counts, extra) in canonical.py; stored on session.capabilities, the splap:session event, manifest.json and sessions.json.

{
  "location":     {"available": true,  "count": 61800, "provenance": "observed"},
  "car_data":     {"available": true,  "count": 60500, "provenance": "observed"},
  "laps":         {"available": true,  "count": 976,   "provenance": "observed"},
  "position":     {...}, "interval": {...}, "pit": {...}, "stint": {...}, "weather": {...},
  "race_control": {...}, "overtake": {...}, "team_radio": {...},
  "drs":          {"available": true,  "count": 60500, "provenance": "observed"},
  "aero":         {"available": false, "count": 0,     "provenance": "unavailable"},
  "telemetry_hz": 3.7,
  "location_hz":  3.7
}

Channel keys are CHANNEL_KEYS. A channel with zero rows is available=false, provenance=unavailable. drs is available only if at least one car_data row has a non-null drs. aero is always unavailable (no provider supplies it). The two _hz values are nominal constants, not measured.

5. Circuit geometry (splap.track/1)

Produced by track.derive(); shipped as splap/appserver/static/sessions/<id>/track.json, bundles/<id>/track.geojson (splap_track member plus GeoJSON features) and as the JSON string in splap:circuit.geometry (subset of keys). All coordinates are decimetres (units: "dm", unit_m: 0.1) in the provider venue frame; distances with _m are metres.

The splap:circuit.geometry string contains: centerline, cum_dist_m, heading, pit_lane, corners, sectors, drs_zones, start_finish, bounds, length_m, width_m, units.

6. Replay frames (splap.replay.frames/1)

frames.build_frames() samples every car every step_ms = 500 ms between the first and last location sample (or the requested window). frames.compact() turns that into a column-oriented payload used by the browser (replay_frames.json.gz in bundles, static/sessions/<id>/replay_frames.json).

{"schema": "splap.replay.frames/1", "step_ms": 500, "n": 1620, "t0_elapsed_ms": 3600234,
 "timestamp_utc_0": "2025-09-07T14:00:00.234Z", "drivers": [1, 4, ...],
 "cols": ["x","y","d","hd","v","g","th","br","drs","pos","gap","int","lap","cmp","age","pit","pp","cov"],
 "cars": {"1": {"x": [...], "y": [...], ...}}}

Coverage semantics: cov=0 means "we do not know where the car is". Renderers must hide the car or show a placeholder; they must not draw it at (0,0) or at its last position as if live. Null telemetry columns with cov=1 mean "position known, telemetry stale/unavailable" and the UI shows TELEMETRY UNAVAILABLE rather than 0 km/h.

The expanded row form (one row per car per frame) is what splap:replay_frame and the splap_monza_2025_frames lookup contain. The lookup additionally carries driver_code, laps_behind, stint_number, telemetry (observed/unavailable) and position_provenance; the indexed sourcetype does not.

7. Splunk-side conventions

• All 17 stanzas in props.conf are identical apart from the name: SHOULD_LINEMERGE=false, LINE_BREAKER=([\r\n]+), KV_MODE=json, TRUNCATE=0, the TIME_* settings from 1.2.
• No indexed fields beyond defaults; everything is search-time JSON extraction. source is splap:<provider> (CLI) or splap:openf1 / splap:browser (browser importer).
• Field names are stable across sourcetypes, so driver_number, lap_number, elapsed_ms and import_id can be used to join across them.

Example SPL (field names as indexed):

# laps per driver, best lap
index=splap sourcetype=splap:lap import_id=imp_17f4a807fa72146b
| stats count AS laps min(lap_duration_s) AS best_s by driver_number driver_code team_name
| sort driver_code

# pit stops with durations
index=splap sourcetype=splap:pit
| table timestamp_utc elapsed_ms driver_code lap_number lane_duration stop_duration pit_duration
| sort elapsed_ms

# running order at a moment in race time (last known position per driver before t)
index=splap sourcetype=splap:position elapsed_ms<=3900000
| stats latest(position) AS position by driver_number driver_code
| sort position

# replay frames for a 60 s window
index=splap sourcetype=splap:replay_frame elapsed_ms>=3600000 elapsed_ms<3660000
| table elapsed_ms driver_number x y dist_m speed_kph position lap_number pit

# what did an import contain?
index=splap import_id=imp_17f4a807fa72146b | stats count by sourcetype

# capability check without reading the session JSON
index=splap sourcetype=splap:session | spath capabilities.car_data.available

elapsed_ms is race-relative; _time is wall clock. Both work for windows; elapsed_ms is what the UI and macros use so a window is portable across imports of the same session.

8. Packaged (lookup-backed) form

The app ships one real session as gzip-CSV lookups so it works with no index and no network. tools/splap_import.py fixture writes them and the static assets; session_backend.js (LookupBackend) reads them. Lookup names come from splap/appserver/static/sessions/monza_2025/session.json (lookups object); stanzas are in splap/default/transforms.conf.

Field names are identical to the indexed form, so | inputlookup splap_monza_2025_events | where event_type="lap" is equivalent to index=splap sourcetype=splap:lap for that import. Circuit geometry, capabilities, palettes, validation results and attribution for the packaged session come from the static session.json / track.json / replay_frames.json, not from the lookups. The registry splap/appserver/static/data/sessions.json (splap.sessions/1) lists packaged sessions with id, label, session_key, import_id, window, coverage, frames, step_ms, capabilities, counts, data_mode, provider, default.

The splap_demo_* lookups (Demo Race 07) are a separate, synthetic schema and are not covered here.

9. Validation checks (canonical.Validator)

Run by splap_import.py validate and during fetch; results go to validation.json in the bundle and into the static session.json. Each result is {check, status, detail} with status pass, fail or warn. Validator.ok is true when nothing is fail.

The shipped Monza window passes 11 checks and warns on nondecreasing_gap_to_leader (214 of 1620 frames, explained by staggered per-driver interval updates).