Databricks Events

Overview Labs

If your player events already live in a Databricks lakehouse, you can connect them to FirstLook without building an export job. FirstLook connects to your workspace as a Delta Sharing recipient (Databricks also calls this OpenSharing; it’s the same protocol). You share your event tables, and FirstLook imports new rows within minutes of them landing in Databricks. Imported events show up in analytics charts after the next scheduled aggregation refresh, typically within the hour.

Ingested events flow into the same analytics, retention charts, and player profiles as data collected through the FirstLook SDK, exactly like the S3 pipeline described on the S3 Events page. Unlike the S3 route, there is no export job to schedule and no CSV format to match, and reads never consume your Databricks compute.

What it does

• Watches your shared tables for new versions about once a minute, reading only the rows added since the last version it processed.
• Imports your history first, ingesting existing rows (optionally from a start date you choose) before switching to incremental reads.
• Resolves player identity from platform IDs to unified FirstLook player profiles, creating new player records when a platform ID appears for the first time.
• Surfaces ingestion status in the dashboard as a log of ingestion runs, so you always know what was imported and when.

How access works

FirstLook is an open recipient: it authenticates to your workspace’s sharing server with a bearer token you generate, receives short-lived download links to your table’s data files, and has no write access to your workspace or storage. You can revoke or rotate the credential whenever you like.

---

Table requirements

Delta Sharing puts a few conditions on tables served to external recipients like FirstLook. Get these right when you create the table; several are awkward to fix afterward.

Requirement	Why
A plain Delta table, not a streaming table or materialized view	Pipeline-managed objects (Lakeflow / Delta Live Tables outputs) are snapshot-only for external recipients: no version history, no change feed. If your events live in one, add a plain Delta table as a final append-only sink in your pipeline and share that.
Change data feed enabled at creation (delta.enableChangeDataFeed = true)	This is how FirstLook reads only new rows each minute instead of re-scanning your table. Enabling it later leaves earlier versions unreadable, so set it from the start.
Deletion vectors disabled at creation (delta.enableDeletionVectors = false)	Recent Databricks runtimes enable deletion vectors by default, but tables carrying the feature can’t be served in the sharing format FirstLook reads, and the feature sticks in the table’s history even if you turn the property off later. An append-only events table gets nothing from deletion vectors anyway.
Target file size set (delta.targetFileSize = '256mb')	FirstLook downloads and processes your table’s data files whole, with a per-file size limit. Databricks’ OPTIMIZE compacts files toward ~1 GB by default, which can exceed that limit; this property keeps compacted files under it.
Data in your own cloud storage (S3, ADLS, or GCS)	Delta Sharing serves external recipients by generating pre-signed links to your storage. Tables in Databricks-managed “default storage” (Free Edition and serverless-default workspaces) can’t be shared externally at all. Use an external table with an explicit LOCATION, or a catalog whose managed location is your bucket.

---

Setting up your Databricks share

You’ll need a Unity Catalog–enabled workspace and permission to create shares and recipients.

1. Create your events table

The reference table below satisfies all five requirements. Adjust the schema to the counter or duration contract (see Table schemas) and the location to your bucket.

CREATE TABLE game_analytics.player_counter_events (
  steam_id      STRING,     -- exactly one player identity column
  session_id    STRING,     -- UUID
  timestamp     TIMESTAMP,
  counter_name  STRING,     -- e.g. 'match.kills'
  counter_value INT,
  build_version STRING      -- optional
)
LOCATION 's3://your-bucket/firstlook/player_counter_events'
TBLPROPERTIES (
  delta.enableChangeDataFeed  = true,
  delta.enableDeletionVectors = false,
  delta.targetFileSize        = '256mb'
);

2. Create a share and recipient

Create a share containing your events table(s), create a recipient representing FirstLook, and grant it access. The WITH HISTORY clause is what allows incremental reads; don’t omit it.

CREATE SHARE firstlook_share;

ALTER SHARE firstlook_share
  ADD TABLE game_analytics.player_counter_events WITH HISTORY;

CREATE RECIPIENT firstlook;

GRANT SELECT ON SHARE firstlook_share TO RECIPIENT firstlook;

Creating the recipient produces an activation link where you download a credential file: a small JSON file containing the sharing endpoint and a bearer token. Treat it like a password. Databricks recommends setting a token expiration on recipients; see Credential lifecycle for how rotation works.

3. Configure in the dashboard

Navigate to Configuration > Databricks Events in your game’s dashboard and enter:

• Credential File — Paste the full contents of the credential file you downloaded.
• Tables — One row per shared table: the share, schema, and table name (e.g. firstlook_share / game_analytics / player_counter_events).
• Backfill from — (Optional, per table) The earliest event date to import. Leave blank to import the table’s full history.
• Enabled — Toggle ingestion on or off at any time.

When you save, FirstLook validates every table against your sharing server on the spot, checking the schema contract, change data feed support, and a test read, so a misconfigured table surfaces a specific error immediately. Each valid table shows a confirmation like “Validated: counter events, steam_id identity — awaiting first ingestion”, and the first import begins within a minute.

---

Table schemas

Each shared table is either a counter table (discrete events with a numeric value) or a duration table (timed activities). The type is detected from the columns, one type per table. Extra columns are ignored, so you can share tables that carry additional fields.

Every table must include exactly one player identifier column, drawn from the same set the S3 pipeline supports. See the player identity column list for the supported identifiers and Identity resolution for how rows become player profiles.

Counter tables

Column	Type	Description
(identifier)	STRING	One player identity column.
session_id	STRING (UUID)	Groups events in a play session.
timestamp	TIMESTAMP	When the event occurred.
counter_name	STRING	Dot-separated event name (e.g. match.kills).
counter_value	INT	The event value (e.g. kill count).
build_version	STRING	(Optional) The game build that generated the event.

Duration tables

Column	Type	Description
(identifier)	STRING	One player identity column.
session_id	STRING (UUID)	Groups events in a play session.
start_time	TIMESTAMP	When the activity started.
end_time	TIMESTAMP	When the activity ended.
duration_name	STRING	Dot-separated event name (e.g. match.round).

---

How ingestion behaves

First import, then incremental

When a table first validates, FirstLook reads a snapshot of it (the full history, or everything after your Backfill from date), then tracks the table’s Delta version and reads only newly added rows on each change. Large histories import in one pass, provided your table follows the delta.targetFileSize guidance above.

Events are append-only

FirstLook ingests inserted rows only. Updates and deletes in your table are ignored: player events are treated as immutable facts, so a correction should be appended as a new event rather than edited in place. Two consequences:

• Full reloads are safe. If an upstream job rewrites your table (INSERT OVERWRITE, a replayed pipeline), FirstLook’s duplicate detection absorbs the re-inserted rows and nothing double-counts. It’s wasteful but harmless.
• Deleting rows in Databricks does not remove events from FirstLook.

When events appear in charts

Imported rows are stored as soon as their run completes, and analytics charts pick them up at the next scheduled aggregation refresh, typically within the hour. A completed import with charts that haven’t moved yet is normal.

Credential lifecycle

Delta Sharing bearer tokens are usually created with an expiration. FirstLook reads the expiry from your credential file and shows a banner on the configuration page starting 14 days before it lapses. If reads fail repeatedly, ingestion pauses.

To rotate, generate a new credential file for the recipient in Databricks (ALTER RECIPIENT firstlook ROTATE TOKEN or via the UI) and paste it into the Credential section of the configuration page. Your table configuration and import history are untouched, and if ingestion was paused by credential failures it resumes on its own.

---