Event Pipeline: From pulse.event() to the Server

Definition

This document explains what happens to a message between the moment the page calls pulse.event() (or attribute(), or a built-in plugin emits something) and the moment event-receiver acknowledges it. It covers pipes, plugins, the MessageQueue, batching, the BatchQueue, HttpService transport selection and retry, IndexedDB durability, priorities, unload behaviour, and how the mini tracker differs.

It goes deeper than Message and Batch Queues. Read Packages and DI first if you have not seen how TrackerModule is wired.

The shape of the pipeline

The tracker never sends a bare event. Every call becomes a PulseMessage, which passes through pipes, is persisted, buffered, grouped by the state header it was created under, cut into batches of at most 60,000 bytes, and posted. Each stage has one job, and each stage can be reasoned about on its own.

pulse.event({ event_type | action | event, data | parameters }, priority?)
  │  tracker.ts processMessages
  │    deep-clone payload → PulseMessage { type, data, priority, state, timestamp }
  │    state = the StateService object *reference* at call time
  ▼
PipeHost                      user transform → default transform → default enrich → user enrich
  ▼
MessageQueue.push             consent gate → persist to IndexedDB eventsStore → rx buffer
  │    buffer flushes on: batchTimeMS | batchSize | High/Force priority | page hidden | flush()
  ▼
getBatches + groupByContext   ≤ 60,000-byte batches; one chunk per distinct state object
  ▼
BatchQueue.pushBatch          events → batchesStore; waits for page-loaded; 100 ms tick
  ▼
PulseTransport.send → HttpService.post
  │    fetch /fetch  |  sendBeacon /beacon  |  XHR /xhr, with retry and backoff
  ▼
on success: delete batch from batchesStore, emit queueEmpty$

The wiring lives in packages/browser/tracker/src/factory/: providers in providers.ts, plugins in plugins.ts, pipes in pipes.ts.

Step 1: creating the message

PulseTracker.processMessages (packages/browser/tracker/src/tracker.ts) is the single entry point for events, attributes and attribute hints. For each item it:

• deep-clones the payload, so later mutation by the caller cannot change what is sent;
• builds { type, data, priority, state, timestamp }, where type is event, attribute or attribute-hint;
• captures state as the reference to the current state object held by StateService.

The state reference matters. StateService.setState() replaces the object whenever the header changes (see Identity, State, Consent and Modes). Messages created before and after a change therefore hold different objects, and groupByContext later uses reference identity to put them in different chunks. That is how an event keeps the header that was true when it fired, even if the batch is sent later.

Step 2: pipes

Pipes are synchronous functions that reshape or enrich a message. PipeHost (packages/browser/tracker/src/pipe/pipe-host.ts) holds pipe sets. A set has a selector ('*', a message type, or a predicate), an optional exclude predicate, and a use list of pipes.

Each message runs through four stages, in this order:

1. user transform pipes (config.transformPipes)
2. default transform pipes
3. default enrichment pipes
4. user enrichment pipes (config.enrichmentPipes)

Rules that are easy to miss:

• Only the first matching set in a host is used. Sets are not combined.
• Transform pipes may fan out. A transform returns an array, so one message can become N messages (or zero, which drops it).
• Enrichment pipes run in reverse of their use order, and each returns exactly one message.

Default pipes

Pipe	Kind	What it does
ActionParamEventPipe	event transform	Normalises {action, parameters} and {event, data} to {event_type, data}
NavigationEventPipe	event transform	Drops a navigation event whose href repeats the previous one
EventGlobalsPipe	event enrich	Adds timestamp, a fresh event_id, data.metadata.event_id_source = 'sdk', customer_id, url, tab_id, opener_tab_id, previous_url, referrer, and the web-to-mobile is_w2m / mobile_attributes
LegacySourcePipe	event enrich	Replaces data.source === true with the computed traffic source
AttributeSpreadPipe	attribute transform	Turns {k: v, …} into N $overwrite attributes
ChangedAttributesPipe	attribute transform	Suppresses an attribute whose value has not changed within attributeTimeoutMS (7 days), using HashStore

Of the EventGlobalsPipe fields, only url, tab_id (together with opener_tab_id), previous_url and referrer are controlled by the eventEnrichment toggle. The others do not depend on it; customer_id is added only when customerID is configured, and is_w2m / mobile_attributes only on Web To Mobile pages.

tip

Because EventGlobalsPipe generates a fresh event_id for each event, that id is also the key under which the message is stored in IndexedDB. Do not reuse ids across events.

Step 3: plugins

Plugins are how built-in events are produced. A plugin implements init(context) and is initialised once when PulseTracker is constructed. The enqueue, beforeBatchSend and batchSent hooks declared in packages/common/src/sdks/tracker/plugin/plugin.ts exist in the interface but are not called by the browser tracker, so do not rely on them.

Most built-in plugins do not call the tracker directly. They push to window.dataLayer with pulse: 'event' (or attribute, attribute_hints, state), and DataLayerPlugin consumes the dataLayer. This gives GTM-style pushes and internal events one path, and it means anything pushed before the SDK loaded is still processed.

Plugin	Emits or does
StatePlugin	Writes the default header (see Identity, State, Consent and Modes)
SessionStartEventPlugin	session_start with High priority; see session_start
NavigationEventPlugin	navigation on init, on session renewal and on history changes
ClickEventPlugin	click; see click
PopupEventPlugin, ContextMenuPlugin	popup, contextmenu
ConsentEventPlugin	consent_change with Force priority; renews the session and resends attributes on opt-in
DeviceAttributesPlugin	Device and UTM attributes plus attribute_hints (user_agent), at most once per 24 hours, forced on a new session
FacebookPixelAttributePlugin	_fbp cookie becomes the fb_pixel_id attribute
InfectedPlugin	Flushes the queue on a new experiment participation
VisibilityChangePlugin	Tracks visibility; DataLayerPlugin uses it to give dataLayer events without an explicit priority High priority while the page is hidden
TabLinkingPlugin	Sibling-tab activity sync and opener tab id
DebugEventPlugin	Live event debug (?debug-leuid)
RUMPlugin	rum events from PerformanceObserver and fetch/XHR collectors
DataLayerPlugin	Consumes window[dataLayerKey]
config.plugins	Your own plugins

Step 4: the MessageQueue

MessageQueue (packages/browser/tracker/src/transport/message-queue.ts) accepts messages, persists them, and decides when to cut a batch.

The consent gate

push() drops a message unless tracking consent is enabled or its priority is Force. Dropped messages are not persisted and are not replayed later when consent is granted. This is intentional: the queue must not hold data the user has not agreed to send. See Identity, State, Consent and Modes for how consent is computed.

Persistence

Accepted messages are written to the IndexedDB database pulseDBN, store eventsStore, before they are buffered. Each record is { id, message, time, tabId, sessionId }. The id is the event's event_id when there is one, otherwise a new UUID (which is also written back onto attribute and attribute-hint messages so they can be deleted later). A failed IndexedDB write is swallowed; delivery continues without durability.

When a batch is cut

Messages go into an rx buffer. The buffer is emitted, and a batch is cut, when any of these happens:

Trigger	Default
batchTimeMS interval elapses	500 ms
batchSize messages accumulated	100
A High or Force message is pushed	immediate
The page becomes hidden (visibilitychange) and tracking is enabled	immediate
pulse.flush() is called	immediate

When the buffer emits, the interval timer and the counter restart.

Step 5: batching and chunks

getBatches (packages/browser/tracker/src/transport/batch-helper.ts) splits the buffered messages into batches whose JSON size stays at or under MAX_REQUEST_BYTES = 60_000. The browser's keepalive request budget is 64 KB, and the margin keeps every normal batch eligible for keepalive. A single message whose own chunk is larger than the limit is sent alone, as a batch with keepAlive: false.

Inside each batch, groupByContext (packages/common/src/sdks/tracker/transport/group-by-context.ts) groups messages by their state object reference and produces one chunk per group:

interface PulseBatchChunk {
    config: PulseBatchConfig; // { deviceId, sessionId, tabId, serverCookies }
    header: PulseSessionState; // the state object the messages were created under
    events: PulseEvent[];
    attributes: PulseAttribute[];
    attributeHints?: PulseAttributeHint;
}

The request body is an array of these chunks. event-receiver merges each chunk's header into every event in it. The config block tells the receiver how each identifier is stored (persistent, tab, memory or none) and whether it may set server cookies; it is built by getTrackerBatchConfig, and serverCookies is forced to false while tracking consent is off. The header fields are listed in Batch State and Contracts.

info

The envelope uses camelCase (config.deviceId, attributeHints) while header and event fields use snake_case. Both conventions are part of the wire contract; keep them as they are.

Step 6: the BatchQueue

BatchQueue (packages/browser/tracker/src/transport/batch-queue.ts) owns delivery of cut batches.

When a batch is pushed, the queue:

1. deletes the batch's events, attributes and attribute hints from eventsStore;
2. writes the whole batch to batchesStore as { id, batch, time, tabId };
3. appends it to the in-memory queue.

Sending does not start until PageLoadedService.whenLoaded() resolves: the DOM has been idle for pageLoadedDomIdleMS (500 ms, measured with a MutationObserver) or pageLoadedMaxWaitMS (5 s) has passed, whichever comes first. This keeps analytics traffic out of the page's critical loading path.

After that, a 100 ms interval takes the next batch when nothing is in flight, or when the bytes currently in flight plus the next batch's size stay below 60,000, so the total of concurrent keepalive requests stays within the browser budget. When a request resolves, its size is released, the batch is removed from batchesStore, and queueEmpty$ emits once nothing is queued or in flight. The interval stops when the queue is empty and restarts on the next push.

Step 7: transport and HttpService

PulseTransport (packages/browser/tracker/src/services/transport/transport.ts) posts batch.chunks to {serverBaseURL}/{app} (default serverBaseURL is https://t.picsart.com/events/v2/web) with headers Content-Type: application/json and receiver-debug. HttpService.post (packages/browser/shared/src/services/http/http.service.ts) then chooses a mechanism and appends the path suffix:

Order	Mechanism	URL suffix	Used when	Notes
1	fetch POST	/fetch	fetch and AbortController exist, and, if the page is hidden, Request.keepalive is supported	credentials: 'include', keepalive set from the batch
2	navigator.sendBeacon	/beacon	fetch is not usable and the batch is a keepalive batch	Cannot set headers, so receiver-debug is lost; falls back to XHR if the beacon is not queued
3	XHR POST	/xhr	Otherwise	withCredentials, timeout transportTimeout

What counts as success

• fetch: any status below 500 whose body parses as JSON is a success. A 4xx is not retried; a client validation error would fail again, so retrying only wastes the user's bandwidth. A 5xx, a network error or an unparseable body is a failure and is retried.
• sendBeacon: success as soon as the browser accepts the beacon for sending.
• XHR: the response is resolved when the request completes; succeeded() (status 200 to 400) only sets the ok flag.

If any response carries a paa-did header, the device id may be replaced; see Identity, State, Consent and Modes.

Retry and backoff

Setting	Default	Meaning
transportRetries	Infinity	Retry count is unlimited…
transportTimeout	60,000 ms	…but the whole attempt gives up after this much time
transportRetryBackoffMS	100 ms	Backoff base
transportRetryBackoffMaxMS	30,000 ms	Backoff ceiling

The wait after failure n is min(30 s, (n+1)^(n+1) × 100 ms). While the page is hidden, the first 9 retries wait 0 ms, because a hidden page may be about to close and has no time to wait. If navigator.onLine is false, the loop waits for the online event before trying again.

When the 60 s budget runs out, the batch is not deleted. It stays in batchesStore and is sent by a later page load.

Durability

The goal is that a closed tab, a crash or a network outage does not lose events. Every message is in IndexedDB from the moment it is accepted until the server confirms its batch.

push ──► eventsStore ──(batch cut)──► batchesStore ──(2xx/4xx)──► deleted
                                             │
                                             └─(timeout / tab closed)──► kept for next load

On start:

• MessageQueue reads eventsStore and re-batches records younger than 5 days, then deletes them. A record carrying a sessionId or tabId gets those values merged back into its header, so it is attributed to the session it came from.
• BatchQueue deletes batchesStore records older than 5 days, then, after a 1 s pause, requeues the rest.
• Both skip records owned by another open tab (TabInfoService.getOpenSiblingTabIds()). The owning tab is still alive and will send them itself; taking them over would send them twice. The 1 s pause gives a sibling tab that is finishing its final delivery time to do so.

IndexedDB is fragile: some browsers hang or corrupt it. Every IndexedDB call in the pipeline catches its own failure, waits are time-boxed, and the pipeline keeps delivering without persistence when IndexedDB is unavailable. pulseDBN is version 2 and also contains settingsStore and attributesStore; a schema change needs an upgrade path (see Contracts).

note

This supersedes the description in Message and Batch Queues, which says unsent batches are saved to localStorage. The current tracker stores both pending messages and pending batches in IndexedDB pulseDBN (eventsStore and batchesStore), and they are picked up by the next page load rather than saved at tab switch.

Priorities

PulseMessagePriority is defined in packages/common/src/sdks/tracker/models/message/event.ts:

Priority	Value	Consent gate	Flush	Used by
Normal	1	applies	waits for time or size	most events
High	2	applies	immediate	session_start; dataLayer events pushed while the page is hidden
Force	3	bypassed	immediate	consent_change

Force exists because the consent decision itself must reach the server even when the answer is "do not track". Use it for nothing else.

Unload behaviour

The pipeline deliberately does not flush on beforeunload or unload. They are not a reliable place to send data, so they are used only for bookkeeping, such as LocationHistoryTracker and TabInfoService, or as a legacy fallback where the Visibility API is missing.

Instead:

• On visibilitychange to hidden, MessageQueue flushes and batches go out with keepalive, which lets the browser finish them after the page is gone.
• Batches are capped at 60,000 bytes so each one qualifies for keepalive, and sendBeacon is the fallback when keepalive fetch is unavailable.
• A batch that is in flight when the tab dies, and is never confirmed, stays in batchesStore and is resent on the next load. There is no abort-and-resend of an in-flight request.

caution

Do not add work that must happen at unload. Anything that is not already a keepalive request under 60 KB when the page becomes hidden should be assumed lost for this page and recovered from IndexedDB.

The mini tracker

packages/browser/mini is a different pipeline for miniapps embedded in a host page that already runs Pulse. It has no session, consent, queue, batching, HTTP or settings of its own:

• Events go through ActionParamEventPipe and a minimal EventGlobalsPipe that sets event_id_source = 'mini_sdk'.
• The transport hands each event to the host's window.pulse._transport({ eventName, params }). The host's full tracker then does the consent check, batching and delivery described above.
• Its SharedModule.onInit deletes the IndexedDB database when pulseMini is set, so a miniapp never leaves its own durable queue behind.

Where to look in the code

Concern	Path
Entry point, processMessages	packages/browser/tracker/src/tracker.ts
Pipe host	packages/browser/tracker/src/pipe/pipe-host.ts
Default pipe registration	packages/browser/tracker/src/factory/pipes.ts
Pipes	packages/browser/tracker/src/pipes/
Plugin registration	packages/browser/tracker/src/factory/plugins.ts
Plugins	packages/browser/tracker/src/plugins/
Plugin interface	packages/common/src/sdks/tracker/plugin/plugin.ts
Message queue, events rehydrate	packages/browser/tracker/src/transport/message-queue.ts
Batch splitting, batch config	packages/browser/tracker/src/transport/batch-helper.ts
Chunk grouping	packages/common/src/sdks/tracker/transport/group-by-context.ts
Batch and chunk models	packages/common/src/sdks/tracker/models/batch.ts
Batch queue, batches rehydrate	packages/browser/tracker/src/transport/batch-queue.ts
Page-loaded gate	packages/browser/tracker/src/services/page-loaded.service.ts
Transport, receiver-debug header	packages/browser/tracker/src/services/transport/transport.ts
HTTP, retry, paa-did	packages/browser/shared/src/services/http/http.service.ts
IndexedDB	packages/browser/shared/src/services/indexedDB/indexeddb.service.ts
Transport defaults	packages/browser/shared/src/configuration.ts
Tracker defaults	packages/browser/tracker/src/tracker.configuration.ts
Mini tracker	packages/browser/mini