Core Concepts

Definition

This page introduces the abstractions the web tracking SDK is built on: the message, the state, the session, pipes, plugins, services, the queue and transport, and the browser facilities the core provides to all of them (document visibility, tabs, location history and storage). Read it before any other page in this section. Each part ends with a link to the page that covers it in depth.

Overview

The tracking SDK is a JavaScript library that, once included in a web page, provides an interface for collecting analytical messages such as behavioral events, device attributes and performance measurements. The library takes care of enriching those messages and delivering them to the server. It also tracks some data automatically, such as page views, browsing sessions, clicks and page visibility.

We can split the SDK's duties into three categories: collecting the data, enriching it, and successfully transferring it to the server. The goal was an SDK that can track any kind of analytics data, so we designed a core that is unaware of what data it is actually tracking. Instead, it works with an abstraction called Message.

The SDK's core contains session management, client-side storage, the transfer queue, tab management, history tracking and so on. None of these is bound to a specific message type such as an event. They are the components you need to track any kind of data. Whether you are tracking a behavioral event or collecting network timings, you need to attach device and session information to your messages. You need contextual information such as the current and previous page URLs, tab identifiers and document visibility. And you need a guarantee that every message that enters the library is eventually transmitted to the server.

The core takes care of all of this, but of course we still need data. That is the job of the collectors. A collector produces the data and teaches the SDK how to handle it, by providing pipes, plugins and services.

   collectors (plugins, dataLayer, pulse.event / attribute calls)
        │  produce raw payloads
        ▼
   PulseTracker.processMessages ── wraps each payload in a Message { type, data, state, ... }
        │
        ▼
   pipes (transform, then enrich) ── may drop, rewrite or multiply messages
        │
        ▼
   MessageQueue ── IndexedDB ── batches of chunks ── BatchQueue ── HTTP transport ── server

   core services used everywhere: session · state · tabs · visibility · location history · storage

Message

Tracking is all about messages. A message carries information about some event or action that took place at a particular point in time. The SDK distinguishes messages by their type field and uses that type to decide how to parse and process the data, by selecting which pipes to apply. A message always carries a timestamp that says when it was produced, even when the data is not about an action in time (a device attribute, for example).

The draft of this design allowed any string as the type. Today the type is a closed union of three values, event, attribute and attribute-hint, defined in common/src/sdks/tracker/models/message/message.ts. Other kinds of data travel as events with their own event_type: RUM performance entries, for example, are events named rum.

{
  "type": "event",
  "timestamp": 1618833526807,
  "priority": 1,
  "state": { "app": "web", "device_id": "a.c.lz3k0q.5b1e…", "session_id": "1618833000000_a.c…" },
  "data": { "event_type": "click", "data": { "click_source": "header" } }
}

Field	Type	Description
type	'event', 'attribute' or 'attribute-hint'	Determines the message data type and which pipe sets apply. event carries a PulseEvent, attribute an attribute operation ($overwrite, $inc, $dec), attribute-hint a small hint object the server turns into attributes.
timestamp	number	The time the message was produced, in milliseconds since the UNIX epoch. EventGlobalsPipe copies it onto the event.
data	unknown (a PulseEvent, PulseAttribute or PulseAttributeHint)	The actual payload. The tracker deep-clones what the caller passed, so later changes to the caller's object do not leak into the queue.
state	Record<string, unknown>	The tracker state the message was produced under. See State below.
priority	PulseMessagePriority (optional)	Delivery priority: Normal (1) waits for its turn, High (2) flushes the queue immediately, Force (3) flushes and also bypasses the consent gate.

pulse.event(), pulse.attribute() and the internal attributeHint() all end in PulseTracker.processMessages (browser/tracker/src/tracker.ts), which builds one message per payload and hands it to the pipes. See Event Pipeline.

State

State is an object that holds information common to a set of messages: app, platform, version, device_id, session_id, experiment participation, and so on. The SDK keeps track of the latest state values and shares them between tabs and sessions. Some values are added automatically by the SDK (mostly by StatePlugin and SessionService), and the application can add and modify values too. Typical examples are setting a user_id after login or the user's subscription status with pulse.set(...).

{ "user_id": 300362254321101, "is_subscribed": true, "app": "web", "data": { "language_code": "en" } }

StateService (browser/shared/src/services/state.service.ts) owns the state. It stores it under paa_ss, shallow-merges every update, ignores updates that are deep-equal to the current value, listens to storage changes from other tabs, and publishes every new value on a stream.

The original design cloned the whole state into every message. The code works differently now, and more cheaply:

• processMessages deep-clones the message data, but not the state.
• message.state is the reference to the current state object at call time.
• StateService.setState never mutates the object in place. Every real change creates a new object.

So messages produced under the same state share one object, and a state change naturally starts a new group. Before transfer the state is separated from the messages, one chunk per distinct state object (see Chunk). On the wire the state is called the header, and event-receiver merges it into every event of the chunk.

See Identity, State, Consent and Modes and Batch State.

Session

A session groups the messages a user produces continuously in one time frame. When a user visits the website for the first time, the SDK creates a new session. The session expires after a period of inactivity, 30 minutes by default (sessionTimeoutMS), measured from the last activity, so a session can last much longer than 30 minutes while the user keeps using the site. Every tracked message counts as activity.

Each session has a unique identifier, session_id, which SessionService adds to the global state. It is therefore sent with all messages. Its format is ${Date.now()}_${deviceId}, and other systems parse it.

What	Where
Session id and last activity	paa_sid and paa_la, in the storage chosen by sessionStorageType (localStorage by default, so all tabs share one session). Last activity is written at most once per second.
Renewal triggers	No session yet; idle longer than sessionTimeoutMS; a traffic source change (new UTM, click id or external referrer); tracking consent accepted.
On renewal	session_start (High priority), then navigation with the renewal reason as its source.

See Identity, State, Consent and Modes and session_start.

Collectors

Collectors extend the SDK's core so it supports new kinds of data. The design describes a collector as a module that registers a set of pipes, plugins and services. Every kind of data, such as behavioral events, network timings, performance metrics or device attributes, has at least one collector behind it.

There is no Collector class or registration API in the code today. A collector is a convention, realised through the dependency-injection modules:

• TrackerModule (browser/tracker) is the main collector. It registers the default pipes in factory/pipes.ts, the built-in plugins in factory/plugins.ts (click, popup, navigation, session start, consent, device attributes, and so on), and the services they need in factory/providers.ts.
• TrackerSharedModule (browser/tracker-shared) provides the pipe and plugin hosts and the RUM collectors: FetchCollectorService and XHRCollectorService wrap fetch and XMLHttpRequest, and PerformanceTimelineService observes the Performance Timeline. RUMService combines them, and RUMPlugin (in the tracker) turns every entry into a rum event.
• The page itself can act as a collector: config.plugins, config.transformPipes and config.enrichmentPipes are registered next to the built-in ones.

Creating a Collector

A new kind of data needs up to three pieces. Pick the ones you need.

1. A producer. Usually a plugin whose init(context) subscribes to some browser source and emits messages. Built-in plugins emit by pushing { pulse: 'event', ... } to window.dataLayer, so that GTM pushes and internal events share one path. A plugin can also call context.tracker.event(...) directly.
2. Pipes that normalise, validate or enrich the new data, registered for the right message type.
3. Services that hold the logic, provided through ProviderBuilder so they can be injected and shared.

Inside the repository, the steps are:

• add the plugin class to browser/tracker/src/plugins/ and register it in factory/plugins.ts (constructor argument, and an entry in the plugins array);
• add pipe classes to pipes/ and put them into a pipe set in factory/pipes.ts;
• provide every new class in factory/providers.ts with .useClass(Token, Class, [deps]) or .useFactory(...), and add it to the PluginsFactory or PipesFactory dependency list.

From a page, the same shape goes into the configuration:

import {
    AnalyticsPlugin,
    EnrichmentPipe,
    IPulsePluginContext,
    PulseEventMessage,
    TransformPipe,
} from '@pulse/common';

// Producer: emits a `web_vital_lcp` event once per page.
class LcpPlugin implements AnalyticsPlugin {
    init(context: IPulsePluginContext): void {
        new PerformanceObserver((list) => {
            const last = list.getEntries().pop();
            if (last) {
                context.tracker.event({ event_type: 'web_vital_lcp', data: { value: last.startTime } });
            }
        }).observe({ type: 'largest-contentful-paint', buffered: true });
    }
}

// Transform pipe: return [] to drop a message, several to fan out.
class DropZeroLcpPipe implements TransformPipe<PulseEventMessage> {
    transform(message: PulseEventMessage): PulseEventMessage[] {
        return message.data.data['value'] === 0 ? [] : [message];
    }
}

// Enrichment pipe: always returns exactly one message.
class PageGroupPipe implements EnrichmentPipe<PulseEventMessage> {
    enrich(message: PulseEventMessage): PulseEventMessage {
        message.data.data['page_group'] = location.pathname.split('/')[1] || 'home';
        return message;
    }
}

const config = {
    plugins: [new LcpPlugin()],
    transformPipes: [{ selector: 'event', use: [new DropZeroLcpPipe()] }],
    enrichmentPipes: [{ selector: 'event', use: [new PageGroupPipe()] }],
};

note

User transform pipes run before the default ones, so they see the payload as the caller passed it. An event sent as { action, parameters } has not yet been normalised to { event_type, data } at that point. Enrichment pipes see the normalised event.

See Packages and DI for ProviderBuilder and module composition.

Pipes

Every message passes through the pipes registered for its type. Pipes have the full power to enrich, transform, validate and filter messages. They can stop a message from being processed, or produce more messages from the one they received. The SDK itself does not modify a message outside of pipes: even the defaults, such as adding event_id, url and tab_id, are implemented as pipes.

Authoring a Pipe

There are two pipe interfaces, both in common/src/sdks/tracker/pipe/pipe.ts:

Interface	Method	Returns	Use it to
TransformPipe	transform(message, context)	T[]	Rewrite, validate, drop (return []) or fan out (return several messages).
EnrichmentPipe	enrich(message, context)	T	Add fields to a message. It cannot drop or multiply it.

context is an IPulsePluginContext, { tracker, config }; the full tracker also adds the current session. Pipes are plain classes, so a built-in pipe gets its services through its constructor.

The built-in pipes show the typical jobs:

Pipe	Stage	Does
ActionParamEventPipe	transform	Normalises {action, parameters} and {event, data} to {event_type, data}.
NavigationEventPipe	transform	Drops a navigation whose URL repeats the previous one.
AttributeSpreadPipe	transform	Fans one { k1: v1, k2: v2 } object out into one $overwrite message per key.
ChangedAttributesPipe	transform	Drops attributes whose value has not changed for 7 days.
EventGlobalsPipe	enrich	Adds timestamp, a fresh event_id, url, tab_id, opener_tab_id, previous_url, referrer and more.
LegacySourcePipe	enrich	Replaces data.source === true with the computed traffic source.

Pipe Sets

Pipes are registered in pipe sets (pipe/pipe-set.ts):

interface TransformPipeSet { selector: PipeSelector; exclude?: PipeSelectorFn; use: TransformPipe[] }
interface EnrichmentPipeSet { selector: PipeSelector; exclude?: PipeSelectorFn; use: EnrichmentPipe[] }
// PipeSelector = 'event' | 'attribute' | 'attribute-hint' | '*' | ((msg) => boolean)

Field	Type	Description
selector	PipeSelector	Which messages the set applies to: a message type, '*' for all, or a predicate function.
exclude	PipeSelectorFn (optional)	A predicate that removes matching messages from the set even when selector matched.
use	array of pipes	The pipes to run, in order. A transform set must contain at least one pipe.

A PipeHost (browser/tracker/src/pipe/pipe-host.ts) checks its sets in registration order and uses only the first set that matches a message. It does not combine sets. If you need several pipes for one type, put them in one set.

Pipe Stages

The tracker owns two pipe hosts: one for the default pipes (DefaultPipesHostToken) and one for the pipes from the configuration (PipesHostToken). processMessages runs every message through four stages:

message
  → user transform       (config.transformPipes)
  → default transform    (AttributeSpread + ChangedAttributes, or ActionParam + Navigation)
  → default enrich       (EventGlobals, LegacySource)
  → user enrich          (config.enrichmentPipes)
  → MessageQueue.push

• Transform stages fan out. Each transform pipe runs on every message produced by the pipe before it, and the outputs are flattened. One input can end up as zero, one or many messages.
• Enrich stages run in reverse. Within a set, the enrichment pipes are applied from last to first, so in [EventGlobalsPipe, LegacySourcePipe] the source pipe runs first.
• Each host picks its own set. The user host and the default host each select their first matching set independently.

See Event Pipeline.

Plugins

Plugins are the extension point for everything that is not a per-message transformation. A plugin implements AnalyticsPlugin (common/src/sdks/tracker/plugin/plugin.ts). All of its methods are optional:

Hook	Status in the browser tracker
init(context)	Called once for every plugin, when the tracker is created. This is where built-in plugins subscribe to the DOM, storage and services.
enqueue, beforeBatchSend, batchSent	Declared, and implemented by PluginHost, but not called by the browser tracker today.
sessionChange, stateChange	Declared only. Plugins that need these subscribe to SessionService or StateService instead.

PluginHost (browser/tracker-shared/src/plugin/plugin-host.ts) keeps the list. The tracker's PluginsFactory registers the built-in plugins first, then config.plugins, then the DataLayerPlugins that consume window.dataLayer and the internal pulse queue. Because events that were pushed to the dataLayer before the SDK loaded are still in the array, they are processed as well.

Most plugins are small producers: ClickEventPlugin, PopupEventPlugin, NavigationEventPlugin, SessionStartEventPlugin, ConsentEventPlugin, DeviceAttributesPlugin, RUMPlugin. Others maintain core state: StatePlugin writes the default header, TabLinkingPlugin starts tab bookkeeping, VisibilityChangePlugin remembers the visibility state. See Event Pipeline for the full list.

Services

Services hold the logic that pipes and plugins share: SessionService, StateService, TabManagerService, ActivityTracker, LocationHistoryTracker, DocumentVisibilityTracker, HttpService, IndexedDBService, the consent services and many more. They are plain classes without decorators. They are wired by hand in each package's providers file with ProviderBuilder:

builder
    .useClass(PopupEventPlugin, PopupEventPlugin, [ActivityTracker, PulseEnvironment])
    .useFactory(
        LocationHistoryTracker,
        (env, selectStorage, config, date) =>
            new LocationHistoryTracker(env, selectStorage(config.tabManagerStorageType), date),
        [PulseEnvironment, StorageSelectorToken, TrackerConfigurations, DateService],
    );

Every provider is a singleton inside its injector, dependencies are resolved asynchronously, and all modules read one merged configuration object. Services that belong to the core live in browser/shared, so the settings module and the mini tracker can use them too. See Packages and DI.

Queue and Transport

After the pipes, the messages end up in the transport queue and wait for their turn to be sent. MessageQueue (browser/tracker/src/transport/message-queue.ts) does four things:

1. Gates on consent. Without tracking consent a message is dropped, unless its priority is Force.
2. Persists. Every message is written to IndexedDB (pulseDBN, eventsStore) before it is buffered, so a closed tab does not lose it. The next page load resends it.
3. Buffers. Messages are held in memory until one of these happens: batchTimeMS (500 ms) passes, batchSize (100) messages have arrived, a High or Force message arrives, the page becomes hidden, or flush() is called.
4. Batches. The buffer is cut into batches and passed to BatchQueue, which moves them to batchesStore, waits until the page has loaded, and sends them through PulseTransport and HttpService (fetch, then sendBeacon, then XHR, with retries and backoff).

See Event Pipeline and Message and Batch Queues.

Chunk

To reduce the number of requests, messages are grouped into batches that are sent in one request. Each batch contains chunks, and a chunk groups messages with the state they were produced under. Because a state change creates a new state object, a new chunk starts whenever a state field changes. groupByContext (common/src/sdks/tracker/transport/group-by-context.ts) builds one chunk per distinct state object.

The design described a chunk as { state, messages }. The shape has evolved: messages are split by type, the state is called header, and every chunk carries the identity configuration the server needs.

{
  "config": { "deviceId": "persistent", "sessionId": "persistent", "tabId": "tab", "serverCookies": true },
  "header": { "app": "web", "device_id": "a.c.lz3k0q.5b1e…", "session_id": "1618833000000_a.c…" },
  "events": [{ "event_type": "click", "event_id": "…", "timestamp": 1618833526807, "data": {} }],
  "attributes": [{ "type": "$overwrite", "name": "is_subscribed", "value": true }],
  "attributeHints": { "user_agent": "…" }
}

Field	Type	Description
config	PulseBatchConfig	How identity is handled for this chunk. See Batch.
header	Record<string, unknown>	The state object the messages were produced under.
events	PulseEvent[]	The data of every event message in the group.
attributes	PulseAttribute[]	The data of every attribute message in the group.
attributeHints	PulseAttributeHint (optional)	The data of the attribute-hint message, if the group has one.

Batch

A batch (PulseBatch, common/src/sdks/tracker/models/batch.ts) is what one HTTP request carries. The request body is the batch's chunks array.

{ "chunks": [ { "config": {}, "header": {}, "events": [], "attributes": [] } ], "size": 18342, "keepAlive": true }

Field	Type	Description
chunks	PulseBatchChunk[]	The chunks to send. This array is the request body.
size	number (optional)	The JSON size in bytes, used by BatchQueue to budget in-flight bytes.
keepAlive	boolean (optional)	Whether the request may use fetch keepalive.

getBatches (transport/batch-helper.ts) keeps each batch at or below 60,000 bytes, below the 64 KB keepalive limit. Every batch within that limit is sent with keepalive, so it survives the page closing. A single message that is larger than the limit on its own goes into a batch of its own, without keepalive.

The chunk config is built by getTrackerBatchConfig:

Field	Type	Description
deviceId	'none', 'persistent', 'tab' or 'memory'	The storage the device id lives in, or none when stateEnrichment.device_id is off.
sessionId	same as above	The same for the session id.
tabId	same as above	The storage of the tab id, or none when eventEnrichment.tab_id is off.
serverCookies	PulseServerCookieMode	Whether the server may set identity cookies. Always false while tracking consent is off.

See Batch State.

Document Visibility

Much of the delivery logic depends on whether the page is visible, because a hidden page may be closed at any moment without further notice. DocumentVisibilityTracker (browser/shared/src/services/visibility.tracker.ts) is the one place that knows this. It listens to visibilitychange, including vendor-prefixed variants. In browsers without the full Visibility API it falls back to pagehide, beforeunload, pageshow and focus. It exposes visible and a de-duplicated onVisibilityChange() stream.

Its main consumers:

• MessageQueue flushes the buffer as soon as the page becomes hidden.
• VisibilityChangePlugin remembers the state, and DataLayerPlugin raises events produced while the page is hidden to High priority, so they are sent at once.
• HttpService skips the retry backoff for the first retries while the page is hidden.
• The fallback lock manager and the RUM performance timeline also watch visibility.

The SDK deliberately does not flush on beforeunload or unload. Those handlers block the back/forward cache. See Design Decisions.

Tabs

Every tab runs its own copy of the SDK over shared storage, so the SDK needs to tell tabs apart.

Piece	Does
TabManagerService (browser/shared/src/services/tab/tab-manager.ts)	Keeps tab_id in sessionStorage (paa_tab_id) and holds the lock tab_id_<id> for it. A duplicated tab copies the sessionStorage but cannot take the lock, so it generates its own id.
TabInfoService (browser/tracker/src/services/tabs/)	Writes a tab_info_<id> record to localStorage with the tab's activity, and lists the sibling tabs that are still open. Queue rehydration uses this to skip records owned by another open tab.
TabOpenerDetector	Works out which tab opened this one, which becomes opener_tab_id on events, and which click caused a navigation: a declared pulse-navigate click first (see Declared Navigation Source), otherwise a recent click within short time windows.
TabLinkingPlugin	Starts the tab info and activity services when tab tracking is on.

tab_id is added to each event by EventGlobalsPipe, and getState() adds it to the returned state. Tab tracking follows eventEnrichment.tab_id, which is off in the anonymous and page-session modes. See Identity, State, Consent and Modes.

Location History

In a single-page application the URL changes without a page load, so the SDK has to watch the history itself. LocationHistoryTracker (browser/tracker/src/services/location.history/location-history.tracker.ts):

• wraps history.pushState, replaceState, back, forward and go, and listens to popstate;
• on every real URL change updates previousUrl and emits on its navigation stream, which NavigationEventPlugin turns into a navigation event;
• keeps the last URL in tab storage (last_location) and writes it on beforeunload. When the next page in the same tab starts within 2 seconds, that URL becomes previous_url. Otherwise document.referrer is used.

EventGlobalsPipe adds previous_url to every event, and NavigationEventPipe drops a navigation whose URL repeats the previous one.

Storage

The core never touches localStorage or cookies directly. Everything goes through the PulseStorage abstraction (common/src/services/storage/pulse-storage.ts): get, set, remove, keys, JSON helpers, and a change(key) stream that reports changes made by this tab or by other tabs.

Implementation	Backed by	Used for
PersistentStorage	localStorage	persistent: session, state, device id, tab info (shared by all tabs)
TabStorage	sessionStorage	tab: tab id, last location (one tab only)
TransientStorage	memory	memory: anonymous modes, and the fallback when localStorage is unavailable
CookieStorage	cookies	Identity recovery (paa-did), the pulse-state cookie, consent cookies
IndexedDBService	IndexedDB pulseDBN	eventsStore, batchesStore, settingsStore, attributesStore

Services do not choose a backend themselves. They ask the StorageSelector (StorageSelectorToken) for a StorageType taken from the configuration, such as sessionStorageType or tabManagerStorageType. That is how the modes switch everything to memory by changing configuration only. PulseSyncStorage prefixes keys (paa_ by default), turns storage events into change notifications, and frees space when the quota is exceeded.

IndexedDB is treated as best effort. Every wait on it is time-boxed and the SDK keeps working without it. See LocalStorage, SessionStorage and IndexedDB.

Where to look in the code

Concept	Code
Message, priorities	packages/common/src/sdks/tracker/models/message/
Message creation	packages/browser/tracker/src/tracker.ts (processMessages)
State	packages/browser/shared/src/services/state.service.ts, tracker/src/plugins/state.plugin.ts
Session	packages/browser/tracker/src/services/session/session.service.ts
Pipe interfaces and sets	packages/common/src/sdks/tracker/pipe/
Pipe host and stages	packages/browser/tracker/src/pipe/pipe-host.ts, tracker/src/factory/pipes.ts
Built-in pipes	packages/browser/tracker/src/pipes/
Plugin interface and host	packages/common/src/sdks/tracker/plugin/, browser/tracker-shared/src/plugin/plugin-host.ts
Built-in plugins	packages/browser/tracker/src/plugins/, tracker/src/factory/plugins.ts
Services and wiring	packages/browser/tracker/src/factory/providers.ts, browser/shared/src/providers/providers.ts
RUM collectors	packages/browser/tracker-shared/src/services/rum/
Queue, batches, chunks	packages/browser/tracker/src/transport/, common/src/sdks/tracker/transport/group-by-context.ts, common/src/sdks/tracker/models/batch.ts
Visibility	packages/browser/shared/src/services/visibility.tracker.ts
Tabs	packages/browser/shared/src/services/tab/tab-manager.ts, browser/tracker/src/services/tabs/
Location history	packages/browser/tracker/src/services/location.history/location-history.tracker.ts
Storage	packages/common/src/services/storage/, browser/shared/src/services/storage/, browser/shared/src/services/indexedDB/