Data Streaming

Data Streaming forwards events from Joryio to external destinations in near real-time. Each stream binds a destination to a set of events, and Joryio delivers matching events to that destination in batches.

Navigate to Data → Data Streaming to manage your streams.

Destinations

Destination	Kind	What it does
Webhook	HTTP endpoint	POST every event as JSON to any HTTPS URL
Snowflake	Data warehouse	Continuously load events into a Snowflake table
BigQuery	Data warehouse	Stream events into a BigQuery dataset
Amazon Redshift	Data warehouse	Load events into an Amazon Redshift cluster
Amazon S3	Object storage	Write batched JSON / NDJSON files to a bucket
Kafka	Event stream	Produce events to a Kafka topic
Mixpanel	Product analytics	Forward events to a Mixpanel project
Amplitude	Product analytics	Forward events to an Amplitude project

Connection fields

Each destination asks for its own connection details. Secrets (passwords, keys, tokens, service-account JSON) are encrypted at rest.

• Webhook - URL, signing Secret (optional), extra Headers (optional)
• Snowflake - Account, Username, Password, Role, Warehouse, Database, Schema, Table
• BigQuery - Project ID, Dataset ID, Table ID, Service Account JSON
• Amazon Redshift - Host, Port, Database, User, Password, Schema, Table
• Amazon S3 - Bucket, Prefix, Region, Endpoint (optional, for S3-compatible storage), Access Key ID, Secret Access Key
• Kafka - Brokers, Topic, SASL Username/Password, SSL
• Mixpanel - Project Token, Region (US / EU / India). Events are sent to Mixpanel's /track API, so a Project Token is all that is needed - no service account or API secret.
• Amplitude - API Key, Region (US / EU)

Creating a Stream

1. Navigate to Data → Data Streaming and click Create Stream.
2. Name the stream (shown in the streams list) and pick a destination.
3. Fill in the destination's connection fields.
4. Choose the payload format - JSON (one object per event) or NDJSON (newline-delimited, one event per line).
5. Select which events to stream (see below).
6. Review the summary bar and click Create Stream.

Choosing events

You control exactly which events a stream receives:

• All events - toggle this on to stream every current and future event. Turn it off to pick specific events.
• By group - built-in events are grouped (Messaging, In-App, Journeys, Campaigns, E-Commerce, and Identity). Tick a whole group or individual events.
• Custom events - under Custom Event, search or type your own tracked events (the ones you send with track(), such as purchase or page_view). The picker lists the events your workspace has actually tracked; you can also type an event that isn't tracked yet and click Add.

Event catalog

These are the built-in events you can stream. You can also stream any custom event you send with track().

Group	Events
Messaging	message.sent, message.delivered, message.opened, message.clicked, message.failed, message.bounced, message.complained, message.spamreport, message.deferred, message.unsubscribed, message.queued, message.skipped
Inbound replies	sms.message.received, whatsapp.message.received
In-App	in_app.displayed, in_app.clicked, in_app.dismissed, in_app.converted
Journeys	canvas.experiment.entered, canvas.ai_decision.entered
Campaigns	campaign.sent, campaign.abtest.winner_declared, campaign.ai_decision.entered
E-Commerce	Order Completed
Identity	$identify, $alias

Messaging events are channel-agnostic

There are no per-channel event names such as email.delivered or sms.delivered. Every send across Email, SMS, Push, WhatsApp, and Webhook emits the same message.* events, and each payload carries a channel property - one of email, sms, push, whatsapp, or webhook. Subscribe once to message.delivered to receive deliveries from all channels, then filter by the channel property downstream.

Narrow channels in the stream config (optional): When you select a message.* event, a per-channel filter appears in the create/edit form - tick any of email, sms, push, whatsapp, or webhook to stream only those channels instead of filtering downstream. Leave it empty for all channels (the default). This applies only to events that carry a channel property; channel-less events (canvas.*, Order Completed, etc.) are unaffected.

Batching

Events are buffered and delivered in batches. You set:

• Batch size - how many events accumulate before a flush.
• Flush interval - the maximum time to wait before flushing a partial batch.

A batch is delivered when it reaches the batch size or the flush interval elapses, whichever comes first.

How It Works

Events are delivered to streams through a durable fan-out bus that is decoupled from analytics storage. Every stream sees the full event firehose across all channels, so a destination keeps receiving events independently of how analytics is written.

When an event matches a stream's event selection, it is queued and delivered to the destination in the next batch. Each delivery includes the event name, properties, timestamp, and user identifier - and, for messaging events, the originating channel. Warehouses and storage destinations load the batch as rows/files; product-analytics destinations forward each event via their ingestion API; webhooks receive a JSON (or NDJSON) body.

Each stream batches (by batch size and flush interval) and retries on failure. If a destination is unreachable, undelivered events are re-queued and redelivered once it recovers, so an outage does not drop your data.

Delivery guarantees and duplicates

Delivery is at-least-once. In rare cases - a retry after a network failure, or a worker restart mid-batch - the same event may be delivered to your destination more than once.

This is deliberate, and it is the same guarantee every event-streaming platform offers. The alternative, at-most-once, would mean silently dropping an event when a delivery's outcome is unknown; we would rather send it twice than lose it.

Every event carries a stable eventId. It is the same value on every delivery of that event, so you can use it to deduplicate:

Destination	What we do
Amplitude	sent as insert_id - Amplitude deduplicates automatically
Mixpanel	sent as $insert_id - Mixpanel deduplicates automatically
S3	a redelivered batch writes to the same object key, overwriting rather than adding a second file
BigQuery, Snowflake, Redshift	deduplicate on eventId when you load, e.g. ROW_NUMBER() OVER (PARTITION BY eventId)
Webhook	ignore an eventId you have already processed

For warehouses and storage, duplicates are harmless once you deduplicate on load. For webhooks that take action on each event - creating a ticket, charging something - check the eventId before acting.

Monitoring

The streams list shows per-stream KPIs and status:

• Active - delivering normally
• Paused - temporarily stopped (resume any time)
• Error - recent deliveries are failing (check the connection)

You can also see total delivered, total failed, delivery rate, and the last delivery time. Open a stream to view its detail page, edit it, pause/resume, or delete it.