Redis Streams

The Redis source reads a Redis stream that your application appends changes to with XADD. Use it when your code already knows what changed and Redis sits next to it.

Entry format

Append one entry per change. Fields:

Field	Meaning
`op`	`upsert` (default; `insert`, `update` and `replace` are aliases), `delete` or `truncate`. Any other value stops the task with `entry … has unknown op`.
`id`	The document key. It is copied into the row as `id` unless the row already has an `id`.
`data`	Optional JSON object with the whole row. Without `data`, every other field except `op` becomes a column, as a string (Redis field values are strings).

XADD items * op upsert id 42 data '{"name":"shoe","price":9.5,"tags":["sale"]}'
XADD items * id 42 name shoe price 9.5          # same row, values as strings
XADD items * op delete id 42
XADD items * op truncate                          # holds the task unless allowTruncate

Prefer data with JSON when you need numbers, booleans or nested values in Meilisearch (for filters and sorting).

Connection string

redis://:password@redis.internal:6379/0
redis://cdc:password@redis.internal:6379/0      # Redis 6+ ACL user
rediss://cdc:password@redis.internal:6380/0     # TLS

The user needs to read the stream: with ACLs, for example ACL SETUSER cdc on >password ~items +ping +type +xrange +xrevrange +xread +xinfo|stream +scan (SCAN is only used by the connector's stream browser). The key must be a stream or not exist yet; any other type fails with key "items" is a hash, not a stream.

Task fields

Field	Meaning
`table`	The stream key, for example `items`.

The primary key is always id. The schema shown in the wizard is inferred from a sample of entries.

How it works

Snapshot. A stream is a log, not a table: the snapshot replays the whole stream from the beginning, keeps the latest image per key (like a compacted log) and writes the rows still alive at the end. That image is held in memory.
Stream. XREAD without a consumer group, 1,000 entries at a time, blocking at most 500 ms per call. The task's groupId is ignored, nothing is acknowledged in Redis, and the service never trims the stream. Trim it yourself (XADD … MAXLEN ~ … or XTRIM) with a margin that covers task downtime.

Positions and resume

Positions are stream entry ids such as 1700000000000-0; 0-0 is the start. On resume the source checks that nothing after the position was trimmed or deleted, and that the stream was not recreated (a position after the last generated id). Otherwise the position is lost and the task resyncs. The precise checks use fields of XINFO STREAM added in Redis 7.