MongoDB

The MongoDB source watches one collection with a change stream and copies it with a collection scan. Each document's _id becomes the Meilisearch id.

Prerequisites

Change streams need a replica set or a sharded cluster. A single-node replica set is enough:

mongod --replSet rs0 --bind_ip_all
mongosh --eval 'rs.initiate()'

On a standalone server the task fails with: mongodb: change streams need a replica set or sharded cluster; start mongod with --replSet and run rs.initiate() (a single node replica set is fine). MongoDB Atlas clusters are replica sets already.

A dedicated user

Reading a collection and opening a change stream on it needs the find and changeStream actions on that collection; the built-in read role on the database grants both:

use admin
db.createUser({
  user: "cdc",
  pwd: "change-me",
  roles: [{ role: "read", db: "shop" }]
})

Connection string

mongodb://cdc:change-me@mongo1:27017,mongo2:27017/?replicaSet=rs0&authSource=admin
mongodb+srv://cdc:change-me@cluster0.example.mongodb.net/

The database is required: set it on the connector (the default database) or in the task, which overrides it. It can also be given in the connection string's path.

Task fields

How changes are applied

• Schema. MongoDB has no fixed schema, so the field list shown in the wizard and the schema check is inferred from a $sample of 100 documents. The primary key is always _id.
• Ids. An ObjectId becomes its 24-character hex string. Other _id types are converted to a string that fits Meilisearch's id alphabet (see document ids). By default the id is written to the field id and _id is not copied; enable Keep key columns to keep it, or set the primary key field to another name.
• Values. Dates become times (formatted by mapping.timeFormat), Decimal128 becomes a number, embedded documents and arrays stay nested.
• Inserts, updates and replaces are written with the full current document (the change stream uses fullDocument: updateLookup). A document deleted before the lookup is skipped; its delete follows.
• Deletes delete by _id.
• Collection drop, rename or database drop are treated as a truncate: the task is held unless allowTruncate is on. An invalidate event (after a drop or rename) also stops the stream with change stream on … invalidated (collection dropped or renamed); recreate the task to resync.

Positions and resume

Positions are change stream resume tokens in extended JSON, for example {"_data":"8263F1…"}. The task resumes right after the stored token. If the token has fallen off the oplog (ChangeStreamHistoryLost, error 286) or is invalid, the position is lost and the task resyncs automatically. Size the oplog to cover the longest time a task may be stopped.

Troubleshooting