Open source · change data capture for Meilisearch

Every database change,
searchable in Meilisearch.

meilisearch-cdc copies your tables once, then streams every insert, update and delete into one or many Meilisearch instances in near real time. Sources, destinations, sync tasks, users, alerts and AI access are all managed from the dashboard, with zero configuration to start.

Download GitHub
  • PostgreSQL
  • MySQL
  • MariaDB
  • MongoDB
  • DynamoDB
  • CouchDB
  • SQL Server
  • CockroachDB
  • Kafka (Debezium)
  • Redis Streams
  • Webhook
  • SQL polling

How it works

Many sources in. Many Meilisearch instances out.

A task joins a connector and a table to a destination and an index. The source position is captured before the snapshot, so streaming picks up exactly where the copy ended, with no gap. Checkpoints advance only after Meilisearch confirms the write.

Architecture: sources to Meili CDC to Meilisearch instances Eleven sources (PostgreSQL, MySQL and MariaDB, MongoDB, DynamoDB, CouchDB, SQL Server, CockroachDB, Kafka with Debezium, Redis Streams, webhook push and SQL polling) feed changes into the single Meili CDC binary, which writes them into several Meilisearch instances. PostgreSQLlogical replication MySQL / MariaDBbinlog MongoDBchange streams DynamoDBDynamoDB Streams CouchDB_changes feed SQL ServerCDC change tables CockroachDBchangefeeds KafkaDebezium JSON Redis StreamsXREAD Webhook pushHTTP ingest SQL pollingcursor column Meilisearchprod-eu Meilisearchprod-us Meilisearchstaging Meili CDCsnapshot → streamcheckpoints · DLQ · guardsdashboard · API · MCP Architecture: eleven sources feed Meili CDC, which writes into several Meilisearch instances PostgreSQL MySQL / MariaDB MongoDB DynamoDB CouchDB SQL Server CockroachDB Kafka Redis Streams Webhook push SQL polling Meilisearchprod-eu Meilisearchprod-us Meilisearchstaging Meili CDCsnapshot → stream
Diagram of data flowing from eleven kinds of sources through Meili CDC into three Meilisearch instances.
  1. 1

    Snapshot

    Keyset-paginated copy of the table, chunked and back-pressured, with the replication position pinned first.

  2. 2

    Stream

    Inserts, updates and deletes are coalesced per batch and sent as pipelined, gzipped NDJSON tasks.

  3. 3

    Confirm & checkpoint

    At-least-once with idempotent writes. Even after kill -9, the task replays from its last checkpoint.

Features

Everything between your database and your search box

Built for production sync: safe by default, observable, and operable without touching a config file.

Snapshot then stream, no gap

The source position is captured before the copy and checkpointed only after Meilisearch confirms. Writes are idempotent, so replays are safe.

11 kinds of sources

PostgreSQL, MySQL/MariaDB, MongoDB, DynamoDB, CouchDB, SQL Server, CockroachDB, Kafka (Debezium), Redis Streams, webhook push and SQL polling.

Zero-downtime resync

Rebuild an index into a shadow index and swap it in. Search never drops while the data is reloaded.

Dead letter queue & guards

Rejected documents are isolated by bisection into a DLQ to replay or discard. Mass-delete and truncate guards hold the task; a primary key lock protects indexes.

Full index settings

Granular filterable attributes, embedders, synonyms, typo tolerance, ranking rules, faceting and more, with drift detection and one-click apply.

Schema check & diff repair

Compare source columns with index fields and coverage. Diff source and index document by document, field by field, and repair missing, stale and extra documents.

Many Meilisearch instances

Destinations with health checks, per-instance indexes, search playground, document browser, swap, compact, task monitor, dumps, snapshots and experimental features.

Reusable connectors

Saved source connections with encrypted credentials. Test them and browse tables, collections or topics before creating a task.

Alerts that reach you

Email, Telegram and webhook delivery for failed, held or retrying tasks, lag, DLQ growth, error bursts, stalled snapshots and Meilisearch outages.

Product tour

A dashboard for the whole sync lifecycle

From the first connection to day-two operations. Scroll through the steps, or click any screenshot to enlarge it.

  1. 01 · Overview

    Live overview

    Every task with its state and throughput, plus the health of each Meilisearch instance, updated live over server-sent events.

  2. 02 · Wizard · connector

    Start from a connector

    The task wizard begins with a saved connector or a new one. Credentials are encrypted at rest and the connection is tested before you continue.

  3. 03 · Wizard · source

    Pick what to sync

    Browse the source's tables, collections or topics, then choose the primary key and the sync mode: snapshot and stream, snapshot only or stream only.

  4. 04 · Wizard · mapping

    Map fields, preview documents

    Include, exclude or rename columns, add static fields, format timestamps, build geo points or composite keys, and preview real transformed sample documents.

  5. 05 · Wizard · index

    Shape the index

    Set filterable attributes with granular features, sortable fields, ranking rules, synonyms, typo tolerance or embedders. Changes are previewed before they apply.

  6. 06 · Task detail

    Watch it run

    Throughput and lag charts, logs, dead letters, schema check, diff and index settings in one place. Pause, resume, restart or resync with zero downtime.

  7. 07 · Dead letters

    Poison documents, isolated

    Documents Meilisearch rejects are isolated by bisection into a dead letter queue, so the rest of the batch goes through. Replay or discard them later. Mass-delete and truncate guards hold the task instead of emptying your index.

  8. 08 · Schema & diff

    Prove it is in sync

    The schema checker compares source columns, index fields and coverage. The diff checker compares documents field by field and repairs missing, stale and extra ones.

  9. 09 · Destinations

    Many Meilisearch instances

    Each destination is health-checked in the background: latency, failures, version, documents, synced tasks and the last sync time.

  10. 10 · Search

    Search what you synced

    A search playground with filters, sort and facets, and a document browser, per index and per instance.

  11. 11 · Alerts

    Know before your users do

    Rules for failed, held or retrying tasks, lag, DLQ growth, error bursts, stalled snapshots and Meilisearch outages, delivered by email, Telegram or webhook.

Performance

Fast enough that Meilisearch is the bottleneck

End to end: a row changes in Postgres and the document is confirmed by Meilisearch and searchable. Every number waits for Meilisearch to finish indexing. Measured by the open-source load test in test/bench; run it yourself with make load.

92k

docs/s Postgres snapshot

83k–92k across runs, 200,000 rows

46k

docs/s Postgres streaming

39k–46k, 500-row transactions

~47 MiB

idle memory

process RSS, dashboard included

~⅓

of one CPU core

used by the service at peak rate; Meilisearch indexing is the bottleneck

  • Hand-written, allocation-light NDJSON encoder: 3.8× faster than reflection with 23× less memory.
  • Changes to the same document are coalesced within a batch; batches are pipelined.
  • Gzip, pooled connections, adaptive buffers, back pressure on the source.
  • Exponential backoff with jitter; oversized payloads are split automatically.

Security & operations

Ready for a team, safe for production

Accounts and roles

Email and password (argon2id), server-side sessions, CSRF protection and brute-force rate limiting. Viewer, moderator and admin roles; nobody changes their own role.

Primary administrator protection

The first admin owns the installation. Its role and account cannot be changed or deleted from the dashboard or the API, only by the operator CLI.

Invitations, password reset, audit log

Invitations are emailed when SMTP is configured. Logins, user changes and settings changes are recorded in the audit log.

Secrets encrypted at rest

AES-256-GCM with a key ring and rotation. Each ciphertext is bound to its record and field; secrets are never returned by the API. Every secret has a *_FILE variant for Docker and Kubernetes.

Observability

Prometheus metrics at /metrics, a live dashboard over SSE, structured logs and optional /debug/pprof for admins.

GitOps friendly

Import and export tasks, or declare them in a YAML file applied at startup with ${VAR} expansion.

Operator CLI on the server

Reset passwords, transfer ownership, sign everyone out, turn MCP off, revoke tokens and purge stored data by scope. It talks to the running service over its admin socket, or works on the data directory while the service is stopped.

meilisearch-cdc admin users reset-password --email you@example.com --generate
meilisearch-cdc admin purge sessions alert-events --yes

AI agents · Model Context Protocol

Let your AI agent operate the sync, within limits you set

A built-in MCP server (Streamable HTTP at /mcp) lets agents inspect tasks, check health and act on your pipelines.

  • Admins turn it on or off.
  • Random bearer tokens carry a role and can be rotated and revoked.
  • Tools and resources are allow-listed per role and checked on every call.
  • Mutating calls are written to the audit log.
mcp client config
{
  "mcpServers": {
    "meilisearch-cdc": {
      "type": "http",
      "url": "http://localhost:8080/mcp",
      "headers": { "Authorization": "Bearer <token>" }
    }
  }
}

Get started

Running in a minute

Open http://localhost:8080. The first visit creates the admin account, then an onboarding flow guides you through connecting Meilisearch, connecting a source and creating your first sync.

Download the archive for your platform (see Download), extract it and run the binary. It needs no runtime and no configuration.

tar -xzf meilisearch-cdc_<version>_<os>_<arch>.tar.gz
cd meilisearch-cdc_<version>_<os>_<arch>
./meilisearch-cdc

Configuration, sources, index settings, alerts, MCP and the operator CLI are covered step by step in the documentation.

Read the docs Download

Download

Get meilisearch-cdc

Pure Go, statically linked, no cgo: one archive per platform with the binary, the README and the API and CLI docs. Or run the multi-arch Docker image.

See releases on GitHub

Docker linux/amd64 · linux/arm64

docker pull ghcr.io/ja7ad/meilisearch-cdc:latest

Verify the download

Every release ships a SHA-256 checksums.txt. Put it next to the archive and run:

sha256sum -c checksums.txt --ignore-missing          # Linux
shasum -a 256 -c checksums.txt --ignore-missing      # macOS
Read the docs All releases and notes →

FAQ

Questions, answered

What does each source need?

PostgreSQL uses logical replication (pgoutput) and needs wal_level=logical. MySQL 8 and MariaDB read the binlog with binlog_format=ROW and binlog_row_image=FULL. MongoDB uses change streams on a replica set; DynamoDB uses DynamoDB Streams; CouchDB and Cloudant the _changes feed; SQL Server its native CDC change tables; CockroachDB changefeeds; Redis Streams XREAD. Kafka carries Debezium JSON envelopes, which also covers Oracle, Db2, Cassandra and others. Anything with a database/sql driver can be polled with an incremental cursor column, and the webhook source accepts HTTP pushes.

What delivery guarantees do I get?

At-least-once with idempotent writes. The source position is captured before the snapshot, and checkpoints advance only after Meilisearch confirms a write. After a crash, even kill -9, the task replays from its last checkpoint. Failed tasks are re-applied without overwriting newer data, and if a source position is lost the task resyncs automatically.

Can one table feed several Meilisearch instances?

Yes. Destinations are named Meilisearch instances with their own URL, key, gzip, timeout and health check interval. Each task maps one connector and table to one destination and index, so several tasks can send one table to several instances.

What happens to documents Meilisearch rejects?

They are isolated by bisection into a dead letter queue, so the rest of the batch goes through. From the dashboard you can replay or discard them.

How do I rebuild an index without downtime?

Use zero-downtime resync: the task rebuilds into a shadow index and swaps it in when it is complete. Search keeps serving the old index until the swap.

Do I need a config file?

No. Everything is optional and managed in the dashboard. Environment variables and flags exist for the listen address, the data directory, pinning to one Meilisearch instance (MEILI_URL), the secret key, a bootstrap admin, and a YAML tasks file for GitOps. See the documentation.

How are credentials stored?

Encrypted at rest with AES-256-GCM, bound to their record and field, decrypted only when a connection opens and never returned by the API. The key is generated automatically if you do not provide one, and can be rotated with CDC_SECRET_KEY_PREVIOUS.

I locked myself out. What now?

On the server, run meilisearch-cdc admin users reset-password --email you@example.com --generate. The operator CLI also transfers the primary administrator, signs everyone out and turns MCP off. See docs/cli.md.

What is it built with, and what is the license?

Go 1.27 with the official meilisearch-go SDK, an embedded state store, and a dashboard in plain ES modules with Preact, embedded in the binary. For the license, see the repository.

Make your database searchable today

One command to start, then a dashboard that walks you through the rest.

Download Read the docs Star on GitHub