Skip to content

Suggestion: Initial sync #47

@calexandre

Description

@calexandre

Hey Richard,
I would like to suggest some sort of initial sync functionality (optional).

Something like when you create the river via the PUT api, some additional options regarding on how the user would like to perform the initial sync.

This would be a "one time" operation. I dont even know if it is possible...

The main issue is that not everything is on the oplog, especially for really large and stale collections...
So it would be nice to implement a set of options that would allow the user to tell the river to pull all data from mongo (much like a GetAll operation).

Of course we could discuss different strategies for pulling the data, such as:

  1. GetAll (easy, but cumbersome for large collections)
  2. via MongoDump, MongoExport of BsonDump
  3. Others..?

It would be nice to support different import strategies, much like as plugins for this river.

Keep up the good work :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions