First I would like to say that you are doing an amacing job with this software :)
This error happens when the oplog window is to small. Normally this should be fine, but it can happen that there is a sudden high spike in oplog usage that could result in this error.
How does monstache handle this?
Will it stop working, so all new changes is lost after this error and we need to do a full resync to get it working again?
Or will it find a new position in the stream and continue?
A full resync is very time consuming when having TB's of data. Since the index is deleted and then created and populated with data, there is a long time where the index is not complete.
It would be nice if there was a possibility to resync an index by creating a temp index, filling this with the data and then rename the index when it was complete. Using alias in elastic makes it easy to move the alias between indexes.
First I would like to say that you are doing an amacing job with this software :)
This error happens when the oplog window is to small. Normally this should be fine, but it can happen that there is a sudden high spike in oplog usage that could result in this error.
How does monstache handle this?
Will it stop working, so all new changes is lost after this error and we need to do a full resync to get it working again?
Or will it find a new position in the stream and continue?
A full resync is very time consuming when having TB's of data. Since the index is deleted and then created and populated with data, there is a long time where the index is not complete.
It would be nice if there was a possibility to resync an index by creating a temp index, filling this with the data and then rename the index when it was complete. Using alias in elastic makes it easy to move the alias between indexes.