I'm facing an issue with a JDBC Source Connector in (strictly) incrementing mode. In my dataset which the connector is pulling from, there are consecutive ids starting from id=1.
On the initial deployment of the connector, it ingests only about 1/3-1/2 of the expected records in the dataset. Note that I am sure that in the "initial deployment of the connector" a brand new consumer group is created. The ids of the records getting skipped over are seemingly random. I'm determining the "expected records" by running the source SQL directly against the db containing the dataset. I've encountered the skipping in datasets where ids go up to 10k, in datasets where ids go over 130k, and a few in between. What's weirder is that when I force re-ingestion of the data (i.e. same exact data/dataset) via sending a message to the consumer_offsets topic and re-deploying the connector, I do not encounter the skipping; all expected records make it to the topic.
How can that be?
Other notes:
-There is no ordering or CTE in my SQL