0

I'm using airbyte to transfer data from MySQL database to Snowflake Database. Airbyte is deploy on EC2 t2.medium (CPU usage is between 60 to 70%) with 16GB storage capacity. When the airbyte connection is use to sync only a few table it's working fine but when I add all them I have this error.

Nested Stack Trace: ** BEGIN NESTED EXCEPTION ** java.io.EOFException MESSAGE: Can not read response from server. Expected to read 1,843 bytes, read 841 bytes before connection was unexpectedly lost

In the same time I have succes logs for all my table, like this :

2023-08-08 16:10:25 [42mnormalization[0m > 184 of 201 START table model <snowflake_schema>.<table_name>..................................................... [RUN]
2023-08-08 16:10:26 [42mnormalization[0m > 180 of 201 OK created table model <snowflake_schema>.<table_name>........................................................ [[32mSUCCESS 1[0m in 1.86s]

Airbyte get the data from MySQL a certain amount of time then stop :

2023-08-08 15:56:31 [44msource[0m > INFO i.a.d.j.s.AdaptiveStreamingQueryConfig(accept):40 Set new fetch size: 138428 rows
2023-08-08 15:56:31 [44msource[0m > INFO i.a.d.j.s.AdaptiveStreamingQueryConfig(accept):40 Set new fetch size: 137718 rows
2023-08-08 15:56:31 [44msource[0m > ERROR i.a.d.j.StreamingJdbcDatabase$1(tryAdvance):107 SQLState: S1000, Message: Error retrieving record: Unexpected Exception: java.io.EOFException message given: Can not read response from server. Expected to read 1,876 bytes, read 735 bytes before connection was unexpectedly lost.
user15915737
  • 165
  • 2
  • 15
  • 1
    all servers have timeouts, so if you can change it increase it, or you need to have lless time consuming queries – nbk Aug 09 '23 at 07:49
  • change the timeouts on airbyte or mysql ? (or both) – user15915737 Aug 09 '23 at 07:56
  • Additional DB information request, please - from MySQL srvr. OS, Version? RAM size, # cores, any SSD or NVME devices on MySQL Host server? Post TEXT data on justpaste.it and share the links. From your SSH login root, Text results of: A) SELECT COUNT(*), sum(data_length), sum(index_length), sum(data_free) FROM information_schema.tables; B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) SHOW FULL PROCESSLIST; E) STATUS; not SHOW STATUS, just STATUS; G) SHOW ENGINE INNODB STATUS; for server workload tuning analysis to provide suggestions. – Wilson Hauck Aug 09 '23 at 19:09

0 Answers0