ElasticSearch import data from mysql

Question

Today I have started working on Elastic Search, in my last project I have done the searching in Solr, and I was importing data from mysql and that was pretty simple.

Now I have to do same work in Elastic search, and successfully done the setup I have started working as per "https://github.com/jprante/elasticsearch-jdbc#quick-links" but getting error.

java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [5.0.0]

I am using Elastic Search version 5.0.0 and using the "wget http://xbib.org/repository/org/xbib/elasticsearch/importer/elasticsearch-jdbc/2.3.4.0/elasticsearch-jdbc-2.3.4.0-dist.zip" which is maximum available version.

Please suggest on this.

According to the link provided, in the given matrix there you could only use the importer till the ES version of 2.3.4, where as you're using the latest. — Kulasangar, Nov 21 '16 at 09:56
Simply you wanted to import data from your mysql db to your index in ES? — Kulasangar, Nov 21 '16 at 09:58
Yes, and any data updated, deleted vai any other program to database then my ES data updated like Solr delta query doing. — Deepesh Uniyal, Nov 21 '16 at 10:00
I'm not sure how Solr delta works. But you could upload MySQL data to your index using `logstash` and update it using a scheduler. — Kulasangar, Nov 21 '16 at 10:02
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/128613/discussion-between-kulasangar-and-deepesh-uniyal). — Kulasangar, Nov 21 '16 at 10:15
This answer should help also: http://stackoverflow.com/questions/34477095/elasticsearch-replication-of-other-system-data/34477639#34477639 — Val, Nov 21 '16 at 10:20

score 2 · Answer 1 · answered Nov 21 '16 at 10:11

You can use the jdbc plugin in your logstash config in order to upload data to your index and then make sure you could keep updating it by adding a scheduler according to your need.

Your logstash config could look like this:

input {
  jdbc {
    jdbc_driver_library => "/mysql-connector-java-5.1.39-bin.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/database_name"
    jdbc_user => "root"
    jdbc_password => "password"
    schedule => "* * * * *"
    statement => "select * from table1"
    type => "table1"
  }
}
output {
    elasticsearch {
        index => "testdb"
        document_type => "%{type}"   # <- use the type from each input
        hosts => "localhost:9200"
    }
}

ElasticSearch import data from mysql

1 Answers1