0

I have a PostgreSQL database. There are 72 tables in the database. The total number of columns is greater than 1600. And I would like to index all this data into Elasticsearch and be able to run bool queries over it.

What is the best way to denormalize my DB data? What tool could be used to join and index all data from the PostgreSQL?

I find it problematic to join data on the PostgreSQL side due to the limits: columns per table and raw size. Also I tried Pandas, but there is a performance problem: Join all PostgreSQL tables and make a Python dictionary

Community
  • 1
  • 1
srgbnd
  • 5,404
  • 9
  • 44
  • 80

1 Answers1

0

You can use the logstash jdbc input and elasticsearch output to synchronize the data: https://www.elastic.co/blog/logstash-jdbc-input-plugin

Jesusaur
  • 594
  • 3
  • 22
  • Do you mean to join all tables (over 250 columns in total) on the PostgreSQL side? If yes, there is a limit for a total number of columns per table in PostgreSQL, and in my case, I reach it. – srgbnd Sep 14 '16 at 22:34