0

I have connected ElasticSearch to SqlServer using Logstash config file,when I am updating or Inserting records in Sqlserver, "ElasticSearch" Index get updated with new / changed records, but the problem is when i am deleting the records from sqlserver ,"ElasticSearch" Index is not get updated with deleted records.. it is still showing deleted record in Index. Is there any config or settings in logstash file for deleted records ? 

Here is my logstash file   

input {  

    jdbc {  

        jdbc_driver_library => "C:\Program Files\Microsoft JDBC Driver 6.4 for SQL Server\sqljdbc_6.4\enu\mssql-jdbc-6.4.0.jre8.jar"  
        jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"  
        jdbc_connection_string => "jdbc:sqlserver://XXXXXXXX;User Id=XXXXXXX;password=XXXXX;databaseName=XXX;"    
        jdbc_user => "XXXXXX"   
        jdbc_password => "XXXXXXXX"     
        statement => "select employeeId,empname,salary,DOB,lastmodifiedDate from                employee where LastModifiedDate>:sql_last_value"     
        use_column_value => true               
        tracking_column => lastmodifieddate     
        tracking_column_type => "timestamp"    
        jdbc_paging_enabled => "true"   
        schedule => "* * * * * *"   
        clean_run => false  }  }      
filter {
}       
output {    
    stdout {  
        codec => rubydebug
    }  
    elasticsearch {  
        hosts => "http://localhost:9200/"  
        index => "emp_index" 
        document_id => "%{employeeId}"     
        document_type => "emp_type"     

    }  
}



  Regards,  
  raj.
Rajneesh
  • 2,009
  • 2
  • 12
  • 6
  • This answer might help: https://stackoverflow.com/questions/34477095/elasticsearch-replication-of-other-system-data/34477639#34477639 – Val Nov 01 '18 at 12:35

1 Answers1

0

There is no out of the box option available in logstash to achieve what you want. Please see Delete old documents from Elastic Search using logstash

ben5556
  • 2,915
  • 2
  • 11
  • 16
  • thanks for response ....I have gone throught the link provided @ben5556 , but it is of no use..in my requirement. We can use some column(ex: status or IsDeleted), i don't want add one more column to this, is there any way -> which can we do from logstash.conf file or from elasticsearch – Rajneesh Nov 19 '18 at 11:41