9

I have filebeats 5.x ship logs to logstash.

How do I reset the “file pointer” in filebeat

This is a similar problem to

I cleaned all elasticsearch's data, delete the /var/lib/filebeat/registry. but filebeat is only shipping the new line.

change the registry_file is invalid, the file's offset saved to new file (delete file is the same problem) filebeat.registry_file: registry

Community
  • 1
  • 1
周宏成
  • 175
  • 1
  • 1
  • 8

3 Answers3

13
  1. Stop filbeat service.
  2. Rename the register file - usually found in /var/lib/filebeat/registry
  3. Start filbeat service.

sudo service filbeat stop

mv /var/lib/filebeat/registry /var/lib/filebeat/registry.old

sudo service filbeat start

Community
  • 1
  • 1
Yasir
  • 4,567
  • 7
  • 22
  • 19
9

The Filebeat agent stores all of its state in the registry file. The location of the registry file should be set inside of your configuration file using the filebeat.registry_file configuration option.

I recommend specifying an absolute path in this option so that you know exactly where the file will be located. If you use a relative path then the value is interpreted relative to the ${path.data} directory. On Linux installations, when started as a service or started using the filebeat.sh wrapper, path.data is set to /var/lib/filebeat.

After deleting this registry file, Filebeat will begin reading all files from the beginning (unless you have configured a prospector with tail_files: true.

If you continue to have problems, I recommend looking at the Filebeat log file which will contain a line stating where the registry file is located. For example:

2017/01/18 18:51:31.418587 registrar.go:85: INFO Registry file set to: /var/lib/filebeat/registry

A J
  • 2,508
  • 21
  • 26
  • stop filebeat's server, then delete the '/var/lib/filebeat/registry'. and it's work. it's my problem, thanks for your reply. – 周宏成 Feb 15 '17 at 03:57
5

As already mentioned here, stopping the filebeat service, deleting the registry file(s) and restarting the service is correct.

I just wanted to add for Windows users, if you haven't specified a unique location for the filebeat.registry_file, it will likely default to ${path.data}/registry which is somewhat confusingly the C:\ProgramData\filebeat directory as mentioned by the folks at Elastic.

In my case I had to show hidden files before it was displayed.

Luke
  • 22,826
  • 31
  • 110
  • 193
StephenSolace
  • 459
  • 5
  • 4
  • Just wanted to point to anyone who's like me, it's the `ProgramData` folder and NOT `Program Files` – Luke Jan 21 '21 at 22:15