Questions tagged [nxlog]

NXLOG is a universal log collector and forwarder supporting different platforms, log sources and protocols.

NXLog can work in a heterogenous environment collecting event logs from thousands of different sources in many formats. NXLog can accept event logs from TCP, UDP, file-based logs, database and various other sources in different formats such as Syslog, Windows Event Log, DNS debug log etc.

It can perform log rewrite, correlation, alerting, pattern matching, execute scheduled jobs, log rotation and log conversion in other formats such as to JSON. It was designed to be able to fully utilize today's multi-core CPU systems. Its multi-threaded architecture enables input, log processing and output tasks to be executed in parallel. Using a high-performance I/O layer, it is capable of handling thousands of simultaneous client connections and process log volumes above the 100.000 EPS range.

NXLog tries hard to minimize losing log messages. It does not drop any unless instructed to. It can process input sources in a prioritized order, meaning that a higher priority source will be always processed before others. This can further help avoiding UDP message loss (as an example). In case of network congestion or other log transmission problems, NXLog can buffer messages on the disk or in memory. Using loadable modules it supports different input sources and log formats, not only limited to Syslog but Windows Event Log, audit logs or even custom binary application logs.

It is possible to further extend its functionality by using custom loadable modules similarly to the Apache Web server. In addition to the online log processing mode, it can be used to process logs in batch mode in an offline fashion. A powerful configuration language with an Apache style configuration file syntax enables it to rewrite logs, send alerts, execute external scripts or do virtually anything based on any criteria specified using the NXLog configuration language.

Please see the NXLog User Guide for more information.

91 questions
7
votes
1 answer

Azure Diagnostics + Logstash

Does anyone have a good solution for moving Azure Diagnostics, primarily Azure Websites logs, via Logstash to Elasticsearch? If Logstash doesnt support it, is there any other tool that does the job? Without too much customization.
7
votes
3 answers

Why do people ship logs to Logstash with NXLog and not Logstash itself?

I read a blog post about setting up Logstash, Elasticsearch and Kibana, and the author suggested to use NXLog to ship logs from different machines. A typical distributed scenario presented in "The Logstash Book" shows how Logstash can be used in…
Vagif Abilov
  • 9,835
  • 8
  • 55
  • 100
4
votes
1 answer

Multiline Log issue with rsyslog

I have an application server (Ubuntu 14.04) which has tomcat server running on top of it. This application server uses "rsyslog" which is configured to send the logs to a NXlog server (on Ubuntu 14.04). Rsyslog sends all its logs, including the…
Nishant Singh
  • 3,055
  • 11
  • 36
  • 74
4
votes
5 answers

Using redis with logstash

I'm wondering what are the pros and cons of using redis as a broker in an infrastructure? At the moment, all my agents are sending to a central NXLog server which proxies the requests to logstash --> ES. What would I gain by using a redis server in…
mike Dumass
  • 93
  • 1
  • 6
3
votes
1 answer

NXLog: Json input to GELF UDP Output

We have a setup where a program logs to a .Json file, in a format that follows the GELF specification. Currently this is sent to a Graylog2 server using HTTP. This works, but due to the nature of HTTP there's a significant latency, which is an issue…
NT93
  • 316
  • 2
  • 15
3
votes
2 answers

Q: log4net file parse with Regex + NXLog

I am trying to parse a log4net file into json. Here's my sample log4net: 2015-01-27 01:06:18,859 [7] ERROR Web.Cms.Content.Base.Taxonomy.TaxonomyDetectionProvider [(null)] - Get taxonomy Type Failed for Tools 2015-01-27 06:34:31,051 [26] ERROR…
Nataraki
  • 97
  • 1
  • 10
2
votes
1 answer

Filter IIS logs in NXlog+GrayLog2 by status code

I have Module xm_csv Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $cs-host, $sc-status, $sc-substatus, $sc-win32-status,…
2
votes
1 answer

Nxlog - how to process multiline log with multiple headers

I would like to pre-process following log structure with nxlog and send it then to graylog. My custom app log structure: timestamp;field1;field2; ---- Start of good event…
nimmie
  • 21
  • 1
2
votes
2 answers

Graylog Collector Sidecar as nonroot

i would like to achieve an architecture like shown on the left side of this picture(because i want to use NXLog): http://docs.graylog.org/en/2.1/_images/sidecar_overview.png. I have already installed Graylog2 on my RedHat Server and now i'm…
Emre Sahin
  • 21
  • 1
2
votes
2 answers

Using nxlog to ship logs in to logstash from Windows using om_ssl

I have been looking at options to ship logs from Windows, I have already got logstash set up, and I currently ship logs from Linux (CentOS) servers to my ELK stack using the logstash-forwarder and ssl encryption. For compliance reasons encryption is…
Rumbles
  • 1,367
  • 3
  • 16
  • 40
1
vote
0 answers

How to log application message using NXLog

I am trying to log an application information using NXLog. Actually i am developing a RESTful web services in Spring Boot and using windows platform, i need to log messages using NXLog so that my client will take the logs from NXLog and display it.…
1
vote
1 answer

IBM AIX - Apache Kafka

I have a case with IBM AIX, need to send syslogs, logs from file and maybe appslogs to Apache Kafka, is it possible ? I try search something about nxlog, but he don't have output to Kafka for AIX any other solutions ?
1
vote
2 answers

Ship only a percentage of logs to logstash

How can I configure filebeat to only ship a percentage of logs (a sample if you will) to logstash? In my application's log folder the logs are chunked to about 20 megs each. I want filebeat to ship only about 1/300th of that log volume to…
red888
  • 27,709
  • 55
  • 204
  • 392
1
vote
2 answers

NxLogs to Fluentd/SSL?

I have a bunch of linux servers configured to push logs to another fluentd server using the fluent-secure-output plugin. Now I'd like to configure our windows servers using nxlogs. I read a bit on the om_ssl nxlogs module, but I am puzzled about the…
Joe
  • 2,496
  • 1
  • 22
  • 30
1
vote
1 answer

Forward .log file via nxlog to Graylog2 VM server

I'm at the phase of just try with Graylog2. My goal is to view the application logs via graylog2 webpage. The application is hosted on windows server, log file is ABC.log. nxlog service is configured with multiline header to force Exception messages…
YDD9
  • 135
  • 2
  • 10
1
2 3 4 5 6 7