securely connect to open distro elastic from packetbeat

Question

I can connect to elastic server using curl as shown below.

# curl --user "root:xxxxx"  https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com/_aliases

I get the expected response and it means the credentials are correct.

{".kibana_3506402_root":{"aliases":{}},".opendistro_security":{"aliases":{}},".kibana_1":{"aliases":{".kibana":{}}}}

But the same credentials are not working in packetbeat

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:80"]

  # Optional protocol and basic auth credentials.
  protocol: "https"
  username: "root"
  password: "xxxxx"

As per logs, it is trying to connect on port 9200 even if port 80 is mentioned in the config file.

# tail /var/log/packetbeat/packetbeat

2020-03-01T13:56:03.234Z        ERROR   pipeline/output.go:100  Failed to connect to backoff(elasticsearch(https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:9200)): Get https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:9200: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2020-03-01T13:56:03.234Z        INFO    pipeline/output.go:93   Attempting to reconnect to backoff(elasticsearch(https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:9200)) with 10 reconnect attempt(s)

How do I connect to AWS elastic using open distro through packetbeat? Similar configuration works without any problem if I use elastic cloud hosted by AWS.

In the curl you hit port 443 which is the default for https schema. Maybe that's the case. — Yuriy Tseretyan, Mar 06 '20 at 15:25
Did you find a cause for this? Could it be related to License? — Haran, Mar 22 '20 at 16:57

score 1 · Accepted Answer · answered Apr 10 '20 at 06:04

There are 2 important points to remember:

1) You need "oss" (Open Source Version) of packetbeat for this to work.

2) Connect to 443 port

Detailed instructions can be found here...

http://oksoft.blogspot.com/2020/03/packetbeat-to-elastic-server-hosted-by.html

securely connect to open distro elastic from packetbeat

1 Answers1