4

I was given a R&D task on running Cassandra on Docker. I'm working on XUbuntu 14.04.2 64 bit (installed on VirtualBOX in Windows7 32 bit OS).

I was following the steps given in Docker on Ubuntu and Pokle cassandra tutorials and I got a ConnectException when I tried to check the status of a particular node using the nodetool.

Following is the list of steps that I did.

asela@teamlk:~$ docker run -d --name cass1 poklet/cassandra start
7b2ddfb60adf1c270a425c4440fe263e1fe56f5dc1c3205372bee3b8626e0624

asela@teamlk:~$ docker run -d --name cass2 poklet/cassandra start $(/data/cassandra/scripts/ipof.sh cass1)
4f98a194214e603242279226bb94cee20774058e84b6d0171a8d14d648e19016

asela@teamlk:~$ docker run -d --name cass3 poklet/cassandra start $(/data/cassandra/scripts/ipof.sh cass1)
394f9ce918121c0b79ed8282ba95c8f1badd4e3536d4ad3f24c4c21651e50b3c

asela@teamlk:~$ docker ps
CONTAINER ID        IMAGE                     COMMAND              CREATED              STATUS              PORTS                                                                      NAMES
394f9ce91812        poklet/cassandra:latest   "start 172.17.0.1"   6 seconds ago        Up 3 seconds        22/tcp, 7000-7001/tcp, 7199/tcp, 8012/tcp, 9042/tcp, 9160/tcp, 61621/tcp   cass3               
4f98a194214e        poklet/cassandra:latest   "start 172.17.0.1"   31 seconds ago       Up 29 seconds       22/tcp, 7000-7001/tcp, 7199/tcp, 8012/tcp, 9042/tcp, 9160/tcp, 61621/tcp   cass2               
7b2ddfb60adf        poklet/cassandra:latest   "start"              About a minute ago   Up About a minute   22/tcp, 7000-7001/tcp, 7199/tcp, 8012/tcp, 9042/tcp, 9160/tcp, 61621/tcp   cass1               

asela@teamlk:~$ docker run -i -t poklet/cassandra nodetool -h $(/data/cassandra/scripts/ipof.sh cass1) status
nodetool: Failed to connect to '172.17.0.1:7199' - ConnectException: 'Connection refused'.

ifconfig shows me that my Docker is running on 172.17.42.1

Can someone please tell me if I have missed any configurations and/or if I have done something wrong?

halfer
  • 19,824
  • 17
  • 99
  • 186
Asela
  • 117
  • 1
  • 3
  • 12
  • Did you manage to fix the issue? I'm struggling with it now :( – Aides Jul 21 '15 at 21:20
  • @Aides I just gave up on using Cassandra with docker. I just installed Cassandra alone. It works fine. Really sorry I can't be of any help to you. :( – Asela Jul 23 '15 at 06:08
  • 1
    actually it looks like I've figured the problem for myself at least. Here are the details. https://github.com/pokle/cassandra/pull/28 I plan to upload the changes into the repository soon. – Aides Jul 23 '15 at 08:21
  • @Aides: would you consider giving an answer for what you have found? It might be helpful to someone. Thanks! – halfer Oct 03 '15 at 08:46
  • In case it's not clear from the language of the answers below, the problem is that you need to use "exec" not "run" when invoking nodetool. Docker (and docker-compose) will start a new container to "run" a command, but connect to an existing, running container when you use "exec". Since you want to connect on localhost to your running cassandra, you need exec. – Kyle VanderBeek May 10 '19 at 18:05

5 Answers5

6

I am using Fedora 22 but I don't see why it shouldn't work on Ubuntu.

I was having the exact same problem and the following command worked for me:

docker exec cass-1 nodetool status

Notice that cass1 is the name of the docker container for the cassandra node, its used instead of the IP Address.

My output:

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address      Load       Tokens  Owns (effective)  Host ID                               Rack
UN  172.17.0.14  51.6 KB    256     100.0%            3dec79f4-4a6f-46e6-b4d5-14b8f56b6c13  rack1
Anand Rockzz
  • 6,072
  • 5
  • 64
  • 71
negative
  • 108
  • 8
3

I am a little late but here are some insights:

If you start a docker container you need to map ports to the outside world or link containers together.

Port Mapping is done by the -p parameter while starting Docker containers. Documentation can be found: https://docs.docker.com/articles/networking/

Containerlinking is done with the --link parameter and it works as if both linked containers are running on the same host. Documentation: https://docs.docker.com/userguide/dockerlinks/

To make your example work (accessing other containers by their IP) you have to expose ports. In case of cassandra this would be

-p 7000:7000 -p 9160:9160 -p 9042:9042 -p 7199:7199

For Reference this is the command I execute on 2 physical hosts to start a cassandra cluster. Because of the port mappings only one instance per host can be started! (to have the container detached from commandline you need to add the -d switch)

docker run --rm --name cassandra-`hostname \
  -v /data/cassandra/projectname:/var/lib/cassandra/data \
  -e CASSANDRA_CLUSTER_NAME=project-database 
  -e CASSANDRA_BROADCAST_ADDRESS=`hostname` 
  -p 7000:7000 -p 9160:9160 -p 9042:9042 -p 7199:7199 \
  -e CASSANDRA_SEEDS=<ip_of_host1>,<ip_of_host2>
  cassandra:2.2.1
Hotstepper13
  • 173
  • 1
  • 1
  • 8
  • Publishing ports is to make them accessible from outside the host. But in the question, Asela deploys 3 cass node containers on the same host. And an extra container which runs nodetool is also on the same host, and on the same default bridge network, so it should normally be able to reach other container ports, no need to export ports on the host. – Nicomak Jun 11 '16 at 08:03
2

In cassandra-env.sh there is a setting "LOCAL_JMX". By default it was "yes", setting it "no" will allow remote connections to port 7199.

Also, you may disable authentication in the same config file by adding: JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=false"

-- Regarding pokle's docker image, I've made a pull request with the above changes to start.sh(github.com/pokle/cassandra/pull/28) which was merged and now the docker image should work as expected in README.md.

Aides
  • 467
  • 2
  • 5
  • 16
2

Use docker's exec instead of run.

Most of the Cassandra images out there in the world ship with a reasonable, local-only JMX configuration. However, it's sometimes easy to get confused about where commands are running. docker run will always spin up a new container (conceptually a tiny VM) to execute a command. docker exec will connect to an already running container and execute a command within it. Since we're talking about local-only JMX control via nodetool, we need exec to run the command inside the same container where cassandra itself is operating.

$ docker run -d cassandra:3.0
6ebda615efd2f997f95aece8f7e8ae0faafa72145a3225831fe31513557dc2bb

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                                                                              NAMES
6ebda615efd2        cassandra:3.0       "docker-entrypoint.s…"   3 seconds ago       Up 2 seconds        7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp                                                                                        confident_margulis

$ docker run cassandra:3.0 nodetool status
nodetool: Failed to connect to '127.0.0.1:7199' - ConnectException: 'Connection refused (Connection refused)'.

$ docker exec 6ebda615efd2 nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load       Tokens       Owns (effective)  Host ID                               Rack
UN  172.17.0.2  103.06 KB  256          100.0%            72bc1f12-538c-4222-9931-40aab96089b3  rack1

Notice that the exec command requires the identifier of the running container, where run takes the image name.

Kyle VanderBeek
  • 1,047
  • 9
  • 7
0

I added the line - "LOCAL_JMX=no" to my docker-compose.yml file, so I have a section

environment:
  - "MAX_HEAP_SIZE=256M"
  - "HEAP_NEWSIZE=128M"
  - "LOCAL_JMX=no"
John Atchison
  • 21
  • 4