Is it possible to start a zookeeper server instance in process, say for unit tests?

Question

Calling org.apache.zookeeper.server.quorum.QuorumPeerMain.main() isn't working.

Also, the curator framework ships with a in memory zookeeper implementation for use in unit tests — manku, Mar 15 '12 at 11:27
curator does ship with a testing zookeeper, but it is _not_ in memory. it stores data on disk, in a temp directory. — Martin Serrano, Aug 13 '15 at 20:25

score 61 · Answer 1 · edited Oct 11 '18 at 11:52

61

Netfix opensourced Curator a framework to make use of Zookeeper even more convenient. It has build in test server class. Just add this test dependency to your project descriptor be it maven, gradle or else:

org.apache.curator:curator-framework:4.0.1
org.apache.curator:curator-test:4.0.1

And here are the test essentials.

TestingServer zkTestServer;
CuratorFramework cli;

@Before
public void startZookeeper() throws Exception {
    zkTestServer = new TestingServer(2181);
    cli = CuratorFrameworkFactory.newClient(zkTestServer.getConnectString(), new RetryOneTime(2000));
    cli.start();
}

@After
public void stopZookeeper() throws IOException {
    cli.close();
    zkTestServer.stop();
}

With cli creating any test data is very easy (requires the curator-framework dependency).

cli.create()
   .creatingParentsIfNeeded()
   .forPath("/a1", "testvalue".getBytes("UTF-8"));

edited Oct 11 '18 at 11:52

bric3

40,072
9
91
111

answered Oct 21 '13 at 13:26

gertas

16,869
1
76
58

2

For me it didn't work without cli.start() using Curator version 2.10.0 – Michael Böckling Nov 14 '16 at 15:34
@MichaelBöckling thanks! I also needed to call `cli.start()` first then it started working – Kevin Zhao Jul 10 '17 at 18:07
1

Hi @gertas. Ahem, could Curator be used to monitor an ensemble of zookeepers, and if 1 of the zookeeper instance went down, could it be used to restart it automatically? – jumping_monkey Dec 19 '19 at 07:55

score 48 · Accepted Answer · answered Feb 15 '12 at 13:36

48

To start ZooKeeper you have to execute ZooKeeperServerMain class.

You can use following code to start ZooKeeper in embedded mode.

Properties startupProperties = ...

QuorumPeerConfig quorumConfiguration = new QuorumPeerConfig();
try {
    quorumConfiguration.parseProperties(startupProperties);
} catch(Exception e) {
    throw new RuntimeException(e);
}

zooKeeperServer = new ZooKeeperServerMain();
final ServerConfig configuration = new ServerConfig();
configuration.readFrom(quorumConfiguration);

new Thread() {
    public void run() {
        try {
            zooKeeperServer.runFromConfig(configuration);
        } catch (IOException e) {
            log.error("ZooKeeper Failed", e);
        }
    }
}.start();

answered Feb 15 '12 at 13:36

Mairbek Khadikov

7,939
3
35
51

4

This is the way to go for unit tests. Using QuorumPeerMain is only for Multi Server (Clustered) Zookeeper. – sourcedelica Sep 11 '12 at 23:50
This is useful in the context of BeforeClass, but shutting this down between tests (to clean out the nodes, for example) is non-trivial -- I gave up and switched to a Curator-based solution. (thread.stop/interrupt in After would stop the JUnitRunner) – Rick Moritz May 11 '16 at 12:21
Any chance you know how to set this up in a C++ project? – serup Nov 14 '16 at 12:16
1

serup, Zookeeper is a Java process – Mairbek Khadikov Apr 12 '17 at 18:54
1

Hi, can you give an example of startup properties to set? – Usr Apr 03 '19 at 13:14

score 12 · Answer 3 · answered Feb 15 '12 at 00:36

You can use something like this.

int clientPort = 21818; // none-standard
int numConnections = 5000;
int tickTime = 2000;
String dataDirectory = System.getProperty("java.io.tmpdir");

File dir = new File(dataDirectory, "zookeeper").getAbsoluteFile();

ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTime);
NIOServerCnxn.Factory standaloneServerFactory = new NIOServerCnxn.Factory(new InetSocketAddress(clientPort), numConnections);

standaloneServerFactory.startup(server); // start the server.

And to shut it down just call standaloneServerFactory.shutdown()

score 4 · Answer 4 · edited May 23 '17 at 10:31

4

Building on 1's answer by adding the use of an ephemeral port (shown by zkPort) and updated for latest ZK API:

int tickTime = 2000;
int numConnections = 5000;
String dataDirectory = System.getProperty("java.io.tmpdir");

File dir = new File(dataDirectory, "zookeeper").getAbsoluteFile();

ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTime);
standaloneServerFactory = ServerCnxnFactory.createFactory(0, numConnections);
int zkPort = standaloneServerFactory.getLocalPort();

standaloneServerFactory.startup(server);

edited May 23 '17 at 10:31

Community

1
1

answered May 08 '13 at 17:30

itzg

1,081
2
13
11

You probably can't start zoo keeper server this way as it will block all subsequent code from executing after standaloneServerFactory.startup(server); – thomas.han May 18 '15 at 12:52

score 2 · Answer 5 · answered Apr 09 '13 at 13:18

2

ServerConfig config = new ServerConfig();
config.parse(new String[] {port, dir});
ZooKeeperServerMain zk = new ZooKeeperServerMain();
zk.runFromConfig(config);

answered Apr 09 '13 at 13:18

Niki Tonsky

1,327
11
19

2

You can't run zookeeper server this way as it will block the following code i.e. code after zk.runFromConfig(config); from getting executed. – thomas.han May 18 '15 at 12:45

8xomaster · Answer 6 · 2016-09-15T15:09:16.843

An updated version of GeoffBourne's answer.

    int clientPort = 2199; // not standard
    int numConnections = 5000;
    int tickTime = 2000;
    String dataDirectory = System.getProperty("java.io.tmpdir");

    File dir = new File(dataDirectory, "zookeeper").getAbsoluteFile();

    ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTime);
    ServerCnxnFactory factory = new NIOServerCnxnFactory();
    factory.configure(new InetSocketAddress(clientPort), numConnections);

    factory.startup(server); // start the server.

    // ...shutdown some time later
    factory.shutdown();

Is it possible to start a zookeeper server instance in process, say for unit tests?

6 Answers6

Linked