Calling a Giraph job from a simple java program

Question

I am new to Giraph and Hadoop Yarn. Following Giraph's quick start leads me to run the example job from jar build from source from command line.

I want to run the job from simple java program. The question is inspired from previous similar MapReduce job question. Looking for similar answers with java's dependencies which would be needed for that.

I have yarn setup locally - needs have a way to feed job to that from java program.

It is evident: https://giraph.apache.org/apidocs/org/apache/giraph/job/GiraphJob.html that there must have a way for this - but I am finding it hard to find examples for it with Yarn.

Answer 1

Found a way to do this from GiraphRunner's source code:

@Test
public void testPageRank() throws IOException, ClassNotFoundException, InterruptedException {

    GiraphConfiguration giraphConf = new GiraphConfiguration(getConf());
    giraphConf.setWorkerConfiguration(1,1,100);
    GiraphConstants.SPLIT_MASTER_WORKER.set(giraphConf, false);

    giraphConf.setVertexInputFormatClass(JsonLongDoubleFloatDoubleVertexInputFormat.class);
    GiraphFileInputFormat.setVertexInputPath(giraphConf,
                                             new Path("/input/tiny-graph.txt"));
    giraphConf.setVertexOutputFormatClass(IdWithValueTextOutputFormat.class);

    giraphConf.setComputationClass(PageRankComputation.class);

    GiraphJob giraphJob = new GiraphJob(giraphConf, "page-rank");       

    FileOutputFormat.setOutputPath(giraphJob.getInternalJob(),
                                   new Path("/output/page-rank2"));
    giraphJob.run(true);
}

private Configuration getConf() {
    Configuration conf = new Configuration();
    conf.set("fs.defaultFS", "hdfs://localhost:9000");

    conf.set("yarn.resourcemanager.address", "localhost:8032");
    conf.set("yarn.resourcemanager.hostname", "localhost");

    // framework is now "yarn", should be defined like this in mapred-site.xm
    conf.set("mapreduce.framework.name", "yarn");
    return conf;
}