Fuseki GC overhead limit exceeded during data import

Question

I'm trying to import LinkedMDB (6.1m triples) into my local version of jena-fuseki at startup:

/path/to/fuseki-server --file=/path/to/linkedmdb.nt /ds

and that runs for a minute, then dies with the following error:

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
    at com.hp.hpl.jena.graph.Node$3.construct(Node.java:318)
    at com.hp.hpl.jena.graph.Node.create(Node.java:344)
    at com.hp.hpl.jena.graph.NodeFactory.createURI(NodeFactory.java:48)
    at org.apache.jena.riot.system.RiotLib.createIRIorBNode(RiotLib.java:80)
    at org.apache.jena.riot.system.ParserProfileBase.createURI(ParserProfileBase.java:107)
    at org.apache.jena.riot.system.ParserProfileBase.create(ParserProfileBase.java:156)
    at org.apache.jena.riot.lang.LangNTriples.tokenAsNode(LangNTriples.java:97)
    at org.apache.jena.riot.lang.LangNTriples.parseOne(LangNTriples.java:90)
    at org.apache.jena.riot.lang.LangNTriples.runParser(LangNTriples.java:54)
    at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:42)
    at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTFactoryImpl$1.read(RDFParserRegistry.java:142)
    at org.apache.jena.riot.RDFDataMgr.process(RDFDataMgr.java:818)
    at org.apache.jena.riot.RDFDataMgr.parse(RDFDataMgr.java:679)
    at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:211)
    at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:104)
    at org.apache.jena.fuseki.FusekiCmd.processModulesAndArgs(FusekiCmd.java:251)
    at arq.cmdline.CmdArgModule.process(CmdArgModule.java:51)
    at arq.cmdline.CmdMain.mainMethod(CmdMain.java:100)
    at arq.cmdline.CmdMain.mainRun(CmdMain.java:63)
    at arq.cmdline.CmdMain.mainRun(CmdMain.java:50)
    at org.apache.jena.fuseki.FusekiCmd.main(FusekiCmd.java:141)

Is there a way that I can bump up the memory limit or import the data in less intensive way?

For comparison's sake, when I used a 1million triple source file, it imports in less than 10 seconds.

Yazan Jaber · Accepted Answer · 2014-01-18T09:44:34.087

6

Increase heap memory, java -Xmx2048M -jar fuseki-sys.jar ......

open fuseki-server with an editor you'll find the line JVM_ARGS=${JVM_ARGS:--Xmx1200M} modify it to JVM_ARGS=${JVM_ARGS:--Xmx2048M}

edited Jan 18 '14 at 09:44

answered Jan 17 '14 at 23:17

Yazan Jaber

2,068
25
36

thanks for the response. when running that i get `Unable to access jarfile fuseki-sys.jar` -- do I need to be in the same directory as that file when I run this? if so, where is that file? – Kristian Jan 18 '14 at 00:09
This does not work for me - I am still getting the *GC overhead limit exceeded* message with that setting when importing [this file](http://queens.db.toronto.edu/~oktie/linkedmdb/linkedmdb-latest-dump.tar.gz). – O. R. Mapper Feb 06 '14 at 17:04

score 2 · Answer 2 · answered Jan 18 '14 at 10:56

2

Set JVM_ARGS when using the fuseki-server script.

Also note that --file=... is reading the file into memory. Maybe this is too big for handling that way. If so, load into TDB and use a TDB database with Fuseki.

answered Jan 18 '14 at 10:56

AndyS

16,345
17
21

Unfortunately, this also won't work. I tried to start fuseki with `--loc=DB` to force it to use the TDB and imported a large data set. Unfortunately it still runs into an java.lang.OutOfMemoryError. – vanthome Mar 23 '14 at 15:35

Fuseki GC overhead limit exceeded during data import

2 Answers2

Linked