Query multiple TDB Datasets

Question

Using: jena-fuseki-1.1.0, apache-jena-2.12.0

What I want to achieve and my current state:

I am trying to set up a local jena-fuseki server with the dbpedia Persondata (English and German), Inter-Language Links, Images and Links to Wikipedia Article downloaded from wiki.dbpedia.org/Downloads2014 as .nt files. I want to run the SPAQRL-Query below on them and get the same result as from dbpedia.org/sparql. This Query should give me all living Persons born in Stuttgart,Germany with their name, birthdate, english and german description text, link to wikipedia, link to a picture and a short description.

PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX : <http://dbpedia.org/resource/>
PREFIX dbpedia2: <http://dbpedia.org/property/>
PREFIX dbpedia: <http://dbpedia.org/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX dbo: <http://dbpedia.org/ontology/>

SELECT ?name ?birth ?description_en ?description_de ?wiki ?description ?pic
WHERE {
   ?person dbo:birthPlace :Stuttgart .
   ?person dbo:birthDate ?birth .
   ?person foaf:name ?name .
   OPTIONAL{
      ?person dc:description ?description .
      FILTER (LANG(?description) = 'en') .
   }
   OPTIONAL{
      ?person foaf:isPrimaryTopicOf ?wiki .
   }
   FILTER NOT EXISTS{
      ?person dbo:deathDate ?death .
   }
   OPTIONAL {
      ?person rdfs:comment ?description_en .
      FILTER (LANG(?description_en) = 'en') .
   }
   OPTIONAL {
      ?person rdfs:comment ?description_de .
      FILTER (LANG(?description_de) = 'de') .
   }
   OPTIONAL {
      ?person dbo:thumbnail ?pic
   }
}
ORDER BY ?name

what i get on dbpedia.org/sparql:

first row:

"Abdulsamed Akin"@en 1991-07-17+02:00 "Abdulsamed Akin (born July 17, 1991) is a Turkish-German footballer who plays for Stuttgarter Kickers."@en "Abdulsamed Akin (* 17. Juli 1991 in Stuttgart) ist ein deutscher Fußballspieler türkischer Abstammung."@de http://en.wikipedia.org/wiki/Abdulsamed_Akin "Footballer"@en http://commons.wikimedia.org/wiki/Special:FilePath/Abdulsamed_Akin.jpg?width=300

what i get on my fuseki:

first row:

"Abdulsamed Akin"@en "1991-07-17"^^<http://www.w3.org/2001/XMLSchema#date> [empty] [empty] [empty] [empty] "Footballer"@en [empty]

As you can see the description text and the links to wikipedia and picture are missing at my local query.

The different attributes are in different TDB-Datasets, because of the seperated .nt-Files from DBpedia. The ?name, ?birth and ?description are in the TDB from "Persondata", the ?wiki in "Links to Wikipedia Articel" and the ?pic in "Images".

So i need to Query across the different TDB-Datasources or combine them some how.

What I did so far:

After downloading the .nt-files and using the tdbloader on them, I got five tdb-folders which I put in my local fuseki. Then I put together this two configs, with the goal of combinig the tdb-datasets, so i can make the above query but none of them worked:

First:

@prefix :        <#> .
@prefix fuseki:  <http://jena.apache.org/fuseki#> .
@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

@prefix rdfs:    <http://www.w3.org/2000/01/rdf-schema#> .
@prefix tdb:     <http://jena.hpl.hp.com/2008/tdb#> .
@prefix ja:      <http://jena.hpl.hp.com/2005/11/Assembler#> .

[] rdf:type fuseki:Server ;
   fuseki:services (
     <#service1>
   ) .

# TDB
[] ja:loadClass "com.hp.hpl.jena.tdb.TDB" .
tdb:DatasetTDB  rdfs:subClassOf  ja:RDFDataset .
tdb:GraphTDB    rdfs:subClassOf  ja:Model .

<#service1> rdf:type fuseki:Service ;
    # URI of the dataset -- http://localhost:3030/ds
    fuseki:name                        "ds" ; 
    fuseki:serviceQuery                "sparql" ;   
    fuseki:serviceReadGraphStore       "data" ;
    fuseki:serviceReadGraphStore       "get" ;
    fuseki:dataset                     <#dataset> ;
    .

## ----------------------------------
## dataset for default graph
<#dataset> rdf:type      ja:RDFDataset ;
     ja:defaultGraph <#dbenGraph> ;
     #ja:namedGraph
     #   [ ja:graphName      <http://localhost:3030/dbenGraph> ;
     #     ja:graph          <#dbenGraph> ] ;
     ja:namedGraph
        [ ja:graphName       <http://localhost:3030/dbdeGraph> ;
          ja:graph           <#dbdeGraph> ] ;
     ja:namedGraph
        [ ja:graphName       <http://localhost:3030/dbinterGraph> ;
          ja:graph           <#dbinterGraph> ] ;
     ja:namedGraph
        [ ja:graphName       <http://localhost:3030/dbpicGraph> ;
          ja:graph           <#dbpicGraph> ] ;
     ja:namedGraph
        [ ja:graphName       <http://localhost:3030/dbwikiGraph> ;
          ja:graph           <#dbwikiGraph> ] ;
     .

## ----------------------------------
## the graph's  
<#dbenGraph> rdf:type tdb:GraphTDB ;
    tdb:dataset <#dbpedia_en> ;
    tdb:unionDefaultGraph   true ;
    .

<#dbdeGraph> rdf:type tdb:GraphTDB ;
    tdb:dataset <#dbpedia_de> ;
    tdb:unionDefaultGraph   true ;
    .

<#dbinterGraph> rdf:type tdb:GraphTDB ;
    tdb:dataset <#dbpedia_inter> ;
    tdb:unionDefaultGraph   true ;
    .

<#dbpicGraph> rdf:type tdb:GraphTDB ;
    tdb:dataset <#dbpedia_wiki> ;
    tdb:unionDefaultGraph   true ;
    .

<#dbwikiGraph> rdf:type tdb:GraphTDB ;
    tdb:dataset <#dbpedia_inter> ;
    tdb:unionDefaultGraph   true ;
    .

## DB of persons in Englisch
<#dbpedia_en> rdf:type      tdb:DatasetTDB ;
    tdb:location            "db" ;
    tdb:unionDefaultGraph   true ;
    .

## DB of persons in German
<#dbpedia_de> rdf:type      tdb:DatasetTDB ;
    tdb:location            "dbde" ;
    tdb:unionDefaultGraph   true ;
    .

## DB of persons inter-language-link
<#dbpedia_inter> rdf:type      tdb:DatasetTDB ;
    tdb:location               "dbinter" ;
    tdb:unionDefaultGraph      true ;
    .

## DB of image-links
<#dbpedia_pic> rdf:type      tdb:DatasetTDB ;
    tdb:location "dbpic" ;
    tdb:unionDefaultGraph true ;
    .

## DB of wiki-links
<#dbpedia_wiki> rdf:type      tdb:DatasetTDB ;
    tdb:location "dbwiki" ;
    tdb:unionDefaultGraph true ;
    .

Second:

@prefix :        <#> .
@prefix fuseki:  <http://jena.apache.org/fuseki#> .
@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

@prefix rdfs:    <http://www.w3.org/2000/01/rdf-schema#> .
@prefix tdb:     <http://jena.hpl.hp.com/2008/tdb#> .
@prefix ja:      <http://jena.hpl.hp.com/2005/11/Assembler#> .

[] rdf:type fuseki:Server ;

   fuseki:services (
     <#service1>
   ) .

# TDB
[] ja:loadClass "com.hp.hpl.jena.tdb.TDB" .
tdb:DatasetTDB  rdfs:subClassOf  ja:RDFDataset .
tdb:GraphTDB    rdfs:subClassOf  ja:Model .

## ---------------------------------------------------------------
## Services.

<#service1> rdf:type fuseki:Service ;
    # URI of the dataset -- http://localhost:3030/ds
    fuseki:name                        "ds" ; 
    fuseki:serviceQuery                "sparql" ;   
    fuseki:serviceReadGraphStore       "data" ;
    fuseki:serviceReadGraphStore       "get" ;
    fuseki:dataset                     <#dataset> ;
    .

<#dataset> rdf:type       ja:RDFDataset ;
    ja:defaultGraph       <#model_inf> ;
    .

<#model_inf> a ja:InfModel ;
    ja:baseModel <#dbenGraph> ;
    ja:reasoner [
        ja:reasonerURL <http://jena.hpl.hp.com/2003/OWLMicroFBRuleReasoner>
    ] 
    .

<#dbenGraph> rdf:type tdb:GraphTDB ;
    tdb:dataset <#dbpedia_en> ;
    tdb:unionDefaultGraph   true ;
    .   

## DB of Persons in Englisch
<#dbpedia_en> rdf:type      tdb:DatasetTDB ;
    tdb:location            "db" ;
    tdb:unionDefaultGraph   true ;
    .

## DB of Persons in German
<#dbpedia_de> rdf:type      tdb:DatasetTDB ;
    tdb:location            "dbde" ;
    tdb:unionDefaultGraph   true ;
    .

## DB of Persons inter-language-link
<#dbpedia_inter> rdf:type      tdb:DatasetTDB ;
    tdb:location               "dbinter" ;
    tdb:unionDefaultGraph      true ;
    .

## DB von Resource auf Image
<#dbpedia_pic> rdf:type      tdb:DatasetTDB ;
    tdb:location "dbpic" ;
    tdb:unionDefaultGraph true ;
    .

## DB von Resource auf Wiki
<#dbpedia_wiki> rdf:type      tdb:DatasetTDB ;
    tdb:location "dbwiki" ;
    tdb:unionDefaultGraph true ;
    .

So why is the local Query missing Attributes? Did I configure or query the fuseki wrong? Is their something missing in the query? Is there another way to achieve what I want?

I hope communicated what I need clearly, if not feel free to ask!

As an aside, it's usually better to filter with `langMatches(lang(?x),'en')` than with `lang(?x) = 'en'`, as it handles complex language tags (e.g., 'en-us') correctly. — Joshua Taylor, Oct 01 '14 at 18:46

score 2 · Accepted Answer · answered Oct 01 '14 at 08:52

2

There is absolutely no need to load each individual file into a separate TDB dataset unless you actually want to keep the data separate for some reason.

From your problem description it seems like you want all the data combined together so you would be better just creating a single TDB dataset and querying that. tdbloader will quite happily allow you to load multiple files into a single TDB database.

As for why your current setup doesn't work it is because you only connect your service to a single one of your TDB datasets.

answered Oct 01 '14 at 08:52

RobV

28,022
11
77
119

yes thats right, i don't need them to be seperated, but i did not know that they could be put in a single tdb, from several .nt-files. i am now trying `tdbloader -loc=location file1 file2 file3..` is this the right usage of tdbloader, to achieve that? (it will take i while so, i thought i ask while waiting) – Fabian Oct 01 '14 at 09:03
1

There's potentially plenty of reason to use separate files. Most obviously, different data sets may be updated at different intervals - some stable, some not. The load into TDB can be time consuming, so it makes sense to separate volatile and less-volatile data into separate data sets. A variety of other reasons suggest themselves. Maybe you run separate servers for security, and certain data sets are simply not to be hosted at all on the externally-acessible site. – PaulMurrayCbr Feb 18 '15 at 03:14

score 0 · Answer 2 · answered Feb 18 '15 at 03:08

0

I have had this working in joseki, but fuseki is giving me grief for some reason.

1 - have multiple TDB data sets

:ds1 a tdb:DatasetTDB ; tdb:location "dir1" .
:ds2 a tdb:DatasetTDB ; tdb:location "dir2" .

2 - make multiple graphs

:g1 a tdb:GraphTDB ; tdb:dataset :ds1 ; tdb:graphName foo:the-dataset .
:g2 a tdb:GraphTDB ; tdb:dataset :ds2 ; tdb:graphName bar:the-dataset .

3 - make a union graph

:g a ja:UnionModel
  ; ja:subModel :g1
  ; ja:subModel :g2

4 - in your dataset, use this union graph as the default graph

:dataset a ja:RDFDataset
 ; ja:defaultGraph :g
 ; ja:namedGraph [ ja:graphName foo:the-graph ; ja:graph :g1 ]
 ; ja:namedGraph [ ja:graphName bar:the-graph ; ja:graph :g2 ]
]

Your queries run against the default, but you can also query specifically agains either graph with a sparql clause

WHERE { GRAPH foo:the-graph { ?S ?p ?o } }

answered Feb 18 '15 at 03:08

PaulMurrayCbr

1,167
12
16

Did you get the above example to work in Fuseki? I like that if you don't know the graph names, you can query everything via the default graph, but if you *do* know the graph names, you can use a particular one. I also wonder what happens if you select both the default and a named graph...does it return duplicate results because the default graph includes the named graph? – Barry NL Nov 15 '16 at 12:06
I never got Fuseki up and running. What we have works - although I see it's not running at the moment. Incidentally - to limit casual select queries attempting to suck the entire contents of our database, our default graph is a static set of terms describing what graphs we have and what's in them. That is, our default graph is server metadata. – PaulMurrayCbr Dec 04 '16 at 11:49
Thanks! I managed to get this working in Apache Jena/Fuseki by creating my own [AssemblerBase](https://jena.apache.org/documentation/javadoc/jena/org/apache/jena/assembler/assemblers/AssemblerBase.html). I use the [MultiUnion](https://jena.apache.org/documentation/javadoc/jena/org/apache/jena/graph/compose/MultiUnion.html) to combine the three named graphs into a single default graph. – Barry NL Dec 05 '16 at 13:28

Query multiple TDB Datasets

So i need to Query across the different TDB-Datasources or combine them some how.

2 Answers2