Dissertation Defense - Managing and Consuming Completeness Information for RDF Data Sources

Web data is mostly incomplete
• Wikidata misses the fact that M. Sottile is an actor of Reservoir Dogs
• YAGO covers only 24,770 hasChild facts
• DBpedia contains 1 out of 29 spouses of Glynn Wolfe1
1 Guinness record holder for world’s most married man

Yet many topics are complete
cast member
publication date
Matt Damon
Casey Affleck
2002
. . . . . . . . . . . . . . . . . . . . . . . . . . .

Yet many topics are complete
. . . . . . . . . . . . . . . . . . . . . . . . . . .

MANAGING AND CONSUMING
COMPLETENESS INFORMATION
FOR RDF DATA SOURCES
Fariz Darari
Supervisors: Werner Nutt and Sebastian Rudolph
PhD Thesis Defense in Bolzano on July 14, 2017

Research questions
• Formalize?
How can we formalize statements about RDF completeness?
• Use?
How can completeness statements be leveraged for checking
the completeness and soundness of SPARQL queries?
• Optimize?
How do we efficiently perform completeness reasoning?
• Create?
How can we collect completeness statements?
Formalize Use Optimize Create

Related Work
• Completeness framework for relational databases
(Motro, 1989),(Levy, 1996),(Razniewski and Nutt, 2011)
• Closed predicates for Description Logics
(Lutz et al, 2013),(Ngo et al, 2016)
• Completeness types on the Semantic Web:
Schema, property, population, interlinking
(Fürber and Hepp, 2011),(Zaveri et al, 2016)

(gerry,actor,matt)
(gerry,actor,casey)
G
Give me Gerry’s actors
SELECT * WHERE
{ gerry actor ?a }
?a/matt
?a/casey
variable
(S,P,O)
Background:
RDF graphs and SPARQL queries
BGP Fragment of SPARQL

(gerry,actor,leo)
(titanic,actor,leo)
G’
Completeness statements
Complete for all of Gerry’s actors
Compl({(gerry,actor,?a)})
(gerry,actor,matt)
(gerry,actor,casey)
X
G
The extension pair (G,G’) satisfies the statement
In general:
>=1 triple pattern

Query completeness: Titanic
(gerry,actor,matt)
(gerry,actor,casey)
Give me Titanic’s actors
SELECT * WHERE
{ titanic actor ?a }
(titanic,actor,leo)
?a/leo- Incomplete!
G G’

Query completeness: Gerry
(gerry,actor,matt)
(gerry,actor,casey)
SELECT * WHERE
{ gerry actor ?a }
(titanic,actor,leo)
?a/matt
?a/casey
?a/matt
?a/casey Complete!
G G’

Recap: Research questions
We have formalized completeness statements
and query completeness by means of extension pairs

Checking query completeness:
Data-agnostic
Query Q
Completeness statements C
Boolean isComplete
For every extension pair (G,G’) satisfying C,
it is the case [[Q]]G’ ⊆ [[Q]]G
Completeness
checking

Data-agnostic completeness checking:
Example
SELECT * WHERE
{ gerry actor ?a }
Data Source
...
Query completeness guarantee: Yes!

Give me Gerry’s actors along with their children and spouses
SELECT * WHERE
{ gerry actor ?a . ?a child ?c . ?a spouse ?s }
?a/a
?c/c
?s/s
G
G’
Assume some answer over G’
(gerry,actor,a)
(a,child,c)
(a,spouse,s)
Compl({(gerry,actor,?a),(?a,child,?c)})
Compl({(?a,spouse,?s)})
.....
?a/a
?c/c
?s/s
Complete!
More example

Data Source
Compl({(matt,child,?c)})
Compl({(casey,child,?c)})
Query completeness guarantee: No!
and their children
SELECT * WHERE
{ gerry actor ?a .
?a child ?c }
Limitation

Checking query completeness:
Data-aware
Query Q
Boolean isComplete
Completeness
checking
Graph G
it is the case [[Q]]G’ ⊆ [[Q]]G

Data-aware completeness checking:
Example
...(gerry,actor,matt)
(gerry,actor,casey)
and their children
SELECT * WHERE
{ gerry actor ?a .
?a child ?c }

...(gerry,actor,matt)
(gerry,actor,casey)
and their children
SELECT * WHERE
{ gerry actor ?a .
?a child ?c }
Give me Matt’s children
SELECT * WHERE
{ matt child ?c } +
Give me Casey’s children
SELECT * WHERE
{ casey child ?c }
Example

Query Q
Boolean isComplete
Completeness
checking
Graph G
Divide-and-conquer Algorithm
Take the body P of the query Q
isComplete(P,C,G):
if P = { } // base case
return true
P’ := FindMatch(P,C,G)
if P’ = { }
return false
else
{ u1, u2, …, un } := [[P’]]G
return isComplete(u1P’’,C,G) ∧ isComplete(u2P’’,C,G) ∧ … ∧ isComplete(unP’’,C,G)
P’’ := P P’
Algorithm

Query Q
Boolean isComplete
Completeness
checking
Graph G
* P is the body of Q
** X denotes “fixed”
Complexity

We have characterized the problem
of checking query completeness
for data-agnostic setting and data-aware setting

Give me the actors who were not acting in Gerry
SELECT * WHERE
{ ?a type Actor .
FILTER NOT EXISTS { gerry actor ?a }}
Queries with negation
- --
Answers
Incomplete data may lead to unsound answers!
In general:
Multiple negation parts

Query Q
Boolean isSound
Soundness
checkingGraph G
Checking answer soundness
Answer u
if u ∈ [[Q]]G then u ∈ [[Q]]G’

Compl({(?m,actor,leo)})
(leo,type,Actor)
(matt,type,Actor)
(titanic,actor,leo)
SELECT * WHERE
{ ?a type Actor .
?a/leo : Answer soundness guarantee?
?a/matt: Answer soundness guarantee?
Checking answer soundness:
Example

(leo,type,Actor)
(matt,type,Actor)
(titanic,actor,leo)
SELECT * WHERE
{ ?a type Actor .
?a/leo : Answer soundness guarantee?
Was Leo acting in Gerry?
SELECT * WHERE
{ gerry actor leo }
Query completeness guarantee: Yes!?a/leo : Answer soundness guarantee? Yes!
Checking “Leo” soundness

(leo,type,Actor)
(matt,type,Actor)
(titanic,actor,leo)
SELECT * WHERE
{ ?a type Actor .
Checking “Matt” soundness
?a/matt: Answer soundness guarantee?
Was Matt acting in Gerry?
SELECT * WHERE
{ gerry actor matt }
Query completeness guarantee: No!?a/matt: Answer soundness guarantee? No!

Query Q
Boolean isSound
Soundness
checkingGraph G
Answer u
The answer u is sound wrt C, Q, and G
iff
each positive query generated from
applying u over each negative part of the query Q
is complete wrt C and G
Reduction Theorem
Checking answer soundness
by checking data-aware completeness

Query Q
Boolean isSound
Soundness
checking
Checking pattern soundness
it is the case [[Q]]G ⊆ [[Q]]G’

Data Source
SELECT * WHERE
{ ?a type Actor .
Pattern soundness guarantee?
Checking pattern soundness:
Example

Data Source
SELECT * WHERE
{ ?a type Actor .
Pattern soundness guarantee?
Give me actors acting in Gerry
SELECT * WHERE
{ gerry actor ?a }
Pattern soundness guarantee? Yes!
Checking pattern soundness:
Example

The query Q is sound wrt C
iff
each negative part of the query Q is complete wrt C
Reduction Theorem
Query Q
Boolean isSound
Soundness
checking
Checking pattern soundness
by checking data-agnostic completeness

We have characterized the problem of checking
query completeness and query soundness

Linked Open Quality Data Cloud
Are we ready to reason with big quality data?

Data-agnostic optimization:
Ignoring irrelevant statements
Data Source
Compl({(?m,actor,?a)})
...
Compl({(titanic,actor,?a)})
Compl({(?p,spouse,?s)})
...
Give Gerry’s actors
SELECT * WHERE
{ gerry actor ?a } Constants
gerry,actor
Constants
gerry,actor
actor
. . .
titanic,actor
spouse
. . .
A statement contributes to query completeness only if
its constants are among the query’s

Data-agnostic optimization:
Evaluation
Setup
Queries: BGP queries from LSQ query logs dataset
Completeness statements: For every query as above, take the body P,
and construct Compl(P)
Reasoning implementation: Java with Apache Jena library
Results
Source NQ NC T-QE T-CR Overhead
DBpedia 334,000 331,000 18 ms 0.08 ms 0.44%
SWC 108,000 44,000 36 ms 0.12 ms 0.33%
LGD 22,000 21,000 8 ms 0.05 ms 0.60%
NQ: Number of queries
NC: Number of non-redundant statements
T-QE: Average runtime of query evaluation
T-CR: Average runtime of completeness reasoning

Data-aware reasoning
cannot reuse data-agnostic optimization
Compl({(titanic,actor,?a),(?a,child,?c)})
Compl({(frozen,actor,?a),(?a,child,?c)})
...
Give Matt’s children
SELECT * WHERE
{ matt child ?c }
(gerry,actor,matt)
(gerry,actor,casey)

. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
We capture similar completeness statements by
completeness templates
Complete for all actors of movies:
Pulp Fiction, Gerry, . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Data-aware optimization:
Completeness templates
Complete for all street names of communities:
Abingdon, Shippon, . . . .
Complete for X:
Instance 1, Instance 2, …
OpenStreetMap

Compl({(titanic,actor,?a),(?a,child,?c)})
Compl({(frozen,actor,?a),(?a,child,?c)})
...
Give Matt’s children
SELECT * WHERE
{ matt child ?c }
(gerry,actor,matt)
(gerry,actor,casey)
Completeness templates enable
simultaneous processing of statements
Compl({(?m,actor,?a),(?a,child,?c)})
s.t. {?m/gerry,?m/titanic,?m/frozen,...}
1 ?a/matt
2 ?m/gerry
StatementsTemplate

Data-aware optimization:
Evaluation
Setup
Graph: Direct-statement fragment of Wikidata (110 mio triples)
Queries and statements: Based on 66 BGP queries from Wikidata query examples page
Each BGP query acts as a base to generate test queries and statements
Reasoning implementation: Java with Apache Jena library
Triple store: Apache Jena TDB
Results
Average reasoning time with 1,100 queries and
• 445,000 statements : 15 s
vs.
• 66 templates : 140 ms

We have seen
how to efficiently check query completeness
by indexing completeness statements

Linked Open Quality Data Cloud
How can we create big quality data?

Creating completeness statements
via cardinality extraction
“Wikidata is complete
for all children of Trump”
Around 19% numbers on Wikipedia
express relation cardinalities!
Count: 5
Machine learning task:
Given text, find the correct cardinalities
of certain relations
|(trump,child,*)| = 5

• Trump has five children
• Trump involved in more than 3500 legal actions
• Manafort is one of his campaign managers.
Texts with numbers on Trump’s Wikipedia page:
Label positive when:
1. Number corresponds to the exact triple count
2. Number modifies a noun
Training data generation for relation “child”
Distantly-supervised training data
Training data is labeled automatically based on some rules
Count: 5

Sequence labeling classifier
learn which context words often occur around the positively labeled data
and apply it to new data
Matt has two children
Child cardinality?
Experimental evaluation gives mixed precision results:
• hasPart for creative work : 58%
• hasPart for musical ensemble : 80%
• contains administrative division : 85%
• child : 38%
• spouse : 84%

We have seen how one can extract cardinality information
from text on the Web that can be leveraged
to generate completeness statements

Other research results
• Data can grow and therefore
completeness statements can become outdated…
Solution: Time-aware completeness reasoning
• How can we create completeness statements
in a collaborative way?
Demonstrator systems: CORNER and COOL-WD

Conclusions
 Formalize
 Use
 Optimize
 Create

Future work
? Richer operators for statements
“Complete for all unibz PhD students who don’t speak German
and who are (above 40) or (below 20) of age”
? OWL incorporation
The statement Compl({(?m,type,Centaur)}) can ensure
the completeness of the query “Give me all men who are also horses”
if we know that: Centaur Man ⨅ Horse
? Completeness case studies with RDF data publishers
What do communities of Wikidata, YAGO, and
DBpedia think of completeness (and its use cases)?
≡

Publications
C1. Paramita Mirza, Simon Razniewski, Fariz Darari, Gerhard Weikum:
Cardinal Virtues: Extracting Relation Cardinalities from Text. ACL (Short Papers) 2017.
C2. Fariz Darari, Simon Razniewski, Radityo Eko Prasojo, Werner Nutt:
Enabling Fine-grained RDF Data Completeness Assessment. ICWE 2016.
W1. Radityo Eko Prasojo, Fariz Darari, Simon Razniewski, Werner Nutt: Managing and
Consuming Completeness Information for Wikidata Using COOL-WD. COLD 2016.
P1. Fariz Darari, Radityo Eko Prasojo, Werner Nutt: Expressing No-Value Information in RDF.
ISWC (Posters & Demos) 2015. (Nominated for Best Poster)
P2. Fariz Darari, Simon Razniewski, Werner Nutt: Bridging the Semantic Gap between RDF and
SPARQL Using Completeness Statements. ISWC (Posters & Demos) 2014.
D1. Fariz Darari, Radityo Eko Prasojo, Werner Nutt: CORNER: A Completeness Reasoner
for SPARQL Queries Over RDF Data Sources. ESWC (Posters & Demos) 2014.

Potential Publications
J1. Fariz Darari, Werner Nutt, Simon Razniewski, Sebastian Rudolph:
Ensuring the Completeness and Soundness of SPARQL Queries
Using Completeness Statements about RDF Data Sources.
To be submitted to a journal.
J2. Fariz Darari, Werner Nutt, Giuseppe Pirro,Simon Razniewski:
Completeness Management for RDF Data Sources. Under revision for a journal.
C3. Paramita Mirza, Simon Razniewski, Fariz Darari, Gerhard Weikum:
Extracting Quantifications of Knowledge Base Facts from Text.
To be submitted to a conference.
D2. Erdal Domi, Simon Razniewski, Fariz Darari, Werner Nutt: ProWD: Profiling WikiData.
To be submitted to ISWC 2017 (Posters & Demos).
D3. Fariz Darari, Radityo Eko Prasojo, Simon Razniewski, Werner Nutt:
COOL-WD: A Completeness Tool for Wikidata. Submitted to ISWC 2017 (Posters & Demos).

Algorithm
for data-aware completeness checking
Query Q
Boolean isComplete
Completeness
checking
Graph G
Divide-and-conquer Algorithm
Take the body P of the query Q
isComplete(P,C,G):
if P = { } // base case
return true
P’ := FindMatch(P,C,G)
if P’ = { }
return false
else if
{ u1, u2, …, un } := [[P’]]G
return isComplete(u1P’’,C,G) ∧ isComplete(u2P’’,C,G) ∧ … ∧ isComplete(unP’’,C,G)
P’’ := P P’
[[P’]]G = { }
return true
else

Time-aware completeness reasoning
Movies of Matt
2002
Actors of Gerry
?
2007
Actors of Gerry Actors of Gerry
. . . . .
Movies of Matt Movies of Matt

Give me Gerry’s actors that have won an Oscar
SELECT * WHERE
{ gerry actor ?a . ?a award Oscar }
Query completeness guarantee: Yes, up to 2016!
Data Source
Compl({(gerry,actor,?a)}, )
Compl({(?a,award,Oscar)},2016)

Creating completeness statements:
CORNER

COOL_-WD

COOL_-WD
. . . . . . . . .
. . . . . . . . .

Cardinality extraction:
ACL 2017 results

Cardinality extraction:
Detailed results

CRF: An illustration
Incorporate the context around the observed token,
that is, that comes before and after the token.
Ingredients:
• Feature functions, with the inputs of:
Sentence, position of word in the sentence,
the label of the current word,
the label of the previos word.
The output: real number
• Sum over weighted feature functions
The first sum runs over each feature function j,
the inner sum runs over each position of the sentence.
• The sum is transformed into probabilities
by exponentiating and normalizing.
Basically the sequential version of logistic regression.

Query soundness experiment cases

Dissertation Defense - Managing and Consuming Completeness Information for RDF Data Sources

RDFS-aware completeness
reasoning

Dissertation Defense - Managing and Consuming Completeness Information for RDF Data Sources

More Related Content

What's hot (20)

More from Fariz Darari (20)

Recently uploaded (20)

Dissertation Defense - Managing and Consuming Completeness Information for RDF Data Sources

Editor's Notes