Skip to content

Commit 75f6e8f

Browse files
fabio-rovaiclaude
andcommitted
The derived-datatype guard names the SPARQL evaluator as its cause, now that the store keeps the datatype (#335)
Cause. `datatype_is_indistinguishable_in_store` and every skip it drives (the direct sh:datatype constraint on node and property shapes, the sh:or, sh:not and qualified members, the sh:node compiler) said the STORE does not preserve the twelve integer-derived XSD types and xsd:dateTimeStamp. Since the previous commit it does. Re-derived by measurement on the new pin (every XSD datatype inserted, DATATYPE() and STR() read back, and the term read back by projection): the store keeps all of them; DATATYPE() in the unpatched evaluator still answers xsd:integer for exactly the twelve and xsd:dateTime for xsd:dateTimeStamp, for stored and query-text literals. Every check here is a DATATYPE() test, so the guard is still needed, for exactly the same thirteen IRIs, and no skip can be removed. Change. Wording only, no verdict or set changes: - the function's doc states the evaluator cause, the #335 history, and the measured set; - the skip reason (shared `unpreserved_datatype_skip` and the node-level one) now reads "the SPARQL evaluator reads this datatype IRI as the type it derives from, so DATATYPE() cannot tell a value carrying it from one carrying that type; the constraint was not evaluated"; - the in-code comments at each guard, the indistinguishable test file's module doc, adversarial_shacl_test's control (whose `contains` check moves from "does not preserve" to "DATATYPE() cannot tell"), and decision 0018's control paragraph say the same. Verification. tests/datatype_preservation_test.rs (previous commit) pins the thirteen in both directions against DATATYPE(); the indistinguishable, adversarial, or/not/qualified/node tests all pass unchanged in verdict. Full cargo test on this tree: 2075 passed, 0 failed, 2 ignored. cargo clippy --all-features --all-targets -D warnings: clean. W3C: gated 61, full tuple 53, report match 51 of 56; baseline byte-identical. Python suite (pyoxigraph 0.5.13, pyshacl 0.40.1, rdflib 7.6.0) against this binary, with tests/test_align.py and test_align_failed_insertion.py left out because numpy crashes on import on this machine: 253 passed, 17 skipped (missing optional extras, and the three-way file's release-binary lookup); then the three-way horn differential with the binary linked where it looks: 16 passed, 1 opt-in skip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 4f08696 commit 75f6e8f

5 files changed

Lines changed: 58 additions & 43 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,12 @@ All notable changes to Open Ontologies are documented here.
176176
of a source that says `"1"` is `ungrounded_in_source`, where it used to come
177177
back `preserved_asserted`. One value, two terms, which is what the checker
178178
has always done.
179+
- **A `sh:datatype` skipped for an integer-derived type or `xsd:dateTimeStamp`
180+
names the real cause.** The skip reason said the store does not preserve the
181+
datatype; since #335 it does, and what cannot report it is `DATATYPE()` in the
182+
SPARQL evaluator. The reason now says so (`... so DATATYPE() cannot tell a
183+
value carrying it from one carrying that type ...`). The set of datatypes and
184+
every verdict are unchanged, by measurement on the new pin.
179185
- **A datatype the store cannot tell apart is refused inside `sh:or`,
180186
`sh:not` and `sh:qualifiedValueShape`, as it already was on its own.** The
181187
store widens xsd:int, xsd:nonNegativeInteger and the other integer-derived

‎docs/decisions/0018-conformance-is-not-truth.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -204,10 +204,11 @@ line between the buckets was drawn.
204204

205205
## The control, and the bound it actually has
206206

207-
`"300"^^xsd:byte` is outside the byte range and is the W3C suite's own ill-formed case. Oxigraph does
208-
not preserve `xsd:byte`, so the literal arrives as `"300"^^xsd:integer`,
209-
`datatype_is_indistinguishable_in_store` routes the constraint to `skipped_constraints`, and the
210-
verdict is `null`. An attack surface where every probe succeeds is not a measurement, so this one is
207+
`"300"^^xsd:byte` is outside the byte range and is the W3C suite's own ill-formed case. Oxigraph
208+
0.5.9 did not preserve `xsd:byte`, so the literal arrived as `"300"^^xsd:integer`. Since #335 the
209+
store keeps it as written, but the SPARQL evaluator still reads `xsd:byte` as `xsd:integer`, so
210+
`DATATYPE()` cannot judge it, `datatype_is_indistinguishable_in_store` routes the constraint to
211+
`skipped_constraints`, and the verdict is `null`. An attack surface where every probe succeeds is not a measurement, so this one is
211212
kept beside the failures rather than in a passing-cases file.
212213

213214
The bound is one-directional and the test now says so. A consumer reading for a PASS is protected,

‎src/shacl.rs‎

Lines changed: 32 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -556,7 +556,7 @@ impl ShaclValidator {
556556
"shape": strip_angle_brackets(&shape_iri),
557557
"constraint": "sh:datatype",
558558
"reason": format!(
559-
"the store does not preserve <{dt}>; a conforming value and a widened one are the same term, so this cannot be decided"
559+
"the SPARQL evaluator reads <{dt}> as the type it derives from, so DATATYPE() cannot tell a conforming value from one of that type; this cannot be decided"
560560
),
561561
}));
562562
continue;
@@ -796,7 +796,7 @@ impl ShaclValidator {
796796
// and the run came back CLEAN. Same defect in sh:in and sh:not.
797797
let mut or_alternatives: HashMap<String, Vec<String>> = HashMap::new();
798798
let mut or_unsupported: HashSet<String> = HashSet::new();
799-
// Property shape -> a member datatype the store does not preserve.
799+
// Property shape -> a member datatype DATATYPE() cannot report.
800800
// The direct sh:datatype path refuses that question; a member
801801
// answering it reported every widened value as matching nothing.
802802
let mut or_unpreserved: HashMap<String, String> = HashMap::new();
@@ -1507,7 +1507,7 @@ impl ShaclValidator {
15071507
// sh:datatype
15081508
if let Some(dt_str) = prop.get("datatype") {
15091509
let dt = strip_angle_brackets(dt_str);
1510-
// The store cannot tell these apart from the type they are
1510+
// DATATYPE() cannot tell these apart from the type they are
15111511
// derived from, so the constraint is not decidable here and
15121512
// must not be answered. Asking anyway produced a violation
15131513
// for every value that satisfied the shape: nine of them in
@@ -2843,10 +2843,10 @@ fn compile_node_shape(
28432843
)),
28442844
"datatype" => {
28452845
// Refused for the same reason as on a property inside the
2846-
// shape, below: the store cannot tell the datatype from the
2846+
// shape, below: DATATYPE() cannot tell the datatype from the
28472847
// one it derives from. Only the property form was guarded, so
28482848
// `sh:node` to a shape declaring `sh:datatype xsd:int` itself
2849-
// reported every value the store had widened to xsd:integer.
2849+
// reported every value DATATYPE() read as xsd:integer.
28502850
let dt = strip_angle_brackets(obj);
28512851
if datatype_is_indistinguishable_in_store(&dt) {
28522852
return None;
@@ -2932,8 +2932,8 @@ fn compile_node_shape(
29322932
}
29332933
if let Some(dt) = row.get("datatype") {
29342934
let dt = strip_angle_brackets(dt);
2935-
// The store cannot tell these apart from the type they derive from,
2936-
// so a nested shape resting on one is not decidable either.
2935+
// DATATYPE() cannot tell these apart from the type they derive
2936+
// from, so a nested shape resting on one is not decidable either.
29372937
if datatype_is_indistinguishable_in_store(&dt) {
29382938
return None;
29392939
}
@@ -2995,21 +2995,25 @@ fn node_kind_test(node_kind: &str, var: &str) -> Option<String> {
29952995
})
29962996
}
29972997

2998-
/// Datatype IRIs the storage layer does not preserve.
2998+
/// Datatype IRIs that `DATATYPE()` cannot report, so no SPARQL test here can
2999+
/// decide them.
29993000
///
3000-
/// oxigraph 0.5 encodes a literal by value, and in `numeric_encoder.rs` twelve
3001-
/// XSD integer-derived datatype IRIs all route to `parse_integer_str`, which
3002-
/// yields `EncodedTerm::IntegerLiteral` — one variant, carrying no datatype IRI.
3003-
/// Reading back can only reconstruct `xsd:integer`. `xsd:dateTimeStamp` collapses
3004-
/// into `xsd:dateTime` the same way. The Turtle parser is correct; the loss is at
3005-
/// storage. That it is a defect rather than a deliberate simplification is settled
3006-
/// by `xsd:yearMonthDuration` and `xsd:dayTimeDuration`, equally derived, which
3007-
/// have their own encodings and survive intact.
3001+
/// The twelve XSD integer-derived types and `xsd:dateTimeStamp`. Since #335 the
3002+
/// store keeps them: the pinned oxigraph (oxigraph/oxigraph#2010) gives
3003+
/// `"5"^^xsd:int` back as written, where 0.5.9 encoded it as
3004+
/// `EncodedTerm::IntegerLiteral` and gave back `"5"^^xsd:integer`. The SPARQL
3005+
/// evaluator, `spareval`, is unpatched and still maps each of these IRIs to its
3006+
/// base type's value before any function sees it, so inside a query
3007+
/// `DATATYPE("5"^^xsd:int)` is `xsd:integer` and `DATATYPE` of an
3008+
/// `xsd:dateTimeStamp` is `xsd:dateTime`, whether the literal came from the
3009+
/// store or the query text. Measured on the new pin over every XSD datatype:
3010+
/// exactly these thirteen; `tests/datatype_preservation_test.rs` pins the set
3011+
/// in both directions. `xsd:yearMonthDuration` and `xsd:dayTimeDuration`,
3012+
/// equally derived, have their own encodings and answer correctly.
30083013
///
3009-
/// A `sh:datatype` constraint naming one of these cannot be decided against the
3010-
/// store: a conforming value and a widened one are the same term by the time the
3011-
/// query runs. Answering anyway reports a violation for every value that in fact
3012-
/// satisfies the shape.
3014+
/// A `sh:datatype` test naming one of these cannot be decided by `DATATYPE()`:
3015+
/// a conforming value and one of the base type answer alike. Answering anyway
3016+
/// reports a violation for every value that in fact satisfies the shape.
30133017
fn datatype_is_indistinguishable_in_store(datatype: &str) -> bool {
30143018
const XSD: &str = "http://www.w3.org/2001/XMLSchema#";
30153019
let Some(local) = datatype.strip_prefix(XSD) else {
@@ -3046,9 +3050,9 @@ fn unpreserved_datatype_skip(shape: &str, constraint: &str, path: &str, datatype
30463050
"constraint": constraint,
30473051
"path": path,
30483052
"datatype": datatype,
3049-
"reason": "the store does not preserve this datatype IRI, so a value \
3050-
carrying it is indistinguishable from one carrying the type \
3051-
it derives from; the constraint was not evaluated",
3053+
"reason": "the SPARQL evaluator reads this datatype IRI as the type it \
3054+
derives from, so DATATYPE() cannot tell a value carrying it \
3055+
from one carrying that type; the constraint was not evaluated",
30523056
})
30533057
}
30543058

@@ -3487,9 +3491,11 @@ fn count_bound(
34873491
///
34883492
/// SHACL 4.2.1, 4.2.2, 4.4.1, 4.4.2 and 4.7.3 require these values to be
34893493
/// literals of datatype `xsd:integer`. The datatype is checked, not only the
3490-
/// lexical form, because the store does not keep the lexical form of a
3491-
/// decimal: Turtle `1.0` comes back as `"1"^^xsd:decimal`, which a lexical
3492-
/// parse reads as 1 (measured; the verified evaluator's `readNat` in
3494+
/// lexical form, because a lexical parse alone reads `"1"^^xsd:decimal` as 1.
3495+
/// Until #335 the store made that common: Turtle `1.0` came back as
3496+
/// `"1"^^xsd:decimal`. The pinned store now keeps `"1.0"`, which no longer
3497+
/// parses, but the spelling `"1"^^xsd:decimal` is still a decimal and still
3498+
/// refused here (the verified evaluator's `readNat` in
34933499
/// `lean/Shacl/Compile.lean` reads only the lexical form and does evaluate it).
34943500
/// A negative value is refused rather than read by the letter of the spec, where
34953501
/// `sh:minCount -1` holds of everything and `sh:maxCount -1` of nothing: no

‎tests/adversarial_shacl_test.rs‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -698,9 +698,10 @@ fn owl_same_as_merges_a_cardinality_that_nothing_here_reports() {
698698
// ═══════════════════════════════════════════════════════════════════════════
699699

700700
/// `"300"^^xsd:byte` is outside the byte range and is the W3C suite's own
701-
/// ill-formed case. Oxigraph does not preserve `xsd:byte`, so the literal
702-
/// arrives as `"300"^^xsd:integer`, `datatype_is_indistinguishable_in_store`
703-
/// routes the constraint to `skipped_constraints`, and the verdict is null.
701+
/// ill-formed case. The store keeps it as written since #335, but the SPARQL
702+
/// evaluator reads `xsd:byte` as `xsd:integer`, so `DATATYPE()` cannot judge
703+
/// it, `datatype_is_indistinguishable_in_store` routes the constraint to
704+
/// `skipped_constraints`, and the verdict is null.
704705
///
705706
/// A probe the existing defence stops, kept here rather than in a
706707
/// passing-cases file so that this set is a measurement and not a list of
@@ -733,7 +734,7 @@ fn a_datatype_the_store_cannot_preserve_yields_no_verdict_rather_than_a_pass() {
733734
gate["skipped_constraints"][0]["reason"]
734735
.as_str()
735736
.unwrap()
736-
.contains("does not preserve"),
737+
.contains("DATATYPE() cannot tell"),
737738
"{gate}"
738739
);
739740

‎tests/shacl_datatype_indistinguishable_test.rs‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,15 @@
1-
//! `sh:datatype` naming a type the store does not preserve.
1+
//! `sh:datatype` naming a type `DATATYPE()` cannot report.
22
//!
3-
//! oxigraph 0.5 encodes twelve XSD integer-derived datatypes, and
4-
//! `xsd:dateTimeStamp`, into the encoding of the type they derive from, dropping
5-
//! the datatype IRI. See `datatype_is_indistinguishable_in_store`, and
6-
//! `datatype_preservation_test` for the isolation showing the parser is correct
7-
//! and the store is not.
3+
//! The twelve XSD integer-derived datatypes and `xsd:dateTimeStamp`. Oxigraph
4+
//! 0.5.9's store dropped their IRIs; the pinned store keeps them since #335, but
5+
//! the SPARQL evaluator still reads each as the type it derives from, so
6+
//! `DATATYPE("0"^^xsd:nonNegativeInteger)` is `xsd:integer`. See
7+
//! `datatype_is_indistinguishable_in_store`, and `datatype_preservation_test`
8+
//! for the isolation of parser, store and evaluator.
89
//!
910
//! The consequence for validation is that a `sh:datatype xsd:nonNegativeInteger`
10-
//! constraint cannot be decided: by the time the query runs, a conforming literal
11-
//! and a widened one are the same term. Asking anyway reported a violation
11+
//! constraint cannot be decided: by the time the test runs, a conforming literal
12+
//! and one of the base type answer alike. Asking anyway reported a violation
1213
//! against every value that satisfied the shape — nine of them in
1314
//! `jsonld-escaping-conformance`, which is how the storage defect was found.
1415
//!
@@ -116,7 +117,7 @@ fn plain_integer_is_still_evaluated() {
116117
// consults `datatype_is_indistinguishable_in_store` and refuses to answer. The
117118
// paths that test a datatype as a MEMBER of something else did not, so the
118119
// question the direct path refuses was answered anyway, wrongly, by them: every
119-
// value the store had widened to `xsd:integer` failed a member of
120+
// value `DATATYPE()` read as `xsd:integer` failed a member of
120121
// `xsd:nonNegativeInteger`. Each test below names its path, asserts the refusal
121122
// the direct path gives, and asserts that no violation is reported.
122123

0 commit comments

Comments
 (0)