Skip to content

Commit

Permalink
Improve instructions in the README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
vemonet authored and JervenBolleman committed Aug 29, 2024
1 parent db6a546 commit df17f6b
Show file tree
Hide file tree
Showing 2 changed files with 50 additions and 49 deletions.
95 changes: 47 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ prefix ex: <https://sparql.uniprot.org/.well-known/sparql-examples/> # <!-- cha
prefix sh: <http://www.w3.org/ns/shacl#>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs:<http://www.w3.org/2000/01/rdf-schema#>
ex:1 # <!-- UniProt, Rhea and Swiss-Lipids are numbered but this can be anything.
ex:001 # <!-- UniProt, Rhea and Swiss-Lipids are numbered but this can be anything.
a sh:SPARQLSelectExecutable, sh:SPARQLExecutable ;
sh:prefixes _:sparql_examples_prefixes ; # <!-- required for the import of the prefix declarations. Note the blank node
rdfs:comment """A comment <em>May have HTML in them</em>. Example: Select all taxa from the <a href=\"https://www.uniprot.org/taxonomy/\">UniProt taxonomy</a>"""^^rdf:HTML ;
Expand All @@ -29,91 +30,89 @@ WHERE
?taxon a up:Taxon .
}""" ;
schema:target <https://sparql.uniprot.org/sparql/> ;
schema:keywords "taxa".
schema:keywords "taxa" .
```

# Quality Assurance (QA).

We use the [SIB SPARQL Examples utils](https://github.com/sib-swiss/sparql-examples-utils/) for testing
## Artifact generation and quality assurance

# Conversion for upload in SPARQL endpoint
We use the [SIB SPARQL Examples utils](https://github.com/sib-swiss/sparql-examples-utils/) for testing and generating artifacts.

To load the examples into a SPARQL endpoint they should be concatenated into one example file. Use the script `convertIntoOneTurtle.sh` provide the project name with a `-p` parameter

This expects the Jena tools to be available in your $PATH. e.g. `export PATH="$JENA_HOME/bin:$PATH"`
First, download the jar file with:

```bash
# e.g. make file examples_uniprot.ttl
./convertToOneTurtle.sh -p uniprot
wget -O sparql-examples-utils.jar 'https://github.com/sib-swiss/sparql-examples-utils/releases/download/v1.0.0/sparql-examples-util-1.0.0-uber.jar'
```

Another option is to build the converter and use that.
### Testing the queries

The queries are parsed and validated but not executed with junit using the Tester

```bash
wget 'https://github.com/sib-swiss/sparql-examples-utils/releases/download/v1.0.0/sparql-examples-util-1.0.0-uber.jar'
java -jar sparql-examples-util-1.0.0-uber.jar -i examples/ -p all -f jsonld
# Or for a specific example folder, as turtle, to a file:
java -jar sparql-examples-util-1.0.0-uber.jar -i examples/ -p Bgee -f ttl > examples_Bgee.ttl
java -cp sparql-examples-utils.jar swiss.sib.rdf.sparql.examples.Tester --input-directory=./examples
```

## Conversion to RQ files
should return no test failures. RDF4j and Jena are both a lot stricter than virtuoso.

The queries can be executed automatically on all endpoints they apply to using ane extra argument for the Tester `--also-run-slow-tests`:

For easier use by other tools we can also generate [rq](https://www.w3.org/TR/2013/REC-sparql11-query-20130321/#mediaType) files. Following the syntax of [grlc](https://grlc.io/) allowing to use these queries as APIs.
```bash
wget 'https://github.com/sib-swiss/sparql-examples-utils/releases/download/v1.0.0/sparql-examples-util-1.0.0-uber.jar'
java -jar sparql-examples-util-1.0.0-uber.jar -i examples/ -p all -r
java -cp sparql-examples-utils.jar swiss.sib.rdf.sparql.examples.Tester --input-directory=./examples/Bgee --also-run-slow-tests
```

## Generate markdown file
> This does change the queries to add a LIMIT 1 if no limit was set in the query. Then check if there is a result it is fetched.
Generate markdown files with the query and a mermaid diagram of the queries, to be used to deploy a static website for the query examples.
### Compile all query files

Compile all query files for a specific example folder, into a local file including the prefixes/namespaces definitions:

```bash
java -jar sparql-examples-util-*-uber.jar -i examples/ -m
java -jar sparql-examples-utils.jar -i examples/ -p UniProt -f ttl > examples_UniProt.ttl
```

# Querying for queries
> You can then load this file to this project SPARQL endpoint!
As the SPARQL examples are themselves RDF, they can be queried for as soon as they are loaded in a SPARQL endpoint.
```sparql
PREFIX sh: <http://www.w3.org/ns/shacl#>
SELECT *
WHERE {
?ex sh:select|sh:ask|sh:construct|sh:describe ?query .
}
```
Or compile for all example folders, as JSON-LD, to the standard output:

# Testing your queries
```bash
java -jar sparql-examples-utils.jar -i examples/ -p all -f jsonld
```

The queries are parsed and validated but not executed with junit using the Tester
### Generate RQ files

For easier use by other tools we can also generate [rq](https://www.w3.org/TR/2013/REC-sparql11-query-20130321/#mediaType) files. Following the syntax of [grlc](https://grlc.io/) allowing to use these queries as APIs.
```bash
wget 'https://github.com/sib-swiss/sparql-examples-utils/releases/download/v1.0.0/sparql-examples-util-1.0.0-uber.jar'
java -cp sparql-examples-util-1.0.0-uber.jar swiss.sib.rdf.sparql.examples.Tester --input-directory=./examples

java -jar sparql-examples-utils.jar -i examples/ -p all -r
```
should return no test failures. RDF4j and Jena are both a lot stricter than virtuoso.

### Generate markdown file

# Labeling queries
Generate markdown files with the query and a mermaid diagram of the queries, to be used to deploy a static website for the query examples.

If you want to add a label to a query please use [schema.org keyword](https://schema.org/keywords)
```bash
java -jar sparql-examples-utils.jar -i examples/ -m
```

# Testing the queries actually work
## Querying for queries

The queries can be executed automatically on all endpoints they apply to using
As the SPARQL examples are themselves RDF, they can be queried for as soon as they are loaded in a SPARQL endpoint.
```sparql
PREFIX sh: <http://www.w3.org/ns/shacl#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
```
mvn test -PallTests
SELECT DISTINCT ?sq ?comment ?query
WHERE {
?sq a sh:SPARQLExecutable ;
rdfs:label|rdfs:comment ?comment ;
sh:select|sh:ask|sh:construct|sh:describe ?query .
} ORDER BY ?sq
```

This does change the queries to add a LIMIT 1 if no limit was set in the query. Then if there is a result it is fetched.

## Finding queries that run on more than one endpoint

```bash
This expects the Jena tools to be available in your $PATH. e.g. `export PATH="$JENA_HOME/bin:$PATH"`

./contvertAllToOneTurtle.sh -a
```bash
java -jar sparql-examples-utils.jar -i examples/ -p all -f ttl > examples_all.ttl

sparql --data examples_all.ttl "SELECT ?query (GROUP_CONCAT(?target ; separator=', ') AS ?targets) WHERE { ?query <https://schema.org/target> ?target } GROUP BY ?query HAVING (COUNT(DISTINCT ?target) > 1) "
```
Expand Down
4 changes: 3 additions & 1 deletion convertToOneTurtle.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
#!/usr/bin/bash


# To load the examples into a SPARQL endpoint they should be concatenated into one example file. Use the script `convertIntoOneTurtle.sh` provide the project name with a `-p` parameter
# This expects the Jena tools to be available in your $PATH. e.g. `export PATH="$JENA_HOME/bin:$PATH"`
# ./convertToOneTurtle.sh -p uniprot

project="uniprot"
while getopts anuhrsgmcbop: option; do
Expand Down

0 comments on commit df17f6b

Please sign in to comment.