WIP: updated branch with changes to support linkml schema

[satra]

@djarecka - adding an updated branch here.

i was testing the pydantic side with a snippet like this. for this snippet i copied the pydantic python file into the reproschema python library as model.py.

from reproschema.model import Protocol, Item, Activity
from reproschema.jsonldutils import to_newformat
import json
data = json.loads(to_newformat("examples/protocols/protocol1.jsonld", format="jsonld", contextfile="contexts/reproschema_new"))
del data["@context"]
Protocol(**data)

i have updated the context in various places to match the current context and also refactored the yaml quite a bit. some major changes:

• dictionary to tuple form of langString (this forms the prefixes in linkml)
• multivalued + inlined for most langString slots
• some classes moved to enums
• descriptions as strings

for now i have left ui as a jsonld @nest still. but did not update the shacl to make corresponding changes. the shacl generation is not deterministic and we should figure a way for patching shacl perhaps using rdflib or something that we can query the object and alter it. also because we are using various classes in linkml, some expressions which should be simply rdf:langString as datatype are turning into class constraints. we may want to ask if there is someway to tell linkml generator that if a class has class_uri: rdf:langString treat it as datatype.

anyway, i hope this gets us closer. the python part also requires some patching (for the enum values), but i think the base libary works.

[satra]

here is a comparison to @djarecka 's branch for clarity:
djarecka/reproschema@linkml_new_tmp...ReproNim:reproschema:ref/linkml

[djarecka]

I also had some updates, but not sure if I will be able to merge with yours...

[djarecka]

I looked quickly at the context and yaml, and I noticed, that:

• sometimes you changed my langString to string, e.g. in description, but I took langString from your shacl
• if you use proper @vocab, like here:

  reproschema/contexts/reproschema_new

  Line 23 in 90d853c

  "@vocab": "http://schema.repronim.org/",

  , you could remove "@id": "reproschema:landingPage",

[djarecka]

@satra - your branch gives shacl errors under validations, let me know if you want me to fix it, and work on your branch

[djarecka]

btw. for the langString and marking this as a class, I was thinking for creating langString under types in linkml, but was trying multiple ways without success. Was planning to create an issue with questions

[satra]

I don't think that I changed description to string. Let's make sure we are looking at the same branch. Regarding id in landingpage it's for a couple of reasons. That one is to indicate that the URL is the identifier for that page.

…

On Sat, Feb 17, 2024, 5:58 PM Dorota Jarecka ***@***.***> wrote: I looked quickly at the context and yaml, and I noticed, that: - sometimes you changed my langString to string, e.g. in description, but I took langString from your shacl - if you use proper @vocab, like here: https://github.com/ReproNim/reproschema/blob/90d853cb8e4cbd974df5175c9d1069f44dbd9c68/contexts/reproschema_new#L23, you could remove ***@***.***": "reproschema:landingPage", — Reply to this email directly, view it on GitHub <#456 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABM57ZSMKFFCC7P567JI4DYUEY2NAVCNFSM6AAAAABDNT2UK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGQ4TQMJQGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[satra]

That's a good idea. Having that as a built in type would be nice.

…

On Sat, Feb 17, 2024, 6:20 PM Dorota Jarecka ***@***.***> wrote: btw. for the langString and marking this as a class, I was thinking for creating langString under types in linkml, but was trying multiple ways without success. Was planning to create an issue with questions — Reply to this email directly, view it on GitHub <#456 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABM573XLMHP3W3NWZJSOG3YUE3KDAVCNFSM6AAAAABDNT2UK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGUZDSMJVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[djarecka]

I don't think that I changed description to string. Let's make sure we are looking at the same branch. Regarding id in landingpage it's for a couple of reasons. That one is to indicate that the URL is the identifier for that page.

Yes, I was wrong about description, but in some other places, e.g. schema:schemaVersion. I think it make sense, but I was just taking it from previous shacl

[satra]

i adjusted it to only be in places where having multiple languages made some sense. regarding the shacl part, as long as it is not just pure hand edit. perhaps first try to read all the protocols/activities/items from the library using the pydantic model and if no model changes are required then generate shacl and edit it.

[djarecka]

That's a good idea. Having that as a built in type would be nice.
…
On Sat, Feb 17, 2024, 6:20 PM Dorota Jarecka @.> wrote: btw. for the langString and marking this as a class, I was thinking for creating langString under types in linkml, but was trying multiple ways without success. Was planning to create an issue with questions — Reply to this email directly, view it on GitHub <#456 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABM573XLMHP3W3NWZJSOG3YUE3KDAVCNFSM6AAAAABDNT2UK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGUZDSMJVGE . You are receiving this because you were mentioned.Message ID: @.>

if I use types python will want to use some basic type, and shacl ignores it (perhaps this is a bug), see example:

• yaml:

name: reproschema
id: http://schema.repronim.org/ # todo: change this
imports:
- linkml:types
prefixes:
  rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#
  linkml: https://w3id.org/linkml/
  reproschema: http://schema.repronim.org/
default_range: string

types:
  type1:
    description: testing type
    base: string
    uri: rdf:langString


classes:
  Participant:
    description: A participant in a study
    class_uri: reproschema:Participant
    slots:
      - slot1


slots:
  slot1:
    description: slot1 description
    range: type1
    slot_uri: reproschema:slot1

• python:

...
class Participant(ConfiguredBaseModel):
    """
    A participant in a study
    """
    slot1: Optional[string] = Field(None, description="""slot1 description""")

• shacl:

reproschema:Participant a sh:NodeShape ;
    sh:closed true ;
    sh:description "A participant in a study" ;
    sh:ignoredProperties ( rdf:type ) ;
    sh:property [ sh:description "slot1 description" ;
            sh:maxCount 1 ;
            sh:order 0 ;
            sh:path reproschema:slot1 ] ;
    sh:targetClass reproschema:Participant .

[djarecka]

I think this list should also contain Skipped or the activity1.jsonld form the examples should be changed

[satra]

this one should really be AllowSkip and the other one in missing enum Skipped. activity1 should be adjusted for the new flag.

btw, for any of the enums, the autogenerated python needs to be updated to have reproschema: in front of the values.

[djarecka]

I'm not sure that I understand, so let me confirm:

• in AllowedType enum I should add one more permissible value: AllowSkip
• MissingType should leave as it is with Skipped
• in the activity1 I should use AllowSkip

[djarecka]

I also noticed that in the pydantic you're changing lang String to str, so perhaps I should move lang String to types as in the example I posted yesterday. However we will loose the check from shacl

[satra]

confirming the first question.

pydantic was auto generated. the only thing i changed were the enums. the class langString in the model now has slots similar to prefix in the linkml model as chris had suggested. so technically it should be a class in pydantic with two fields. can you give an example of where you are seeing the change?

[djarecka]

yes, it's very inconsistent..

but actually having type Dict[str, str] makes this version valid with the data. If you use really langString it will fail since the keys do not match

[djarecka]

maybe we will simply create a class that we list all possible languages we use as attributes, so you can get in pydantic something like this:

class LangString(ConfiguredBaseModel):
    en: str = Field(..., description="""The english version of the string""")
    es: str = Field(..., description="""The spanish version of the sting.""")

This would allow to keep you short version of langstring in the jsonld

[djarecka]

i understand we can also set a handful of languages in yaml and add extra=Extra.allow to the pydantic version

[djarecka]

btw. I think you had Dict[str, str] instead of langString because you used key: true. I haven't used it before, but in a way it makes sense that it translated to Dcit[str, str]. Still should be consistent though

[satra]

i don't know how we would deal with hyphenated langs like the ones here: https://github.com/ReproNim/reproschema-library/blob/9be140d0297ae0671d50466448ae67dea86d3bfd/activities/PHQ-9/PHQ9_schema#L13

for now let's keep it as key, value pairs like linkml prefixes and then we can adjust later.

[djarecka]

you want to keep this still as a separate branch?

[satra]

we can merge this, but if we do, we should remove/filter out the python file, unless there is a clear reason to keep one in this repo. we should either remove the shacl file and add a set of TODOs to the README before we release.

[djarecka]

so what about I will try to fix the shacl and in the meantime i will create a new branch in repronim-py with the model.py?

[djarecka]

@satra - working on test for reproschema-py I got some additional errors from the model from Responses files. Both Participant and SoftwareAgent doesn't have id in the new model, so pydantic complain, and that make me wondering whether I should id to all classes in the model?

Some of the classes, e.g. Activity, Item, have id and category fields because they inherit from CreativeWork, but other classes do not have id.

[satra]

all classes should inherit from creativework/namedthing .

[satra]

@djarecka - can we make citation just a string? is there a reason it's a langstring? also could we make it be any of url or string (so we can point to dois, urls, etc.,.)

[djarecka]

@satra @yibeichan - i think this can be merged, but unfortunately the python package workflow depends on reproschema-py and will be failing with old version of validate

[Remi-Gau]

The validation folder is gone.
This is giving us a dead link in the doc.