Support units in Table Schema #1002
Replies: 22 comments
-
@rufuspollock before we move it here, it would be good to discuss how it is to be implemented. Is the proposal to implement this spec, as is, as part of Table Schema? |
Beta Was this translation helpful? Give feedback.
-
@pwalsh open to suggestions. I was thinking of keeping it separate but adding support for referencing it from table schema but not sure what is best. |
Beta Was this translation helpful? Give feedback.
-
I've drafted a pattern #607 and started a discussion on the forum https://discuss.okfn.org/t/table-schema-units-pattern/6573 |
Beta Was this translation helpful? Give feedback.
-
Notes from thread in the PR #607 @Stephen-Gates wrote:
@rufuspollock wrote back
@dr-shorthair wrote
|
Beta Was this translation helpful? Give feedback.
-
@Stephen-Gates (/cc @dr-shorthair) I think our aim is to see if we can extract a subset of UCUM that gives us 80/20 and then say if you want more go to UCUM. I have to say i think this could / should go in 2 stages:
This keeps things clean. wdyt? And if so would that mean we could merge #607 as it is? |
Beta Was this translation helpful? Give feedback.
-
@Stephen-Gates any luck here on progressing this? |
Beta Was this translation helpful? Give feedback.
-
Sorry been focussed a new release of Data Curator. Haven’t forgotten |
Beta Was this translation helpful? Give feedback.
-
Hi @Stephen-Gates (and also @rufuspollock + @pwalsh)! I wanted to let you know that @mbomhoff from Planet Microbe has been working with data packages for their oceanographic data and has been thinking about what units specs would work best for them. I wanted to tag Matt so he can keep updated on the specs units conversation, and also intro y'all in case you want to connect and discuss what units ideas Matt has. Thanks both 😄 |
Beta Was this translation helpful? Give feedback.
-
@lwinfree Thanks Lilly! Our data packages are in https://github.com/hurwitzlab/planet-microbe-datapackages. For the time being we added a custom property |
Beta Was this translation helpful? Give feedback.
-
@mbomhoff are you ok with the direction the draft PR was taking if we address the comments above? |
Beta Was this translation helpful? Give feedback.
-
@rufuspollock do you have any concerns about using UCUM given its licence? |
Beta Was this translation helpful? Give feedback.
-
Item 2. in the license is problematic:
Unfortunately UCUM now appears to be an infrastructure orphan - I've not been able to make contact with Guenther Schadow for a couple of years now. Possibly retired. I'll try again. |
Beta Was this translation helpful? Give feedback.
-
@Stephen-Gates It looks like the draft spec is capable of describing all of the units that we use in our project, but I think our application falls under the case of using an existing spec. One of the goals of our project is to use ontologies to unify disparate datasets from various sources. To describe a field we supply an Environment Ontology (ENVO, http://environmentontology.org/) purl in the
For us the UO purl provides stronger semantics and some additional info such as aliases (meter, metre) and a text description. |
Beta Was this translation helpful? Give feedback.
-
@Stephen-Gates any chance to look at this further. It sounds like we have to steer around UCUM atm. |
Beta Was this translation helpful? Give feedback.
-
UCUM only provides the terminal symbols, and a grammar to combine them into any UoM. So it is a mistake to talk about 80:20 provide by UCUM with respect to some finite set. UCUM probably provides 95%+, but by utilising the grammar. Meanwhile, I have now tracked down the owner of UCUM so it's not dead yet. |
Beta Was this translation helpful? Give feedback.
-
Lilly Winfree directed me to this discussion after I asked her how you handle units in your project in PyData Asutin. We have been dealing with Unit standardization for over a year and can connect you to some of unit specs - at least in the medical domain. |
Beta Was this translation helpful? Give feedback.
-
@Jacob-Barhak this is great info - if you could share your experience and links that would help esp any key pointers. Your tip re UCUM is also very helpful. We will look at https://clinicalunitmapping.com/ |
Beta Was this translation helpful? Give feedback.
-
So @rufuspollock , all documentation associated with the project is available in the about page: https://clinicalunitmapping.com/about |
Beta Was this translation helpful? Give feedback.
-
If adding support to units was done: frictionless-py could output the values with its units (maybe optionally) using https://pint.readthedocs.io/en/stable/ Obviously if the units used in the spec were available in the Pint library. |
Beta Was this translation helpful? Give feedback.
-
made a comment about units over here, acep-uaf/aetr-web-book-2024#40 but maybe this issue is a more appropriate place so I've copied below: another units good standard: https://www.qudt.org/doc/DOC_VOCAB-UNITS.html FYI in case its useful / interesting: |
Beta Was this translation helpful? Give feedback.
-
Are you still looking for solutions for units? You may want to check advances in clinicalunitmapping.com There is now AI behind this that is pretty good already. This project is still in beta and there is still work to do and its use is limited to demonstrate feasibility, yet it is getting better. You can find recent publications in: What are your unit needs? Why exactly do you need them? Will be happy to talk via video. |
Beta Was this translation helpful? Give feedback.
-
Also see https://si-digital-framework.org/ from BIPM who are the authority on SI. |
Beta Was this translation helpful? Give feedback.
-
Move the units draft spec http://specs.okfnlabs.org/units/ back to FD specs
/cc @pwalsh
Beta Was this translation helpful? Give feedback.
All reactions