gCubeSystem
/
grsf-publisher
15
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Improving profile definition

This commit is contained in:
Luca Frosini 2023-03-23 15:20:00 +01:00
parent e8c0d8d102
commit 89ff50c077
1 changed files with 324 additions and 72 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

396
src/test/resources/profiles/AssessmentUnit.xml
View File

@ -32,10 +32,12 @@
<topField>title</topField> <topField>title</topField>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "stock_name" : "Cancer pagurus - Central North Sea (Division 27.4.b)". Example:
"stock_name" : "Cancer pagurus - Central North Sea (Division 27.4.b)".
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example: "stock_name" : "Sebastes mentella - Atlantic, Northwest / 21.2 - Atlantic, Northwest / 21.1 - East Greenland (Subarea 27.14) - Iceland and Faroes Grounds (Subarea 27.5) - North of Azores (Subarea 27.12)" Example:
"stock_name" : "Sebastes mentella - Atlantic, Northwest / 21.2 - Atlantic, Northwest / 21.1 - East Greenland (Subarea 27.14) - Iceland and Faroes Grounds (Subarea 27.5) - North of Azores (Subarea 27.12)"
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
</grsf> </grsf>
@ -60,10 +62,12 @@
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "short_name" : "Edible crab - Central North Sea -". Example:
"short_name" : "Edible crab - Central North Sea -".
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example: "short_name" : "Beaked Redfish Areas V-XII-XIV and NAFO Subareas 1-2", Example:
"short_name" : "Beaked Redfish Areas V-XII-XIV and NAFO Subareas 1-2",
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
</grsf> </grsf>
@ -87,10 +91,12 @@
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "grsf_semantic_identifier" : "asfis:CRE+fao:27.4.b". Example:
"grsf_semantic_identifier" : "asfis:CRE+fao:27.4.b".
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example: "grsf_semantic_identifier" : "asfis:REB+fao:21.1;fao:21.2;fao:27.12;fao:27.14;fao:27.5" Example:
"grsf_semantic_identifier" : "asfis:REB+fao:21.1;fao:21.2;fao:27.12;fao:27.14;fao:27.5"
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
</grsf> </grsf>
@ -121,13 +127,14 @@
</dstTypes> </dstTypes>
<comments> <comments>
A list of String each one containing the assessment area. A list of String each one containing the assessment area.
It generate a Field for each element of the Array. It generates a Field for each element of the Array.
Example: "assessment_area" : [ "Code: 27.4.b, System: fao, Name: Central North Sea (Division 27.4.b)" ]. Example:
"assessment_area" : [ "Code: 27.4.b, System: fao, Name: Central North Sea (Division 27.4.b)" ].
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example:
Example: "assessment_area" : [ "assessment_area" : [
"Code: 21.2, System: fao, Name: Atlantic, Northwest / 21.2", "Code: 21.2, System: fao, Name: Atlantic, Northwest / 21.2",
"Code: 27.5, System: fao, Name: Iceland and Faroes Grounds (Subarea 27.5)", "Code: 27.5, System: fao, Name: Iceland and Faroes Grounds (Subarea 27.5)",
"Code: 27.14, System: fao, Name: East Greenland (Subarea 27.14)", "Code: 27.14, System: fao, Name: East Greenland (Subarea 27.14)",
@ -164,13 +171,14 @@
</dstTypes> </dstTypes>
<comments> <comments>
A list of URL of connected records. A list of URL of connected records.
It generate a Field for each element of the Array. It generates a Field for each element of the Array.
Example: "connected" : [ "https://data.d4science.org/ctlg/GRSF_Admin/ae42d466-4a1b-3615-9686-dca1f1606bb7" ] Example:
"connected" : [ "https://data.d4science.org/ctlg/GRSF_Admin/ae42d466-4a1b-3615-9686-dca1f1606bb7" ]
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example:
Example: "connected" : [ "https://data.d4science.org/ctlg/GRSF_Admin/d67aa4b6-3222-345a-bb6d-8dd74545e529", "https://data.d4science.org/ctlg/GRSF_Admin/22a56c1c-d485-3280-beb4-728e1772496e", "https://data.d4science.org/ctlg/GRSF_Admin/a25d7b36-b9fe-3ce6-a912-2c625703c823", "https://data.d4science.org/ctlg/GRSF_Admin/f52b3673-2a70-3b31-83fd-b5a21708ec8c", "https://data.d4science.org/ctlg/GRSF_Admin/34df0b47-380c-3ad9-9722-aad6185f7e25", "https://data.d4science.org/ctlg/GRSF_Admin/9fca104f-bc6c-3fcd-83a2-a28ec7ff7239", "https://data.d4science.org/ctlg/GRSF_Admin/1b614323-8336-305d-bb07-526f4ea5d255" ], "connected" : [ "https://data.d4science.org/ctlg/GRSF_Admin/d67aa4b6-3222-345a-bb6d-8dd74545e529", "https://data.d4science.org/ctlg/GRSF_Admin/22a56c1c-d485-3280-beb4-728e1772496e", "https://data.d4science.org/ctlg/GRSF_Admin/a25d7b36-b9fe-3ce6-a912-2c625703c823", "https://data.d4science.org/ctlg/GRSF_Admin/f52b3673-2a70-3b31-83fd-b5a21708ec8c", "https://data.d4science.org/ctlg/GRSF_Admin/34df0b47-380c-3ad9-9722-aad6185f7e25", "https://data.d4science.org/ctlg/GRSF_Admin/9fca104f-bc6c-3fcd-83a2-a28ec7ff7239", "https://data.d4science.org/ctlg/GRSF_Admin/1b614323-8336-305d-bb07-526f4ea5d255" ],
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
</grsf> </grsf>
@ -230,13 +238,15 @@
"name" : "FishSource", "name" : "FishSource",
"description" : "FishSource is an online information resource about the status of stocks and fisheries, that compiles and summarizes all the information that is needed from analysts to evaluate the sustainability.", "description" : "FishSource is an online information resource about the status of stocks and fisheries, that compiles and summarizes all the information that is needed from analysts to evaluate the sustainability.",
"url" : "http://www.fishsource.com" "url" : "http://www.fishsource.com"
}]. }]
The generated field is
It generates the following Metadata:
"Database Source": "FishSource" "Database Source": "FishSource"
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example: "database_sources" : [{ Example:
"database_sources" : [{
"name" : "FishSource", "name" : "FishSource",
"description" : "FishSource is an online information resource about the status of stocks and fisheries, that compiles and summarizes all the information that is needed from analysts to evaluate the sustainability.", "description" : "FishSource is an online information resource about the status of stocks and fisheries, that compiles and summarizes all the information that is needed from analysts to evaluate the sustainability.",
"url" : "http://www.fishsource.com" "url" : "http://www.fishsource.com"
@ -249,9 +259,26 @@
"description" : "RAM Legacy Stock Assessment Database is a compilation of stock assessment results for commercially exploited marine populations from around the world.", "description" : "RAM Legacy Stock Assessment Database is a compilation of stock assessment results for commercially exploited marine populations from around the world.",
"url" : "http://ramlegacy.org" "url" : "http://ramlegacy.org"
}] }]
The generated field is: It generates the following Metadata:
"Database Source": RAM FishSource FIRMS" "Database Source": "RAM FishSource FIRMS"
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
According to the ticket https://support.d4science.org/issues/24817
this property will be changed as following
"database_sources": [
"FishSource",
"Fisheries and Resources Monitoring System (FIRMS)",
"RAM Legacy Stock Assessment Database"
]
No Resource will be generated and there will be a Metadata for each element of the array.
So the Record will have the following Metadata:
"Database Source": "FishSource"
"Database Source": "Fisheries and Resources Monitoring System (FIRMS)"
"Database Source": "RAM Legacy Stock Assessment Database"
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -286,10 +313,14 @@
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "grsf_type" : "assessment unit". // TODO also system:type
Example:
"grsf_type" : "assessment unit".
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example: "grsf_type" : "assessment unit". Example:
"grsf_type" : "assessment unit".
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
</grsf> </grsf>
@ -313,6 +344,9 @@
<dstTypes> <dstTypes>
<dstType maxOccur="*">Field</dstType> <dstType maxOccur="*">Field</dstType>
</dstTypes> </dstTypes>
<comments>
TODO
</comments>
</grsf> </grsf>
</extra> </extra>
</metadatafield> </metadatafield>
@ -345,10 +379,12 @@
Used for tagging purpose only. Used for tagging purpose only.
No field or resource is generated. No field or resource is generated.
Example: "similarities_indicator" : "without similar records". Example:
"similarities_indicator" : "without similar records".
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example: "similarities_indicator" : "without similar records". Example:
"similarities_indicator" : "without similar records".
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
</grsf> </grsf>
@ -379,8 +415,10 @@
</dstTypes> </dstTypes>
<comments> <comments>
A list of String each one containing the specie. A list of String each one containing the specie.
It generate a Field for each element of the Array. It generates a Field for each element of the Array.
Example: "species" : [ "Code: CRE, Classification System: ASFIS, Scientific Name: Cancer pagurus" ].
Example:
"species" : [ "Code: CRE, Classification System: ASFIS, Scientific Name: Cancer pagurus" ].
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
</comments> </comments>
</grsf> </grsf>
@ -447,8 +485,8 @@
}] }]
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
Example of produced metadata is: It generates the following Metadata:
Abundance Level: 6.15 [Unit: Male SSBtrp ('000 t) - Rep. Year or Assessment Id: 2019 - Ref. Year: 2019 - Data Owner: UK Centre for Environment, Fisheries and Aquaculture Science - DB Source: FishSource] "Abundance Level": "6.15 [Unit: Male SSBtrp ('000 t) - Rep. Year or Assessment Id: 2019 - Ref. Year: 2019 - Data Owner: UK Centre for Environment, Fisheries and Aquaculture Science - DB Source: FishSource]"
Example: Example:
@ -472,8 +510,8 @@
] ]
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
Example of produced metadata is: It generates the following Metadata:
"Abundance Level": MSY BTrigger: Undefined [Rep. Year or Assessment Id: 2016 - Ref. Year: 2015 - Data Owner: International Council for the Exploration of the Sea (ICES) - DB Source: FIRMS] "Abundance Level": "MSY BTrigger: Undefined [Rep. Year or Assessment Id: 2016 - Ref. Year: 2015 - Data Owner: International Council for the Exploration of the Sea (ICES) - DB Source: FIRMS]"
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -534,8 +572,8 @@
] ]
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
Example of produced metadata is: It generates the following Metadata:
"Abundance Level (FIRMS Standard)": Uncertain/Not assessed [Rep. Year or Assessment Id: 2009 - Ref. Year: 2008 - Data Owner: International Council for the Exploration of the Sea (ICES) - DB Source: FIRMS] "Abundance Level (FIRMS Standard)": "Uncertain/Not assessed [Rep. Year or Assessment Id: 2009 - Ref. Year: 2008 - Data Owner: International Council for the Exploration of the Sea (ICES) - DB Source: FIRMS]"
Please note that here we have a controlled vocabulary for "value": Please note that here we have a controlled vocabulary for "value":
- not applicable -> Not applicable - not applicable -> Not applicable
@ -572,7 +610,17 @@
<dstType>Resource</dstType> <dstType>Resource</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "Analytical assessment" Example:
"assessment_methods": [
"Unknown [Rep. Year or Assessment ID: NWWG-REDDEEPSH1-2-V-XII-XIV-1982-2016-ICESIMP2016, Ref. Year: 2016, DB Source: RAM]"
]
It generates the following Metadata:
"Assessment Method": "Unknown [Rep. Year or Assessment ID: NWWG-REDDEEPSH1-2-V-XII-XIV-1982-2018-ICESIMP2021, Ref. Year: 2018, DB Source: RAM]"
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
TODO It must generate a Resource see https://support.d4science.org/issues/24034
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -598,11 +646,11 @@
<comments> <comments>
It generate a metadata with as value the string value of the JSON property It generate a metadata with as value the string value of the JSON property
Example Example:
"assessor": "NWWG", "assessor": "NWWG",
It generated the following metadata: It generates the following Metadata:
"Assessor": NWWG "Assessor": "NWWG"
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
@ -634,7 +682,23 @@
<dstType>TimeSeries</dstType> <dstType>TimeSeries</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "biomass": [{"year":2014, "value": 123, "unit": "..."}, {"year":2015, "value": 172, "unit": "..."}, ...] It generates
- a Resource with name 'Abundance Level' containing a CSV where
- the first row is the CSV header which is generated by taking the key of the element
of the JsonArray.
- the subsequent rows are generated with the value of each element of the JSONArray.
- a metadata field for each element of the array where the value has the following format:
- the value of 'value' property
- A beginning '['
- the key (titled) - the value
- A '-' which separate each element
- A terminating ']'
Example:
"biomass": [{"year":2014, "value": 123, "unit": "..."}, {"year":2015, "value": 172, "unit": "..."}, ...]
TODO add a real example
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -664,9 +728,46 @@
<dstType>TimeSeries</dstType> <dstType>TimeSeries</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
A time series of value, unit and date. Example: "catches": [{"unit" : "...", "value": "...", "year": "..."}, ...] It generates
It generate a Resource containing all the content of the input field (i.e. a TimeSeries) - a Resource with name 'Abundance Level' containing a CSV where
and as many Field as the number of element in the JsonArray. - the first row is the CSV header which is generated by taking the key of the element
of the JsonArray.
- the subsequent rows are generated with the value of each element of the JSONArray.
- a metadata field for each element of the array where the value has the following format:
- the value of 'value' property
- A beginning '['
- the key (titled) - the value
- A '-' which separate each element
- A terminating ']'
Example:
"catches": [
{
"value": "5350",
"unit": "tonnes",
"reference_year": 2009,
"db_source": "FIRMS",
"reporting_year_or_assessment_id": "2016",
"data_owner": "Food and Agriculture Organization (FAO)"
},
...
{
"value": "4083",
"unit": "tonnes",
"reference_year": 1994,
"db_source": "FIRMS",
"reporting_year_or_assessment_id": "2016",
"data_owner": "Food and Agriculture Organization (FAO)"
}
]
It generates the following Metadata:
"Catch": "5350 [Unit: tonnes - Rep. Year or Assessment Id: 2016 - Ref. Year: 2009 - Data Owner: Food and Agriculture Organization (FAO) - DB Source: FIRMS]"
....
"Catch": "4083 [Unit: tonnes - Rep. Year or Assessment Id: 2016 - Ref. Year: 1994 - Data Owner: Food and Agriculture Organization (FAO) - DB Source: FIRMS]"
https://data.d4science.org/ctlg/GRSF_Admin/2a001ef0-e820-3d55-a330-5058f4701b4b
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -692,6 +793,16 @@
</dstTypes> </dstTypes>
<comments> <comments>
It generates a Resource for each element of the array. It generates a Resource for each element of the array.
Example:
"data_owner": [
"Food and Agriculture Organization (FAO) [DB Source: [FIRMS]]"
]
It generates the following Metadata:
"Data Owner": "Food and Agriculture Organization (FAO) [DB Source: [FIRMS]]"
https://data.d4science.org/ctlg/GRSF_Admin/2a001ef0-e820-3d55-a330-5058f4701b4b
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -773,7 +884,45 @@
<dstType>TimeSeries</dstType> <dstType>TimeSeries</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "fishing_pressure" : [{"year":2014, "value":"..."}, {"year":2015, "value":"..."}, ...] It generates
- a Resource with name 'FAO Stock Status Category' containing a CSV where
- the first row is the CSV header which is generated by taking the key of the element
of the JsonArray.
- the subsequent rows are generated with the value of each element of the JSONArray.
- a metadata field for each element of the array where the value has the following format:
- the value of 'value' property
- A beginning '['
- the key (titled) - the value
- A '-' which separate each element
- A terminating ']'
"fishing_pressure": [
{
"value": "FMSY: Undefined",
"reference_year": 2015,
"unit": "",
"db_source": "FIRMS",
"reporting_year_or_assessment_id": "2016",
"data_owner": "International Council for the Exploration of the Sea (ICES)"
},
...
{
"value": "Undefined",
"reference_year": 2008,
"unit": "",
"db_source": "FIRMS",
"reporting_year_or_assessment_id": "2009",
"data_owner": "International Council for the Exploration of the Sea (ICES)"
}
]
Example of produced metadata is:
"Fishing Pressure": "FMSY: Undefined [Rep. Year or Assessment Id: 2016 - Ref. Year: 2015 - Data Owner: International Council for the Exploration of the Sea (ICES) - DB Source: FIRMS]"
...
"Fishing Pressure": "Undefined [Rep. Year or Assessment Id: 2009 - Ref. Year: 2008 - Data Owner: International Council for the Exploration of the Sea (ICES) - DB Source: FIRMS]"
https://data.d4science.org/ctlg/GRSF_Admin/cb468fb0-fed5-370f-9bc6-08fa99c54f3c
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -803,7 +952,21 @@
<dstType>TimeSeries</dstType> <dstType>TimeSeries</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Example: "firms_standard_fishing_pressure" : [{"year":2014, "value":"..."}, {"year":2015, "value":"..."}, ...] It generates
- a Resource with name 'Abundance Level' containing a CSV where
- the first row is the CSV header which is generated by taking the key of the element
of the JsonArray.
- the subsequent rows are generated with the value of each element of the JSONArray.
- a metadata field for each element of the array where the value has the following format:
- the value of 'value' property
- A beginning '['
- the key (titled) - the value
- A '-' which separate each element
- A terminating ']'
Example: "firms_standard_fishing_pressure" : [{"year":2014, "value":"..."}, {"year":2015, "value":"..."}, ...]
// TOD add a real example
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -833,9 +996,21 @@
<dstType>TimeSeries</dstType> <dstType>TimeSeries</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
A time series of value, unit and date. Example: "landings": [{"unit" : "...", "value": "...", "year": "..."}, ...] It generates
It generate a Resource containing all the content of the input field (i.e. a TimeSeries) - a Resource with name 'Abundance Level' containing a CSV where
and as many Field as the number of element in the JsonArray. - the first row is the CSV header which is generated by taking the key of the element
of the JsonArray.
- the subsequent rows are generated with the value of each element of the JSONArray.
- a metadata field for each element of the array where the value has the following format:
- the value of 'value' property
- A beginning '['
- the key (titled) - the value
- A '-' which separate each element
- A terminating ']'
TODO add a real example
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -876,15 +1051,14 @@
- A '-' which separate each element - A '-' which separate each element
- A terminating ']' - A terminating ']'
TODO The example does not report any reference_year instance the record has been properly generated.
Ask to Yannis.
Example: Example:
"scientific_advice" : [ "scientific_advice": [
"As a precautionary measure not to increase fishing effort as a series of more complete and better quality data are not available. The catch should not exceed the 2008 level (5 000 tonnes). [Rep. Year or Assessment ID: 2016, Data Owner: Food and Agriculture Organization (FAO), DB Source: FIRMS]" {
"value": "As a precautionary measure not to increase fishing effort as a series of more complete and better quality data are not available. The catch should not exceed the 2008 level (5 000 tonnes). [Data Owner: Food and Agriculture Organization (FAO), DB Source: FIRMS]",
"reference_year": "2016"
}
] ]
Example of produced metadata is: Example of produced metadata is:
"Scientific Advice": As a precautionary measure not to increase fishing effort as a series of more complete and better quality data are not available. The catch should not exceed the 2008 level (5 000 tonnes). [Data Owner: Food and Agriculture Organization (FAO), DB Source: FIRMS] [Ref. Year: 2016] "Scientific Advice": As a precautionary measure not to increase fishing effort as a series of more complete and better quality data are not available. The catch should not exceed the 2008 level (5 000 tonnes). [Data Owner: Food and Agriculture Organization (FAO), DB Source: FIRMS] [Ref. Year: 2016]
https://data.d4science.org/ctlg/GRSF_Admin/2a001ef0-e820-3d55-a330-5058f4701b4b https://data.d4science.org/ctlg/GRSF_Admin/2a001ef0-e820-3d55-a330-5058f4701b4b
@ -964,8 +1138,18 @@
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments> <comments>
Exmaple: {"annotation_message":"checked together with Paula Anton","admin":"Aureliano Gentile","time":"2018-07-27 14:58:13 CET"}. Example:
the above example of from Record: https://data.d4science.org/ctlg/GRSF_Admin/22657fbf-41fb-3c10-90bd-6244b38d4e05 "annotations": [
{
"admin": "Aureliano Gentile",
"annotation_message": "checked together with Paula Anton",
"time": "2018-07-27 14:58:13 CET"
}
]
Generated Metadata
"Annotation": {"annotation_message":"checked together with Paula Anton","admin":"Aureliano Gentile","time":"2018-07-27 14:58:13 CET"}
https://data.d4science.org/ctlg/GRSF_Admin/22657fbf-41fb-3c10-90bd-6244b38d4e05
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -973,7 +1157,7 @@
<metadatafield> <metadatafield>
<fieldName>Citation</fieldName> <fieldName>Citation</fieldName>
<mandatory>false</mandatory> <mandatory>true</mandatory>
<dataType>String</dataType> <dataType>String</dataType>
<remove>false</remove> <remove>false</remove>
<extra> <extra>
@ -982,11 +1166,21 @@
<srcType> <srcType>
<simpleType>String</simpleType> <simpleType>String</simpleType>
</srcType> </srcType>
<mandatory>false</mandatory> <mandatory>true</mandatory>
<isSensitive>false</isSensitive> <isSensitive>false</isSensitive>
<dstTypes> <dstTypes>
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments>
Example:
"citation": "© D4Science.org. Cynoglossus senegalensis - Cynoglossus monodi - Cynoglossus canariensis - Cynoglossus spp - Sherbro - Atlantic, East central / 34.3.13. In: Global Record of Stocks and Fisheries (GRSF). Consiglio Nazionale delle Ricerche, CNR. Version: 1.0. Updated: June 08, 2021. Accessed: [Date accessed and/or downloaded]. http://data.d4science.org/ctlg/GRSF/2a001ef0-e820-3d55-a330-5058f4701b4b. Database sources: [FIRMS]"
Generated Metadata:
"Citation": © D4Science.org. Cynoglossus senegalensis - Cynoglossus monodi - Cynoglossus canariensis - Cynoglossus spp - Sherbro - Atlantic, East central / 34.3.13. In: Global Record of Stocks and Fisheries (GRSF). Consiglio Nazionale delle Ricerche, CNR. Version: 1.0. Updated: June 08, 2021. Accessed: [Date accessed and/or downloaded]. http://data.d4science.org/ctlg/GRSF/2a001ef0-e820-3d55-a330-5058f4701b4b. Database sources: [FIRMS]
https://data.d4science.org/ctlg/GRSF_Admin/2a001ef0-e820-3d55-a330-5058f4701b4b
</comments>
</grsf> </grsf>
</extra> </extra>
</metadatafield> </metadatafield>
@ -1015,7 +1209,11 @@
<dstTypes> <dstTypes>
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments>Used for tagging purpose only.</comments> <comments>
Used for tagging purpose only.
The Metadata field is generated for tagging purposed
but it is removed before sending the record to Ckan.
</comments>
</grsf> </grsf>
</extra> </extra>
</metadatafield> </metadatafield>
@ -1033,18 +1231,15 @@
<remove>false</remove> <remove>false</remove>
<extra> <extra>
<grsf> <grsf>
<srcField>null</srcField> <comments>
<mandatory>aggregated</mandatory> TODO The Metadata is generated from the main type i.e. Stock
<isSensitive>false</isSensitive> </comments>
<dstTypes>
<dstType maxOccur="*">Field</dstType>
</dstTypes>
</grsf> </grsf>
</extra> </extra>
</metadatafield> </metadatafield>
<metadatafield> <metadatafield>
<fieldName>Geospatial</fieldName> <fieldName>spatial</fieldName>
<mandatory>false</mandatory> <mandatory>false</mandatory>
<dataType>String</dataType> <dataType>String</dataType>
<remove>false</remove> <remove>false</remove>
@ -1059,6 +1254,18 @@
<dstTypes> <dstTypes>
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments>
The CKAN plugin requires that this field is called 'spatial'.
The Ckan UI will be patched to show the requested label 'Geospatial'
Example:
"spatial": "{\"type\":\"Polygon\",\"coordinates\":[[[-1.23415719831099,54.8962264819267],[-1.11637942107783,54.6609253712494],[-0.608943696714849,54.4606925047925],[-0.460263892355364,54.3029928424199],[-0.0901738226846061,54.1394234161846],[-0.189191528293974,53.9899681118183],[-0.0377280072033337,53.7899320462288],[0.596695521040569,53.8282733567903],[1.19641847970505,53.8377856582369],[1.99966157981981,53.8241249582213],[2.16392788596666,53.9787618687905],[1.80624052344258,54.1337742836553],[1.46016642141069,54.2604142726476],[0.582926902489878,54.4360169123072],[-0.0252546202590009,54.5431837732809],[-0.643827431217429,54.7418256663802],[-1.10384703627582,54.9790722173552],[-1.53172481421045,55.3204666897879],[-1.23415719831099,54.8962264819267]]]}"
Generated Metadata:
"spatial": {"type":"Polygon","coordinates":[[[-1.23415719831099,54.8962264819267],[-1.11637942107783,54.6609253712494],[-0.608943696714849,54.4606925047925],[-0.460263892355364,54.3029928424199],[-0.0901738226846061,54.1394234161846],[-0.189191528293974,53.9899681118183],[-0.0377280072033337,53.7899320462288],[0.596695521040569,53.8282733567903],[1.19641847970505,53.8377856582369],[1.99966157981981,53.8241249582213],[2.16392788596666,53.9787618687905],[1.80624052344258,54.1337742836553],[1.46016642141069,54.2604142726476],[0.582926902489878,54.4360169123072],[-0.0252546202590009,54.5431837732809],[-0.643827431217429,54.7418256663802],[-1.10384703627582,54.9790722173552],[-1.53172481421045,55.3204666897879],[-1.23415719831099,54.8962264819267]]]}
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
</comments>
</grsf> </grsf>
</extra> </extra>
</metadatafield> </metadatafield>
@ -1080,11 +1287,15 @@
<dstTypes> <dstTypes>
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments>
Example:
"grsf_uuid": "2a001ef0-e820-3d55-a330-5058f4701b4b"
</comments>
</grsf> </grsf>
</extra> </extra>
</metadatafield> </metadatafield>
<!-- Record URL --> <!-- TODO Record URL -->
<metadatafield> <metadatafield>
<fieldName>Refers To</fieldName> <fieldName>Refers To</fieldName>
@ -1115,12 +1326,40 @@
</dstTypes> </dstTypes>
<comments> <comments>
It generates a Resource with the content of refers_to input field. It generates a Resource with the content of refers_to input field.
A list of objects of the format {"url": "http://", "id": "..."} that allows the aggregated GRSF records to point to their source records already published in the catalogue.
The URL and the id are both mandatory and are the ones returned by the services when a source record is published.
Example: Check the record http://data.d4science.org/ctlg/GRSF_Admin/f82393dd-d23a-3c26-98d3-ccc5f7910266.
The JSON submitted by KB contains: "refers_to" : [ { "id" : "c29b2485-c33f-3c03-8790-55a1dcf045ed", "url" : "http://data.d4science.org/ctlg/GRSF_Admin/c29b2485-c33f-3c03-8790-55a1dcf045ed"}].
The reference is in the generated resource FIRMS.
No metadata field is created. No metadata field is created.
A list of objects of the format {"url": "http://", "id": "..."} that allows the aggregated
GRSF records to point to their source records already published in the catalogue.
The name of the resource is get by performing a lookup to the target record and getting the original source.
The URL and the id are both mandatory and are the ones returned by the services when a source record is published.
Example:
"refers_to": [
{
"id": "eb7ddeb0-ecc3-393b-b98a-fdb90be2c706",
"url": "https://data.d4science.org/ctlg/GRSF_Admin/eb7ddeb0-ecc3-393b-b98a-fdb90be2c706"
}
]
It generate the Resource with name FishSource which point to https://data.d4science.org/ctlg/GRSF_Admin/eb7ddeb0-ecc3-393b-b98a-fdb90be2c706
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
According to ticket https://support.d4science.org/issues/24816 this field will be modified as following
"refers_to": [
{
"name": "FishSource",
"url": "https://data.d4science.org/ctlg/GRSF_Admin/eb7ddeb0-ecc3-393b-b98a-fdb90be2c706"
}
]
This avoid the lookup and made the process faster.
// TODO move out from metadatafield
</comments> </comments>
</grsf> </grsf>
</extra> </extra>
@ -1151,6 +1390,15 @@
<dstTypes> <dstTypes>
<dstType>Field</dstType> <dstType>Field</dstType>
</dstTypes> </dstTypes>
<comments>
Example:
"sdg_flag": false
It generates the following Metadata:
"SDG Flag": "false"
https://data.d4science.org/ctlg/GRSF_Admin/00a1f849-1b43-3fd1-9255-1d1134c969bb
</comments>
</grsf> </grsf>
</extra> </extra>
</metadatafield> </metadatafield>
@ -1160,7 +1408,7 @@
<mandatory>false</mandatory> <mandatory>false</mandatory>
<dataType>String</dataType> <dataType>String</dataType>
<maxOccurs>1</maxOccurs> <maxOccurs>1</maxOccurs>
<remove>false</remove> <remove>true</remove>
<extra> <extra>
<grsf> <grsf>
<srcField>source_of_information</srcField> <srcField>source_of_information</srcField>
@ -1193,12 +1441,16 @@
E.g. Resource Name: "a name" - Resource Content: "http://...." E.g. Resource Name: "a name" - Resource Content: "http://...."
No metadata field is created. No metadata field is created.
Example: "source_of_information" : [{ Example:
"source_of_information" : [{
"name" : "http://www.bluebridge-vres.eu/ram/CSALMGOODNEWSR", "name" : "http://www.bluebridge-vres.eu/ram/CSALMGOODNEWSR",
"description" : "", "description" : "",
"url" : "http://www.bluebridge-vres.eu/ram/CSALMGOODNEWSR" "url" : "http://www.bluebridge-vres.eu/ram/CSALMGOODNEWSR"
}] }]
https://i-marine.d4science.org/group/grsf_admin/data-catalogue?path=/dataset/e7b40e6c-6767-3a9e-8745-bdb89d2703b5
https://data.d4science.org/ctlg/GRSF_Admin/e7b40e6c-6767-3a9e-8745-bdb89d2703b5
TODO move outside from metadatafield
</comments> </comments>
</grsf> </grsf>
</extra> </extra>