Claudio Atzori
|
741bc99c47
|
Merge branch 'beta' into datasource_pdf_consent
|
2022-03-28 09:20:48 +02:00 |
Claudio Atzori
|
61319b2e83
|
updated dhp-schema version; set entity-level dataInfo before & after merging the fields from the group of duplicates
|
2022-03-25 16:38:33 +01:00 |
Antonis Lempesis
|
d8503cd191
|
added moooar organizations
|
2022-03-24 14:02:36 +02:00 |
Miriam Baglioni
|
7b8f85692e
|
[Enrichment country] fixed issues with parameters and workflow args
|
2022-03-23 17:20:23 +01:00 |
Claudio Atzori
|
48d32466e4
|
instances grouped by URL expose only one refereed
|
2022-03-23 14:52:03 +01:00 |
Claudio Atzori
|
f10066547b
|
increased spark.sql.shuffle.partitions in affiliation_from_semrel_propagation
|
2022-03-23 12:22:26 +01:00 |
Claudio Atzori
|
43733c1a18
|
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
|
2022-03-23 12:14:27 +01:00 |
Enrico Ottonello
|
f11dfc51f7
|
fix resolved url format, added alternate identifier from original pid
|
2022-03-22 16:39:21 +01:00 |
Antonis Lempesis
|
62f91b0869
|
cleanup
|
2022-03-22 16:17:49 +02:00 |
Antonis Lempesis
|
2e8394ecf8
|
creating aaall tables as parquet
|
2022-03-22 16:16:08 +02:00 |
Antonis Lempesis
|
dcfbeb8142
|
yet more typos
|
2022-03-21 12:36:03 +02:00 |
Miriam Baglioni
|
89fd275480
|
[HostedByMap] added left over from PR and fixed issue on workflow
|
2022-03-21 09:54:45 +01:00 |
Enrico Ottonello
|
afe84c4244
|
added subjects to oaf generation
|
2022-03-18 18:10:39 +01:00 |
Enrico Ottonello
|
db831e6f43
|
removed dynamic allocation on wf
|
2022-03-18 17:43:53 +01:00 |
Enrico Ottonello
|
861f2a3306
|
added titles merging title page and protein identifier
|
2022-03-18 14:51:57 +01:00 |
Enrico Ottonello
|
f43bfdb594
|
added subjects
|
2022-03-17 19:24:07 +01:00 |
miconis
|
c763aded70
|
dependency updated to the new pace-core version
|
2022-03-16 16:41:50 +01:00 |
Enrico Ottonello
|
3ef5eec3a6
|
added bmuse and rdfconverter modules - added repository for bmuse jars
|
2022-03-16 12:07:36 +01:00 |
Enrico Ottonello
|
41284ec2f9
|
retrieving vocabulary terms from nquads
|
2022-03-16 11:26:50 +01:00 |
Enrico Ottonello
|
e53a606afc
|
added date of collection, resource type as workflow parameter
|
2022-03-15 17:36:48 +01:00 |
miconis
|
c959639bd5
|
dependency updated to the new pace-core version
|
2022-03-15 16:33:03 +01:00 |
Miriam Baglioni
|
0f7d8ca2e0
|
[HostedByMap] change on master to align to PR 201 on beta merged as 9f3036c847
|
2022-03-11 15:16:02 +01:00 |
Claudio Atzori
|
f430029596
|
cleanup
|
2022-03-11 14:28:28 +01:00 |
Miriam Baglioni
|
12de9acb0d
|
[Country Propagation] left out from previous commit
|
2022-03-11 14:17:02 +01:00 |
Miriam Baglioni
|
2fbb35ade5
|
mergin with branch beta
|
2022-03-11 13:58:10 +01:00 |
Miriam Baglioni
|
4437f9345d
|
[Country Propagation] left out from previous commit
|
2022-03-11 13:57:47 +01:00 |
Miriam Baglioni
|
2b643059fa
|
[Country Propagation] changed the logic to get the collectedfrom at the result level. To fix issue when no instance is created for a result that should have the country associated. Change the code to use spark instead of hive to prepare the data needed for the propagation step. Added new tests for the intermediate steps and new verification for the propagation itself
|
2022-03-11 13:56:48 +01:00 |
Claudio Atzori
|
f25407bbe2
|
added mapping for datasource consent fields to integrate them in the graph
|
2022-03-11 09:32:42 +01:00 |
Miriam Baglioni
|
2c5087d55a
|
[HostedByMap] download of doaj from json, modification of test resources, deletion of class no more needed for the CSV download
|
2022-03-04 15:18:21 +01:00 |
Miriam Baglioni
|
5d608d6291
|
[HostedByMap] changed the model to include also oaStart date and review process that could be possibly used in the future
|
2022-03-04 11:06:09 +01:00 |
Miriam Baglioni
|
b7c2340952
|
[HostedByMap - DOIBoost] changed to use code moved to common since used also from hostedbymap now
|
2022-03-04 11:05:23 +01:00 |
Miriam Baglioni
|
8a41f63348
|
[HostedByMap] update to download the json instead of the csv
|
2022-03-04 10:38:43 +01:00 |
Miriam Baglioni
|
44b0c03080
|
[HostedByMap] update to download the json instead of the csv
|
2022-03-04 10:37:59 +01:00 |
Enrico Ottonello
|
bd37f14941
|
added working ocean configuration
|
2022-03-03 14:38:21 +01:00 |
Enrico Ottonello
|
29ee1b9d82
|
added datasource key to workflow parameter to properly choose collected from and id values
|
2022-03-03 12:31:29 +01:00 |
Antonis Lempesis
|
ad78e505da
|
yet another fix
|
2022-03-03 12:28:12 +02:00 |
Enrico Ottonello
|
e57216a1fa
|
added oozie workflow to generate bioschema dataset on hdfs
|
2022-03-02 16:58:10 +01:00 |
Miriam Baglioni
|
3be8737c32
|
[graph-stats] fixed query after the change in the indicator table related to PR#200
|
2022-03-02 14:09:05 +01:00 |
Miriam Baglioni
|
3970651ee1
|
Merge pull request 'fixed query after the change in the indicator table' (#200) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: D-Net/dnet-hadoop#200
|
2022-03-02 14:05:58 +01:00 |
Antonis Lempesis
|
efeeebfee1
|
fixed query after the change in the indicator table
|
2022-03-02 13:29:25 +02:00 |
Enrico Ottonello
|
f28d7e3b9d
|
added spark dataset creation
|
2022-03-02 12:12:37 +01:00 |
Enrico Ottonello
|
8f281846a4
|
update test file
|
2022-02-28 13:37:53 +01:00 |
Enrico Ottonello
|
f833de8a75
|
added relatedIdentifierType, fix IsCitedBy value
|
2022-02-28 13:37:28 +01:00 |
Enrico Ottonello
|
7f9636ef00
|
added alternateIdentifiers to oaf
|
2022-02-25 14:42:08 +01:00 |
Claudio Atzori
|
580d904aae
|
manually merging PR#199 D-Net/dnet-hadoop#199
|
2022-02-25 12:22:50 +01:00 |
Claudio Atzori
|
1932a65d1c
|
Merge pull request '[Stats wf] sprint 6 indicators' (#198) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: D-Net/dnet-hadoop#198
|
2022-02-25 12:09:18 +01:00 |
Miriam Baglioni
|
f5b0a6f89c
|
[master to beta] fixed issues in test files
|
2022-02-25 10:21:57 +01:00 |
miconis
|
8991d097b4
|
bug fix in the DedupRecordFactory, DataInfo set before merge
|
2022-02-24 17:13:12 +01:00 |
miconis
|
fe1c966cbf
|
Merge branch 'master_202203' of code-repo.d4science.org:D-Net/dnet-hadoop into master_202203
|
2022-02-24 17:08:38 +01:00 |
miconis
|
b0f369dc78
|
bug fix in the DedupRecordFactory, DataInfo set before merge
|
2022-02-24 17:08:24 +01:00 |
Enrico Ottonello
|
2f5caef77b
|
resolution of generated relations url to uniprot and pubmed datasources
|
2022-02-24 16:59:50 +01:00 |
Miriam Baglioni
|
859cb7ac9d
|
[DoiBoost AR] changed test resource to be sure the result will always have EMBARGO as value for AccessRight
|
2022-02-24 16:55:32 +01:00 |
Miriam Baglioni
|
a40b59b7d5
|
[ResultToOrgFromInstRepoTest] fixed issue in model of the input resources
|
2022-02-24 16:05:57 +01:00 |
Claudio Atzori
|
66c09b1bc7
|
code formatting
|
2022-02-24 12:58:07 +01:00 |
Claudio Atzori
|
a87c070447
|
conflicts resolved, merged from beta
|
2022-02-24 12:51:31 +01:00 |
Claudio Atzori
|
86cdb7a38f
|
[provision] serialize measures defined on the result level
|
2022-02-23 15:54:18 +01:00 |
Alessia Bardi
|
9d6203f79b
|
test mapping datasource
|
2022-02-23 15:00:53 +01:00 |
Antonis Lempesis
|
3b92a2ab9c
|
added the rest of spring 6 in monitor db
|
2022-02-23 12:05:57 +02:00 |
Antonis Lempesis
|
87c91f70a2
|
added sprint 6 indicators to monitor db
|
2022-02-22 14:41:48 +02:00 |
Enrico Ottonello
|
4975278558
|
added modules for downloading data with bmuse and converting to datacite json format
|
2022-02-22 12:53:19 +01:00 |
Enrico Ottonello
|
2bc79c50f8
|
mapping bioschema to oaf
|
2022-02-22 11:46:29 +01:00 |
Enrico Ottonello
|
446f81ee60
|
wf to generate oaf from bioschema json datacite
|
2022-02-22 11:42:57 +01:00 |
Claudio Atzori
|
5226d0a100
|
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
|
2022-02-18 15:21:07 +01:00 |
Claudio Atzori
|
99f5b14469
|
[graph raw] invisible records stored among the raw graph rather than the claimed subgraph
|
2022-02-18 15:20:57 +01:00 |
Claudio Atzori
|
401dd38074
|
code formatting
|
2022-02-18 15:19:15 +01:00 |
Claudio Atzori
|
cf8443780e
|
added processingchargeamount to the result view
|
2022-02-18 15:17:48 +01:00 |
Sandro La Bruzzo
|
891781ee3f
|
Merge branch 'beta' of code-repo.d4science.org:D-Net/dnet-hadoop into beta
|
2022-02-18 11:11:32 +01:00 |
Sandro La Bruzzo
|
d3f03abd51
|
fixed wrong json path
|
2022-02-18 11:11:17 +01:00 |
Claudio Atzori
|
89c7313fc5
|
Merge branch 'beta' into hierarchical_orgs_relations
|
2022-02-17 10:30:04 +01:00 |
dimitrispie
|
58c59f46eb
|
Added Sprint 6
|
2022-02-17 10:21:09 +02:00 |
Antonis Lempesis
|
393a4ee956
|
fixed yet another typo...
|
2022-02-15 12:56:50 +02:00 |
Sandro La Bruzzo
|
3aa2020b24
|
added script to regenerate hostedBy Map following instruction defined on ticket #7539
updated hosted By Map
|
2022-02-15 11:05:27 +01:00 |
Miriam Baglioni
|
be64055cfe
|
[OpenCitation] changed the name of destination folders
|
2022-02-14 15:49:44 +01:00 |
Miriam Baglioni
|
1490867cc7
|
[OpenCitation] cleaning of the COCI model
|
2022-02-14 14:52:12 +01:00 |
Miriam Baglioni
|
c191080965
|
mergin with branch beta
|
2022-02-14 14:49:39 +01:00 |
Alessia Bardi
|
600ede1798
|
serialisation of APCs int he XML records
|
2022-02-11 11:00:20 +01:00 |
Miriam Baglioni
|
5c4043dba8
|
[OpenCitation] refactoring
|
2022-02-08 16:23:05 +01:00 |
Miriam Baglioni
|
759ed519f2
|
[OpenCitation] added logic to avoid the genration of self citations relations
|
2022-02-08 16:15:34 +01:00 |
Miriam Baglioni
|
b071f8e415
|
[OpenCitation] change to extract in json format each folder just onece
|
2022-02-08 15:37:28 +01:00 |
Miriam Baglioni
|
fbc28ee8c3
|
[OpenCitation] change the integration logic to consider dois with commas inside
|
2022-02-07 18:32:08 +01:00 |
Miriam Baglioni
|
78be2975f0
|
[stats-wf]fixed another typo related to PR#193
|
2022-02-07 11:22:08 +01:00 |
Miriam Baglioni
|
1f8302dc37
|
Merge pull request '[stats-wf]fixed yet another typo' (#193) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: D-Net/dnet-hadoop#193
|
2022-02-07 11:19:26 +01:00 |
Antonis Lempesis
|
5f762cbd09
|
fixed yet another typo
|
2022-02-07 12:09:12 +02:00 |
Alessia Bardi
|
ac8b8f224f
|
Merge branch 'beta' into extendResult
|
2022-02-04 16:43:27 +01:00 |
Miriam Baglioni
|
493caef358
|
[stats-wf]fixed the result_result table related to PR#191
|
2022-02-04 14:51:25 +01:00 |
Miriam Baglioni
|
0547fd6ee7
|
Merge pull request '[stats-wf]fixed the result_result table' (#191) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: D-Net/dnet-hadoop#191
|
2022-02-04 14:47:31 +01:00 |
Antonis Lempesis
|
ae633c566b
|
fixed the result_result table
|
2022-02-04 15:04:19 +02:00 |
Miriam Baglioni
|
aae667e6b6
|
[APC at the result level] added the APC at the level of the result and modified test class
|
2022-02-04 12:34:25 +01:00 |
Sandro La Bruzzo
|
bcfdf9a0d7
|
iis repository with https
|
2022-02-03 16:49:31 +01:00 |
Miriam Baglioni
|
3c60e53a96
|
[stats-wf]fixed the result_result creation for monitor PR#190 on beta
|
2022-02-03 14:47:08 +01:00 |
Miriam Baglioni
|
89922156c9
|
Merge pull request '[stats-wf]fixed the result_result creation for monitor' (#190) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: D-Net/dnet-hadoop#190
|
2022-02-03 13:00:56 +01:00 |
Antonis Lempesis
|
c2b44530a3
|
typo...
|
2022-02-03 13:44:07 +02:00 |
Antonis Lempesis
|
dbd2646d59
|
fixed the result_result creation for monitor
|
2022-02-03 12:37:10 +02:00 |
Alessia Bardi
|
2e215abfa8
|
test for instances with URLs for OpenAPC
|
2022-02-02 17:27:44 +01:00 |
Miriam Baglioni
|
37784209c9
|
[dhp-schemas-] updated the version of dhp-schema to 2.10.27 for APC name and id modification
|
2022-02-02 12:46:31 +01:00 |
Miriam Baglioni
|
73eba34d42
|
[UnresolvedEntities] Changed the way to merge the unresolved because the new merge removed the dataInfo from the merged result. Added also data info for subjects
|
2022-02-01 08:38:41 +01:00 |
Miriam Baglioni
|
dce7f5fea8
|
[BULK TAGGING] changed to fix issue that should have been fixed already
|
2022-01-31 08:20:28 +01:00 |
Claudio Atzori
|
8eb75ca169
|
adapted GenerateEntitiesApplicationTest behaviour
|
2022-01-27 16:24:37 +01:00 |
Claudio Atzori
|
af61e44acc
|
ported changes to the GraphCleaningFunctionsTest from 8de9788308
|
2022-01-27 16:19:14 +01:00 |
Claudio Atzori
|
1322379741
|
Merge branch 'beta' into delegated_authorities
|
2022-01-25 14:28:25 +01:00 |
Claudio Atzori
|
59a250337c
|
[graph resolution] drop output path at the beginning
|
2022-01-24 18:02:39 +01:00 |
Claudio Atzori
|
97ad94d7d9
|
[graph resolution] drop output path at the beginning
|
2022-01-24 18:02:07 +01:00 |
Claudio Atzori
|
8de9788308
|
applied fix for avoiding ruling out the invisible (APC) records during the graph cleaning
|
2022-01-24 11:29:22 +01:00 |
Claudio Atzori
|
2f385b3ac6
|
updated dnet workflow profile definitions
|
2022-01-21 13:59:46 +01:00 |
Claudio Atzori
|
dd52bf1bb8
|
copy relations to the graphOutputPath
|
2022-01-21 13:59:29 +01:00 |
Claudio Atzori
|
4983d6536d
|
Merge branch 'beta' into delegated_authorities
|
2022-01-21 13:02:48 +01:00 |
Claudio Atzori
|
f0ea2410e5
|
improved mapping titles from datacite records to consider title types
|
2022-01-21 10:50:34 +01:00 |
Claudio Atzori
|
b37bc277c4
|
reintroduced the hostedby patching to the datacite records
|
2022-01-21 09:15:13 +01:00 |
Claudio Atzori
|
f2fde5566b
|
using helper method from ModelSupport to find the inverse relation descriptor
|
2022-01-20 09:19:07 +01:00 |
Claudio Atzori
|
3b9020c1b7
|
added unit test for the DispatchEntitiesJob
|
2022-01-19 18:15:55 +01:00 |
Claudio Atzori
|
abfa9c6045
|
code formatting
|
2022-01-19 17:17:11 +01:00 |
Claudio Atzori
|
391aa1373b
|
added unit test
|
2022-01-19 17:13:21 +01:00 |
Claudio Atzori
|
44a937f4ed
|
factored out entity grouping implementation, extended to consider results from delegated authorities rather than identical records from other sources
|
2022-01-19 12:24:52 +01:00 |
Miriam Baglioni
|
a7c4d0d16d
|
[DoiBoost Organizations] added parameter to specify the action in the wf raw_organizations to be able to load the openorgs organization as in the loading step for the construction of the graph
|
2022-01-13 13:52:00 +01:00 |
Miriam Baglioni
|
a75fb8c47a
|
[BipFinderInstanceLevel] change pom to align to the dhp-schema release 2.10.24 and refactoring
|
2022-01-12 18:06:26 +01:00 |
Miriam Baglioni
|
e7d5a39c03
|
[BipFinderInstanceLevel] added tests in test class
|
2022-01-12 17:25:04 +01:00 |
Miriam Baglioni
|
4993666d73
|
[BipFinderInstanceLevel] changed creation of the instance to allow to enrich existing instances with same pid
|
2022-01-12 16:53:47 +01:00 |
Claudio Atzori
|
9acc32faa6
|
[stats wf] final touches for the integration of PRs #166, #179 in the master branch
|
2022-01-12 12:04:31 +01:00 |
dimitrispie
|
b053b0178e
|
Sprint 5 and other changes
|
2022-01-12 11:23:37 +01:00 |
Antonis Lempesis
|
b6b4bc0df9
|
added first indicator of sprint 5
|
2022-01-12 11:20:28 +01:00 |
Antonis Lempesis
|
e91f06f39b
|
fixed typos in indicators. Added extra views in monitor
|
2022-01-12 11:18:28 +01:00 |
Antonis Lempesis
|
3ce1976627
|
fixed column names
|
2022-01-12 11:14:41 +01:00 |
Antonis Lempesis
|
4878d7485c
|
added usage stats
|
2022-01-12 11:13:25 +01:00 |
Antonis Lempesis
|
a4316bafed
|
fixed a typo
|
2022-01-12 11:12:53 +01:00 |
Antonis Lempesis
|
bb17e070d8
|
added result_result relations
|
2022-01-12 11:09:38 +01:00 |
Claudio Atzori
|
a30a98a716
|
Applying PR#166 in the master branch (Added sprint 3&4 of indicators). Merge commit '0df9574a6f5d9d75bc840decb023561ae941f9d6'
|
2022-01-12 10:57:19 +01:00 |
Sandro La Bruzzo
|
57e2c4b749
|
formatted code
|
2022-01-12 09:40:28 +01:00 |
Claudio Atzori
|
0f2144b5e0
|
scalafmt: code formatting
|
2022-01-11 17:03:44 +01:00 |
Claudio Atzori
|
dcd282977c
|
pulled from beta
|
2022-01-11 16:59:41 +01:00 |
Claudio Atzori
|
4f212652ca
|
scalafmt: code formatting
|
2022-01-11 16:57:48 +01:00 |
Sandro La Bruzzo
|
0163dadb7f
|
[doiboost]
- update MAG schema, new filed added on version dec-2021
|
2022-01-11 11:05:44 +01:00 |
Miriam Baglioni
|
904e1c2667
|
Merge pull request 'Affiliation Propagation through semantic relation' (#183) from enrichment into beta
Reviewed-on: D-Net/dnet-hadoop#183
|
2022-01-07 19:18:16 +01:00 |
Miriam Baglioni
|
064f9bbd87
|
[AFFPropSR] added new paprameter for the number of iterations and new code for just one iteration
|
2022-01-07 18:58:51 +01:00 |
Miriam Baglioni
|
b7e450070b
|
[SDG-FOS] to import SDG file not considering the header
|
2022-01-07 12:13:26 +01:00 |
Miriam Baglioni
|
639190370a
|
mergin with branch beta
|
2022-01-07 11:29:25 +01:00 |
Miriam Baglioni
|
adccc2346a
|
[SDG-FOS] to lower case for the doi
|
2022-01-07 11:28:50 +01:00 |
Claudio Atzori
|
8ae46ca789
|
OAF-store-graph mdstores: firther fix for PR#180
|
2022-01-05 15:52:15 +01:00 |
Claudio Atzori
|
908294d86e
|
OAF-store-graph mdstores: firther fix for PR#180
|
2022-01-05 15:49:05 +01:00 |
Claudio Atzori
|
3bd3653be9
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 16:39:39 +01:00 |
Claudio Atzori
|
3dc48c7ab5
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 16:39:27 +01:00 |
Claudio Atzori
|
f82db765db
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 16:39:15 +01:00 |
Claudio Atzori
|
8d13effa31
|
test for the tolerant deserialisation utility method
|
2022-01-04 16:38:26 +01:00 |
Claudio Atzori
|
9458ee7938
|
serialise records in the OAF-store-graph mdstores in json format. Read them again in the graph construction phase using a tolerant parser to support backward compatible changes in the evolution of the schema
|
2022-01-04 16:38:09 +01:00 |
Claudio Atzori
|
58f8998e3d
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 15:02:09 +01:00 |
Claudio Atzori
|
174c3037e1
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 14:40:16 +01:00 |
Claudio Atzori
|
045d767013
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 14:23:01 +01:00 |
Claudio Atzori
|
bd59b58efb
|
test for the tolerant deserialisation utility method
|
2022-01-04 11:26:56 +01:00 |
Claudio Atzori
|
a6977197b3
|
serialise records in the OAF-store-graph mdstores in json format. Read them again in the graph construction phase using a tolerant parser to support backward compatible changes in the evolution of the schema
|
2022-01-03 17:25:26 +01:00 |
Miriam Baglioni
|
4c60ee1718
|
mergin with branch beta
|
2022-01-03 15:24:02 +01:00 |
Miriam Baglioni
|
92fd69e25d
|
[SDG-FOS] alternative way to get input data to avoid OOM error while getting csv
|
2022-01-03 15:23:06 +01:00 |