Claudio Atzori
|
c4a23c2f4d
|
fix: preserving the old identifier among the originalIds in the doiboost construction process, trying to avoid UnsupportedOperationException while adding elements to the originalIds
|
2021-05-19 16:01:52 +02:00 |
Claudio Atzori
|
ba03f549d7
|
fix: preserving the old identifier among the originalIds in the doiboost construction process
|
2021-05-19 15:43:26 +02:00 |
Claudio Atzori
|
2cbf15f4fb
|
using ModelConstants
|
2021-05-17 09:54:45 +02:00 |
Claudio Atzori
|
f19feceaf0
|
set the old identifier before switching to the new one
|
2021-05-14 12:53:40 +02:00 |
Claudio Atzori
|
1bd70fa2c6
|
preserving the old identifier among the originalIds in the doiboost construction process
|
2021-05-14 11:30:41 +02:00 |
Claudio Atzori
|
ca3f3a7687
|
using ModelConstants
|
2021-05-14 11:29:49 +02:00 |
Claudio Atzori
|
23b8883ab1
|
applied intellij code cleanup
|
2021-05-14 10:58:12 +02:00 |
Claudio Atzori
|
5afa7d3e0c
|
core utilities in dhp-common moved in external module dhp-schemas
|
2021-04-27 15:44:01 +02:00 |
Sandro La Bruzzo
|
a16e5299f9
|
applied unique function on the final dataset
|
2021-04-16 17:36:48 +02:00 |
Sandro La Bruzzo
|
67085da305
|
fixed NPE
|
2021-04-16 11:05:58 +02:00 |
Sandro La Bruzzo
|
7d6a80e2f2
|
added new type on MAG mapping
|
2021-04-16 09:14:15 +02:00 |
Sandro La Bruzzo
|
479abd10cb
|
Add into ORCID workflow a method that extracts orcid directly to the dump generated by Enrico
|
2021-04-13 17:47:43 +02:00 |
Claudio Atzori
|
e686b8de8d
|
[ORCID-no-doi] integrating PR#98 #98
|
2021-04-01 17:11:03 +02:00 |
Claudio Atzori
|
ee34cc51c3
|
[ORCID-no-doi] integrating PR#98 #98
|
2021-04-01 17:07:49 +02:00 |
Claudio Atzori
|
7941d7be29
|
WIP: using common definitions from ModelConstants
|
2021-03-31 18:33:57 +02:00 |
Sandro La Bruzzo
|
616d2ecce2
|
splitted workflow collecting datacite into two workflows.
Released on beta
|
2021-03-31 15:45:58 +02:00 |
Sandro La Bruzzo
|
1dfda3624e
|
improved workflow importing datacite
|
2021-03-26 13:56:29 +01:00 |
Sandro La Bruzzo
|
625e4c29c4
|
added model constants
|
2021-03-23 09:39:56 +01:00 |
Sandro La Bruzzo
|
c392936b97
|
fixed error on best access right
|
2021-03-23 09:23:22 +01:00 |
Sandro La Bruzzo
|
c73072079d
|
fix conflicts
|
2021-03-22 16:36:31 +01:00 |
Sandro La Bruzzo
|
098914dcff
|
fix wrong relation with source null
|
2021-03-22 11:35:02 +01:00 |
Sandro La Bruzzo
|
25d5663d97
|
added filter
|
2021-03-18 10:24:42 +01:00 |
Sandro La Bruzzo
|
5f98ea74a9
|
Added fix for pid generation in stableIds
|
2021-03-17 15:53:24 +01:00 |
Sandro La Bruzzo
|
cc5bbafa5d
|
some fix to make workflows runs
|
2021-03-17 12:12:56 +01:00 |
Sandro La Bruzzo
|
4bb3bcafa5
|
add author sequence number
|
2021-03-11 11:32:32 +01:00 |
Sandro La Bruzzo
|
a8e5d0ea0d
|
updated test and fixed assign of access right
|
2021-03-11 10:41:24 +01:00 |
Sandro La Bruzzo
|
f5e7c57654
|
Fixed ticket 6282
|
2021-03-11 10:32:45 +01:00 |
Claudio Atzori
|
d525785497
|
[#6282 open access status in the Graph] Result.Instance.accessRight defined with dedicated data type that includes the open access color.
|
2021-03-09 11:12:55 +01:00 |
Sandro La Bruzzo
|
a2169ccf07
|
// implemented Ticket #6281 added pid to Instance in doiBoost
|
2021-03-09 10:46:36 +01:00 |
Claudio Atzori
|
8d2bb24512
|
merged from master
|
2021-03-08 15:44:34 +01:00 |
Sandro La Bruzzo
|
99cf3a8ea4
|
Merged Datacite transfrom into this branch
|
2021-01-28 16:34:46 +01:00 |
Claudio Atzori
|
8879704ba0
|
[DOIBoost] configurable ES server url and index name in crossref importer
|
2021-01-05 10:00:13 +01:00 |
Sandro La Bruzzo
|
7834a35768
|
avoid to save intermediate dataset before generation of Sequence file
|
2021-01-04 17:54:57 +01:00 |
Sandro La Bruzzo
|
8765020b85
|
minor fix
|
2021-01-04 17:37:08 +01:00 |
Sandro La Bruzzo
|
b0dc92786f
|
defined a single oozie workflow for the generation of doiboost
|
2021-01-04 17:01:35 +01:00 |
Claudio Atzori
|
28460c2cd1
|
using com.fasterxml.jackson.databind.ObjectMapper instead of org.codehaus.jackson.map.ObjectMapper
|
2020-12-23 16:59:52 +01:00 |
Sandro La Bruzzo
|
1f6c8a9e83
|
added orcid_pending type to records coming from Crossref
|
2020-12-15 11:47:15 +01:00 |
Claudio Atzori
|
d9532446eb
|
imported more diffs from master branch; code formatting
|
2020-12-10 16:14:16 +01:00 |
Claudio Atzori
|
12e2f930c8
|
resolved conflicts
|
2020-12-10 10:57:39 +01:00 |
Sandro La Bruzzo
|
302baab67b
|
fixed doiboost mapping and workflows
|
2020-12-07 19:59:33 +01:00 |
Sandro La Bruzzo
|
b31dd126fb
|
fixed crossref workflow added common ORCID Class
|
2020-12-07 10:42:38 +01:00 |
Sandro La Bruzzo
|
7da679542f
|
fixed wrong projectId
|
2020-12-02 14:28:09 +01:00 |
Claudio Atzori
|
cfb55effd9
|
code formatting
|
2020-12-02 11:23:49 +01:00 |
Enrico Ottonello
|
f2df3ead74
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop into orcid-no-doi
|
2020-11-30 14:22:46 +01:00 |
Enrico Ottonello
|
40c4559e92
|
added datainfo on authors pid with "sysimport:crosswalk:entityregistry",
|
2020-11-30 14:19:22 +01:00 |
Claudio Atzori
|
db0181b8af
|
Merge pull request 'added bidirectionality to relations from project and result coming from crossref' (#60) from miriam.baglioni/dnet-hadoop:sxBidirectionality into master
|
2020-11-25 17:17:40 +01:00 |
Sandro La Bruzzo
|
ec3e238de6
|
Fixed problem on duplicated identifier
|
2020-11-25 17:15:54 +01:00 |
Enrico Ottonello
|
99a086f0c6
|
max concurrent executors set to 10, according to ORCID Director of Technology mail request
|
2020-11-24 17:49:32 +01:00 |
Miriam Baglioni
|
00874a8ce6
|
added bidirectionality to relations from project and result
|
2020-11-24 15:17:23 +01:00 |
Enrico Ottonello
|
5c17e768b2
|
set wf configuration with spark.dynamicAllocation.maxExecutors 20 over 20 input partitions
|
2020-11-23 16:01:23 +01:00 |
Enrico Ottonello
|
97c8111847
|
action to convert lambda file in seq file; spark action to download updated authors
|
2020-11-23 09:49:22 +01:00 |
Enrico Ottonello
|
c0c2e05eae
|
added wf to extracting authors and works xml data from orcid dump to hdfs; added wf to download the lamda file (containing last orcid update informations) from orcid to hdfs
|
2020-11-17 18:23:12 +01:00 |
Enrico Ottonello
|
005f849674
|
added compression to output dataset
|
2020-11-13 12:45:31 +01:00 |
Enrico Ottonello
|
13f28fa225
|
moved AuthorData to dhp-schemas; added other names to author data
|
2020-11-12 17:43:32 +01:00 |
Enrico Ottonello
|
1f861f2b0d
|
now wf output is a sequence file with the format seq("eu.dnetlib.dhp.schema.oaf.Publication",eu.dnetlib.dhp.schema.action.AtomicActions)
|
2020-11-11 17:38:50 +01:00 |
Enrico Ottonello
|
1513174d7e
|
added further test case
|
2020-11-10 11:44:55 +01:00 |
Sandro La Bruzzo
|
8e1d43aab2
|
Implemented ID generation using IdentifierRecordFactory on DOIBoost
|
2020-11-09 11:53:55 +01:00 |
Enrico Ottonello
|
6bc7dbeca7
|
first version of dataset successful generated from orcid dump 2020
|
2020-11-06 13:47:50 +01:00 |
Enrico Ottonello
|
9818e74a70
|
added dependency version in main pom.xml for orcid no doi
|
2020-10-22 16:38:00 +02:00 |
Enrico Ottonello
|
210a50e4f4
|
replaced null value
|
2020-10-22 16:24:42 +02:00 |
Enrico Ottonello
|
1139d6568d
|
replaced null value with a more safe empty string as return value
|
2020-10-22 15:32:26 +02:00 |
Enrico Ottonello
|
c58db1c8ea
|
added filter on null value after map function
|
2020-10-22 15:11:02 +02:00 |
Enrico Ottonello
|
846ba30873
|
if typologies mapping fails, an exception will be propagated
|
2020-10-22 14:36:18 +02:00 |
Enrico Ottonello
|
c3114ba0ae
|
replaced null as return value with a more safe empty string
|
2020-10-22 14:21:31 +02:00 |
Enrico Ottonello
|
c295c71ca0
|
added comment
|
2020-10-22 14:07:26 +02:00 |
Enrico Ottonello
|
ab083f9946
|
propagate exception on parsing work (PR request)
|
2020-10-22 14:02:32 +02:00 |
sandro
|
3a81a940b7
|
solved bug on merge publication
|
2020-10-21 22:41:55 +02:00 |
Sandro La Bruzzo
|
cd9c377d18
|
adpted scholexplorer Dump generation to the new Dataset definition
|
2020-10-08 10:10:13 +02:00 |
Sandro La Bruzzo
|
c4a3c52e45
|
fixed Doiboost bug in the identifier
|
2020-10-01 15:46:44 +02:00 |
Enrico Ottonello
|
a97ad20c7b
|
exception is now propagated (PR review)
|
2020-09-22 10:46:34 +02:00 |
Enrico Ottonello
|
9e8e7fe6ef
|
add comments
|
2020-09-15 11:32:49 +02:00 |
Enrico Ottonello
|
0377b40fba
|
output to one parquet file
|
2020-07-30 18:38:07 +02:00 |
Enrico Ottonello
|
196f36c6ed
|
fix publication dataset creation
|
2020-07-30 13:38:33 +02:00 |
Enrico Ottonello
|
c82b15b5f4
|
migrate configuration to ocean, fix publication dataset creation
|
2020-07-28 15:23:52 +02:00 |
Enrico Ottonello
|
ca37d3427b
|
separate workflow to parse orcid summaries, activities and generate dataset with no doi publications; test
|
2020-07-03 23:30:31 +02:00 |
Enrico Ottonello
|
1729cc5cf3
|
publication conversion from json to oaf test
|
2020-07-02 18:46:20 +02:00 |
Enrico Ottonello
|
5525f57ec8
|
converter from orcid work json to oaf
|
2020-07-01 18:36:14 +02:00 |
Enrico Ottonello
|
b7b6be12a5
|
fixed enriched works generation
|
2020-06-29 18:03:16 +02:00 |
Enrico Ottonello
|
b2213b6435
|
merged with dnet version
|
2020-06-26 17:27:34 +02:00 |
Enrico Ottonello
|
c5e149c46e
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop into orcid-no-doi
|
2020-06-26 16:15:38 +02:00 |
Enrico Ottonello
|
d6498278ed
|
added workflow to generate seq(orcidId,work) and seq(orcidId,enrichedWork)
|
2020-06-25 18:43:29 +02:00 |
Sandro La Bruzzo
|
a6c0faac70
|
added test to verify secondary sorting
|
2020-06-25 10:48:15 +02:00 |
Enrico Ottonello
|
fcbb4c1489
|
parser of orcid publication data from xml original dump
|
2020-06-24 16:29:32 +02:00 |
Claudio Atzori
|
67c7b31ba6
|
Merge branch 'master' into graph_cleaning
|
2020-06-10 15:00:35 +02:00 |
Claudio Atzori
|
a2fdf85ba1
|
WIP: graph cleaner implementation
|
2020-06-09 19:52:53 +02:00 |
Alessia Bardi
|
2d3f7d1eb4
|
fixed log classes to make the ORCID test run
|
2020-06-09 18:07:14 +02:00 |
Alessia Bardi
|
fc4d220964
|
updated function name for SNSF
|
2020-06-09 17:05:31 +02:00 |
Alessia Bardi
|
d6de406e11
|
fixed classid for subjects
|
2020-06-09 14:43:34 +02:00 |
Alessia Bardi
|
f072125152
|
map volume and issue in journal information from MAG
|
2020-06-09 14:32:10 +02:00 |
Alessia Bardi
|
b7cb1163ea
|
identifiers always start with 50
|
2020-06-09 10:39:11 +02:00 |
Alessia Bardi
|
9fd25887f7
|
Result identifiers all start with 50|
|
2020-06-08 19:32:24 +02:00 |
Alessia Bardi
|
16cb073b15
|
set the instance datepfacceptance with the Crossref createdDate in case the issuedDate is blank
|
2020-06-08 19:06:03 +02:00 |
Sandro La Bruzzo
|
7ac1ba2e35
|
improvement DOIBoost
|
2020-06-04 14:39:20 +02:00 |
Sandro La Bruzzo
|
13815d5d13
|
improvement DOIBoost
|
2020-06-01 17:52:12 +02:00 |
Sandro La Bruzzo
|
b87b3ddb6b
|
changed mapping ORCIDToOAF
|
2020-05-29 09:32:04 +02:00 |
Sandro La Bruzzo
|
7d29b61c62
|
code refactor
|
2020-05-28 09:57:46 +02:00 |
Sandro La Bruzzo
|
25f52e19a4
|
implemented generation of ActionSet
|
2020-05-26 09:15:33 +02:00 |
Sandro La Bruzzo
|
2408083566
|
implemented filtering step
|
2020-05-23 08:46:49 +02:00 |
Sandro La Bruzzo
|
147dd389bf
|
minor fix
|
2020-05-22 20:51:42 +02:00 |
Sandro La Bruzzo
|
22936d0877
|
Merge branch 'doiboost' of code-repo.d4science.org:D-Net/dnet-hadoop into doiboost
|
2020-05-22 15:15:17 +02:00 |