 Claudio Atzori
|
2e503ee101
|
code formatting
|
2020-12-17 13:47:38 +01:00 |
|
Claudio Atzori
|
5a3e2199b2
|
Merge pull request 'Creation of the action set to include the bipFinder! score' (#80) from miriam.baglioni/dnet-hadoop:bipFinder into bipFinder_master_test
|
2020-12-17 12:26:38 +01:00 |
|
Claudio Atzori
|
03319d3bd9
|
Revert "Merge pull request 'Creation of the action set to include the bipFinder! score' (#62) from miriam.baglioni/dnet-hadoop:bipFinder into master"
This reverts commit add7e1693b, reversing
changes made to f9a8fd8bbd.
|
2020-12-17 12:23:58 +01:00 |
|
Claudio Atzori
|
add7e1693b
|
Merge pull request 'Creation of the action set to include the bipFinder! score' (#62) from miriam.baglioni/dnet-hadoop:bipFinder into master
|
2020-12-17 12:09:03 +01:00 |
 Alessia Bardi
|
f9a8fd8bbd
|
updated test record for textgrid
|
2020-12-17 11:59:45 +01:00 |
|
Claudio Atzori
|
4766495f5b
|
[orcid_to_result_from_semrel_propagation] fixed typo in SQL
|
2020-12-17 09:15:50 +01:00 |
|
Claudio Atzori
|
de00094ebc
|
Merge pull request 'FIX on the creation of subject based broker enrichments' (#79) from broker into master
|
2020-12-15 14:58:31 +01:00 |
 Michele Artini
|
f9dc1e45fd
|
fixed a bug with invalid subject topic
|
2020-12-15 14:54:11 +01:00 |
 Sandro La Bruzzo
|
f92bd56f56
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-12-15 11:47:29 +01:00 |
|
Sandro La Bruzzo
|
1f6c8a9e83
|
added orcid_pending type to records coming from Crossref
|
2020-12-15 11:47:15 +01:00 |
|
Claudio Atzori
|
9f1181290e
|
Merge pull request 'broker' (#78) from broker into master
The changes look good to me.
|
2020-12-15 10:03:45 +01:00 |
|
Michele Artini
|
0a0f62bd01
|
Merge branch 'master' into broker
|
2020-12-15 08:30:52 +01:00 |
|
Michele Artini
|
12fa5d122a
|
fixed a problem with join
|
2020-12-15 08:30:26 +01:00 |
|
Michele Artini
|
991e675dc6
|
validation in claim rels
|
2020-12-14 15:41:25 +01:00 |
|
Michele Artini
|
3e19cf7b4a
|
openaireId
|
2020-12-14 15:24:33 +01:00 |
|
Claudio Atzori
|
b6f08ce226
|
re-adding the old junit:junit dep as solr-test-framework needs it
|
2020-12-14 15:07:31 +01:00 |
|
Claudio Atzori
|
7d325e2c57
|
using actual result subclasses instead of their parent class
|
2020-12-14 14:40:54 +01:00 |
|
Claudio Atzori
|
152916890f
|
renamed test name
|
2020-12-14 14:40:05 +01:00 |
|
Michele Artini
|
a203aee32a
|
ES wf properties
|
2020-12-14 12:02:33 +01:00 |
|
Claudio Atzori
|
1506f49052
|
Xml record serialization for author PIDs: 1) only one value per PID type is allowed; 2) orcid prevails over orcid_pending
|
2020-12-14 11:14:03 +01:00 |
|
Michele Artini
|
d03756c962
|
mkdir of output dir
|
2020-12-14 11:11:41 +01:00 |
|
Michele Artini
|
399548f221
|
whitelist of topics
|
2020-12-14 11:03:55 +01:00 |
|
Michele Artini
|
38da1c282a
|
Merge branch 'master' into broker
|
2020-12-14 09:14:02 +01:00 |
Dimitris
|
dc9c2f3272
|
Commit 12122020
|
2020-12-12 12:00:14 +02:00 |
|
Claudio Atzori
|
61cd129ded
|
XML serialisation test
|
2020-12-11 12:44:53 +01:00 |
|
Claudio Atzori
|
ce7a319e01
|
using the correct assertion import
|
2020-12-11 12:44:17 +01:00 |
|
Claudio Atzori
|
7fe2433137
|
excluded transitive older junit dependencies, they can compromise the unit test executions
|
2020-12-11 12:42:55 +01:00 |
|
Michele Artini
|
933b4c1ada
|
workingDir and outputDir
|
2020-12-10 14:47:51 +01:00 |
|
Michele Artini
|
2e7df07328
|
workingDir and outputDir
|
2020-12-10 14:47:22 +01:00 |
|
Michele Artini
|
94bfed1c84
|
gzipped output
|
2020-12-10 11:59:28 +01:00 |
 Miriam Baglioni
|
b7adbc7c3e
|
merge branch with master
|
2020-12-10 10:35:27 +01:00 |
|
Alessia Bardi
|
112da6d76a
|
in theory, just auto-formatting after mvn compile
|
2020-12-09 20:00:27 +01:00 |
|
Alessia Bardi
|
bece04b330
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2020-12-09 19:54:43 +01:00 |
|
Alessia Bardi
|
426b76ee8e
|
more asserts for TextGrid record
|
2020-12-09 19:46:11 +01:00 |
|
Claudio Atzori
|
ff72fcd91a
|
allow orcid_pending to be percolate to the XML graph serialization
|
2020-12-09 19:04:50 +01:00 |
|
Claudio Atzori
|
4705144918
|
Merge pull request 'rel_project_validation' (#69) from rel_project_validation into master
LGTM
|
2020-12-09 19:01:20 +01:00 |
|
Claudio Atzori
|
211aa04726
|
allow orcid_pending to be percolate to the XML graph serialization
|
2020-12-09 18:08:51 +01:00 |
|
Claudio Atzori
|
ada21ad920
|
Merge pull request 'dump of the results related to at least one project' (#61) from miriam.baglioni/dnet-hadoop:dump into master
LGTM
|
2020-12-09 17:22:56 +01:00 |
|
Michele Artini
|
1bc9adc10d
|
default trust for validated rels
|
2020-12-09 16:18:37 +01:00 |
|
Michele Artini
|
5f21a356fd
|
reindent
|
2020-12-09 11:24:30 +01:00 |
|
Michele Artini
|
370a5e650b
|
validation attributes in resultProject relations
|
2020-12-09 11:18:26 +01:00 |
|
Claudio Atzori
|
27e96767e0
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2020-12-07 21:53:22 +01:00 |
|
Claudio Atzori
|
fba11eef2a
|
cleanup
|
2020-12-07 21:53:13 +01:00 |
|
Sandro La Bruzzo
|
7f8b93de72
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-12-07 19:59:39 +01:00 |
|
Sandro La Bruzzo
|
302baab67b
|
fixed doiboost mapping and workflows
|
2020-12-07 19:59:33 +01:00 |
|
Michele Artini
|
d6934f370e
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-12-07 14:56:23 +01:00 |
|
Michele Artini
|
5de8a7276f
|
wf to partition opendoar events
|
2020-12-07 14:56:06 +01:00 |
|
Claudio Atzori
|
5e8509bef7
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2020-12-07 13:50:08 +01:00 |
|
Claudio Atzori
|
026ad40633
|
disabled test
|
2020-12-07 13:50:01 +01:00 |
|
Sandro La Bruzzo
|
620e585b63
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-12-07 10:42:53 +01:00 |
|
Sandro La Bruzzo
|
b31dd126fb
|
fixed crossref workflow added common ORCID Class
|
2020-12-07 10:42:38 +01:00 |
|
Claudio Atzori
|
a104a632df
|
cleanup
|
2020-12-04 16:32:47 +01:00 |
|
Claudio Atzori
|
5b4e1142a8
|
Merge pull request 'added last step to update cache' (#64) from antonis.lempesis/dnet-hadoop:master into master
Looks good to me, thanks!
|
2020-12-04 14:42:31 +01:00 |
 Antonis Lempesis
|
b1ed1afdcc
|
added the new parameter (stats_tool_api_url) in the workflow parameters
|
2020-12-04 13:07:18 +02:00 |
|
Antonis Lempesis
|
7cb113e088
|
added the new parameter (stats_tool_api_url) in the workflow parameters
|
2020-12-04 13:04:25 +02:00 |
|
Antonis Lempesis
|
d23ccae0d5
|
ignoring deletedbyinference relations
|
2020-12-04 12:42:17 +02:00 |
|
Miriam Baglioni
|
5fb65ffc4a
|
merge branch with master
|
2020-12-03 11:24:35 +01:00 |
|
Miriam Baglioni
|
ea88dc3401
|
fixed issue in property name
|
2020-12-03 11:24:23 +01:00 |
|
Miriam Baglioni
|
4c58bd1c93
|
merge with upstream
|
2020-12-03 11:24:00 +01:00 |
|
Miriam Baglioni
|
05c452f58d
|
merge with upstream
|
2020-12-03 10:26:45 +01:00 |
|
Antonis Lempesis
|
413afcfed5
|
finished first implementation of wf
|
2020-12-02 15:57:17 +02:00 |
|
Antonis Lempesis
|
0948536614
|
initial implementation of the promote wf
|
2020-12-02 15:41:56 +02:00 |
|
Sandro La Bruzzo
|
7da679542f
|
fixed wrong projectId
|
2020-12-02 14:28:09 +01:00 |
|
Sandro La Bruzzo
|
6ba8037cc7
|
fixed failure to test due to changing of input
|
2020-12-02 11:34:46 +01:00 |
|
Claudio Atzori
|
cfb55effd9
|
code formatting
|
2020-12-02 11:23:49 +01:00 |
|
Claudio Atzori
|
74242e450e
|
using constants from ModelConstants
|
2020-12-02 11:23:35 +01:00 |
|
Miriam Baglioni
|
d5efa6963a
|
using constants in ModelCOnstants
|
2020-12-02 11:20:26 +01:00 |
|
Miriam Baglioni
|
cd285e98bc
|
usoing the constants defined in the ModelConstants class
|
2020-12-02 11:13:23 +01:00 |
|
Miriam Baglioni
|
4b0d1530a2
|
merge upstream
|
2020-12-02 11:05:00 +01:00 |
|
Claudio Atzori
|
faa977df7e
|
Merge pull request 'orcid-no-doi' (#43) from enrico.ottonello/dnet-hadoop:orcid-no-doi into master
The dataset was generated and is now part of the actionsets available in BETA
|
2020-12-02 10:55:12 +01:00 |
|
Claudio Atzori
|
57f448b7a4
|
graph cleaning workflow separate orcid_pending from orcid, depending on the author pid provenance
|
2020-12-02 10:44:05 +01:00 |
|
Alessia Bardi
|
2d15667b4a
|
testing XML generation from json object (case AMS ACTA)
|
2020-12-02 10:16:26 +01:00 |
|
Alessia Bardi
|
a417624670
|
tests for raw graph mapping
|
2020-12-02 10:15:26 +01:00 |
|
Miriam Baglioni
|
f8468c9c22
|
added extention for new author pid (orcid_pending)
|
2020-12-01 20:09:35 +01:00 |
|
Miriam Baglioni
|
888175baf7
|
added java doc
|
2020-12-01 18:36:29 +01:00 |
|
Miriam Baglioni
|
3d62d99d5d
|
fixed issue in workflow variable
|
2020-12-01 15:02:49 +01:00 |
|
Miriam Baglioni
|
17680296b9
|
removed unnecessary variable and unused method
|
2020-12-01 15:02:31 +01:00 |
|
Miriam Baglioni
|
5b3ed70808
|
refactoring
|
2020-12-01 14:31:34 +01:00 |
|
Miriam Baglioni
|
62ff4999e3
|
added workflow and last step of collection and save
|
2020-12-01 14:30:56 +01:00 |
|
Miriam Baglioni
|
45d06c45c7
|
collecting all the atoic actions for result type and save them all in the AS path
|
2020-12-01 14:29:18 +01:00 |
|
Miriam Baglioni
|
0051ebede5
|
extending test
|
2020-12-01 12:43:03 +01:00 |
|
Miriam Baglioni
|
719da15f04
|
added test resources
|
2020-12-01 12:42:30 +01:00 |
|
Miriam Baglioni
|
db36e11912
|
classes test classes and resources for production of the actionset to include bipFinder score in results
|
2020-11-30 20:14:23 +01:00 |
 Enrico Ottonello
|
f2df3ead74
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop into orcid-no-doi
|
2020-11-30 14:22:46 +01:00 |
|
Enrico Ottonello
|
40c4559e92
|
added datainfo on authors pid with "sysimport:crosswalk:entityregistry",
|
2020-11-30 14:19:22 +01:00 |
|
Antonis Lempesis
|
815d6b25d9
|
added last step to update cache
|
2020-11-30 00:48:10 +02:00 |
|
Claudio Atzori
|
e731a7658d
|
cleaning texts to remove tab characters too
|
2020-11-27 09:00:04 +01:00 |
|
Claudio Atzori
|
a104d2b6ad
|
cleanup
|
2020-11-26 11:12:00 +01:00 |
|
Miriam Baglioni
|
124591a7f3
|
refactoring
|
2020-11-25 18:23:28 +01:00 |
|
Miriam Baglioni
|
1a89f8211c
|
#61 (comment)
|
2020-11-25 18:12:40 +01:00 |
|
Miriam Baglioni
|
5fbe54ef54
|
#61 (comment)
|
2020-11-25 18:10:28 +01:00 |
|
Miriam Baglioni
|
ed01e5a5e1
|
#61 (comment)
|
2020-11-25 18:09:34 +01:00 |
|
Miriam Baglioni
|
d4ddde2ef2
|
changed because of #61 (comment)
|
2020-11-25 18:01:01 +01:00 |
|
Miriam Baglioni
|
f5e5e92a10
|
changed because of #61 (comment)
|
2020-11-25 17:58:53 +01:00 |
|
Miriam Baglioni
|
1df94b85b4
|
changed because of #61 (comment)
|
2020-11-25 17:57:43 +01:00 |
|
Claudio Atzori
|
db0181b8af
|
Merge pull request 'added bidirectionality to relations from project and result coming from crossref' (#60) from miriam.baglioni/dnet-hadoop:sxBidirectionality into master
|
2020-11-25 17:17:40 +01:00 |
|
Sandro La Bruzzo
|
ec3e238de6
|
Fixed problem on duplicated identifier
|
2020-11-25 17:15:54 +01:00 |
|
Miriam Baglioni
|
90d4369fd2
|
added test to verify the compression in writing community info on hdfs
|
2020-11-25 14:34:58 +01:00 |
|
Miriam Baglioni
|
6750e33d69
|
merge branch with master
|
2020-11-25 14:09:01 +01:00 |
|
Miriam Baglioni
|
b2c455f883
|
added java doc
|
2020-11-25 14:08:09 +01:00 |
|
Miriam Baglioni
|
1f130cdf92
|
changed the relation (produces -> isProducedBy) due to the change in the code
|
2020-11-25 14:04:26 +01:00 |
|
Miriam Baglioni
|
e758d5d9b4
|
refactoring
|
2020-11-25 13:46:39 +01:00 |
|
Miriam Baglioni
|
87a9f616ae
|
refactoring and addition of the funder nsp first part as nome for the dump insteasd of the whole nsp
|
2020-11-25 13:45:41 +01:00 |
|
Miriam Baglioni
|
e7e418e444
|
added decision node to verify if to upload in Zenodo
|
2020-11-25 13:44:10 +01:00 |
|
Miriam Baglioni
|
305e3d0c9c
|
added resource file for relation with relClass = isProducedBy
|
2020-11-25 13:43:41 +01:00 |
|
Miriam Baglioni
|
21ce175d17
|
added FilterFunction specification if filter operation
|
2020-11-25 13:42:31 +01:00 |
|
Miriam Baglioni
|
bde6d337dd
|
test classes for dump of results related to funders
|
2020-11-25 13:42:01 +01:00 |
|
Miriam Baglioni
|
b37b9352d7
|
added constant value for semantic relationship between projects and results
|
2020-11-25 13:41:08 +01:00 |
|
Sandro La Bruzzo
|
264723ffd8
|
updated stuff for zenodo upload
|
2020-11-25 11:56:07 +01:00 |
|
Claudio Atzori
|
eeebd5a920
|
Cleanig workflow: remove newlines from titles, descriptions, subjects
|
2020-11-24 18:40:25 +01:00 |
|
Enrico Ottonello
|
99a086f0c6
|
max concurrent executors set to 10, according to ORCID Director of Technology mail request
|
2020-11-24 17:49:32 +01:00 |
|
Miriam Baglioni
|
72bb0fe360
|
changed directory name
|
2020-11-24 16:47:07 +01:00 |
|
Miriam Baglioni
|
00874a8ce6
|
added bidirectionality to relations from project and result
|
2020-11-24 15:17:23 +01:00 |
|
Miriam Baglioni
|
39f4a20873
|
chenged the path and the name for saving the communities_infrastructures dump file
|
2020-11-24 14:47:32 +01:00 |
|
Miriam Baglioni
|
7e14452a87
|
final versione of the wf to get the dump of results associated to at least one funder per funder
|
2020-11-24 14:46:34 +01:00 |
|
Miriam Baglioni
|
c167a18057
|
added new parameter for the dumpType
|
2020-11-24 14:45:50 +01:00 |
|
Miriam Baglioni
|
54a309bb6b
|
refactoring
|
2020-11-24 14:45:30 +01:00 |
|
Miriam Baglioni
|
35ecea8842
|
changed to consider the modification for the specification of the type of dump
|
2020-11-24 14:45:15 +01:00 |
|
Miriam Baglioni
|
b9b6bdb2e6
|
fixing issue on previous implementation
|
2020-11-24 14:44:53 +01:00 |
|
Miriam Baglioni
|
7e940f1991
|
changed to consider the modification for the specification of the type of dump
|
2020-11-24 14:43:34 +01:00 |
|
Miriam Baglioni
|
62928ef7a5
|
changed to save the communities_infrastructures information as the other entity dumps: in a json.gz file
|
2020-11-24 14:42:41 +01:00 |
|
Miriam Baglioni
|
3319440c53
|
changed the direction of the relation between projects and result considered to select the results linked to projects
|
2020-11-24 14:41:09 +01:00 |
|
Miriam Baglioni
|
00c377dac2
|
added specification of MapFunction types in map
|
2020-11-24 14:40:22 +01:00 |
|
Miriam Baglioni
|
44db258dc4
|
added enumerated for the dump type
|
2020-11-24 14:38:06 +01:00 |
|
Miriam Baglioni
|
1832708c42
|
modified boolean variable with string one whcih specify the type of dump we are performing: complete, community or funder
|
2020-11-24 14:37:36 +01:00 |
|
Enrico Ottonello
|
5c17e768b2
|
set wf configuration with spark.dynamicAllocation.maxExecutors 20 over 20 input partitions
|
2020-11-23 16:01:23 +01:00 |
|
Enrico Ottonello
|
5c9a727895
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop into orcid-no-doi
|
2020-11-23 09:49:53 +01:00 |
|
Enrico Ottonello
|
97c8111847
|
action to convert lambda file in seq file; spark action to download updated authors
|
2020-11-23 09:49:22 +01:00 |
|
Miriam Baglioni
|
259c67ce36
|
fixed issue in path name
|
2020-11-20 12:32:23 +01:00 |
|
Miriam Baglioni
|
0a9db67eec
|
-
|
2020-11-20 12:21:33 +01:00 |
|
Miriam Baglioni
|
d362f2637d
|
merge branch with master
|
2020-11-19 19:17:20 +01:00 |
|
Miriam Baglioni
|
cf3f47563f
|
new parameter files
|
2020-11-19 19:16:05 +01:00 |
|
Miriam Baglioni
|
24c56fa7a3
|
new logic and workflow for dump of results with link to projects. In this implementation the result match the model of the communityresult.
|
2020-11-19 19:15:39 +01:00 |
|
Claudio Atzori
|
d48f388fb2
|
Merge branch 'provision_indexing'
|
2020-11-19 15:59:55 +01:00 |
|
Claudio Atzori
|
46bde9c13f
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2020-11-19 15:26:27 +01:00 |
|
Claudio Atzori
|
7c9feaf9e7
|
project attributes removed from the XML record serialization: contactfullname, contactfax, contactphone, contactemail
|
2020-11-19 15:26:20 +01:00 |
|
Michele Artini
|
293da47ad9
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-11-19 10:42:31 +01:00 |
|
Michele Artini
|
ab08d12c46
|
considering abstract > MIN_LENGTH in ENRICH_MISSING_ABSTRACT
|
2020-11-19 10:42:10 +01:00 |
|
Claudio Atzori
|
e503271abe
|
fixed notification workflow name
|
2020-11-19 10:41:38 +01:00 |
|
Claudio Atzori
|
0374d34c3e
|
introduced configuration param outputFormat: HDFS | SOLR
|
2020-11-19 10:34:28 +01:00 |
|
Miriam Baglioni
|
fafb688887
|
-
|
2020-11-18 18:56:48 +01:00 |
|
Miriam Baglioni
|
906db690d2
|
-
|
2020-11-18 17:43:08 +01:00 |
|
Claudio Atzori
|
ede7fae6c8
|
Merge pull request 'XML record indexing test' (#58) from provision_indexing into master
|
2020-11-18 17:04:34 +01:00 |
|
Miriam Baglioni
|
5402062ff5
|
changed parameter file with the ono associated to the job
|
2020-11-18 16:58:20 +01:00 |
|
Miriam Baglioni
|
a172a37ad1
|
fixed typo
|
2020-11-18 16:55:07 +01:00 |
|
Miriam Baglioni
|
46ba3793f6
|
code, workflow and parameters for the dump of the results associated to funders
|
2020-11-18 16:47:31 +01:00 |
|
Claudio Atzori
|
5218718e8b
|
updated set of fields from the MDFormatDSResourceType on PROD
|
2020-11-18 15:00:41 +01:00 |
|
Claudio Atzori
|
d9e07a242b
|
extended XmlIndexingJob to accept an optional parameter: outputPath. When present, forces the job to write its output on the specified HDFS location
|
2020-11-18 14:34:55 +01:00 |
|
Claudio Atzori
|
29dcff0f34
|
spark complains about missing classes, so here they are again
|
2020-11-18 14:32:32 +01:00 |
|
Miriam Baglioni
|
57cac36898
|
changed the workflow name
|
2020-11-18 13:38:03 +01:00 |