Claudio Atzori
10696f2a44
reverted procedure for creating the UsageCounts actionset
2024-10-01 15:54:13 +02:00
Claudio Atzori
5734b80861
Merge pull request 'datasource table creation split in steps' ( #489 ) from antonis.lempesis/dnet-hadoop:beta into beta
...
Reviewed-on: #489
2024-09-30 16:34:38 +02:00
Antonis Lempesis
f3c179658a
datasource table creation split in steps
2024-09-30 17:12:21 +03:00
Miriam Baglioni
b18ad035c1
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
2024-09-30 15:10:44 +02:00
Miriam Baglioni
e430826e00
[ImportOC] fix to move original folder instead of extracted ones
2024-09-30 15:10:10 +02:00
Claudio Atzori
3fcafc7ed6
Merge pull request 'Latest institutions in monitor dbs' ( #472 ) from antonis.lempesis/dnet-hadoop:beta into beta
...
Reviewed-on: #472
2024-09-26 09:49:01 +02:00
Miriam Baglioni
599e56dbc6
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
2024-09-25 17:28:23 +02:00
Claudio Atzori
6397141e56
code formatting
2024-09-25 15:27:32 +02:00
Claudio Atzori
e354f9853a
[OpenCitations] move the extracted contents under a backup path to avoid needing to re-download it in case of errors
2024-09-25 15:27:02 +02:00
Sandro La Bruzzo
6a097abc89
as described on ticket #9525
...
1. Changed the mapping applied to Crossref records: anything that has a relationship "is-review-of" must be mapped as publication of type "Review".
2. Force the hostedby of Crossref records with DOI prefix 10.3410 and 10.12703 to the H1 Connect data source.
2024-09-25 11:32:54 +02:00
Michele Artini
9754521847
Merge pull request 'fixed a bug with id' ( #486 ) from osfPreprints_plugin into beta
...
Reviewed-on: #486
2024-09-25 10:02:24 +02:00
Michele Artini
fa2532db30
fixed a bug with id
2024-09-25 09:38:50 +02:00
Michele Artini
54f8b4da39
Merge pull request 'fixed a bug with 'null' string' ( #484 ) from osfPreprints_plugin into beta
...
Reviewed-on: #484
2024-09-24 15:19:54 +02:00
Michele Artini
b35d046fd2
fixed a bug with 'null' string
2024-09-24 15:18:54 +02:00
Claudio Atzori
4f0463d779
[graph provision] person serialisation, limit the number of authorships and coauthorships before expanding the payloads
2024-09-24 14:54:34 +02:00
Miriam Baglioni
4d3e079590
Merge remote-tracking branch 'origin/beta' into beta
2024-09-24 14:26:29 +02:00
Claudio Atzori
d1cadc77c9
[graph provision] person serialisation, limit the number of authorships and coauthorships before expanding the payloads
2024-09-24 10:57:20 +02:00
Michele Artini
0e89d4a1cf
fixed a bug with topic ENRICH/MORE/SUBJECT/ARXIV
2024-09-24 08:57:49 +02:00
Michele Artini
e941adbe2b
fixed a bug with topic ENRICH/MORE/SUBJECT/ARXIV
2024-09-24 08:57:37 +02:00
Michele Artini
7f81673f3c
removed the deletedByInference=true filter
2024-09-23 15:27:43 +02:00
Michele Artini
fdbe629f49
removed the deletedByInference=true filter
2024-09-23 15:27:28 +02:00
Antonis Lempesis
619aa34a15
Merge branch 'beta' of https://code-repo.d4science.org/antonis.lempesis/dnet-hadoop into beta
2024-09-23 15:25:59 +03:00
Antonis Lempesis
dbea7a4072
removed duplicate line
2024-09-23 14:57:11 +03:00
Antonis Lempesis
c9241dba0d
Merge pull request 'convert_hive_to_spark_actions' ( #1 ) from convert_hive_to_spark_actions into beta
...
Reviewed-on: antonis.lempesis/dnet-hadoop#1
2024-09-23 13:53:28 +02:00
Claudio Atzori
e0ff84baf0
[graph provision] person serialisation, limit the number of authorships and coauthorships before expanding the payloads
2024-09-23 10:29:46 +02:00
Michele Artini
2d7a7a962d
unit test @Disabled
2024-09-23 10:19:36 +02:00
Michele Artini
6b0f7cc8b0
skip urls with authentication
2024-09-23 10:16:53 +02:00
Claudio Atzori
5f86c93be6
[graph provision] person serialisation
2024-09-20 12:20:00 +02:00
Michele Artini
339d8124f2
osf plugin: links to contributors and primaty_file
2024-09-20 08:44:05 +02:00
Michele Artini
52bb7af03b
use of dom4j
2024-09-19 14:59:05 +02:00
Michele Artini
9073b1159d
partial implementation of osfPreprints plugin + tests
2024-09-19 13:58:53 +02:00
Michele Artini
dcf09811a2
partial implementation of osfPreprints plugin
2024-09-19 12:42:45 +02:00
Claudio Atzori
23e0ab3a7c
run mergeResultsOfDifferentTypes only when checkDelegatedAuthority is true
2024-09-17 15:36:10 +02:00
Claudio Atzori
bfd05cdab2
run mergeResultsOfDifferentTypes only when checkDelegatedAuthority is true
2024-09-17 10:49:32 +02:00
Michele Artini
a2fac78dcc
fixed a problem in incremental harvesting
2024-09-17 10:16:28 +02:00
Michele Artini
99b7adda0c
gtr2 unit test
2024-09-16 15:13:44 +02:00
Michele Artini
bb9cee4f40
implementation of gtr2Publications plugin
2024-09-16 14:16:56 +02:00
Michele De Bonis
6df6b4583e
blacklist filtering moved before the cleanup phase in order to have case sensitive regex
2024-09-16 14:04:59 +02:00
Alessia
07e6e7b4d6
#9839 : include claimed affiliation relationships
2024-09-16 13:41:56 +02:00
Antonis Lempesis
37ad259296
cleanup
2024-09-05 16:02:44 +03:00
Antonis Lempesis
b64c144abf
added new institutions
2024-09-05 16:00:09 +03:00
Serafeim Chatzopoulos
b043f8a963
Remove redundant error messages from impact indicators workflow
2024-09-04 14:28:43 +03:00
Serafeim Chatzopoulos
db03f85366
Remove steps for updating BIP! from the impact indicators workflow
2024-09-04 14:25:44 +03:00
Miriam Baglioni
468f2aa5a5
[AffiliationAffRo]align beta with new affiliation from publisher webpage introduced in production. AffRo collectedfrom OpenAIRE to discriminate against WebCrawl
2024-08-12 18:10:46 +02:00
Miriam Baglioni
89fcf4086c
[Person]fix issue in affiliation relation id construction for person (missing ::)
2024-08-12 18:04:43 +02:00
Miriam Baglioni
45605f93ae
merging with branch beta
2024-08-12 18:03:10 +02:00
Miriam Baglioni
5a7ba77271
[Person]fix issue in affiliation relation id construction for person (missing ::)
2024-08-12 18:01:15 +02:00
Miriam Baglioni
8c185a7b1a
resolving conflicts
2024-08-05 17:14:11 +02:00
Claudio Atzori
e16616b964
added dataInfo to person records
2024-08-05 15:57:37 +02:00
Claudio Atzori
8e7ef79ce0
[bip affiliations] considers only DOI based records
2024-08-05 12:13:48 +02:00
Miriam Baglioni
985ca15264
[openaire-affiliation]removes matchings without DOI
2024-08-05 12:10:40 +02:00
Claudio Atzori
0bf76f2a34
[graph provision] added person to the graph2hive workflow
2024-08-05 09:35:07 +02:00
Claudio Atzori
975d44cac7
[graph provision] added person to the provision workflow
2024-08-02 16:14:10 +02:00
Claudio Atzori
fecbf93e0e
Merge pull request 'FoS L1 & L2' ( #465 ) from fos_l1l2 into beta
...
Reviewed-on: #465
2024-08-01 13:58:04 +02:00
Claudio Atzori
6bdb8643e6
ActionManager promote: allow to ingest person records in a graph that did not contain them, bumped dhp-schemas version
2024-07-31 11:02:22 +02:00
Claudio Atzori
9486e21a44
copy or process the person records throughout the graph pipeline
2024-07-30 14:25:31 +02:00
Claudio Atzori
64740475d0
depending on dhp-schemas:7.0.1
2024-07-29 11:51:42 +02:00
Miriam Baglioni
1af6571474
merging with branch beta
2024-07-25 15:48:05 +02:00
Claudio Atzori
a81c555fe6
[graph provision] include only FoS L1..L2 in the record serialization
2024-07-25 15:26:47 +02:00
Claudio Atzori
359b8ebda8
[graph provision] include only FoS L1..L2 in the record serialization
2024-07-25 15:22:29 +02:00
Miriam Baglioni
c7f6669f1a
[webcrawl] the blacklist is now in json and no more in csv after the normalization process
2024-07-25 15:20:18 +02:00
Miriam Baglioni
7cff281d3e
[webcrawl] the blacklist is now in json and no more in csv after the normalization process
2024-07-25 15:16:42 +02:00
Claudio Atzori
d4bf449e8c
minor
2024-07-25 14:53:06 +02:00
Miriam Baglioni
fc60661ac5
[webcrawl] added code and test (code/resource) to verify the deletion of the relations related to results put in blacklist
2024-07-25 12:25:14 +02:00
Claudio Atzori
d771a883f9
[dedup] updated sql query used to read organizations from the OpenOrgs DB to include their typology
2024-07-25 09:53:48 +02:00
Claudio Atzori
01958a3e07
[graph provision] addded filter to exclude records marked with datainfo.deletedbyinference = true
2024-07-24 10:00:10 +02:00
Miriam Baglioni
6f1801d7d1
[webcrawl]-
2024-07-23 17:34:48 +02:00
Miriam Baglioni
19806c2ae3
[SDG]fixed switch of methods
2024-07-23 17:12:55 +02:00
Antonis Lempesis
d0590e0e49
added latest institutions
2024-07-23 15:17:15 +03:00
Antonis Lempesis
7d2c0a3723
added new institutions
2024-07-23 15:10:17 +03:00
Miriam Baglioni
62649dc5c4
merging with branch beta
2024-07-23 12:50:12 +02:00
Miriam Baglioni
9573bf576d
[SDG]added code to ingest also the SDG without DOI
2024-07-23 12:47:57 +02:00
Michele Artini
d27e9ea50f
added ODF invisible stores in raw_all workflow
2024-07-23 09:56:27 +02:00
Michele De Bonis
4f4c73d65b
minor change: addition of missing parameter in sql query
2024-07-22 15:19:02 +02:00
Miriam Baglioni
79985ad197
[Crossref]added mapping for DFG versus the unidentified project [ https://support.openaire.eu/issues/9926?next_issue_id=9924&prev_issue_id=9927#note-4 ]
2024-07-17 18:30:24 +02:00
Claudio Atzori
06e3985b77
merged from beta
2024-07-17 12:01:40 +02:00
Claudio Atzori
83327239de
fixed pom definitions, bumped dependency version for the dhp-schema module, removed unnecessary dependencies
2024-07-17 11:58:48 +02:00
Claudio Atzori
db9c54c944
Revert "removed legacy actionmanager dependencies"
...
This reverts commit bb12d0b4df
.
2024-07-17 11:27:43 +02:00
Claudio Atzori
e39e8bbd47
Merge pull request '[WebCrawlAffiliation]remove from the creation of the action set the relations for pmc and pmid. Only doi are allowed' ( #462 ) from affiliationFromWebCrawlOnlyDOI into beta
...
Reviewed-on: #462
2024-07-17 11:12:32 +02:00
Claudio Atzori
e94ae771ff
Merge pull request '[BulkTag]added tagging for the organization relevant for the community.' ( #461 ) from tagOrganization into beta
...
Reviewed-on: #461
2024-07-17 11:11:52 +02:00
Claudio Atzori
78b5e4bb6f
reverted changed contens under dhp-graph-provision
2024-07-17 10:48:20 +02:00
Claudio Atzori
40c5d87645
Merge pull request '[graph provision] entity level contexts' ( #460 ) from entity_contexts into beta
...
Reviewed-on: #460
2024-07-17 10:43:21 +02:00
Claudio Atzori
a65241fcaf
Merge pull request 'implementation of the new collector plugin: research_fi' ( #456 ) from research_fi_collector_plugin into beta
...
Reviewed-on: #456
2024-07-17 10:25:38 +02:00
Claudio Atzori
6665976604
Merge pull request 'Optimizations for the Openorgs Dedup: normalization and inference of strings and implementation of new general-purpose comparators' ( #455 ) from openorgs_optimization into beta
...
Reviewed-on: #455
2024-07-17 10:25:20 +02:00
Claudio Atzori
c99f92efaa
Merge pull request '[beta] OpenAIRE Affiliation Inference' ( #452 ) from affRoFromRawString into beta
...
Reviewed-on: #452
2024-07-17 10:24:39 +02:00
Claudio Atzori
f17e1243ba
reverted changed contens under dhp-graph-provision
2024-07-17 10:23:50 +02:00
Claudio Atzori
6a19337dab
Merge pull request 'removed legacy actionmanager dependencies' ( #454 ) from cleanup_actionmanager_deps into beta
...
Reviewed-on: #454
2024-07-17 10:20:44 +02:00
Miriam Baglioni
d96215cb9b
[UnpayWall]added othe : in the identifier construction
2024-07-16 18:17:32 +02:00
Miriam Baglioni
9246bdec1c
[WebCrawlAffiliation]remove from the creation of the action set the relations for pmc and pmid. Only doi are allowed
2024-07-16 14:07:37 +02:00
Miriam Baglioni
9d27910144
[BulkTag]added tagging for the organization relevant for the community. Added test. Changed the tagging variables.
2024-07-16 13:48:48 +02:00
Claudio Atzori
beb93cdfe9
[graph provision] expand the context info for each entity type
2024-07-16 11:43:48 +02:00
Claudio Atzori
38f8ed27fd
[graph provision] log the Solr admin application operations for alias deletion and creation
2024-07-15 16:30:43 +02:00
Claudio Atzori
1fb44198fb
renamed workflow to better reflect its purpose
2024-07-15 15:24:38 +02:00
Claudio Atzori
6f6e85ddf4
code formatting
2024-07-15 09:32:04 +02:00
Claudio Atzori
7fa3d51200
renamed class, updated criteria to consider the ORCIDs used in the matchers
2024-07-15 09:18:58 +02:00
Michele Artini
f99fb21040
tests
2024-07-15 09:18:46 +02:00
Claudio Atzori
e17edb2581
[broker] fine tuned the workflow memory settings
2024-07-12 10:27:50 +02:00
Claudio Atzori
61d1fa9b9f
[metadata collection] added -Dcom.sun.security.enableAIAcaIssuers=true as a default for metadata collection
2024-07-12 10:26:45 +02:00
Claudio Atzori
f9ed2ae33c
[metadata collection] added the possibility to specify the JAVA_HOME and the JAVA_OPTS parameters
2024-07-11 15:32:36 +02:00
Michele Artini
bbe52584f7
log message
2024-07-11 15:14:34 +02:00