Miriam Baglioni
9246bdec1c
[WebCrawlAffiliation]remove from the creation of the action set the relations for pmc and pmid. Only doi are allowed
2024-07-16 14:07:37 +02:00
Claudio Atzori
38f8ed27fd
[graph provision] log the Solr admin application operations for alias deletion and creation
2024-07-15 16:30:43 +02:00
Claudio Atzori
1fb44198fb
renamed workflow to better reflect its purpose
2024-07-15 15:24:38 +02:00
Claudio Atzori
6f6e85ddf4
code formatting
2024-07-15 09:32:04 +02:00
Claudio Atzori
7fa3d51200
renamed class, updated criteria to consider the ORCIDs used in the matchers
2024-07-15 09:18:58 +02:00
Michele Artini
f99fb21040
tests
2024-07-15 09:18:46 +02:00
Claudio Atzori
e17edb2581
[broker] fine tuned the workflow memory settings
2024-07-12 10:27:50 +02:00
Claudio Atzori
61d1fa9b9f
[metadata collection] added -Dcom.sun.security.enableAIAcaIssuers=true as a default for metadata collection
2024-07-12 10:26:45 +02:00
Claudio Atzori
f9ed2ae33c
[metadata collection] added the possibility to specify the JAVA_HOME and the JAVA_OPTS parameters
2024-07-11 15:32:36 +02:00
Miriam Baglioni
814e650e12
[Irish Tender]changed the irish.json file according to comments #26 , #29 , and #34 for 9635
2024-07-04 12:24:28 +02:00
Michele De Bonis
a10e8d9f05
implementation of countryMatch and addition of workflow parameters
2024-06-28 16:46:52 +02:00
Claudio Atzori
14539f9c8b
[graph provision] publicFormat worfklow parameter defined as optional
2024-06-28 14:55:18 +02:00
Claudio Atzori
1bc8c5d173
[graph provision] fixed serialization of the instancetypes
2024-06-28 14:54:28 +02:00
Claudio Atzori
1ccf01cdb8
Using the updated Solr JSON payload model classes
2024-06-28 12:38:07 +02:00
Claudio Atzori
b79cb155ba
Merge pull request 'Fix permissions-issue in Stats-workflow, step22a-createPDFsAggregated.' ( #450 ) from antonis.lempesis/dnet-hadoop:beta into beta
...
Reviewed-on: D-Net/dnet-hadoop#450
2024-06-26 10:11:34 +02:00
Claudio Atzori
33a02c5b9e
Merge pull request 'Change the selection criteria for the pivot record of a group so that by best pid type becomes the first criteria. This will have the effect to converge to records having DOI pid' ( #446 ) from pivotselectionbypid into beta
...
Reviewed-on: D-Net/dnet-hadoop#446
2024-06-26 10:10:13 +02:00
Claudio Atzori
1c30eacac2
updated index feeding procedure to exploit the collection aliases
2024-06-25 15:27:38 +02:00
Claudio Atzori
6055212f77
merged from the json_payload branch
2024-06-25 12:39:02 +02:00
Claudio Atzori
0031cf849e
Merge branch 'beta' into 9872-create-solr-collection-aliases
2024-06-25 09:58:01 +02:00
Serafeim Chatzopoulos
9f6e16a03c
Add support to cretate/update solr collection aliases
2024-06-20 16:03:15 +03:00
Lampros Smyrnaios
66cd28f70a
- Fix not using the "export HADOOP_USER_NAME" statement in "createPDFsAggregated.sh", which caused permission-issues when creating tables with Impala.
...
- Remove unused "--user" parameter in "impala-shell" calls.
- Code polishing.
2024-06-20 14:33:46 +03:00
Miriam Baglioni
d35edac212
[IrishFunderList]make changed according to 9635 comment 20, 21, 22 and 23
2024-06-20 12:28:28 +02:00
Miriam Baglioni
6421f8fece
Merge remote-tracking branch 'origin/beta' into beta
2024-06-19 11:12:15 +02:00
Miriam Baglioni
ac270f795b
[IrishFunderList]make changed according to 9635 comment 14, 15 and 16
2024-06-19 11:11:52 +02:00
Lampros Smyrnaios
285416c74e
Merge branch 'beta' into beta
2024-06-18 13:50:38 +02:00
Lampros Smyrnaios
3095047e5e
Miscellaneous updates to the copying operation to Impala Cluster:
...
- Fix not breaking out of the VIEWS-infinite-loop when the "SHOULD_EXIT_WHOLE_SCRIPT_UPON_ERROR" is set to "false".
- Exit the script when no HDFS-active-node was found, independently of the "SHOULD_EXIT_WHOLE_SCRIPT_UPON_ERROR".
- Fix view_name-recognition in a log-message, by using the more advanced "Perl-Compatible Regular Expressions" in "grep".
- Add error-handling for "compute stats" errors.
2024-06-18 14:40:41 +03:00
Antonis Lempesis
0456f1b788
Merge remote-tracking branch 'origin/beta' into beta
2024-06-14 15:11:30 +03:00
Antonis Lempesis
38636942c7
filtering out deletedbyinference and invinsible results from accessroute
2024-06-14 15:11:19 +03:00
Lampros Smyrnaios
d942a1101b
Miscellaneous updates to the copying operation to Impala Cluster:
...
- Show some counts and the elapsed time for various sub-tasks.
- Code polishing.
2024-06-14 12:14:38 +03:00
Giambattista Bloisi
9bf2bda1c6
Fix: next returned a null value at end of stream
2024-06-12 13:28:51 +02:00
Giambattista Bloisi
d90cb099b8
Fix for paginationStart parameter management
2024-06-11 20:23:44 +02:00
Giambattista Bloisi
4f2a61e10f
Change the selection criteria for the pivot record of a group so that by best pid type becomes the first criteria. This will have the effect to slowly converge to records having DOI pid
2024-06-11 15:33:56 +02:00
Claudio Atzori
11fe3a4fe0
[graph resolution] use sparkExecutorMemory to define also the memoryOverhead
2024-06-11 14:21:17 +02:00
Miriam Baglioni
8fe934810f
Merge remote-tracking branch 'origin/beta' into beta
2024-06-11 10:28:51 +02:00
Miriam Baglioni
9da006e98c
[SDGFoSActionSet]remove datainfo for the result. It is not needed (qualifier.classid = UPDATE) useless since subject do not go at the level of the instance
2024-06-11 10:28:32 +02:00
Giambattista Bloisi
85c1eae7e0
Fixes for pagination strategy looping at end of download
2024-06-10 19:03:58 +02:00
Claudio Atzori
b0eba210c0
[actionset promotion] use sparkExecutorMemory to define also the memoryOverhead
2024-06-10 16:15:24 +02:00
Claudio Atzori
3776327a8c
hostedby patching to work with the updated Crossref contents, resolved conflict
2024-06-10 15:24:12 +02:00
Claudio Atzori
0139f23d66
Merge pull request 'organization type from OpenOrgs' ( #445 ) from import_openorg_type into beta
...
Reviewed-on: D-Net/dnet-hadoop#445
2024-06-07 12:17:31 +02:00
Michele Artini
c726572418
changed some parameters in OSF test
2024-06-07 12:03:26 +02:00
Claudio Atzori
ec79405cc9
[graph raw] set organization type from openorgs
2024-06-07 11:30:31 +02:00
Miriam Baglioni
1477406ecc
[bulkTag] fixed issue that made project disappear in graph_10_enriched
2024-06-06 10:45:41 +02:00
Claudio Atzori
92c3abd5a4
[graph cleaning] use sparkExecutorMemory to define also the memoryOverhead
2024-06-06 10:44:33 +02:00
Claudio Atzori
ce2364743a
applying changes from PR#442: Fix for missing collectedfrom after dedup
2024-06-06 10:43:43 +02:00
Claudio Atzori
f70dc76b61
minor
2024-06-06 10:43:10 +02:00
Claudio Atzori
73bd1938a5
[graph2hive] use sparkExecutorMemory to define also the memoryOverhead
2024-06-05 12:17:35 +02:00
Claudio Atzori
da5c1e73a4
Merge pull request 'Irish oaipmh exporter' ( #443 ) from irish-oaipmh-exporter into beta
...
Reviewed-on: D-Net/dnet-hadoop#443
2024-06-05 10:55:09 +02:00
Claudio Atzori
81090ad593
[IE OAIPHM] added oozie workflow, minor changes, code formatting
2024-06-05 10:03:33 +02:00
Claudio Atzori
a02f3f0d2b
code formatting
2024-05-30 10:21:18 +02:00
Alessia Bardi
05ee783c07
Merge branch 'beta' into dblp_collection_plugin
2024-05-29 16:04:39 +02:00