55f39f7850[graph provision] adds the possibility to validate the XML records before storing them via the validateXML parameterClaudio Atzori2024-05-09 14:06:04 +0200
39a2afe8b5[graph provision] fixed XML serialization of the usage counts measures, renamed workflow actions to better reflect their roleClaudio Atzori2024-05-09 13:54:42 +0200
908ed9da7aMerge pull request 'Various fixes in the stats wf' (#430) from antonis.lempesis/dnet-hadoop:beta into betaClaudio Atzori2024-05-08 13:41:02 +0200
0cada3cc8fevery step is run in the analytics queue. Hardcoded for now, will make a parameter later
#430
Antonis Lempesis2024-05-08 13:42:53 +0300
e1a0fb8933fixed id prefix creation for the fosnodoi recordsClaudio Atzori2024-05-03 14:14:18 +0200
69c5efbd8bFix: when applying enrichments with no instance information the resulting merge entity was generated with no instance instead of keeping the original informationGiambattista Bloisi2024-05-03 13:57:56 +0200
9cd3bc0f10Added a new generation of the dump for scholexplorer tested with last version of spark, and strongly refactoredSandro La Bruzzo2024-04-26 16:02:07 +0200
c08a58bba8Merge pull request 'Miscellaneous related to changes in MergeUtils' (#429) from misc_fixes_merge_entities into betaClaudio Atzori2024-04-24 08:55:37 +0200
1878199daeMiscellaneous fixes: - in Merge By ID pick by preference those records coming from delegated Authorities - fix various tests - close spark session in SparkCreateSimRelsGiambattista Bloisi2024-04-24 08:12:45 +0200
49af2e5740Miscellaneous updates to the copying operation to Impala Cluster: - Update the algorithm for creating views that depend on other views; overcome some bash-instabilities. - Upon any error, fail the whole process, not just the current DB-creation, as those errors usually indicate a bug in the initial DB-creation, that should be fixed immediately. - Enhance parallel-copy of large files by "hadoop distcp" command. - Reduce the "invalidate metadata" commands to just the current DB's tables, in order to eliminate the general overhead on Impala. - Show the number of tables and views in the logs. - Fix some log-messages.
#423
#248
#238
Lampros Smyrnaios2024-04-23 17:15:04 +0300
6189879643[NOAMI] removed entry for Irish Research eLibray (IReL) Care Board from the list of funders.Miriam Baglioni2024-04-23 11:09:18 +0200
c57cff2d6dMerge pull request '[WebCrawl] adding affiliation relations from web information' (#428) from WebCrowlBeta into betaClaudio Atzori2024-04-23 09:36:15 +0200
3a027e97a7[graph indexing] sets spark memoryOverhead in the join operations to the same value used for the memory executorClaudio Atzori2024-04-19 16:57:55 +0200
795e1b2629Merge pull request '[graph indexing] sets spark memoryOverhead in the join operations to the same value used for the memory executor' (#426) from provision_memoryOverhead into master
master
#59
Claudio Atzori2024-04-19 16:59:45 +0200
5ab8cd1794Various fixes for the stats DB update workflow, step16-createIndicatorsTables.sqlClaudio Atzori2024-04-18 11:28:18 +0200
8fdd0244adMerge pull request 'Various fixes for the stats DB update workflow, step16-createIndicatorsTables.sql' (#425) from stats_step16_fix into masterClaudio Atzori2024-04-18 11:25:24 +0200
0db7e4ae9aMerge pull request 'Refinements to PR #404: refactoring the Oaf records merge utilities into dhp-common' (#422) from revised_merge_logic into betaClaudio Atzori2024-04-17 11:58:26 +0200
589bce3520Merge pull request '[pBETA] Improvements to copying data from ocean to impala' (#421) from antonis.lempesis/dnet-hadoop:beta into betaClaudio Atzori2024-04-16 14:22:32 +0200