dnet-hadoop

Commit Graph

Select branches

Hide Pull Requests

beta

convert_hive_to_spark_actions

deduptesting

dhp_oaf_model

dump

forceOrginalId_claims

graph_cleaning

graph_db

hive

master

monitor_dbs

observatory_country

organization_pids

promotion-prod-only

stable_ids

unique_field_in_lists

usage-stats-export-wf-v2

wf-promotion

#1

1.0.1

archive/bipFinder_master_test

archive/dedupTest

archive/islookup_timeout

archive/junit5

archive/master_pre_stable_ids

dhp-1.0.2

dhp-1.0.3

dhp-1.0.4

dhp-1.1.5

dhp-1.1.6

dhp-1.1.7

dhp-1.2.0

dhp-1.2.1

dhp-1.2.2

dhp-1.2.3

dhp-1.2.4

7b7520850b fixed an invalid char Michele Artini 2023-01-11 08:35:53 +0100
4953ae5649 fixed an invalid char Michele Artini 2023-01-11 08:35:53 +0100
d6895f0387 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta Miriam Baglioni 2023-01-09 17:28:38 +0100
c60d3a2b46 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Miriam Baglioni 2023-01-09 17:28:27 +0100
7a7eb30a3f Changes 05012023 dimitrispie 2023-01-05 10:37:33 +0200
0654343479 Added usage-stats-promote dimitrispie 2023-01-05 10:36:32 +0200
becb242c17 Monitor DB only Workflow dimitrispie 2023-01-04 16:50:29 +0200
dcb958e146 Changes to execute the stats wf only in hive dimitrispie 2023-01-04 11:39:01 +0200
7becdaf31d Merge pull request 'Workaround to use new version of intellij on Master' (#266) from master_intellij into master Claudio Atzori 2022-12-23 10:32:21 +0100
18a7aa2d78 Merge pull request 'Workaround to use new version of intellij on Beta' (#267) from beta_intellij into beta Claudio Atzori 2022-12-23 10:32:01 +0100
592013d5dd Added more steps in decision node dimitrispie 2022-12-23 09:43:16 +0200
2a4bf32d4c Merge branch 'hive' of https://code-repo.d4science.org/antonis.lempesis/dnet-hadoop into hive dimitrispie 2022-12-22 10:22:46 +0200
6449ff4207 1. Added a decision node to enables the workflow to make a selection on the execution path to follow 2. Added new organization 3. Added 5 new tables from Eurostast dimitrispie 2022-12-22 10:18:21 +0200
b713132db7 [Cleaning] adding missing classes Miriam Baglioni 2022-12-21 12:49:08 +0100
8893389895 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta Miriam Baglioni 2022-12-21 12:42:27 +0100
11f2b470d3 [Cleaning] adding missing classes Miriam Baglioni 2022-12-21 12:42:19 +0100
c8309fe18e addded command line params to allow hive actions to run Antonis Lempesis 2022-07-29 16:36:20 +0300
028873cc51 added new hive opts Antonis Lempesis 2022-04-29 12:49:27 +0300
1ddea4f442 removed 'stored as parquet' from views.. Antonis Lempesis 2021-12-13 23:05:00 +0200
2754c3dd62 moving data to impala cluster and creating shadow databases there Antonis Lempesis 2021-12-13 16:26:14 +0200
778a1a724f finished migration to hive only Antonis Lempesis 2021-11-30 19:01:48 +0200
e84dd5fe26 first Antonis Lempesis 2021-11-26 15:40:40 +0200
3c9826f186 updated lines function to it's implementation linesWithSeparators.map(l => l.stripLineEnd) in this way we force scala plugin compiler to consider this pipeline scala code and not java.string.lines() pipeline Sandro La Bruzzo 2022-12-21 11:21:17 +0100
91c70b15a5 updated lines function to it's implementation linesWithSeparators.map(l => l.stripLineEnd) in this way we force scala plugin compiler to consider this pipeline scala code and not java.string.lines() pipeline Sandro La Bruzzo 2022-12-21 11:14:42 +0100
f910b7379d [cleaning] recovering missing resources from D-Net/dnet-hadoop#265 Claudio Atzori 2022-12-21 09:26:34 +0100
33bdad104e [cleaning] align parameter names Claudio Atzori 2022-12-20 21:43:59 +0100
6aa91204a5 [orcid propagation] skip empty directories Claudio Atzori 2022-12-20 14:15:46 +0100
5816ded93f code formatting Claudio Atzori 2022-12-20 10:41:40 +0100
46972f8393 [orcid propagation] skip empty directory Claudio Atzori 2022-12-20 10:28:22 +0100
9cf0a98699 [cleaning] set the common subject classid/name Claudio Atzori 2022-12-20 10:17:33 +0100
da85ca697d Merge pull request 'cleanCountryOnMaster' (#265) from cleanCountryOnMaster into master Claudio Atzori 2022-12-16 15:58:44 +0100
059e100ec7 [Clean Country] moving other resources for testing purposes Miriam Baglioni 2022-12-16 15:48:21 +0100
fc95a550c3 [Clean Country] moving other resources for testing purposes Miriam Baglioni 2022-12-16 15:46:32 +0100
6901ac91b1 [Clean Country] moving source and resources to master Miriam Baglioni 2022-12-16 15:42:49 +0100
6674cccb94 [BulkTag] description of parameters more comprehensive for those who do not implement it Miriam Baglioni 2022-12-16 15:33:20 +0100
f37113a941 [BulkTag] moving xquery to get community configuration in dedicated file Miriam Baglioni 2022-12-16 15:32:26 +0100
8685eaa706 [Clean Country] added test to verify remove of country Miriam Baglioni 2022-12-16 15:31:25 +0100
dc0ec88a58 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta Miriam Baglioni 2022-12-16 13:18:32 +0100
d791840b82 [Clean Country] added test to verify remove of country: Miriam Baglioni 2022-12-16 13:18:29 +0100
7b80b24f82 [cleaning] country cleaning must use both PID and AlternateIdentifier fields Claudio Atzori 2022-12-15 14:49:04 +0100
b8bafab8a0 [cleaning] improved vocabulary based mapping, specialization for the strict vocab cleaning Claudio Atzori 2022-12-12 14:43:03 +0100
5e4866d033 implemented synch for single mdstore Sandro La Bruzzo 2022-12-12 11:29:46 +0100
c18b8048c3 [cleaning] avoid NPE Claudio Atzori 2022-12-10 11:41:38 +0100
8b44afe5e5 [cleaning] avoid NPE Claudio Atzori 2022-12-09 15:44:57 +0100
389dd25430 [cleaning] avoid NPE Claudio Atzori 2022-12-08 18:40:48 +0100
730228d73d [cleaning] align wf parameter names in test Claudio Atzori 2022-12-08 18:40:22 +0100
2094fa6db0 [cleaning] align wf parameter names Claudio Atzori 2022-12-08 17:22:26 +0100
a485a94956 [Cleaning] fixed parameter name in property file Miriam Baglioni 2022-12-08 16:59:34 +0100
3d99b78d94 [Cleaning] fixed error in parameter (workingPath to workingDir) Miriam Baglioni 2022-12-08 10:25:02 +0100
08c4588d47 Merge pull request 'Changes from beta stats wf to prod' (#264) from antonis.lempesis/dnet-hadoop:beta into master Claudio Atzori 2022-12-07 15:56:22 +0100
1b8488976b code formatting Claudio Atzori 2022-12-07 10:45:38 +0100
cd1b58483e [bulk tag] fixed Community configuration parsing to void NPE Claudio Atzori 2022-12-07 10:38:42 +0100
062abfd669 fixed NPE, removed unused stuff Claudio Atzori 2022-12-06 12:04:00 +0100
2a52a42169 Added 4 institutions: -University of Modena and Reggio Emilia -Bilkent University -Saints Cyril and Methodius University of Skopje -University of Milan dimitrispie 2022-12-06 10:10:21 +0200
71b121e9f8 Merge pull request '[graph cleaning] update collectedfrom & hostedby references as consequence of the datasource deduplication' (#260) from graph_cleaning into beta Claudio Atzori 2022-12-02 14:49:15 +0100
8248da40d9 Merge branch 'beta' into graph_cleaning Claudio Atzori 2022-12-02 14:49:00 +0100
ddf065756f Merge pull request 'Two organizations are added for monitor' (#258) from antonis.lempesis/dnet-hadoop:beta into beta Claudio Atzori 2022-12-02 14:45:27 +0100
41f7f1bbc5 Merge pull request '[graph dedup] records stability and testing' (#44) from deduptesting into beta Claudio Atzori 2022-12-02 14:43:05 +0100
5a48a2fb18 implemented synch for single mdstore Sandro La Bruzzo 2022-12-01 11:34:43 +0100
a38116546d Merge branch 'beta' into deduptesting Claudio Atzori 2022-11-30 11:27:29 +0100
ce020f2c83 [EOSC FUTURE] added resources and test for review Miriam Baglioni 2022-11-30 09:57:30 +0100
bb0ddc1c44 [BulkTag] adding verb starts_with Miriam Baglioni 2022-11-30 09:56:24 +0100
8e3edba318 [graph cleaning] testing the collectedfron and hostedby patch procedure Claudio Atzori 2022-11-29 16:07:09 +0100
58c05731f9 [graph cleaning] WIP: testing the collectedfron and hostedby patch procedure Claudio Atzori 2022-11-29 11:21:51 +0100
7d264a1d69 Merge pull request 'horizontalConstraints' (#259) from horizontalConstraints into beta Miriam Baglioni 2022-11-28 18:20:17 +0100
9c70c5dbd6 [Bulk Tag horizontal] added new path in definition of constraint (to recognize fos subjects) - changed test and resource class to test this new aspect Miriam Baglioni 2022-11-28 14:51:20 +0100
0628df7a3a resolving conflicts Miriam Baglioni 2022-11-28 10:44:56 +0100
11695ba649 [graph cleaning] patch also the result's collectedfrom and hostedby datasource name according to the datasource master-duplicate mapping Claudio Atzori 2022-11-28 10:18:43 +0100
6082d235d3 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into graph_cleaning Claudio Atzori 2022-11-28 09:54:48 +0100
24ef301cc1 [graph cleaning] patch the result's collectedfrom and hostedby identifiers according to the datasource master-duplicate mapping Claudio Atzori 2022-11-28 09:53:23 +0100
29d3da85f1 [EOSC DUMP] added resources needed for the review as test Miriam Baglioni 2022-11-25 17:16:20 +0100
90c8f9cb61 tests for EOSC Future Alessia Bardi 2022-11-23 12:18:44 +0100
33a2b1b5dc [Bulk Tag] fixed typo in test configuration Miriam Baglioni 2022-11-23 11:31:17 +0100
c6df8327b3 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Miriam Baglioni 2022-11-23 11:26:57 +0100
0e3edc5018 [Bulk Tag] fixed issue in verb name Miriam Baglioni 2022-11-23 11:26:36 +0100
935aa367d8 [BulkTag] removed commented code Miriam Baglioni 2022-11-23 11:16:39 +0100
43aedbdfe5 [BulkTag] changed verb name in configuration Miriam Baglioni 2022-11-23 11:14:23 +0100
b6da9b67ff [BulkTag] fixed typo in annotation for verb name Miriam Baglioni 2022-11-23 11:13:58 +0100
a79c47522d updated ORCID datasource identifier Claudio Atzori 2022-11-23 10:17:49 +0100
2832117f23 added eoscifguidelines in test Alessia Bardi 2022-11-22 18:01:12 +0100
14f6346676 implementation of the new software configuration Michele De Bonis 2022-11-22 17:48:34 +0100
3c08269a4d Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta Alessia Bardi 2022-11-22 17:31:00 +0100
2687fc9f73 tests for EOSC Future review - ROhub Alessia Bardi 2022-11-22 17:30:56 +0100
a34c8b6f81 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Claudio Atzori 2022-11-22 10:22:31 +0100
1d5143b0b6 Merge branch 'beta' into deduptesting Claudio Atzori 2022-11-22 10:21:30 +0100
122e75aa17 fixed conflicts Miriam Baglioni 2022-11-21 18:13:12 +0100
cee7a45b1d [Bulk Tag Datasource] fixed issue with verb name and add new test for neanias selection for orcid Miriam Baglioni 2022-11-21 18:10:20 +0100
9fee2ed611 minor changes Michele De Bonis 2022-11-21 14:35:46 +0100
ed64618235 increased spark.sql.shuffle.partitions in the last join phase of the result (publication) to community through semantic relation propagation Claudio Atzori 2022-11-18 16:06:51 +0100
8742934843 added spark.sql.shuffle.partitions in the last join phase of the result to community through semantic relation propagation Claudio Atzori 2022-11-18 11:32:22 +0100
0aa725083f extended dedup testing Claudio Atzori 2022-11-17 16:13:43 +0100
3dbc637d3e code formatting Claudio Atzori 2022-11-17 09:55:41 +0100
13cc592f39 code formatting Claudio Atzori 2022-11-15 09:37:57 +0100
af15b1e48d [eosc tag] extending criteria for Jupyter Notebook (adding to ORP the same constraint) Claudio Atzori 2022-11-14 18:30:43 +0100
eb45ba7af0 extended mapping from ODF relations (PR#251) Claudio Atzori 2022-11-14 18:26:13 +0100
a929dc5fee integrated changes for mapping ROHub contents in the Graph Claudio Atzori 2022-11-14 18:15:35 +0100
24f99d7310 Merge pull request 'Map oaf:eoscifguidelines from mdstore to the graph' (#256) from eoscifguidelines-from-mdstores into beta Claudio Atzori 2022-11-14 15:40:34 +0100
ddff0e8999 merging duplicates using IdentifierComparator Claudio Atzori 2022-11-11 16:10:25 +0100
5f9383b2d9 [EOSC TAG] remove reduntant check for jupyter notebook Miriam Baglioni 2022-11-11 14:06:19 +0100
b18bbca8af [EOSC TAG] adding search in orp for jupyter notebook criteria Miriam Baglioni 2022-11-11 12:42:58 +0100