dnet-hadoop

Commit Graph

Select branches

Hide Pull Requests

deduptesting

dhp_oaf_model

doiboost

enrichment_wfs

graph_cleaning

graph_db

import_openorgs

islookup_timeout

master

merge_graph

oaf-store-interpretation

provision_indexing

przemyslawjacewicz_jenkins_deployment

scholexplorer

trunc_trust

unique_field_in_lists

1.0.1

archive/bipFinder_master_test

archive/dedupTest

archive/islookup_timeout

archive/junit5

dhp-1.0.2

dhp-1.0.3

dhp-1.0.4

dhp-1.1.5

dhp-1.1.6

dhp-1.1.7

dhp-1.2.0

dhp-1.2.1

dhp-1.2.2

dhp-1.2.3

dhp-1.2.4

85766a02d8 added dependency to use hive on local machine Miriam Baglioni 2020-04-11 16:34:22 +0200
79b8ea4fed prepared information to be used in actual country propagation. Subset of info Miriam Baglioni 2020-04-11 16:29:41 +0200
1822476613 Test for country propagation Miriam Baglioni 2020-04-11 16:28:09 +0200
7783b09c5b new implementation for result to project propagation. Prepare some info to be used in propagation Miriam Baglioni 2020-04-11 16:26:23 +0200
6b5f9ca9cb raw graph creation workflow moved under dhp-graph-mapper, claims integration is included Claudio Atzori 2020-04-10 17:53:07 +0200
90469789b9 two new classes fro new implementation of project to result propagation Miriam Baglioni 2020-04-09 13:29:01 +0200
627ad58a8b new wf definition Miriam Baglioni 2020-04-09 11:33:19 +0200
9c63c4840d new workflow and parameters for country propagation Miriam Baglioni 2020-04-08 19:13:42 +0200
a2d309545b new parametrized implementation for country propagation Miriam Baglioni 2020-04-08 19:12:59 +0200
6dfdba9ef7 new parametrized implementation for country propagation Miriam Baglioni 2020-04-08 18:14:37 +0200
03f7cb6402 new parametrized implementation for country propagation Miriam Baglioni 2020-04-08 18:08:41 +0200
df2fc4a6d7 Merge remote-tracking branch 'upstream/master' Miriam Baglioni 2020-04-08 18:07:26 +0200
fcfef4632f input parameters for country propagation preparation job Miriam Baglioni 2020-04-08 18:07:18 +0200
0be2e72be5 further implementation of tests for the deduplication of each entity. publication dump added, empty entity files created miconis 2020-04-08 18:02:30 +0200
61045e84d9 merged conflict in pom Miriam Baglioni 2020-04-08 14:23:30 +0200
47f3d9b757 unit test for GraphHiveImporterJob Claudio Atzori 2020-04-08 13:24:43 +0200
ba9f07a6fe fixed wrong test Sandro La Bruzzo 2020-04-08 13:18:20 +0200
540da4ab61 new busuness logic with prepared info before actual job run Miriam Baglioni 2020-04-08 13:04:04 +0200
8438702b3d addition in propagation constants Miriam Baglioni 2020-04-08 10:54:01 +0200
2afe971816 new implementation for country propagatio Miriam Baglioni 2020-04-08 10:49:09 +0200
beebbcf66b new config for countrypropagation Miriam Baglioni 2020-04-08 10:31:29 +0200
d74e128aa6 Utility classes moved in dhp-common and dhp-schemas Claudio Atzori 2020-04-07 11:56:22 +0200
c57cf679ca Merge branch 'provision_dataset' Claudio Atzori 2020-04-07 08:56:58 +0200
1a1a026a18 we do expect to find field bestaccessright already defined. No need to add it again Claudio Atzori 2020-04-07 08:55:33 +0200
fbdd18a96b using dataset based relation preparation procedure Claudio Atzori 2020-04-07 08:54:39 +0200
77f59b1b10 dataset based provision WIP Claudio Atzori 2020-04-06 19:37:27 +0200
6177cf36fb Merge pull request 'Closes #4: New action manager implementation' (#5) from przemyslaw.jacewicz/dnet-hadoop:przemyslawjacewicz_actionmanager_impl_prototype into master Claudio Atzori 2020-04-06 17:35:07 +0200
e355961997 dataset based provision WIP Claudio Atzori 2020-04-06 17:34:25 +0200
56fbe689f0 implementation of the tests for each spark action miconis 2020-04-06 16:30:31 +0200
ca345aaad3 dataset based provision WIP Claudio Atzori 2020-04-06 15:33:31 +0200
c8f4b95464 dataset based provision WIP Claudio Atzori 2020-04-06 08:59:58 +0200
eb2f5f3198 dataset based provision WIP Claudio Atzori 2020-04-04 17:41:31 +0200
3d1b637cab dataset based provision WIP Claudio Atzori 2020-04-04 14:03:43 +0200
53fd624c34 implemented test for sparkcreatesimrels miconis 2020-04-03 18:32:25 +0200
24b2c9012e dataset based provision WIP Claudio Atzori 2020-04-02 18:44:09 +0200
a61763d149 structure for sparksimrel changed to be compliant with mockito testing miconis 2020-04-02 18:37:53 +0200
daa26acc9d dataset based provision WIP, fixed spark2EventLogDir Claudio Atzori 2020-04-02 16:15:50 +0200
7b2a7e2417 [dhp-actionmanager] missing descriptions added and minor naming and formatting fixes Przemysław Jacewicz 2020-04-02 11:48:40 +0200
1ab97bbe00 Adding the full stats workflow to the dnet-hadoop hierarchy Spyros Zoupanos 2020-04-01 22:05:35 +0300
9c7092416a dataset based provision WIP Claudio Atzori 2020-04-01 19:07:30 +0200
bfa5bc74df minor changes miconis 2020-04-01 19:05:48 +0200
80cf43b9c8 [dhp-actionmanager] promoting workflow added Przemysław Jacewicz 2020-04-01 18:51:25 +0200
5b459bcc47 [dhp-actionmanager] promoting spark job added Przemysław Jacewicz 2020-04-01 18:44:35 +0200
9802bcb9fe dedup testing miconis 2020-04-01 18:48:31 +0200
e21bb89dbd [dhp-actionmanager] partitioning spark job added Przemysław Jacewicz 2020-04-01 18:41:29 +0200
f9f7350bb9 [dhp-actionmanager] common package added with utility classes supporting hadoop and spark envs Przemysław Jacewicz 2020-04-01 18:39:26 +0200
ad70c23b2e [dhp-actionmanager] pom updated Przemysław Jacewicz 2020-04-01 18:36:00 +0200
4e910a78d4 [dhp-workflows] spark 2 connection properties added Przemysław Jacewicz 2020-04-01 18:29:26 +0200
51ff3b4e81 [dhp-schemas] added safeguard against casting exception in mergeFrom methods and null-safe handling of collectedfrom collection for relation Przemysław Jacewicz 2020-04-01 18:28:23 +0200
1402eb1fe7 cleanup Claudio Atzori 2020-04-01 15:38:50 +0200
7061d07727 ActionSets migration serialize the output as plain text files instead of SequenceFiles Claudio Atzori 2020-04-01 14:58:22 +0200
adcdd2d05e WIP: reimplementing the adjacency list construction process using spark Datasets Claudio Atzori 2020-04-01 14:56:57 +0200
205e9521c6 implemented import crossref job Sandro La Bruzzo 2020-04-01 14:12:33 +0200
201d79021e Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Sandro La Bruzzo 2020-03-31 14:41:41 +0200
cd7416ae4c first implementation of incremental update of scholix index Sandro La Bruzzo 2020-03-31 14:41:35 +0200
9d1d18d4b9 Merge branch 'master' into przemyslawjacewicz_actionmanager_impl_prototype przemek 2020-03-31 12:04:58 +0200
377e1ba840 [maven-release-plugin] prepare for next development iteration Claudio Atzori 2020-03-30 20:06:00 +0200
76d9315129 [maven-release-plugin] prepare release dhp-1.1.6 dhp-1.1.6 Claudio Atzori 2020-03-30 20:05:56 +0200
ef429010ee removed log file and job-override.properties Claudio Atzori 2020-03-30 20:00:58 +0200
0fbec69b82 use oozie prepare statement to cleanup working directories Claudio Atzori 2020-03-30 19:48:41 +0200
3af2b8d700 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Claudio Atzori 2020-03-30 13:12:21 +0200
f3f9affd49 allow dynamic executors to build XML records Claudio Atzori 2020-03-30 13:12:11 +0200
2e2d4c4c68 adjusted path to template resource Claudio Atzori 2020-03-30 13:11:49 +0200
dd011f4a95 to make them visible to Claudio Miriam Baglioni 2020-03-30 10:55:47 +0200
b1af90a45f to make it visible to Claudio Miriam Baglioni 2020-03-30 10:50:03 +0200
62cc257e5c fixed step1 workflow Sandro La Bruzzo 2020-03-27 17:07:34 +0100
1a7a866861 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Sandro La Bruzzo 2020-03-27 15:11:48 +0100
7cef698f36 reformat code Sandro La Bruzzo 2020-03-27 15:11:34 +0100
1767dfaa3f method can be protected, it is meant to be used only in tests Claudio Atzori 2020-03-27 14:31:26 +0100
a4b6a51168 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Sandro La Bruzzo 2020-03-27 13:48:56 +0100
15d9106b3f FIxed merge of dhp dedup Sandro La Bruzzo 2020-03-27 13:48:44 +0100
589d1c1443 ignore *.log files Claudio Atzori 2020-03-27 13:45:25 +0100
e196fff212 adjusted path for source resource in unit test Claudio Atzori 2020-03-27 13:45:10 +0100
8c9a56a0c8 refactored package name Sandro La Bruzzo 2020-03-27 13:19:33 +0100
2bd2d6f202 Merge branch 'master' of code-repo.d3science.org:D-Net/dnet-hadoop Sandro La Bruzzo 2020-03-27 13:16:36 +0100
a9935f80d4 refactor class name and workflow name for graph mapper, added javadoc Sandro La Bruzzo 2020-03-27 13:16:24 +0100
ae03948eed Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Michele Artini 2020-03-27 11:47:07 +0100
f6e86b44a6 tests Michele Artini 2020-03-27 11:46:37 +0100
408be3c632 test and fixed a problem with datacite namespaces Michele Artini 2020-03-27 11:44:50 +0100
673e744649 moved openaire specific implementations under dedicated package eu.dnetlib.dhp.oa Claudio Atzori 2020-03-27 10:42:17 +0100
098fabab3f reorganizing content under dhp-workflows/dhp-graph-mapper Claudio Atzori 2020-03-26 19:44:19 +0100
77c4294924 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Claudio Atzori 2020-03-26 18:26:52 +0100
43cbcda7ef unit test for SparkGraphImporterJob Claudio Atzori 2020-03-26 18:26:40 +0100
e04da6d66a merged all oozie wf in one scholexplorer Sandro La Bruzzo 2020-03-26 14:17:07 +0100
e71e001b58 commented test that doesn't work Sandro La Bruzzo 2020-03-26 14:15:21 +0100
0cd022ad6a merge with master Sandro La Bruzzo 2020-03-26 14:08:29 +0100
abcd3f5bf5 added sample data for unit tests Claudio Atzori 2020-03-26 11:12:52 +0100
d5f11e27be renamed wf Sandro La Bruzzo 2020-03-26 09:49:23 +0100
9a37ad0127 renamed modules Sandro La Bruzzo 2020-03-26 09:46:46 +0100
a768226e52 updated generate scholix to generate json Sandro La Bruzzo 2020-03-26 09:40:50 +0100
4753662edd removed unnecessary dependency Claudio Atzori 2020-03-26 09:03:43 +0100
9dff4adbc3 dhp-graph-mapper workflow tests upgraded to junit5 archive/junit5 Claudio Atzori 2020-03-25 18:25:12 +0100
cd7dc3e1ae dhp-dedup-openaire workflow tests upgraded to junit5 Claudio Atzori 2020-03-25 18:04:23 +0100
c0e825e713 dhp-aggregation workflow tests upgraded to junit5 Claudio Atzori 2020-03-25 17:59:45 +0100
19b2048109 code formatting Claudio Atzori 2020-03-25 17:40:38 +0100
3e8f6981c4 dhp-schemas tests upgraded to junit5 Claudio Atzori 2020-03-25 17:38:58 +0100
56f5a4eeab dhp-build tests upgraded to use junit5 Assertions Claudio Atzori 2020-03-25 17:38:31 +0100
c02718c10b dhp-build tests upgraded to junit5 Claudio Atzori 2020-03-25 17:23:22 +0100
23668d4a6a WIP adopting junit5 Claudio Atzori 2020-03-25 16:49:45 +0100
a226198a13 WIP adopting junit5 Claudio Atzori 2020-03-25 16:47:39 +0100