dnet-hadoop

Commit Graph

Select branches

Hide Pull Requests

dedupTest

deduptesting

doiboost

hive_timeout

impala_changes_by_alexiou

junit5

master

migration_wfs_regular_all_steps

provision_dataset

provision_indexing

scholexplorer

stats_wf_extensions_and_corrections

usage-stats-export-wf

#1

#2

1.0.1

archive/bipFinder_master_test

archive/dedupTest

archive/islookup_timeout

archive/junit5

dhp-1.0.2

dhp-1.0.3

dhp-1.0.4

dhp-1.1.5

dhp-1.1.6

dhp-1.1.7

dhp-1.2.0

dhp-1.2.1

dhp-1.2.2

dhp-1.2.3

dhp-1.2.4

e7167b996a logs and closeable Michele Artini 2020-03-04 10:46:36 +0100
25ceec29ab code formatting Claudio Atzori 2020-03-04 10:44:24 +0100
63c00c5e88 fixed typo Claudio Atzori 2020-03-04 10:43:44 +0100
c37f2bd1b5 moved some classes to package to make code clearer Miriam Baglioni 2020-03-03 16:42:23 +0100
d9d2060561 implementation for bulk tagging Miriam Baglioni 2020-03-03 16:38:50 +0100
e80f80ca93 properties and workflow for new propagation Miriam Baglioni 2020-03-02 17:03:31 +0100
9cf5ce2e66 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Claudio Atzori 2020-03-02 17:03:10 +0100
bc7cfd5975 indexing workflow WIP: fixed projects fundingtree xml conversion, prioritized links between results and projects when limiting them to 100 in the join procedure Claudio Atzori 2020-03-02 17:03:07 +0100
50080c1b3c changed the implementation of addAll method. Before adding all the items in a collection, we check if the accumulator set is not empty Miriam Baglioni 2020-03-02 16:41:37 +0100
02815dd2cf update result for community moved in propagationconstants Miriam Baglioni 2020-03-02 16:40:56 +0100
95f8c3092f update for new propagation implementation and moving of updateResult for community business logic since the same can be used for result to community from organization and result to community from semrel Miriam Baglioni 2020-03-02 16:40:17 +0100
3d63f35dcb implementation of new propagation. Result to community for results linked to given organization. We exploit the hasAuthorInstitution semantic link to discover which results are related to institutions Miriam Baglioni 2020-03-02 16:39:03 +0100
4b29a121b0 migration using spark in step2 Michele Artini 2020-03-02 16:12:14 +0100
5445a57102 migration using spark in step2 Michele Artini 2020-03-02 16:11:59 +0100
3a4ccb26c0 New properties for the orcid to result propagation through semantic relation Miriam Baglioni 2020-02-28 18:26:04 +0100
b50166b9ad None Miriam Baglioni 2020-02-28 18:25:28 +0100
550cb21c23 None Miriam Baglioni 2020-02-28 18:24:39 +0100
b098ee0bae Changed the structure of typed row to conatain also list of authors with orcid Miriam Baglioni 2020-02-28 18:23:51 +0100
841f5523fe Added information and methods for the new propagation of orcid to result through semrel Miriam Baglioni 2020-02-28 18:23:16 +0100
2b7b05fb29 New propagation of ORCID to result exploiting the semantic relation connecting them. R has author with orcid o, R is bounf by strong semantic relationship with R1 that has the same author withouth orcid, then o is also associated to the author in R1 Miriam Baglioni 2020-02-28 18:22:41 +0100
833c83c694 Wrong file name Miriam Baglioni 2020-02-28 18:21:01 +0100
a86426776a Changed from Oaf to Result the type of the updateResult method parameter, not to be forced to cast each time Miriam Baglioni 2020-02-28 18:20:19 +0100
b32655e48e changed code to save intermediate result Sandro La Bruzzo 2020-02-27 10:18:46 +0100
60bc2b1a20 drop the hive DB before populating it from scratch Claudio Atzori 2020-02-27 10:10:55 +0100
f09e065865 incremented number of repartition Sandro La Bruzzo 2020-02-26 19:26:19 +0100
071f5c3e52 fixed NPE Sandro La Bruzzo 2020-02-26 15:42:20 +0100
a1a6fc8315 fixed NPE Sandro La Bruzzo 2020-02-26 15:42:13 +0100
1edf02a3ce added log Sandro La Bruzzo 2020-02-26 15:25:03 +0100
c3ecabd8e8 fixed NPE Sandro La Bruzzo 2020-02-26 14:40:02 +0100
5d0f46651b fixed NPE Sandro La Bruzzo 2020-02-26 14:31:34 +0100
bc342bf73a fixed wrong generation type in summary Sandro La Bruzzo 2020-02-26 12:49:47 +0100
3112e21858 fixed typo Sandro La Bruzzo 2020-02-26 12:22:43 +0100
119ae6eef5 fixed wrong loop in the workflow Sandro La Bruzzo 2020-02-26 12:18:50 +0100
7936583a3d added generation of Scholix collection Sandro La Bruzzo 2020-02-26 12:09:06 +0100
02db368dc5 Merge branch 'master' into przemyslawjacewicz_actionmanager_impl_prototype Przemysław Jacewicz 2020-02-26 11:50:20 +0100
2ef3705b2c Added Provision workflow Sandro La Bruzzo 2020-02-26 10:51:35 +0100
689908b2e9 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Michele Artini 2020-02-25 16:00:51 +0100
93665773ea Fixed a problem with JavaRDD Union Michele Artini 2020-02-25 15:59:21 +0100
b021b8a2e1 Added index wf Sandro La Bruzzo 2020-02-24 10:15:55 +0100
6a73fd5da5 in order to reuse the same XmlRecordFactory across different tasks, the state of contexts must be one per record built Claudio Atzori 2020-02-21 09:17:19 +0100
4c94e74a84 Added a missing dependency Michele Artini 2020-02-20 11:43:32 +0100
d49cd2fdc6 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Michele Artini 2020-02-20 11:21:54 +0100
3f941a2af4 Merge branch 'master' into propagationCommunityToResult Miriam Baglioni 2020-02-19 18:05:22 +0100
b2bdc9b99b merging project to result propagation logic to master Miriam Baglioni 2020-02-19 18:04:59 +0100
a153a07997 none Miriam Baglioni 2020-02-19 18:03:13 +0100
d0279af630 start to implement the business logic Miriam Baglioni 2020-02-19 17:59:24 +0100
5f63ab1416 to query the information system to get the list of comunities up to now. It will have a more general usage when introducing bulk tagging Miriam Baglioni 2020-02-19 17:59:02 +0100
d42dde52ba implemented method to merge relations Claudio Atzori 2020-02-19 17:29:05 +0100
5ceb174d24 Merge branch 'master' into propagationCommunityToResult Miriam Baglioni 2020-02-19 17:13:38 +0100
e8af7a6b64 Merge remote-tracking branch 'upstream/master' Miriam Baglioni 2020-02-19 17:03:10 +0100
79ff79b0cd propagation of result to community through semantic relation: C -> R and R -> isSupplementedBy R1 => C -> R1 Miriam Baglioni 2020-02-19 17:02:39 +0100
5e5e32cb48 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Claudio Atzori 2020-02-19 16:56:52 +0100
33185fd0b7 ISLookupClientFactory moved in dhp-common Claudio Atzori 2020-02-19 16:56:38 +0100
5d3739b5cf migration of claims Michele Artini 2020-02-19 15:11:17 +0100
ab84163bb3 added set accumulator in TypedRow and used it to acucmulate country information in Country Propagation Miriam Baglioni 2020-02-19 15:02:50 +0100
bb0fdf5e0a fix wrong source target in new relation Miriam Baglioni 2020-02-19 15:00:46 +0100
9e1678ccf8 fix error in workflow name Miriam Baglioni 2020-02-19 14:59:24 +0100
8aa3b4d7c0 adding to propagation constants the ones needed for propagation of project to result and addition of new accumulator Set in typed row to collect values of a type Miriam Baglioni 2020-02-19 14:55:54 +0100
7167673a58 implementation and configuration for propagation of project to result through semantic relation: P -> R1 and R1 -> supplemented by -> R2 => P -> R2 Miriam Baglioni 2020-02-19 14:54:18 +0100
173f1df1e5 saved a query for openaire production database Michele Artini 2020-02-19 10:15:08 +0100
9a2d74ac82 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Sandro La Bruzzo 2020-02-19 10:13:45 +0100
e5d7cdf422 fixed sql query Sandro La Bruzzo 2020-02-19 10:13:36 +0100
2b8675462f refactoring code Sandro La Bruzzo 2020-02-19 10:07:08 +0100
b81e6af429 added config for new propagation Miriam Baglioni 2020-02-18 17:30:44 +0100
b736a9581c changed relclass and reltype in reelation specification for country propagation and implementation of propagation of result affiliation through institutional repositories Miriam Baglioni 2020-02-18 17:27:28 +0100
ed262293a6 aligned to new snapshot version 1.1.6 Miriam Baglioni 2020-02-18 17:25:32 +0100
2688a89c21 changed relclass and reltype in relation specification Miriam Baglioni 2020-02-18 17:24:40 +0100
c0022fec9f moved on upper package to serve other propagations Miriam Baglioni 2020-02-18 17:24:11 +0100
e0a777028a fix problem in parameters Miriam Baglioni 2020-02-18 17:23:34 +0100
ed76521d9b removed stale test resources, will be re-added later on Claudio Atzori 2020-02-18 11:50:39 +0100
0f364605ff removed stale tests, need to reimplemente them anyway Claudio Atzori 2020-02-18 11:48:19 +0100
5868ff8a86 synch fork with master Miriam Baglioni 2020-02-17 18:22:27 +0100
958f0693d6 WIP: logic for promoting action sets added Przemysław Jacewicz 2020-02-17 18:18:24 +0100
18e4092d5c change name of properties dir Miriam Baglioni 2020-02-17 18:07:06 +0100
bd0e504b42 changes to the wf configuration Miriam Baglioni 2020-02-17 18:04:15 +0100
3a9d723655 adding default parameters in code Miriam Baglioni 2020-02-17 16:30:52 +0100
bea1a94346 Merge branch 'master' into przemyslawjacewicz_actionmanager_impl_prototype Przemysław Jacewicz 2020-02-17 15:07:23 +0100
6a288625e5 fixed workflow outgoing node Claudio Atzori 2020-02-17 15:04:33 +0100
a5517eee35 adding the mkdirs for creation of propagation folder under provision on tmp Miriam Baglioni 2020-02-17 14:20:42 +0100
9abde5cfac removed outputPath from job parameters Miriam Baglioni 2020-02-17 14:19:53 +0100
1b18fd4d54 sync with master branch provision_indexing Claudio Atzori 2020-02-17 13:49:46 +0100
5bae30f399 adding readme for dhp-schema Claudio Atzori 2020-02-17 13:38:33 +0100
4f04759738 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Sandro La Bruzzo 2020-02-17 12:31:58 +0100
76ee85141a added oozie job for DNET migration and implemented Spark job for extracting entities Sandro La Bruzzo 2020-02-17 12:31:44 +0100
be2421d5d8 removed wrongly pushed file Miriam Baglioni 2020-02-17 12:07:26 +0100
c460e2d281 Aggiornare 'dhp-workflows/docs/oozie-installer.markdown' Claudio Atzori 2020-02-17 11:54:48 +0100
c7bc73aedf country propagation for results collected from institutional repositories Miriam Baglioni 2020-02-17 11:44:48 +0100
fe93c709f1 Merge branch 'master' of michele.artini/dnet-hadoop into master Sandro La Bruzzo 2020-02-17 10:43:08 +0100
176c5606bd aligned with origin/master, aligned model and mapping Michele Artini 2020-02-17 10:40:53 +0100
56d1810a66 working procedure for records indexing using Spark, via lib com.lucidworks.spark:spark-solr Claudio Atzori 2020-02-14 12:28:52 +0100
1ee1baa8c0 Merge branch 'master' into provision_indexing Claudio Atzori 2020-02-13 18:17:07 +0100
a3d0b57b25 [maven-release-plugin] prepare for next development iteration Claudio Atzori 2020-02-13 18:11:33 +0100
6ed9a15bc8 [maven-release-plugin] prepare release dhp-1.1.5 dhp-1.1.5 Claudio Atzori 2020-02-13 18:11:31 +0100
49e648f7c3 bumped version Claudio Atzori 2020-02-13 18:09:31 +0100
f9fae97e09 test json files aligned with the latest model changes Claudio Atzori 2020-02-13 18:05:59 +0100
11cfd6bd9a integrated changes from master branch Claudio Atzori 2020-02-13 17:27:07 +0100
bbf1b611b9 refereed, processingchargeamount and processingchargecurrency moved inside the Instance element. Introduced specific type to model Result's countries Claudio Atzori 2020-02-13 17:21:11 +0100
1fee6e2b7e implemented XML records construction and serialization, indexing WIP Claudio Atzori 2020-02-13 16:53:27 +0100
956da2f923 added Saxon-HE extension functions and Transformer factory class Claudio Atzori 2020-02-13 16:49:45 +0100
80cb52593f bug fixing Michele Artini 2020-02-13 15:34:13 +0100