dnet-hadoop

lsmyrnaios

dnet-hadoop

forked from D-Net/dnet-hadoop

Fork 0

Commit Graph

Select branches

Hide Pull Requests

7096-fileGZip-collector-plugin

8172_impact_indicators_workflow

8232-mdstore-synch-improve

8876

9078_xml_records_irish_tender

SWH_integration

UsageStatsRecordDS

advConstraintsInBeta

apc_affiliation

beta

beta_dedup_configuration

bulkTag

bulkTagRefactor

citations_monodirectional

cleanCountryOnMaster

clean_context_master

clean_license_publisher

compatibility_order

continuous-validation

continuous_validation2

dedup-with-dataframe-spark34

dedup_increasenumofblocks

deduptesting

discard-non-wellformed

distinct_pids_from_openorgs_beta

doiboostFunderExtention

doiboostMappingExtention

doiboost_authormerger

doiboost_refactor

dump

dump_delta_projects

dump_new_funded_products

enrichment

eosc_context_tagging

fc4e-rsac

fulltext_url_validation

graph_cleaning

graph_cleaning_refactoring

h2020classification

horizontalConstraints

hostedByMap_oastartdate

hostedByMap_update

importpoci

instance_group_by_url

mag_citation_relation

master

master_july23

monitoring

native_records_migration

oaf_relation_mapping

orcid_import

organizationToRepresentative

propagationProjectThroughParentChils

pubmed_to_production

pubmed_update

removeDump

removeTaggingCondition

scholix_data_type_openaire

scholix_flat_indexing

scholix_to_solr

spark34-integration

subjectPropagation

sygma_indexing

ticket_8369

transformation_wf

tweaking_spark_params

updateCopyScripts

validation

1.0.1

archive/bipFinder_master_test

archive/dedupTest

archive/islookup_timeout

archive/junit5

archive/master_pre_stable_ids

dhp-1.0.2

dhp-1.0.3

dhp-1.0.4

dhp-1.1.5

dhp-1.1.6

dhp-1.1.7

dhp-1.2.0

dhp-1.2.1

dhp-1.2.2

dhp-1.2.3

dhp-1.2.4

september-2023

bea1a94346 Merge branch 'master' into przemyslawjacewicz_actionmanager_impl_prototype Przemysław Jacewicz 2020-02-17 15:07:23 +0100
6a288625e5 fixed workflow outgoing node Claudio Atzori 2020-02-17 15:04:33 +0100
a5517eee35 adding the mkdirs for creation of propagation folder under provision on tmp Miriam Baglioni 2020-02-17 14:20:42 +0100
9abde5cfac removed outputPath from job parameters Miriam Baglioni 2020-02-17 14:19:53 +0100
1b18fd4d54 sync with master branch Claudio Atzori 2020-02-17 13:49:46 +0100
5bae30f399 adding readme for dhp-schema Claudio Atzori 2020-02-17 13:38:33 +0100
4f04759738 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop Sandro La Bruzzo 2020-02-17 12:31:58 +0100
76ee85141a added oozie job for DNET migration and implemented Spark job for extracting entities Sandro La Bruzzo 2020-02-17 12:31:44 +0100
be2421d5d8 removed wrongly pushed file Miriam Baglioni 2020-02-17 12:07:26 +0100
c460e2d281 Aggiornare 'dhp-workflows/docs/oozie-installer.markdown' Claudio Atzori 2020-02-17 11:54:48 +0100
c7bc73aedf country propagation for results collected from institutional repositories Miriam Baglioni 2020-02-17 11:44:48 +0100
fe93c709f1 Merge branch 'master' of michele.artini/dnet-hadoop into master Sandro La Bruzzo 2020-02-17 10:43:08 +0100
176c5606bd aligned with origin/master, aligned model and mapping Michele Artini 2020-02-17 10:40:53 +0100
56d1810a66 working procedure for records indexing using Spark, via lib com.lucidworks.spark:spark-solr Claudio Atzori 2020-02-14 12:28:52 +0100
1ee1baa8c0 Merge branch 'master' into provision_indexing Claudio Atzori 2020-02-13 18:17:07 +0100
a3d0b57b25 [maven-release-plugin] prepare for next development iteration Claudio Atzori 2020-02-13 18:11:33 +0100
6ed9a15bc8 [maven-release-plugin] prepare release dhp-1.1.5 dhp-1.1.5 Claudio Atzori 2020-02-13 18:11:31 +0100
49e648f7c3 bumped version Claudio Atzori 2020-02-13 18:09:31 +0100
f9fae97e09 test json files aligned with the latest model changes Claudio Atzori 2020-02-13 18:05:59 +0100
11cfd6bd9a integrated changes from master branch Claudio Atzori 2020-02-13 17:27:07 +0100
bbf1b611b9 refereed, processingchargeamount and processingchargecurrency moved inside the Instance element. Introduced specific type to model Result's countries Claudio Atzori 2020-02-13 17:21:11 +0100
1fee6e2b7e implemented XML records construction and serialization, indexing WIP Claudio Atzori 2020-02-13 16:53:27 +0100
956da2f923 added Saxon-HE extension functions and Transformer factory class Claudio Atzori 2020-02-13 16:49:45 +0100
80cb52593f bug fixing Michele Artini 2020-02-13 15:34:13 +0100
cdea0dae75 bug fixing Michele Artini 2020-02-12 16:34:00 +0100
69336195d3 simplifications Michele Artini 2020-02-12 11:12:38 +0100
06c2fd6df9 bug fixing Michele Artini 2020-02-11 15:29:50 +0100
5fc09b179c bug fixing Michele Artini 2020-02-11 12:48:03 +0100
95740767e0 Ready for tests Michele Artini 2020-02-10 16:04:06 +0100
7f11d06a1f upgraded version of dnet-pace-core in pom.xml Sandro La Bruzzo 2020-02-10 12:58:59 +0100
8e4211708e [maven-release-plugin] prepare for next development iteration Sandro La Bruzzo 2020-02-10 12:51:04 +0100
24e2ab9092 [maven-release-plugin] prepare release dnet-dedup-4.0.0 Sandro La Bruzzo 2020-02-10 12:50:45 +0100
46727f5c76 upgraded maven version of commons-lang Sandro La Bruzzo 2020-02-10 12:38:40 +0100
181e8498d4 ... Michele Artini 2020-02-07 16:02:49 +0100
86b60268bb actionmanager implementation prototyping Przemysław Jacewicz 2020-02-06 19:14:41 +0100
bb1533a07e partial commit Michele Artini 2020-02-05 15:35:40 +0100
fbb0fc140b partial implementation of migration Michele Artini 2020-02-04 15:25:47 +0100
d3b96f102b builder pattern screws up the Parquet schema inference method, avoid using it in the bean definitions Claudio Atzori 2020-02-04 14:10:58 +0100
ed290ca8d7 builder pattern Claudio Atzori 2020-02-03 10:35:51 +0100
7ba0f44d05 WIP Claudio Atzori 2020-01-30 18:21:07 +0100
49ef2f4eb1 removed input parameter specification, SparkXmlRecordBuilderJob doesn't need hive Claudio Atzori 2020-01-30 18:20:26 +0100
b5e1e2e5b2 reintegrated changes from fcbc4ccd70 Claudio Atzori 2020-01-30 18:11:04 +0100
7bacd6812e Merge branch 'provision_indexing' of https://code-repo.d4science.org/D-Net/dnet-hadoop into HEAD Claudio Atzori 2020-01-30 17:59:46 +0100
b2691a3b0a save adjacency list as JoinedEntity Claudio Atzori 2020-01-30 17:46:29 +0100
1ecca69f49 added annotation to ignore method during the serialization Claudio Atzori 2020-01-30 17:45:28 +0100
8c2aff99b0 joining entities using T x R x S, WIP: last representation based on LinkedEntity type Claudio Atzori 2020-01-29 15:40:33 +0100
ad4387dd38 added property to gitignore Sandro La Bruzzo 2020-01-27 10:56:40 +0100
24219d1204 Merge branch 'master' of https://code-repo.d3science.org/D-Net/dnet-hadoop Sandro La Bruzzo 2020-01-27 10:54:11 +0100
0dff14b28e added property to gitignore Sandro La Bruzzo 2020-01-27 10:53:54 +0100
5c8f6febee minor changes in comparators miconis 2020-01-24 10:01:11 +0100
19a80e4638 implemented workfow for aggregation and generation of infospace graph Sandro La Bruzzo 2020-01-24 09:58:55 +0100
fcbc4ccd70 a bit of docs doesn't hurt Claudio Atzori 2020-01-24 08:43:23 +0100
a55f5fecc6 joining entities using T x R x S method with groupByKey, WIP: making target objects (T) have lower memory footprint Claudio Atzori 2020-01-24 08:17:53 +0100
6bfe2dc96e partial implementation Michele Artini 2020-01-22 16:00:23 +0100
799929c1e3 joining entities using T x R x S method with groupByKey Claudio Atzori 2020-01-21 16:35:44 +0100
f6eccdde33 partial implementation Michele Artini 2020-01-21 14:17:05 +0100
cd114f1c3b partial update Michele Artini 2020-01-21 12:32:10 +0100
b35c59eb42 partial implementation of entities from db Michele Artini 2020-01-20 16:04:19 +0100
fa7504bf29 removed DLI stuff should be in a branch Sandro La Bruzzo 2020-01-20 10:28:00 +0100
81f82b5d34 partial implementation of applications to migrate entities Michele Artini 2020-01-17 15:26:21 +0100
1cd6899480 merged from master Claudio Atzori 2020-01-17 14:25:57 +0100
749b0660ab instance URLs must be repeatable Claudio Atzori 2020-01-17 14:22:15 +0100
63c0db4ff8 instance URLs must be repeatable Claudio Atzori 2020-01-16 15:54:53 +0200
97c239ee0d WIP: trying to find a way to build the records for the index Claudio Atzori 2020-01-16 12:02:28 +0200
4955be0197 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop miconis 2020-01-14 15:03:44 +0200
f61adfc2bb minor changes miconis 2020-01-14 15:03:27 +0200
9bdcb02179 minor changes and update of the configuration for publications miconis 2020-01-14 15:01:03 +0200
4dce785375 update in the implementation of the tree: addition of new logic aggregations and statistics miconis 2020-01-14 11:42:43 +0200
f7b9a7a9af entity migration (partial implementation) Michele Artini 2020-01-10 15:55:23 +0100
731f9b64e6 Merge branch 'master' of michele.artini/dnet-hadoop into master Claudio Atzori 2019-12-20 14:22:37 +0100
7229fecbcf fix warnings in poms Michele Artini 2019-12-20 13:41:08 +0100
dd21db7036 fixed stuff Sandro La Bruzzo 2019-12-18 16:28:22 +0100
b3748b8d77 minor changes miconis 2019-12-18 16:20:35 +0100
b21b1b8f61 implementation of new aggregation in the tree node processing miconis 2019-12-18 16:19:36 +0100
20fcfe6328 implementation of new aggregation in the tree node processing miconis 2019-12-18 16:19:26 +0100
d924f28b93 fixed wrong use of jspath Sandro La Bruzzo 2019-12-18 09:29:44 +0100
7ba586d2e5 oozie workflow aimed to build the adjacency lists representation of the graph, needed to build the records to be indexed Claudio Atzori 2019-12-17 16:24:49 +0100
84aaa65501 implementation of new json comparator and update of the publication configuration miconis 2019-12-17 09:16:26 +0100
76efcde4fd using new branch decisionTreeDedup Sandro La Bruzzo 2019-12-13 12:20:35 +0100
5c01ae4c92 merged JqMapping branch into tree2 Sandro La Bruzzo 2019-12-13 11:30:02 +0100
b4392f9f43 implemented DedupRecord factory for missing entities Sandro La Bruzzo 2019-12-13 09:40:02 +0100
545e940007 implementation of the mergeFrom for the Datasources miconis 2019-12-12 15:36:41 +0100
39367676d7 implemented DedupRecord factory with the merge of project Sandro La Bruzzo 2019-12-12 15:18:48 +0100
6b45e37e22 implemented DedupRecord factory with the merge of organizations Sandro La Bruzzo 2019-12-11 16:57:37 +0100
abd9034da0 implemented DedupRecord factory with the merge of publications Sandro La Bruzzo 2019-12-11 15:43:24 +0100
4b66b471a4 implementation of the sorting by trust mechanism and the merge of oaf entities miconis 2019-12-10 14:57:16 +0100
35008fdbf9 fix stuff Sandro La Bruzzo 2019-12-06 15:28:30 +0100
cc63706347 Implemented deduplication on spark Sandro La Bruzzo 2019-12-06 13:38:00 +0100
16c670a5d5 Improved deduplication Sandro La Bruzzo 2019-12-05 14:14:25 +0100
49f9beb4a8 implementation of romansmatch and re-implementation of the getNumber function. New terms in the translation map and update of the configuration miconis 2019-11-28 16:54:44 +0100
f791730330 addition of one term to the translation maps in the configurations miconis 2019-11-27 15:48:37 +0100
d2278fe358 minor change in the citymatch miconis 2019-11-21 10:54:02 +0100
8c0d346005 the param map has been updated: now it accepts string parameters miconis 2019-11-21 09:37:56 +0100
ddd40540aa jarowinklernormalizedname splitted in 3 different comparators: citymatch, keywordmatch and jarowinkler. Implementation of the TreeStatistic support functions miconis 2019-11-20 10:45:00 +0100
6a7bee5e43 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop Claudio Atzori 2019-11-14 15:43:07 +0100
0c4b316f82 align Result model with the latest OpenAIRE schema changes introduced in the protobuf model Claudio Atzori 2019-11-14 15:42:52 +0100
aad0cb40b7 Added schema Scholexplorer Sandro La Bruzzo 2019-11-14 10:34:09 +0100
c687956371 code cleaning and implementation of the TreeDedup + minor changes miconis 2019-11-14 10:01:21 +0100
5711e75f67 use ${project.version} whenever possible Claudio Atzori 2019-11-08 17:41:51 +0100
245b4cbbb3 removed import limit Claudio Atzori 2019-11-08 17:41:01 +0100