dnet-hadoop

Commit Graph

Select branches

Hide Pull Requests

datepicker

deduptesting

dhp_oaf_model

graph_cleaning

graph_db

import_new_mdstores

master

openorgswf

organization_pids

prepare_ror_actionset

scholexplorer_model_update

scholexplorer_resolve_on_hadoop

stable_id_scholexplorer

stable_ids

unique_field_in_lists

1.0.1

archive/bipFinder_master_test

archive/dedupTest

archive/islookup_timeout

archive/junit5

archive/master_pre_stable_ids

dhp-1.0.2

dhp-1.0.3

dhp-1.0.4

dhp-1.1.5

dhp-1.1.6

dhp-1.1.7

dhp-1.2.0

dhp-1.2.1

dhp-1.2.2

dhp-1.2.3

dhp-1.2.4

88b73f4f49 mergin with branch beta Miriam Baglioni 2021-11-10 17:00:52 +0100
c371b23077 - Miriam Baglioni 2021-11-10 17:00:37 +0100
fc8fceaac3 create direct link to WT projects as well Alessia Bardi 2021-11-10 14:11:52 +0100
6cd91004e3 fixed DOI for Wellcome Trust in mapping relationships from Crossref Alessia Bardi 2021-11-09 12:22:57 +0100
9e214ce0eb [BypassAS] addition of OC relations Miriam Baglioni 2021-11-09 12:07:19 +0100
b9d4f115cc fixed Crossref mappign for SFI projects Alessia Bardi 2021-11-09 12:04:45 +0100
6477a40670 implement filter of openCitation Sandro La Bruzzo 2021-11-09 11:27:12 +0100
6f7ca539c6 [BypassAS] update of results for bipFinder and FOS Miriam Baglioni 2021-11-09 11:25:41 +0100
a7d50c499b [BypassAS] prepare FOS subject, test and model for FOS and BipFinder scores Miriam Baglioni 2021-11-08 16:44:19 +0100
91354c6068 - fetching all context related results - storing tables as parquet Antonis Lempesis 2021-11-08 15:15:46 +0200
94918a673c [Graph DUMP] Fix issue for empty origilaId list Miriam Baglioni 2021-11-08 10:25:28 +0100
9cb8e4ad21 Merge branch 'beta' into hierarchical_orgs_relations Claudio Atzori 2021-11-08 09:40:24 +0100
4c70201412 mergin with branch beta Miriam Baglioni 2021-11-05 12:29:56 +0100
8442efd8d1 [Graph DUMP] Filtering out from the originalIds the id of the result in OpenAIRE Miriam Baglioni 2021-11-05 12:29:22 +0100
5681e89544 Update 'dhp-workflows/dhp-graph-mapper/src/main/resources/eu/dnetlib/dhp/oa/graph/dump/schemas/result_schema.json' Claudio Atzori 2021-11-05 12:18:24 +0100
a22c29fba1 [Graph DUMP] Filtering out from the originalIds the id of the result in OpenAIRE Miriam Baglioni 2021-11-05 12:08:33 +0100
c10ff6928c [Graph DUMP] add schema of the dump related to the model as in dhp-schemas.2.8.31. Note the measere element at the level of the result has been removed because of issues on where to display it: at the level of the result or at the level of the entity Miriam Baglioni 2021-11-05 11:36:21 +0100
0857849a86 [Graph DUMP] Remove dump of measure until it will be clear where to put it (at the level of result or at the level of the instance) Miriam Baglioni 2021-11-05 11:02:37 +0100
df7ee77c7a [DOIBoost Mapping] removed not needed comments Miriam Baglioni 2021-11-04 16:24:07 +0100
de63d29b6f [DOIBoost Mapping] Fix to avoid to produce results with null as identifier (probably due to the filtering function in the factory for the creation of the id) Miriam Baglioni 2021-11-04 16:16:40 +0100
d50057b2d9 [DOIBoost Mapping] changed the way to create the url for the instance: we use the crooref guidelines https://doi.org/doi Miriam Baglioni 2021-11-03 16:59:37 +0100
edf55395e9 added test resourse Miriam Baglioni 2021-11-03 16:49:30 +0100
d97ea82a29 [DOIBoost Mapping] Added test to verify the instance created for Crossref will have just the url related to the doi Miriam Baglioni 2021-11-03 16:45:15 +0100
96769b4481 [DOIBoost - Mapping] Changed the logic which brought in in the instance urls that should not be there: The urld of the doi in the json is reachable from the root (json/"URL") other urls where added from the links element. Now the mapping from the link element has been removed Miriam Baglioni 2021-11-03 16:43:36 +0100
683fe093cf [DOIBoost - Mapping] Remove the addition of the instance to the MAG publication record Miriam Baglioni 2021-11-03 15:51:26 +0100
b2bb8d9d79 [DOIBoost - Mapping] selecting the url from Crossref containing the doi Miriam Baglioni 2021-11-03 15:44:57 +0100
779318961c [DOIBoost - Mapping] removed the url from crossref containing the api.elsevier.com... string in the url Miriam Baglioni 2021-11-03 14:38:52 +0100
2480e590d1 [DOIBoost - Mapping] changed the type on which to map dissertation from Crossref: from 006 Doctoral thesis to 0044 Thesis since dissertation could be either Doctoral or master thesis Miriam Baglioni 2021-11-03 14:25:23 +0100
b9d124bb7c [Enrichment: Propagation through parent-child relationships] Added counters, and changed constraint to verify if filtering out the relation (from classname = harvested to classid != propagation) Miriam Baglioni 2021-11-03 13:55:37 +0100
7bd224f051 implement first version of scholexplorer integration for the generation of final graph Sandro La Bruzzo 2021-11-02 15:58:15 +0100
b97b78f874 removed hardcoded reference Antonis Lempesis 2021-11-01 15:42:29 +0200
7fa49f6956 Merge pull request 'removed hardcoded reference' (#154) from antonis.lempesis/dnet-hadoop:beta into beta Claudio Atzori 2021-11-02 09:11:30 +0100
f78afb5ef9 removed hardcoded reference Antonis Lempesis 2021-11-01 15:42:29 +0200
2aca6bfa0a mergin with branch beta Miriam Baglioni 2021-10-29 11:20:45 +0200
09f36cffb8 [Enrichment: Propagation through parent-child relationships] First implementation, testing, and wf for propagation of result to organization through semantic relation Miriam Baglioni 2021-10-29 11:20:03 +0200
1225ba0b92 [resolution] increasing number of partitions to avoid OOM Claudio Atzori 2021-10-28 16:18:17 +0200
d9cbca83f7 moved filter on next phase Sandro La Bruzzo 2021-10-28 16:13:24 +0200
d02caef185 Merge branch 'beta' into hierarchical_orgs_relations Claudio Atzori 2021-10-27 15:36:29 +0200
1be9aa0a5f Removed filter of datacite items from the raw graph merging phase, Datacite is not an actionset anymore in beta Sandro La Bruzzo 2021-10-26 17:52:20 +0200
4acfa8fa2e Scholexplorer Datasource Aggregation: - Added collectedfrom in the inverse relation generated Relation resolution: - increased number of partitions in workflow.xml - using classid instead of classname to build the pid-dnetId mapping Sandro La Bruzzo 2021-10-26 17:51:20 +0200
d0ef7d91c5 adding test resource Miriam Baglioni 2021-10-26 17:34:11 +0200
aafdffa6b3 resolved conflict Sandro La Bruzzo 2021-10-26 09:45:46 +0200
034304b33a conflict resolved on merge Sandro La Bruzzo 2021-10-26 09:40:47 +0200
d66e20e7ac added hierarchy rel in ROR actionset Michele Artini 2021-10-21 15:51:48 +0200
6b34ba737e minor Claudio Atzori 2021-10-21 14:16:18 +0200
d147295c2f avoiding java.io.NotSerializableException: java.util.HashMap Claudio Atzori 2021-10-21 14:15:57 +0200
3702fe478d cleanup Claudio Atzori 2021-10-21 12:05:02 +0200
ac36aa7d1c fixed wrong Encoding during a map phase Sandro La Bruzzo 2021-10-21 11:35:02 +0200
aeeebd573b code refactor renamed datacite package Sandro La Bruzzo 2021-10-20 17:37:42 +0200
ab3a99d3e9 removed old datacite oozie workflow Sandro La Bruzzo 2021-10-20 17:19:47 +0200
ae4e99a471 Adapted workflow of resolution of PID to work into OpenAIRE data workflow Sandro La Bruzzo 2021-10-20 17:12:08 +0200
cece432adc [stats] reducing the step22 wait time Claudio Atzori 2021-10-20 14:14:53 +0200
a7376907c2 invalidating medatadata before context thingies Antonis Lempesis 2021-10-15 13:42:55 +0300
43f4eb492b fetching affiliated results for 4 orgs in monitor. fixed affiliated orgs in stats db Antonis Lempesis 2021-10-14 18:53:35 +0300
4f8970f8ed [stats] reducing the step22 wait time Claudio Atzori 2021-10-20 14:14:53 +0200
00b78b9c58 cleanup: mapping contents in the graph already defined in the OAF graph model doesn't require to be aware of the vocabularies Claudio Atzori 2021-10-20 14:04:45 +0200
c01dd0c925 registered oaf model classes for the KryoSerializer Claudio Atzori 2021-10-20 13:55:07 +0200
652114c641 [affiliationPropagation] first try. preparetion Miriam Baglioni 2021-10-20 11:44:23 +0200
d0cf2963f0 Merge pull request 'hierarchical_orgs_relations' (#150) from hierarchical_orgs_relations into beta Claudio Atzori 2021-10-20 10:13:47 +0200
59f76b50d4 Merge branch 'beta' into hierarchical_orgs_relations Claudio Atzori 2021-10-20 09:42:35 +0200
bc3372093e Merge pull request '[stats] affiliations in stats and monitor dbs' (#152) from antonis.lempesis/dnet-hadoop:beta into beta Claudio Atzori 2021-10-20 09:40:34 +0200
241dcf6df1 Merge branch 'beta' into beta Antonis Lempesis 2021-10-19 23:54:21 +0200
515e068a78 Merge branch 'beta' into hierarchical_orgs_relations Claudio Atzori 2021-10-19 16:46:06 +0200
512e7b0170 code formatting Claudio Atzori 2021-10-19 16:19:29 +0200
c4fce785ab fixed a compilation problem of a unit test Michele Artini 2021-10-19 16:18:26 +0200
d517c71458 Merge branch 'dump' into beta Claudio Atzori 2021-10-19 16:15:42 +0200
e9157c67aa Merge branch 'beta' into dump Claudio Atzori 2021-10-19 16:15:03 +0200
98f37c8d81 WIP: worflow nodes for including Scholexplorer records in the RAW graph Claudio Atzori 2021-10-19 16:14:40 +0200
c8850456e9 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta Claudio Atzori 2021-10-19 16:09:54 +0200
172363e7f1 [broker] integrating PR#147, notification record creation phase separated from indexing on ES Claudio Atzori 2021-10-19 15:56:27 +0200
bdffa86c2f undo last commit Claudio Atzori 2021-10-19 15:39:38 +0200
c9870c5122 code formatted Sandro La Bruzzo 2021-10-19 15:24:59 +0200
f8329bc110 since dhp-schemas changed, introducing new Relation inverse model, this class has been updated Sandro La Bruzzo 2021-10-19 15:24:22 +0200
e471f12d5e hotfix: recovered implementation removing the hardcoded working_dirs Claudio Atzori 2021-10-19 12:35:38 +0200
7a73010acd WIP: worflow nodes for including Scholexplorer records in the RAW graph Claudio Atzori 2021-10-19 11:59:16 +0200
c7f6cd2591 added again the setting for saXReader Miriam Baglioni 2021-10-19 10:15:26 +0200
a894d7adf3 updated version of dhp-schemas Sandro La Bruzzo 2021-10-19 10:02:55 +0200
5f780a6ba1 bug fix in migrate entities: parameter name was wrong miconis 2021-10-18 23:30:40 +0200
1315952702 merge with branch beta Miriam Baglioni 2021-10-18 14:17:09 +0200
1cc09adfaa Opencitations: chenaged the test class to mirror the creation or not of duplicate dois for .refs oc original plus added optional parameter to duplicate the relation Miriam Baglioni 2021-10-18 14:11:27 +0200
76d41602be Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta Miriam Baglioni 2021-10-18 10:53:22 +0200
46f82c7c8f removed not needed folder deletion Miriam Baglioni 2021-10-18 10:53:16 +0200
7b15b88d4c renamed wrong package, implemented last aggregation workflow for scholexplorer Sandro La Bruzzo 2021-10-15 15:00:15 +0200
41ecb1eb61 invalidating medatadata before context thingies Antonis Lempesis 2021-10-15 13:42:55 +0300
4b7c8dff2d fetching affiliated results for 4 orgs in monitor. fixed affiliated orgs in stats db Antonis Lempesis 2021-10-14 18:53:35 +0300
e15a1969a5 applying fix on the DOIBoost construction process that somehow wasn't part of the merge done in 83c90c7180 Claudio Atzori 2021-10-14 14:33:56 +0200
51a03c0a50 refactor code for EBI from dhp-graph-mapper into dhp-aggregation Sandro La Bruzzo 2021-10-14 14:23:05 +0200
dd568ec88b Merge pull request 'Refactoring Solr Configuration' (#148) from beta_solr_config into beta Claudio Atzori 2021-10-14 12:45:11 +0200
14fbf92ad6 Merge branch 'beta' into beta_solr_config Claudio Atzori 2021-10-14 11:08:44 +0200
4b1920f008 changed the working path parameter value as dependant from the dnet-workflow working dir parameter Miriam Baglioni 2021-10-13 15:33:50 +0200
8db39c86e2 added new parameter in the doiboost process workflow to specify a folder for the process of MAG dataset Miriam Baglioni 2021-10-13 11:34:24 +0200
b292e4a700 [stats wf] added extra logging in the context data retrieval phase Claudio Atzori 2021-10-13 17:31:53 +0200
995c1eddaf minor change miconis 2021-10-13 17:07:10 +0200
5d9cc2452d changed the working path parameter value as dependant from the dnet-workflow working dir parameter Miriam Baglioni 2021-10-13 15:33:50 +0200
326bf63775 integration of parent child orgs relations miconis 2021-10-13 12:24:48 +0200
16b28494a9 added new parameter in the doiboost process workflow to specify a folder for the process of MAG dataset Miriam Baglioni 2021-10-13 11:34:24 +0200
63933808d4 added fix for mixing result types, added configuration default to funder subworkflow Miriam Baglioni 2021-10-13 11:28:28 +0200
f2c8356ccf Merge branch 'beta' of code-repo.d4science.org:D-Net/dnet-hadoop into beta Sandro La Bruzzo 2021-10-12 12:36:40 +0200
7387416e90 added params skip update to direct transform in OAF, this should be set to true in production Sandro La Bruzzo 2021-10-12 12:36:30 +0200
914b3e92cb updating graph schema module dependency to version 2.8.20 to include organization parent/child relation constants Claudio Atzori 2021-10-12 12:00:45 +0200