lsmyrnaios
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Pull Requests Activity

Commit Graph

  • f2352e8a78 changed in the classes the path for the property files for the propagation of community from project Miriam Baglioni 2023-12-22 11:43:34 +0100
  • 009730b3d1 added properties file in the forlder for the workflow of orcid propagation. Changes the path in the classes implementing the propagationchanged the path to the parameter file in the class for entitytoorganization propagation Miriam Baglioni 2023-12-22 11:42:09 +0100
  • 89f269c7f4 changed the path to the parameter file in the class for entitytoorganization propagation Miriam Baglioni 2023-12-22 11:37:50 +0100
  • b06aea0adf adding the bulkTag parameter file in the folder for the oozie workflow for bulkTagging. Changes the path in the class Miriam Baglioni 2023-12-22 11:35:37 +0100
  • 3afd4aa57b adjustments for country propagation Miriam Baglioni 2023-12-22 11:27:30 +0100
  • ffdd03d2f4 Monitor Irish Stats WF dimitrispie 2023-12-22 11:05:24 +0200
  • 40b98d8182 Changes to indicators and funders definition dimitrispie 2023-12-22 10:29:20 +0200
  • 62104790ae added metaresourcetype to the result hive DB view Claudio Atzori 2023-12-21 12:26:19 +0100
  • 5011c4d11a refactoring after compiletion Miriam Baglioni 2023-12-20 15:57:26 +0100
  • 4740c808f7 - Miriam Baglioni 2023-12-20 14:26:54 +0100
  • 17282ea8fc - Fix the "is not NULL" checks inside "spark.filter()" - Make sure the "outputPath" ends with a "/", in any case. - Fix a parameter-description. Lampros Smyrnaios 2023-12-20 15:15:56 +0200
  • d410ea8a41 added needed parameter Miriam Baglioni 2023-12-19 12:15:01 +0100
  • 22fa60c3dd removed lib biinary Claudio Atzori 2023-12-18 15:50:29 +0100
  • 24173d7a0b continuous validation WIP Claudio Atzori 2023-12-18 15:46:36 +0100
  • 624f5f3f21 [Transformative Agreement] added check to verify the APC were paid byu the IReL funder Miriam Baglioni 2023-12-18 15:28:19 +0100
  • 354e02e6a9 [Transformative Agreement] removed not needed class. Read directly the json and no need to pass from the csv Miriam Baglioni 2023-12-18 15:20:27 +0100
  • b00771c7cc [Transformative Agreement] added code to extract relations from the transformative agreement file for the IE products got from OpenAPC Miriam Baglioni 2023-12-18 15:12:44 +0100
  • 15fd93a2b6 uploaded input parameters on CreateBaseline WF Sandro La Bruzzo 2023-12-18 12:21:55 +0100
  • 9d342a47da updated the transformation Baseline workflow to include mdstore rollback/commit action Sandro La Bruzzo 2023-12-18 11:48:57 +0100
  • a2feda6c07 - Fix acquiring the "openaire_guidelines" parameter. - Use the right Guidelines-profile, depending on the "openaire_guidelines" version. - Update log-levels. - Optimize imports. continuous-validation Lampros Smyrnaios 2023-12-18 10:57:40 +0200
  • 3eca5d2e1c - Miriam Baglioni 2023-12-18 09:55:27 +0100
  • b71633fd7f - Fix the location of the "input_continuous_validator_parameters.json" file. - Fix handing the "isSparkSessionManaged" parameter. - Add the "provided" scope for some dependencies. They do not inherit it from the main pom, since the "version" tag is declared, even though the value is the same as the one from the main pom. - Code polishing / cleanup. Lampros Smyrnaios 2023-12-15 18:29:38 +0200
  • 9e6a03e4e2 Initial commit of the "dhp-continuous-validation" module. Lampros Smyrnaios 2023-12-15 15:53:31 +0200
  • 01ce0b9c76 [doiboost - preprocess] remove transition to orcid preparation from sequence of steps at the beginning of the workflow Miriam Baglioni 2023-12-15 12:24:55 +0100
  • 0d8e496a63 - Miriam Baglioni 2023-12-15 12:16:43 +0100
  • a59be5779e Merge pull request '9078_xml_records_irish_tender' (#368) from 9078_xml_records_irish_tender into beta Claudio Atzori 2023-12-12 12:34:43 +0100
  • ff924215b8 [graph provision] added tests for new peerreviewed field Claudio Atzori 2023-12-12 11:21:30 +0100
  • a6d635e695 Merge branch 'beta' into 9078_xml_records_irish_tender Claudio Atzori 2023-12-12 11:06:42 +0100
  • 98cce5bfb2 code formatting Claudio Atzori 2023-12-12 09:59:05 +0100
  • 84d54643cf [cleaning] allow enriched orcids to pass the cleaning, rule out non-orcid author pids Claudio Atzori 2023-12-12 09:57:00 +0100
  • 7e8eff40c1 [graph provision] added tests for the new model fields 9078_xml_records_irish_tender Claudio Atzori 2023-12-12 08:54:15 +0100
  • a6c7217df1 Do no longer use dedupId information from pivotHistory Database dedup_increasenumofblocks Giambattista Bloisi 2023-12-11 21:26:05 +0100
  • 8752d275fa removed not needed parameter Miriam Baglioni 2023-12-09 15:24:45 +0100
  • d4eedada71 adjusting workflow definition Miriam Baglioni 2023-12-09 15:20:11 +0100
  • aba95ed1d1 code formatting Claudio Atzori 2023-12-08 17:06:19 +0100
  • 2877839df0 Merge pull request '[graph cleaning] added cleaning for result.publisher and result.instance.license' (#366) from clean_license_publisher into beta Claudio Atzori 2023-12-08 16:58:37 +0100
  • 34abd0fc43 Merge branch 'beta' into clean_license_publisher clean_license_publisher Claudio Atzori 2023-12-08 16:58:27 +0100
  • cb71a7936b [graph cleaning] avoid stack overflow error when navigating Oaf objects declaring an Enum Claudio Atzori 2023-12-07 23:09:54 +0100
  • 70eb1796b2 logging typo Claudio Atzori 2023-12-07 14:08:04 +0100
  • c381bacee0 [enrichment] passing the community API base URL Claudio Atzori 2023-12-07 14:07:11 +0100
  • 336fb31d87 [community_result_propagation] adjusting starting poit of workflow Miriam Baglioni 2023-12-07 10:27:25 +0100
  • c0cde53bf6 [bulktagging] setting first step of bulktaggin as the copy of the entities and relations not involved in the tagging' Miriam Baglioni 2023-12-07 10:08:35 +0100
  • 616622d2bb first version of the workflow single step Miriam Baglioni 2023-12-07 09:59:52 +0100
  • 259c69e446 [orcid enrichment] fixed workflow definition Claudio Atzori 2023-12-06 19:41:53 +0100
  • 431c6bb08a [dedup] added isLookupUrl to the graph consistency workflow definition, required now by the entity grouping phase Claudio Atzori 2023-12-06 11:06:46 +0100
  • 613ec5ffce Add profiles for different spark versions: spark-24, spark-34, spark-35 spark34-integration Giambattista Bloisi 2023-09-21 14:23:37 +0200
  • 52495f2cd2 used javax.xml.stream.XMLEventReader instead of deprecated scala.xml.pull.XMLEventReader Sandro La Bruzzo 2023-09-18 13:58:22 +0200
  • 8c3e9a09d3 added repository openaire-third-parties Sandro La Bruzzo 2023-09-18 12:51:18 +0200
  • 2fa78f6071 Changes requires to build and run tests with Java 17 Giambattista Bloisi 2023-09-07 11:58:59 +0200
  • 326c9dc08c Changes in maven poms to build and test the project using Spark 3.4.x and scala 2.12 Giambattista Bloisi 2023-08-02 18:05:53 +0200
  • 982c0c110b Merge pull request '[graph provision] added serialization for the new fields imported from the stats DB' (#365) from 9078_xml_records_irish_tender into beta Claudio Atzori 2023-12-05 16:39:44 +0100
  • 321922772b added serialization for the new fields imported for the Irish tender Claudio Atzori 2023-12-05 16:37:04 +0100
  • c5b7253130 [community_organization propagation] fixed workflow parameters Claudio Atzori 2023-12-05 09:13:33 +0100
  • 3c3bdb8318 [bulktagging] fixed workflow parameters Claudio Atzori 2023-12-05 09:08:48 +0100
  • b0fc113749 SparkCreateSimRels: - Create dedup blocks from the complete queue of records matching cluster key instead of truncating the results - Clean titles once before clustering and similarity comparisons - Added support for filtered fields in model - Added support for sorting List fields in model - Added new JSONListClustering and numAuthorsTitleSuffixPrefixChain clustering functions - Added new maxLengthMatch comparator function - Use reduced complexity Levenshtein with threshold in levensteinTitle - Use reduced complexity AuthorsMatch with threshold early-quit - Use incremental Connected Component to decrease comparisons in similarity match in BlockProcessor - Use new clusterings configuration in Dedup tests Giambattista Bloisi 2023-10-02 09:25:12 +0200
  • 7c3041b276 avoid NPEs Claudio Atzori 2023-12-03 16:49:49 +0100
  • 74b185d07b avoid NPEs Claudio Atzori 2023-12-03 16:18:20 +0100
  • e6086efc53 avoid NPEs in Vocabulary.getTermBySynonym Claudio Atzori 2023-12-03 13:33:20 +0100
  • 2a233a89aa [graph grouping] added isLookupUrl to the workflow definition, passed to the grouping spark aciton Claudio Atzori 2023-12-03 13:32:52 +0100
  • 178a14c491 code formatting Claudio Atzori 2023-12-03 13:31:58 +0100
  • 3caf6ff27e Extracted the correct original type to pass to instanceTypeMapping in Crossref Mapping Sandro La Bruzzo 2023-12-01 16:33:56 +0100
  • 511a98dd80 fixed doiboost process workflow, removed references to the ProcessORCID step Claudio Atzori 2023-12-01 16:21:53 +0100
  • d33f578e54 code formatting Claudio Atzori 2023-12-01 15:14:17 +0100
  • c5ac593c07 Merge pull request 'ORCID Enrichment and Download' (#364) from orcid_import into beta Claudio Atzori 2023-12-01 15:05:44 +0100
  • 09d061e90b Merge branch 'beta' into orcid_import orcid_import Claudio Atzori 2023-12-01 15:05:35 +0100
  • 93a700742a Merge pull request 'Changes for tables and creation of the new indicator indi_is_result_accessible' (#363) from antonis.lempesis/dnet-hadoop:beta into beta Claudio Atzori 2023-12-01 15:05:23 +0100
  • 0c3c9ea43d Merge pull request 'StatsDB workflow to export actionsets about OA routes, diamond, and publicly-funded' (#355) from dimitris.pierrakos/dnet-hadoop:beta into beta Claudio Atzori 2023-12-01 15:03:56 +0100
  • 33cb483c75 using objectSubType as originalType in Crossref2Oaf, code formatting Claudio Atzori 2023-12-01 15:03:05 +0100
  • c9d995dde0 New institutions added dimitrispie 2023-12-01 15:44:35 +0200
  • a397112cb8 Add new indicator dimitrispie 2023-12-01 15:00:18 +0200
  • 76594ded23 Changes to indicators dimitrispie 2023-12-01 13:38:19 +0200
  • 622fafbd2e Merge branch 'beta' into orcid_import Claudio Atzori 2023-12-01 12:28:14 +0100
  • bf0fd27c36 Removed unused function Applied PR Comment of Giambattista in the PR Sandro La Bruzzo 2023-12-01 12:16:42 +0100
  • 48430a32a6 Update StatsAtomicActionsJob.java dimitrispie 2023-12-01 11:35:01 +0200
  • cdfb7588dd code formatting Sandro La Bruzzo 2023-11-30 15:31:42 +0100
  • 5e22b67b8a Merge remote-tracking branch 'origin/beta' into orcid_import Sandro La Bruzzo 2023-11-30 15:27:46 +0100
  • f718caaac9 Added copy of the untouched entities of the graph Sandro La Bruzzo 2023-11-30 14:51:00 +0100
  • 7b5e04f37e removed Orcid intersection on DOIBoost Sandro La Bruzzo 2023-11-30 14:36:50 +0100
  • 4cbabc9fbc Merge pull request '[ENRICHMENT][BETA] Use of community API in enrichment process AND addition to tagging result for communities through projects' (#359) from propagationapi into beta Claudio Atzori 2023-11-30 14:20:33 +0100
  • 6f10791e77 Merge branch 'beta' into propagationapi Claudio Atzori 2023-11-30 14:20:18 +0100
  • 4e1aac2e2f resolved conflict in pom.xml before applying the changes from [COAR based resource types & Irish tender] #350 Claudio Atzori 2023-11-29 14:37:52 +0100
  • 86b5775e08 added vocabulary in instanceTypeMapping for - DOIBoost - Datacite - PubMed - Scholexplorer Datasource Sandro La Bruzzo 2023-11-29 13:15:43 +0100
  • c96ff54b45 Merge remote-tracking branch 'origin/resource_types' into resource_types Sandro La Bruzzo 2023-11-29 12:45:41 +0100
  • af1c2634b3 added instanceTypeMapping original field in the mapping of - DOIBoost - Datacite - PubMed - Scholexplorer Datasource Sandro La Bruzzo 2023-11-29 12:45:30 +0100
  • 279100fa52 added test Sandro La Bruzzo 2023-11-29 11:17:58 +0100
  • aa239ec673 Changed implementation of check similarity to verify exact match of name instead of the first char Sandro La Bruzzo 2023-11-29 11:17:41 +0100
  • 59111713fa added comment Sandro La Bruzzo 2023-11-28 09:00:48 +0100
  • 6f4d0c05ea Implemented Author MErger for ORCID that takes in account the case when name and surname are swapped Sandro La Bruzzo 2023-11-28 08:43:56 +0100
  • 8eb70e6657 refactoring Miriam Baglioni 2023-11-27 15:13:15 +0100
  • e3cce9a5a0 mergin with branch beta Miriam Baglioni 2023-11-27 15:10:55 +0100
  • 48e0427a23 changed the parameter from production to baseURL. Fixed issue in tagging configuration Miriam Baglioni 2023-11-27 15:10:27 +0100
  • 34a4b3cbdf Implemented ORCID Enrichment Sandro La Bruzzo 2023-11-24 12:39:58 +0100
  • 1763d377ad code formatting master Claudio Atzori 2023-11-23 16:33:24 +0100
  • 1ba582de3c [graph cleaning] added cleaning for result.publisher and result.instance.license Claudio Atzori 2023-11-23 16:27:19 +0100
  • 359e81b7a6 Update StatsAtomicActionsJob.java dimitrispie 2023-11-23 10:48:55 +0200
  • a0311e8a90 Merge pull request 'Clear working dir in bipranker workflow' (#360) from 9120_bipranker_clean_working_dir into master Claudio Atzori 2023-11-22 14:10:39 +0100
  • 8fb05888fd Merge branch 'master' into 9120_bipranker_clean_working_dir Claudio Atzori 2023-11-22 14:10:30 +0100
  • a21617732a Merge pull request 'graph cleaning, suggestions from ticket 8898 - round 2' (#356) from cleaning_8898 into beta Claudio Atzori 2023-11-22 14:00:37 +0100
  • 2c77638bf5 Merge branch 'beta' into cleaning_8898 Claudio Atzori 2023-11-22 14:00:10 +0100
  • 836d7ec724 Merge pull request 'Add Pubmed affiliations (inferred by BIP) as actionsets' (#353) from 9117_pubmed_affiliations into beta Claudio Atzori 2023-11-22 13:53:07 +0100