sabeel
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Pull Requests Activity
4,028 Commits 55 Branches 18 Tags 649 MiB
Commit Graph

1256 Commits

Author SHA1 Message Date
Claudio Atzori 09ccc7b472 Merge branch 'beta' into project_organization_contribution 2022-07-28 09:49:59 +02:00
Miriam Baglioni 5968ec018d [Clean Country] modified workflow and added param file 2022-07-22 16:48:38 +02:00
Miriam Baglioni a12d28c644 [Clean Country] added logic not to remove country from result if it exist a hosting datasource with that country. Moreover the country will be removed only if added with propagation 2022-07-22 16:23:12 +02:00
Miriam Baglioni 2c933f1158 mergin with branch beta 2022-07-22 14:57:41 +02:00
Sandro La Bruzzo ddc414b258 fixed wrong json param 2022-07-22 09:43:15 +02:00
Sandro La Bruzzo 5f651f2316 changed filter relation on SubRelType 2022-07-21 10:11:48 +02:00
Miriam Baglioni 65cc736e2f [Clean Country] first implementation to remove country NL from results collected from NARCIS when doi starts with mendely prefix 2022-07-20 17:05:56 +02:00
Sandro La Bruzzo 5b76321d9c implemented oozie workflow to generate scholix dump filtering relclass semantic 2022-07-20 16:34:32 +02:00
Claudio Atzori 1138b2ac8e code formatting 2022-07-19 14:15:49 +02:00
Claudio Atzori 0c1cfee396 mapping oaf:fulltext elements in the result.fulltext field 2022-07-11 17:34:59 +02:00
Claudio Atzori 0cb1c70788 code formatting 2022-07-01 10:44:08 +02:00
Claudio Atzori 4ec13e2b66 Merge branch 'master' into dump_new_funded_products 2022-07-01 10:30:28 +02:00
Claudio Atzori 7da24c1dec added more logging 2022-06-28 13:47:49 +02:00
Miriam Baglioni 71744a1f52 [DUMP DELTA PROJECTS] refactoring 2022-06-27 18:07:58 +02:00
Miriam Baglioni 1d1fe3b151 [DUMP DELTA PROJECTS] refactoring 2022-06-27 18:04:59 +02:00
Claudio Atzori a8773af0cb Merge branch 'beta' into project_organization_contribution 2022-06-27 09:37:40 +02:00
Claudio Atzori 5130eac247 mapping by participant project contribution 2022-06-24 17:16:42 +02:00
Miriam Baglioni edddfc6c63 [DUMP DELTA PROJECTS] adding test and resource 2022-06-21 18:28:53 +02:00
Miriam Baglioni f561f13dd9 [Funder Products Dump] fixed names of parameters in workflow 2022-06-21 18:18:17 +02:00
Miriam Baglioni ff74e73369 [DUMP NEW FUNDED PRODUCTS] change in resources 2022-06-21 18:02:51 +02:00
Miriam Baglioni b98f904d48 [Funder Products Dump] new way to avoid using hive 2022-06-21 17:52:27 +02:00
Miriam Baglioni 7423577a08 [Graph DUMP] add code to produce the delta of new projects with respect to the previous delta/dump 2022-06-21 14:51:38 +02:00
Claudio Atzori b295a40d9c restored use of name_particles when parsing author names 2022-06-16 12:20:43 +02:00
Claudio Atzori 4c8e820ff0 mapping relationship from trasformed records based on oaf:relation 2022-06-14 08:49:02 +02:00
Claudio Atzori 116902c028 mapping relationship from trasformed records based on oaf:relation 2022-06-13 14:31:48 +02:00
Alessia Bardi 68bd58d6a4 tests for ROHub 2022-06-10 17:29:11 +02:00
Claudio Atzori 52cb086506 [graph grouping] drop relation target path before copying from source 2022-05-16 12:08:36 +02:00
Claudio Atzori 997c50078e [graph grouping] drop relation target path before copying from source 2022-05-16 12:07:40 +02:00
Claudio Atzori 6031acb2e3 [openorgs] fixed parent/child query, using the correct semantic labels 2022-05-16 09:20:48 +02:00
Claudio Atzori 0dc33ea391 [openorgs] fixed parent/child query, using the correct semantic labels 2022-05-16 09:20:30 +02:00
Miriam Baglioni e4eac1d20b [EOSC TAG] added code to remove EOSC Jupyter Notebook from subjects and put EOSC as classid in the qualifier 2022-05-13 11:01:33 +02:00
Sandro La Bruzzo 22f65680b9 Merge branch 'beta' of code-repo.d4science.org:D-Net/dnet-hadoop into beta 2022-05-11 15:30:12 +02:00
Sandro La Bruzzo ca8d26bcb4 added better filter for openCitations 2022-05-11 15:29:57 +02:00
Claudio Atzori 5d3b4a9c25 [graph merge beta] merge datasource originalid, collectedfrom, and pid lists 2022-05-11 14:13:06 +02:00
Claudio Atzori 2a8e0fb72f [openorgs] mapping parent/child relations without massaging the semantic labels 2022-05-10 08:45:53 +02:00
Claudio Atzori 77bc9863e9 [openorgs] mapping parent/child relations without massaging the semantic labels 2022-05-09 16:06:04 +02:00
Claudio Atzori 378020e30a [eosc_services] unit test adaptation 2022-05-09 16:05:06 +02:00
Claudio Atzori 846975c886 [eosc_services] using the correct 'keyword' subject type, as declared in the dnet:subject_classification_typologies vocabulary 2022-05-05 11:37:58 +02:00
Claudio Atzori da611cfbbd [eosc_services] resolved merge conflicts 2022-05-03 13:37:15 +02:00
Claudio Atzori 2ade69dea6 EOSC Services - minor 2022-05-02 17:03:31 +02:00
Claudio Atzori b6a7ff3a99 EOSC Services - removed fields from mapping, testing preparation 2022-05-02 15:52:33 +02:00
Claudio Atzori a8c51f6f16 EOSC Services - fixed query and testing preparation 2022-05-02 11:09:03 +02:00
Claudio Atzori 05c1ea92e9 EOSC Services - added Service-specific fields in the XML record serialization 2022-04-29 15:56:55 +02:00
Claudio Atzori f5f532d134 EOSC Services - ongoing update 2022-04-29 12:25:24 +02:00
Claudio Atzori 5ffc24d1ba EOSC Services - ongoing update 2022-04-26 16:18:41 +02:00
Miriam Baglioni 19d90658fc [Clean Context] added description to parameters 2022-04-22 15:41:23 +02:00
Miriam Baglioni e0915061c2 [Clean Context] fixed issue in param name 2022-04-21 16:32:40 +02:00
Miriam Baglioni 9a961a0092 [Clean Context] fixed issue in param name 2022-04-21 15:12:24 +02:00
Miriam Baglioni 5b7d9e741c [Clean Context] added logic to cleaning workflow to accomodate also context cleaning 2022-04-21 13:02:14 +02:00
Miriam Baglioni ccba1a3db1 [Clean Context] added logic to cleaning workflow to accomodate also context cleaning 2022-04-21 13:00:06 +02:00
Miriam Baglioni a38f0f5ea7 mergin with branch beta 2022-04-20 15:44:18 +02:00
Miriam Baglioni dbfbe8841a [Clean Context] changed the description in input parameters 2022-04-20 15:41:03 +02:00
Michele Artini c96a8613f8 update SQL queries 2022-04-20 12:07:49 +02:00
Michele Artini 4314db55c8 migration to services: update sql queries 2022-04-19 15:05:02 +02:00
Claudio Atzori c26222623f [maven-release-plugin] prepare for next development iteration 2022-04-07 13:32:22 +02:00
Claudio Atzori 86585a6b27 [maven-release-plugin] prepare release dhp-1.2.4 2022-04-07 13:32:19 +02:00
Claudio Atzori ad85d88eaf [maven-release-plugin] rollback the release of dhp-1.2.4 2022-04-07 13:28:35 +02:00
Claudio Atzori 598e11dfd7 [maven-release-plugin] prepare for next development iteration 2022-04-07 13:27:02 +02:00
Claudio Atzori db3d9877a5 [maven-release-plugin] prepare release dhp-1.2.4 2022-04-07 13:26:58 +02:00
Claudio Atzori 3bba6d6e38 [maven-release-plugin] rollback the release of dhp-1.2.4 2022-04-07 12:23:17 +02:00
Claudio Atzori 2ac2d928bd [maven-release-plugin] prepare for next development iteration 2022-04-07 12:18:47 +02:00
Claudio Atzori 85bc722ff4 [maven-release-plugin] prepare release dhp-1.2.4 2022-04-07 12:18:43 +02:00
Claudio Atzori bc05b6168a [maven-release-plugin] rollback the release of dhp-1.2.4 2022-04-07 11:49:06 +02:00
Claudio Atzori 505420fd61 [maven-release-plugin] prepare for next development iteration 2022-04-07 11:34:06 +02:00
Claudio Atzori 66e718981e [maven-release-plugin] prepare release dhp-1.2.4 2022-04-07 11:34:02 +02:00
Claudio Atzori 05fafa1408 [graph raw] avoid NPEs importing datasource consent fields 2022-04-06 15:23:50 +02:00
Claudio Atzori 8c457f1b2c conflicts resolved, merged from beta 2022-04-06 10:27:52 +02:00
Miriam Baglioni 79336d46c5 [Clean Context] first naive implementation of a functionality to clean not wanted contextes from one result. This implementation simply verifies the main title of the results start with a given string 2022-04-04 15:52:31 +02:00
Claudio Atzori 0a0ae84c22 [graph raw] DOI based instance URLs on https 2022-03-29 10:52:58 +02:00
Claudio Atzori 741bc99c47 Merge branch 'beta' into datasource_pdf_consent 2022-03-28 09:20:48 +02:00
Miriam Baglioni 89fd275480 [HostedByMap] added left over from PR and fixed issue on workflow 2022-03-21 09:54:45 +01:00
Miriam Baglioni 0f7d8ca2e0 [HostedByMap] change on master to align to PR 201 on beta merged as 9f3036c847 2022-03-11 15:16:02 +01:00
Claudio Atzori f25407bbe2 added mapping for datasource consent fields to integrate them in the graph 2022-03-11 09:32:42 +01:00
Miriam Baglioni 2c5087d55a [HostedByMap] download of doaj from json, modification of test resources, deletion of class no more needed for the CSV download 2022-03-04 15:18:21 +01:00
Miriam Baglioni 5d608d6291 [HostedByMap] changed the model to include also oaStart date and review process that could be possibly used in the future 2022-03-04 11:06:09 +01:00
Miriam Baglioni 8a41f63348 [HostedByMap] update to download the json instead of the csv 2022-03-04 10:38:43 +01:00
Miriam Baglioni 44b0c03080 [HostedByMap] update to download the json instead of the csv 2022-03-04 10:37:59 +01:00
Claudio Atzori a87c070447 conflicts resolved, merged from beta 2022-02-24 12:51:31 +01:00
Claudio Atzori 99f5b14469 [graph raw] invisible records stored among the raw graph rather than the claimed subgraph 2022-02-18 15:20:57 +01:00
Claudio Atzori cf8443780e added processingchargeamount to the result view 2022-02-18 15:17:48 +01:00
Alessia Bardi 600ede1798 serialisation of APCs int he XML records 2022-02-11 11:00:20 +01:00
Miriam Baglioni 493caef358 [stats-wf]fixed the result_result table related to PR#191 2022-02-04 14:51:25 +01:00
Miriam Baglioni aae667e6b6 [APC at the result level] added the APC at the level of the result and modified test class 2022-02-04 12:34:25 +01:00
Alessia Bardi 2e215abfa8 test for instances with URLs for OpenAPC 2022-02-02 17:27:44 +01:00
Claudio Atzori 8eb75ca169 adapted GenerateEntitiesApplicationTest behaviour 2022-01-27 16:24:37 +01:00
Claudio Atzori af61e44acc ported changes to the GraphCleaningFunctionsTest from 8de9788308 2022-01-27 16:19:14 +01:00
Claudio Atzori 1322379741 Merge branch 'beta' into delegated_authorities 2022-01-25 14:28:25 +01:00
Claudio Atzori 59a250337c [graph resolution] drop output path at the beginning 2022-01-24 18:02:39 +01:00
Claudio Atzori 97ad94d7d9 [graph resolution] drop output path at the beginning 2022-01-24 18:02:07 +01:00
Claudio Atzori 8de9788308 applied fix for avoiding ruling out the invisible (APC) records during the graph cleaning 2022-01-24 11:29:22 +01:00
Claudio Atzori dd52bf1bb8 copy relations to the graphOutputPath 2022-01-21 13:59:29 +01:00
Claudio Atzori 4983d6536d Merge branch 'beta' into delegated_authorities 2022-01-21 13:02:48 +01:00
Claudio Atzori f0ea2410e5 improved mapping titles from datacite records to consider title types 2022-01-21 10:50:34 +01:00
Claudio Atzori 3b9020c1b7 added unit test for the DispatchEntitiesJob 2022-01-19 18:15:55 +01:00
Claudio Atzori abfa9c6045 code formatting 2022-01-19 17:17:11 +01:00
Claudio Atzori 391aa1373b added unit test 2022-01-19 17:13:21 +01:00
Claudio Atzori 44a937f4ed factored out entity grouping implementation, extended to consider results from delegated authorities rather than identical records from other sources 2022-01-19 12:24:52 +01:00
Miriam Baglioni a7c4d0d16d [DoiBoost Organizations] added parameter to specify the action in the wf raw_organizations to be able to load the openorgs organization as in the loading step for the construction of the graph 2022-01-13 13:52:00 +01:00
Sandro La Bruzzo 57e2c4b749 formatted code 2022-01-12 09:40:28 +01:00
Claudio Atzori 4f212652ca scalafmt: code formatting 2022-01-11 16:57:48 +01:00