Commit Graph

143 Commits

Author SHA1 Message Date
Miriam Baglioni 5ff50d115a [dumpCSV] ading double quotes enclosing all the fileds 2023-07-17 16:21:20 +02:00
Miriam Baglioni 81b55dc20b merging with master 2023-07-15 11:15:24 +02:00
Miriam Baglioni 25be584028 [dumpSubset] aligned with master 2023-07-15 11:12:27 +02:00
Miriam Baglioni 21a521b97c changed the API to consider the upload only of an already open version 2023-07-15 10:36:54 +02:00
Miriam Baglioni b74d6f1c23 resolved conflict 2023-07-13 18:20:31 +02:00
Miriam Baglioni 787d4d0b4a changed the pom reference to the dho schema 2023-07-13 18:19:12 +02:00
Miriam Baglioni b01573e201 [dumpCSV] removed output directory before starting the jobs 2023-07-12 07:38:53 +02:00
Miriam Baglioni 95125d704a [dump] removed usage stats info from the datasource and project 2023-07-11 12:40:21 +02:00
Miriam Baglioni abc30756e4 - 2023-07-07 18:42:41 +02:00
Miriam Baglioni ab791fe424 [master] update reference to ZenodoAPI 2023-07-07 18:12:17 +02:00
Miriam Baglioni 3bfac8bc6e [dumpCSV] addressing the issues fointed out by the Dare Lab people. Repeated relations from author to result due to the author repeated in the data. Repeated relations from result to result due to the same pid present in more that one result. Author table not properly formatted due to the bad formatting of the input data 2023-07-07 18:01:26 +02:00
Miriam Baglioni 9d1b708a89 [dumpCSV] addressing the issues fointed out by the Dare Lab people. Repeated relations from author to result due to the author repeated in the data. Repeated relations from result to result due to the same pid present in more that one result. Author table not properly formatted due to the bad formatting of the input data 2023-07-07 17:44:19 +02:00
Miriam Baglioni 8a44653dbe [DumpCSV] fixing issues 2023-07-05 09:58:55 +02:00
Miriam Baglioni 29b81bef26 refactoring 2023-07-01 11:54:48 +02:00
Miriam Baglioni 3fba247c38 refactoring 2023-07-01 11:07:41 +02:00
Miriam Baglioni 2ac5c4a9ab moved also the model and other linked classes to the dump project 2023-07-01 11:06:41 +02:00
Miriam Baglioni 766288d1c9 Merge branch 'dump_zenodo_2' of https://code-repo.d4science.org/D-Net/dhp-graph-dump into dump_zenodo_2 2023-07-01 10:39:37 +02:00
Miriam Baglioni b9d4d67c72 - 2023-06-30 19:06:15 +02:00
Sandro La Bruzzo d746390b9f new implementatiton with okhttp 2023-06-23 15:15:09 +02:00
Miriam Baglioni 72ead1bd85 added okhttp3 again 2023-06-23 14:16:15 +02:00
Sandro La Bruzzo 6ace388cff fixed method 2023-06-23 14:16:10 +02:00
Sandro La Bruzzo d472050ad4 Added new implementation upload huge file 2023-06-22 17:43:53 +02:00
Sandro La Bruzzo 5d0d14528f Added new implementation upload huge file 2023-06-22 16:54:17 +02:00
Miriam Baglioni e87b790a60 - 2023-06-22 16:54:13 +02:00
Miriam Baglioni 8661bc0c90 aligned with the last version of pom for production 2023-06-02 16:13:18 +02:00
Miriam Baglioni 2e8639f22d added test to verify the dump for indicators at the level of project and datasource. Fixed issue on identifier with the prefix 2023-06-01 15:10:00 +02:00
Miriam Baglioni 32983e90d1 change to the model of the Relation -> flatten: remove the node and add source, sourceType, target, targetType. Adding indicators at the level of Projects and Datasources. Removing the prefix from the identifier of the entities 2023-06-01 12:58:56 +02:00
Miriam Baglioni 2e0999a1df First implementation of the csv dump 2023-05-29 10:16:47 +02:00
Miriam Baglioni f79b9d5c0d [DUMP CSV] sligth modification 2023-05-17 16:58:04 +02:00
Miriam Baglioni 21599598ae [DUMP CSV] test and resources for the result dumps 2023-05-17 16:57:25 +02:00
Miriam Baglioni 66873c1744 [DUMP CSV] Dumping of the results, of the authors and the relationships between results and authors and results and pids 2023-05-17 16:56:28 +02:00
Miriam Baglioni 7563499740 [DUMP CSV] - 2023-05-16 14:29:31 +02:00
Miriam Baglioni f79c06209e [DUMP CSV] test and resources for the SelectResultAndDumpRelation job 2023-05-16 14:21:39 +02:00
Miriam Baglioni 2ed76d4662 [DUMP CSV] tested file to execute the dump of the relations with semantcis Cites from nodes belonging to a selected communities. It also dumps the relationships result_communities and prepare the ground for the dump of the results. 2023-05-16 14:20:45 +02:00
Miriam Baglioni 44a256fc90 [DUMP CSV] refactoring 2023-05-16 14:10:14 +02:00
Miriam Baglioni 636945a5c5 [DUMP CSV] refactoring 2023-05-16 14:09:21 +02:00
Miriam Baglioni b9076f9aa8 [DUMP CSV] model classes to mirror the attributes of the tables to be dumped 2023-05-16 14:06:25 +02:00
Miriam Baglioni acb3c691bc [DUMP CSV] added query and method to get the information to dump in the CSV regarding the selected communities 2023-05-16 14:04:44 +02:00
Miriam Baglioni d0f144d422 first implementation for the dump in csv of the subset of the graph related to specific communities. The only relations considered are the cites. the source must be within the set of communties, the target con be outside => we also have to map nodes not related to the communities of interest. These communities are given as parameter 2023-05-11 16:44:54 +02:00
Miriam Baglioni 1fb840ff28 added test classes and resources. removed one step from the workflow since it was not needed 2023-05-04 12:05:10 +02:00
Miriam Baglioni 011b7737ad - 2023-05-02 15:47:06 +02:00
Miriam Baglioni 6ba43a1b67 selects actual result per result type associated with the given country and saves them 2023-04-27 18:16:01 +02:00
Miriam Baglioni 7f57f3cd1e selection of the results id having the given country among the countries, or being in relation with other entities associated with the given country 2023-04-27 18:14:48 +02:00
Miriam Baglioni 1671e78e59 - 2023-04-21 11:32:07 +02:00
Miriam Baglioni 563c5d8527 - 2023-04-19 15:19:03 +02:00
Miriam Baglioni b6e0c7d660 changed the interaction with Zenodo since the API chenaged 2023-04-19 09:40:45 +02:00
Miriam Baglioni 43e9286db2 Changed the code for the production of the dump for FCT 2023-04-05 19:00:10 +02:00
Miriam Baglioni 80d51cea56 change dipendency from the workflow (leftover with old library name 2023-03-30 10:29:00 +02:00
Miriam Baglioni f738db860a refactoring 2023-01-25 11:52:51 +01:00
Miriam Baglioni 4dcd03b78e minor and fixed wronf number is test because of change in the input resource 2022-12-31 13:00:00 +01:00
Miriam Baglioni b743dc2960 removed class 2022-12-29 15:19:36 +01:00
Miriam Baglioni 5e36b80dc1 merge with changeMeasure 2022-12-29 15:14:20 +01:00
Miriam Baglioni ad1ba563cd update of the schema of the dump 2022-12-29 14:46:51 +01:00
Miriam Baglioni 8ec02787f2 minor changes 2022-12-28 23:00:37 +01:00
Miriam Baglioni 2d2b62386f removed indicators from Instance 2022-12-28 21:50:48 +01:00
Miriam Baglioni 71862838b0 [dump] removed relations extracted from products where the datasource was not in the graph 2022-12-27 10:00:47 +01:00
Miriam Baglioni b26ecd74ea merging with dumpSubset 2022-12-23 09:43:30 +01:00
Miriam Baglioni dc5e79dc64 [dumpSubset] added test to verify why sdsn-gr disappears from the community set 2022-12-23 09:42:49 +01:00
Miriam Baglioni 4bedecaa60 [dumpSubset] added the correct path to the context relations 2022-12-22 13:48:15 +01:00
Miriam Baglioni 62d8180891 [ChangeMeasure] semplified workflow 2022-12-22 09:54:21 +01:00
Miriam Baglioni db36a9be2e [Dump Subset] issue on the relations 2022-12-22 09:38:09 +01:00
Miriam Baglioni 45cc165e92 [Dump Subset] moved one step ahead the change of master in hosted by, collectedfrom 2022-11-30 09:54:45 +01:00
Miriam Baglioni 0a0e2cfc9c refactoring 2022-11-29 16:09:10 +01:00
Miriam Baglioni 054103ae70 [Dump Subset] fixed issue in workflow parameter 2022-11-29 15:32:08 +01:00
Miriam Baglioni 99fb3dc1d0 [Dump Subset] fixed issue in parameter file 2022-11-28 14:58:57 +01:00
Miriam Baglioni f26378f426 [Dump Subset] change code to read from db 2022-11-25 17:52:46 +01:00
Miriam Baglioni 67d48763fa [Dump subset] change the class to read from the db and added needed parameters in the workflow 2022-11-24 10:24:22 +01:00
Miriam Baglioni 0bb97fead7 [Dump Subset] fixing issue with missing datasource - serach in collectedfrom at the level of the result and select master id if duplicate id is inserted in hostedby or collectedfrom in the result-. Added new test 2022-11-22 15:58:50 +01:00
Miriam Baglioni d3da9ab2c6 [Dump Subset] fixing issue and finalizing workflow 2022-11-21 14:10:46 +01:00
Miriam Baglioni 8878b96204 [Dump Subset] first try fro dump subset and refactoring 2022-11-17 16:13:10 +01:00
Miriam Baglioni 31ce13ffb4 [DUMP INDICATORS ] refactoring 2022-11-10 09:40:42 +01:00
Miriam Baglioni 0a53c29a8f [DUMP INDICATORS ] added code and resource to test the serialization of indicators 2022-11-10 09:39:09 +01:00
Miriam Baglioni bdd1cfc1e0 [DUMP INDICATORS ] added code to serialize the indicators 2022-11-10 09:37:54 +01:00
Miriam Baglioni e222c2c4d7 [DUMP INDICATORS ] added new constants for the indicators 2022-11-10 09:37:28 +01:00
Miriam Baglioni 5e8cd02acd [DUMP INDICATORS ] adding a step of mapping to string with object mapper to support decorator in getter and setter to have 'class' as value for a serialized variable 2022-11-10 09:37:05 +01:00
Miriam Baglioni 4b339df43f [DUMP INDICATORS ] refactoring 2022-11-10 09:32:05 +01:00
Miriam Baglioni db657d79a6 [Dump] fixed issue on dump of datasource pids 2022-10-13 11:50:48 +02:00
Miriam Baglioni 3cc2802a75 [Dump] removing all EOSC related addition from master and fixed issue on dump of datasource pids 2022-10-13 11:49:37 +02:00
Miriam Baglioni 6ed41a9276 [Dump] removing all EOSC related addition from master 2022-10-11 11:55:32 +02:00
Miriam Baglioni 8a574fee2a [Dump] removing all EOSC related addition from master 2022-10-11 11:41:19 +02:00
Miriam Baglioni cebee39c22 [Dump] removing all EOSC related addition from master 2022-10-11 10:58:44 +02:00
Miriam Baglioni 746a06d4ea [Dump] removing all EOSC related addition from master 2022-10-11 10:57:50 +02:00
Miriam Baglioni 80e525e0c1 Changed the jar from dhp-graph-dump to dump 2022-10-04 12:37:24 +02:00
Miriam Baglioni 67ee82535a Changed the jar from dhp-graph-dump to dump 2022-10-04 12:20:08 +02:00
Miriam Baglioni 3fe35345c3 minor changes 2022-10-04 12:13:18 +02:00
Claudio Atzori 6a4589aa2f Merge branch 'master' into changeMeasure 2022-09-27 15:10:57 +02:00
Miriam Baglioni eb06474106 [Extend Measure] added test to verify the new serialization model and the serialization at the level of the result 2022-09-22 18:02:37 +02:00
Miriam Baglioni b5ee457969 added measure at the level of the result. Changed the way the measures are dumped since the previous serialization was not able to describe in the correct way the current association measure and value for the indicators (for BipFinder) 2022-09-22 15:50:39 +02:00
Miriam Baglioni 3b16d9040b change the dependency to dhp-schemas to 2.13.2-SNAPSHOT' 2022-09-22 15:13:13 +02:00
Miriam Baglioni 97f26a205a added test for the dump of the EOSC IF Tag. Fixed issue in serialization on the cluster 2022-08-04 14:23:19 +02:00
Miriam Baglioni 3905afa0c2 fixed format in measure after modification of the model 2022-08-03 15:40:28 +02:00
Miriam Baglioni e7eb17f73e first attempt at changing the measure element 2022-08-03 12:25:32 +02:00
Miriam Baglioni 5b32bf1482 added the schema of the dump 2022-08-03 11:42:56 +02:00