Commit Graph

70 Commits

Author SHA1 Message Date
Miriam Baglioni 292c69d819 [SKG-IF-EOSC] added new step in the resumeFrom 2024-04-04 12:23:53 +02:00
Miriam Baglioni 059b275a06 [SKG-IF-EOSC] fixed issue in selecting relevant eosc results. Applied static mapping from old eoscDsId and new eoscDsId 2024-03-29 11:46:46 +01:00
Miriam Baglioni 4c7e24df81 [SKG-IF-EOSC] added id for the eosc datasource at the level of the materialization 2024-03-27 09:45:13 +01:00
Miriam Baglioni 8fe452b61d [SKG-IF denormalized] refactoring 2024-03-26 11:45:59 +01:00
Miriam Baglioni 9f9ade077b [SKG-IF] changed the implementation to reduce the number of needed joins. Reduced the memory usage by defining specific flat beans 2024-03-18 09:30:05 +01:00
Miriam Baglioni 3c4c4e8ce0 [SKG-IF] tries to make the process finish. need to change the strategy 2024-03-16 08:44:10 +01:00
Miriam Baglioni 98bec3d2d2 [SKG-IF] changed workflow parametrization to avoid OOM error 2024-03-14 15:54:12 +01:00
Miriam Baglioni 187b91a699 [SKG-IF] fixing issue in deserialization 2024-03-14 13:02:47 +01:00
Miriam Baglioni e8f19ad003 [SKG-IF] selection of subset of relevant results from the set provided via input 2024-03-13 15:22:56 +01:00
Miriam Baglioni a6a6922f11 [SKG-IF] added first implementation for denormalization 2024-03-04 16:28:52 +01:00
Miriam Baglioni 752fd896e4 [SKG-IF] refactoring and fixing issues 2024-03-01 09:35:15 +01:00
Miriam Baglioni 0c887ca015 [SKG-IF] mapping to version latest in date 27 february 2024 2024-02-27 12:35:34 +01:00
Miriam Baglioni e4b56a4f88 [Organization] dump for organizations only 2024-02-06 14:13:12 +02:00
Miriam Baglioni 3ad0d6edfc - 2024-01-18 11:18:53 +01:00
Miriam Baglioni 5dcb7019a5 fixed left over 2024-01-08 12:26:16 +01:00
Miriam Baglioni 253ffb42f6 merge origin master after PR merge branch 'master' of https://code-repo.d4science.org/D-Net/dhp-graph-dump 2024-01-08 12:10:45 +01:00
Miriam Baglioni ac8ef53d02 removed last islookupur. Added new data for the irish tender 2023-12-19 11:08:04 +01:00
Miriam Baglioni 08f0b1c84c removed not needed parameters 2023-11-27 11:00:07 +01:00
Miriam Baglioni b2ca6b3bb9 modified the .gitignore file 2023-11-22 12:24:48 +01:00
Miriam Baglioni 566c1a9e4e - 2023-11-21 16:57:15 +01:00
Miriam Baglioni d170789adc removed not needed parameter 2023-11-13 18:20:34 +01:00
Miriam Baglioni 0a2c00ce29 - 2023-11-13 18:13:08 +01:00
Miriam Baglioni cc86f24372 removed not needed parameter 2023-11-13 11:38:56 +01:00
Miriam Baglioni d9ca135c1f removed not needed parameter 2023-11-13 10:33:31 +01:00
Miriam Baglioni 332c02c2c1 removed not needed parameter 2023-11-10 14:59:16 +01:00
Miriam Baglioni 998048b494 - 2023-11-10 14:42:07 +01:00
Miriam Baglioni 818bb4b11c removing interaction with the IS. Using communityAPIs instead 2023-10-30 14:28:55 +01:00
Miriam Baglioni e91636817c - 2023-10-25 17:32:09 +02:00
Miriam Baglioni 5529bbe3cc add retry with exponential backof and delay between the calls 2023-09-29 15:53:08 +02:00
Miriam Baglioni 32d64dd7a1 added possibility to copy the graph from hive 2023-09-22 15:25:53 +02:00
Miriam Baglioni f6677429c7 fixed conflicts 2023-08-17 15:03:41 +02:00
Miriam Baglioni d1f41b8e28 removed organization without legalname and legalshortname from the dump 2023-08-17 10:14:20 +02:00
Miriam Baglioni 5fb58362c5 moved parameter file. Added 40| as prefix on projects for computing the delta 2023-08-04 17:18:15 +02:00
Miriam Baglioni 097905171a adding master duplicate to avoid join of relation. Changed the model for the indicators 2023-08-04 16:22:23 +02:00
Miriam Baglioni 6b113961c1 - 2023-07-28 10:26:22 +02:00
Miriam Baglioni 0482648131 merg 2023-07-17 16:24:57 +02:00
Miriam Baglioni 25be584028 [dumpSubset] aligned with master 2023-07-15 11:12:27 +02:00
Miriam Baglioni b01573e201 [dumpCSV] removed output directory before starting the jobs 2023-07-12 07:38:53 +02:00
Miriam Baglioni e87b790a60 - 2023-06-22 16:54:13 +02:00
Miriam Baglioni 2e0999a1df First implementation of the csv dump 2023-05-29 10:16:47 +02:00
Miriam Baglioni 66873c1744 [DUMP CSV] Dumping of the results, of the authors and the relationships between results and authors and results and pids 2023-05-17 16:56:28 +02:00
Miriam Baglioni b9076f9aa8 [DUMP CSV] model classes to mirror the attributes of the tables to be dumped 2023-05-16 14:06:25 +02:00
Miriam Baglioni d0f144d422 first implementation for the dump in csv of the subset of the graph related to specific communities. The only relations considered are the cites. the source must be within the set of communties, the target con be outside => we also have to map nodes not related to the communities of interest. These communities are given as parameter 2023-05-11 16:44:54 +02:00
Miriam Baglioni 1fb840ff28 added test classes and resources. removed one step from the workflow since it was not needed 2023-05-04 12:05:10 +02:00
Miriam Baglioni 011b7737ad - 2023-05-02 15:47:06 +02:00
Miriam Baglioni 1671e78e59 - 2023-04-21 11:32:07 +02:00
Miriam Baglioni 43e9286db2 Changed the code for the production of the dump for FCT 2023-04-05 19:00:10 +02:00
Miriam Baglioni 80d51cea56 change dipendency from the workflow (leftover with old library name 2023-03-30 10:29:00 +02:00
Miriam Baglioni 4dcd03b78e minor and fixed wronf number is test because of change in the input resource 2022-12-31 13:00:00 +01:00
Miriam Baglioni 5e36b80dc1 merge with changeMeasure 2022-12-29 15:14:20 +01:00