79 Commits

Author	SHA1	Message	Date
Miriam Baglioni	c0bebb7c35	code to compute the prepared information used in the actual propagation step. This step will produce who files: one with potential updates (association between projects and a list of results), the other already linked entities (association between projects and the list of results already linked to them)	2020-04-14 15:31:26 +02:00
Miriam Baglioni	f47ee5b78e	directory where to store the prepared infor before actual propagation will take place	2020-04-14 15:29:21 +02:00
Miriam Baglioni	36cc9516d8	the starting relation set for testing	2020-04-14 15:28:34 +02:00
Miriam Baglioni	4b01dc60e6	test unit for result to project propagation	2020-04-14 15:28:00 +02:00
Miriam Baglioni	8f12292daa	changed the way to save the results on filesystem	2020-04-11 16:47:34 +02:00
Miriam Baglioni	87f802821e	new workflow for country propagation: it is composed of the preparation step and in the propagation. The propagation part runs in parallel on the result types	2020-04-11 16:40:22 +02:00
Miriam Baglioni	a562080b0b	parameters to be used in the prepared Job and in the actual country propagation job	2020-04-11 16:39:17 +02:00
Miriam Baglioni	1251ad4455	removed unuseful class	2020-04-11 16:38:13 +02:00
Miriam Baglioni	aef9b3aa90	new parametric implementation of country propagation. Exploits information compute before and broadcasts it to each executor	2020-04-11 16:36:59 +02:00
Miriam Baglioni	a2d833d5dd	step of data preparation before actual country propagation will take palce	2020-04-11 16:36:03 +02:00
Miriam Baglioni	6897c920a2	classes in support of new implementation of country propagation	2020-04-11 16:35:26 +02:00
Miriam Baglioni	85766a02d8	added dependency to use hive on local machine	2020-04-11 16:34:22 +02:00
Miriam Baglioni	79b8ea4fed	prepared information to be used in actual country propagation. Subset of info	2020-04-11 16:29:41 +02:00
Miriam Baglioni	1822476613	Test for country propagation	2020-04-11 16:28:09 +02:00
Miriam Baglioni	7783b09c5b	new implementation for result to project propagation. Prepare some info to be used in propagation	2020-04-11 16:26:23 +02:00
Miriam Baglioni	90469789b9	two new classes fro new implementation of project to result propagation	2020-04-09 13:29:01 +02:00
Miriam Baglioni	627ad58a8b	new wf definition	2020-04-09 11:33:19 +02:00
Miriam Baglioni	9c63c4840d	new workflow and parameters for country propagation	2020-04-08 19:13:42 +02:00
Miriam Baglioni	a2d309545b	new parametrized implementation for country propagation	2020-04-08 19:12:59 +02:00
Miriam Baglioni	6dfdba9ef7	new parametrized implementation for country propagation	2020-04-08 18:14:37 +02:00
Miriam Baglioni	03f7cb6402	new parametrized implementation for country propagation	2020-04-08 18:08:41 +02:00
Miriam Baglioni	fcfef4632f	input parameters for country propagation preparation job	2020-04-08 18:07:18 +02:00
Miriam Baglioni	540da4ab61	new busuness logic with prepared info before actual job run	2020-04-08 13:04:04 +02:00
Miriam Baglioni	8438702b3d	addition in propagation constants	2020-04-08 10:54:01 +02:00
Miriam Baglioni	2afe971816	new implementation for country propagatio	2020-04-08 10:49:09 +02:00
Miriam Baglioni	beebbcf66b	new config for countrypropagation	2020-04-08 10:31:29 +02:00
Miriam Baglioni	dd011f4a95	to make them visible to Claudio	2020-03-30 10:55:47 +02:00
Miriam Baglioni	b1af90a45f	to make it visible to Claudio	2020-03-30 10:50:03 +02:00
Miriam Baglioni	19d7f8b51d	decommented execution for some of the result type for testing purposes	2020-03-24 16:49:46 +01:00
Miriam Baglioni	ad24c8478f	added missing parameter	2020-03-24 16:19:59 +01:00
Miriam Baglioni	46094a3eec	bug fixing for implementation with dataset	2020-03-24 16:19:36 +01:00
Miriam Baglioni	ad712f2d79	added the needed variables in the config and read the variables in the workflow	2020-03-23 17:11:36 +01:00
Miriam Baglioni	f1e9fe9752	changed implementation using dataset and query on hive	2020-03-23 17:11:00 +01:00
Miriam Baglioni	f09cd1e911	removed unuseful variable in the configuration	2020-03-23 17:10:14 +01:00
Miriam Baglioni	9418e3d4fa	read dataset from files instead of using hive tables	2020-03-23 17:09:27 +01:00
Miriam Baglioni	a7bf037306	remove unused class	2020-03-23 14:36:43 +01:00
Miriam Baglioni	8ab8b6b0bf	minor	2020-03-23 14:35:23 +01:00
Miriam Baglioni	30d58fd98c	change the configuration of the workflow	2020-03-23 14:32:49 +01:00
Miriam Baglioni	a440152b46	refactoring	2020-03-23 14:30:56 +01:00
Miriam Baglioni	47561f3597	changed the implementation from rdd to dataset got from sql queries (on hive)	2020-03-23 11:58:32 +01:00
Miriam Baglioni	67ea3cf3ed	changed the way to read the file with info on resource or relation. From sequenceFile to textFile	2020-03-17 16:32:05 +01:00
Miriam Baglioni	b4652d018c	moved the creation of new dir to common class.	2020-03-17 16:31:24 +01:00
Miriam Baglioni	e80f80ca93	properties and workflow for new propagation	2020-03-02 17:03:31 +01:00
Miriam Baglioni	50080c1b3c	changed the implementation of addAll method. Before adding all the items in a collection, we check if the accumulator set is not empty	2020-03-02 16:41:37 +01:00
Miriam Baglioni	02815dd2cf	update result for community moved in propagationconstants	2020-03-02 16:40:56 +01:00
Miriam Baglioni	95f8c3092f	update for new propagation implementation and moving of updateResult for community business logic since the same can be used for result to community from organization and result to community from semrel	2020-03-02 16:40:17 +01:00
Miriam Baglioni	3d63f35dcb	implementation of new propagation. Result to community for results linked to given organization. We exploit the hasAuthorInstitution semantic link to discover which results are related to institutions	2020-03-02 16:39:03 +01:00
Miriam Baglioni	3a4ccb26c0	New properties for the orcid to result propagation through semantic relation	2020-02-28 18:26:04 +01:00
Miriam Baglioni	b50166b9ad	None	2020-02-28 18:25:28 +01:00
Miriam Baglioni	550cb21c23	None	2020-02-28 18:24:39 +01:00