626 Commits

Author	SHA1	Message	Date
Miriam Baglioni	c29d142087	-	2020-11-16 10:53:12 +01:00
Miriam Baglioni	0f1a4f6637	added collectedfrom information on record	2020-11-09 16:07:17 +01:00
Miriam Baglioni	0ef5e7dc34	fixed issue for authors with no name	2020-11-09 16:06:52 +01:00
Miriam Baglioni	902b0db85a	try to make workflow and sub-workflow for making report and actual orcid cleaning	2020-11-06 17:19:28 +01:00
Miriam Baglioni	c56a43c90b	-	2020-11-06 15:46:31 +01:00
Miriam Baglioni	863ce76820	merge branch with master	2020-11-06 15:30:19 +01:00
Miriam Baglioni	a1aed813a5	done workflow for report and actual cleaning in the results. Renamed and moved some files	2020-11-06 15:29:28 +01:00
Claudio Atzori	d10447e747	re-packaged graph dump workflow sources	2020-11-05 17:38:18 +01:00
Miriam Baglioni	f8e9bda24c	merge branch with master	2020-11-05 16:31:18 +01:00
Miriam Baglioni	be5ed8f554	added check to avoid sending empty metadata.	2020-11-05 16:10:17 +01:00
Claudio Atzori	2148a51fae	minor changes	2020-11-05 11:24:12 +01:00
Miriam Baglioni	fff512a87a	added one level of checking (search all the words of name surname in orcid and in paper)	2020-11-04 18:30:09 +01:00
Claudio Atzori	4625b7486e	code formatting	2020-11-04 18:12:43 +01:00
Miriam Baglioni	e9ac471ae9	removed dependency from classes for the pid graph dump	2020-11-04 18:04:42 +01:00
Miriam Baglioni	b90a945c49	removed property files for pid graph dump	2020-11-04 17:28:33 +01:00
Miriam Baglioni	bac307155a	removed properties specific for pid graph dump	2020-11-04 17:28:04 +01:00
Miriam Baglioni	9c9d50f486	removed code specific for pid graph dump	2020-11-04 17:26:22 +01:00
Miriam Baglioni	5669890934	removed commented lines	2020-11-04 17:15:21 +01:00
Miriam Baglioni	6a89f59be9	removed commented lines	2020-11-04 17:13:59 +01:00
Miriam Baglioni	56150d7e5e	removed all code related to the dump of pids graph	2020-11-04 17:13:12 +01:00
Miriam Baglioni	16c54a96f8	removed pid dump	2020-11-04 17:11:32 +01:00
Miriam Baglioni	44cf0b712f	added repartition(1) to have all the output in a single json file	2020-11-04 17:01:08 +01:00
Miriam Baglioni	1293dd276a	merge branch with master	2020-11-04 13:37:34 +01:00
Miriam Baglioni	0cac5436ff	Merge branch 'dump' of code-repo.d4science.org:miriam.baglioni/dnet-hadoop into dump	2020-11-04 13:21:11 +01:00
Miriam Baglioni	b610d08399	added test for the report generation	2020-11-04 13:20:16 +01:00
Miriam Baglioni	c694457acc	added new attribute to store the orcid fullname when provided	2020-11-04 13:19:57 +01:00
Miriam Baglioni	72abbb0510	added the link to the property file	2020-11-04 13:19:25 +01:00
Miriam Baglioni	fd00d44e1e	new classes for generating the report for the orcid cleaning plus new property file	2020-11-04 13:18:49 +01:00
Alessia Bardi	51808b5afd	Updated descriptions	2020-11-04 12:29:48 +01:00
Alessia Bardi	e6becf8659	Updated descriptions	2020-11-04 12:17:57 +01:00
Alessia Bardi	0abe0eee33	Updated descriptions	2020-11-04 12:15:30 +01:00
Alessia Bardi	f6ab238f5d	Updated descriptions	2020-11-04 11:50:47 +01:00
Miriam Baglioni	c010a8442f	fixed issue on test code	2020-11-03 17:26:51 +01:00
Miriam Baglioni	8ec7a61188	merge branch with master	2020-11-03 16:59:08 +01:00
Miriam Baglioni	c209284ca7	new schemas for the entities in the dump with added descriptions	2020-11-03 16:58:08 +01:00
Miriam Baglioni	08806deddf	added the splitSize non mandatory parameter. Default size 10G	2020-11-03 16:57:34 +01:00
Miriam Baglioni	7d2eda43ca	added new non mandatory property publish to determine if to publish the upload or leave it pending. Default value flase	2020-11-03 16:57:01 +01:00
Miriam Baglioni	cbbb1bdc54	moved business logic to new class in common for handling the zip of hte archives	2020-11-03 16:55:50 +01:00
Miriam Baglioni	d4382b54df	moved the tar archive with maz size on common module	2020-11-03 16:54:50 +01:00
Claudio Atzori	5310e56dba	remove empy PIDs	2020-11-03 11:52:10 +01:00
Miriam Baglioni	4ee84ae724	added files for testing purposes	2020-11-02 18:25:41 +01:00
Miriam Baglioni	7dcb2eff02	added needed dependency	2020-11-02 18:25:07 +01:00
Miriam Baglioni	43ddeedd6a	first part of the test	2020-11-02 18:24:25 +01:00
Miriam Baglioni	72fb425787	new logis for the cleaning of the authors orcid	2020-11-02 18:23:16 +01:00
Miriam Baglioni	967d839ba1	merge branch with master	2020-11-02 10:23:11 +01:00
Sandro La Bruzzo	754c86f33e	fixed test to work on jenkins	2020-11-02 09:35:01 +01:00
Miriam Baglioni	dabb33e018	changed the discriminant for which split the file	2020-10-30 17:52:22 +01:00
Miriam Baglioni	0fba08eae4	max allowed size per file 10 Gb	2020-10-30 16:05:55 +01:00
Miriam Baglioni	f3de9c02ae	first classes for the orcid cleaning	2020-10-30 16:02:28 +01:00
Miriam Baglioni	b828587252	prevent the code to cicle indefinetly	2020-10-30 15:01:25 +01:00