Claudio Atzori
|
128c3bf1c8
|
restored Author bean with simple getter/setter, author pid addition moved into dedicated implementation SparkOrcidToResultFromSemRelJob3
|
2020-05-07 11:14:56 +02:00 |
Miriam Baglioni
|
b2fec32c87
|
new workflow for job3
|
2020-05-07 10:01:57 +02:00 |
Miriam Baglioni
|
29bc8c44b1
|
changes in the construction of new country set
|
2020-05-07 10:01:34 +02:00 |
Miriam Baglioni
|
55e825acd4
|
chenged the test according to changes in SparkCOuntryPropagationJob2
|
2020-05-07 10:01:00 +02:00 |
Miriam Baglioni
|
16193cf0ba
|
new workflow and parameter for country propagation
|
2020-05-07 09:59:58 +02:00 |
Miriam Baglioni
|
5a476c7a13
|
chenged the xquery for the cfhb table
|
2020-05-07 09:58:17 +02:00 |
Miriam Baglioni
|
42ad51577a
|
new implementation with one more serialization step
|
2020-05-07 09:57:49 +02:00 |
Claudio Atzori
|
17860d3ab6
|
general changes in the RAW graph mapping: missing collectedfrom/hostedby causes records to be skipped; factored out most of the constants in ModelConstants class (dhp-schemas)
|
2020-05-06 13:20:02 +02:00 |
Claudio Atzori
|
fdfecc9578
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2020-05-06 11:28:01 +02:00 |
Claudio Atzori
|
c79e2f5977
|
drop workingPath before starting the dedup workflow
|
2020-05-06 11:27:44 +02:00 |
Michele Artini
|
8f30a09d84
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-05-05 17:12:22 +02:00 |
Michele Artini
|
ccc609f909
|
new module for the production of broker events
|
2020-05-05 17:09:00 +02:00 |
Miriam Baglioni
|
dd2e698a72
|
added a sequentialization step on the spark job. Addedd new parameter
|
2020-05-05 17:03:43 +02:00 |
Claudio Atzori
|
0825321d0b
|
improved unit tests in dhp-aggregation
|
2020-05-05 12:39:04 +02:00 |
Miriam Baglioni
|
252b219dd5
|
chanced the name of some properties
|
2020-05-05 10:03:32 +02:00 |
Claudio Atzori
|
4a8487165c
|
using long param names in wf definition
|
2020-05-04 19:19:29 +02:00 |
Claudio Atzori
|
a2fc37df5f
|
adjusted parameters
|
2020-05-04 19:18:59 +02:00 |
Claudio Atzori
|
f1b7e14036
|
code formatting
|
2020-05-04 19:18:34 +02:00 |
Miriam Baglioni
|
78578c3ccf
|
fixed wrong trnasition name in workflow
|
2020-05-04 15:46:24 +02:00 |
Miriam Baglioni
|
cc7d9b6b19
|
merge upstream
|
2020-05-04 13:59:09 +02:00 |
Miriam Baglioni
|
3957c815b9
|
changed the name of some parameters
|
2020-05-04 13:58:52 +02:00 |
Miriam Baglioni
|
e218360f8a
|
changed code for the mode of DbClient and also removed the dependency to graph-mapper
|
2020-05-04 12:26:17 +02:00 |
Miriam Baglioni
|
31ea05297d
|
moved the DbClient to common and added needed dependency to pom
|
2020-05-04 12:22:28 +02:00 |
miconis
|
085cf173d7
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-05-04 12:08:20 +02:00 |
miconis
|
3df703f67d
|
mergerels added to propagate relations
|
2020-05-04 12:08:12 +02:00 |
Claudio Atzori
|
bac37b3973
|
fixed children expansion in XML records
|
2020-05-04 11:51:17 +02:00 |
Claudio Atzori
|
077ccd8743
|
stats wf properties cleanup
|
2020-05-04 11:41:46 +02:00 |
Miriam Baglioni
|
b7dd400e51
|
added check if author.pid exists or is null
|
2020-05-01 15:09:02 +02:00 |
Miriam Baglioni
|
dbf3ba051a
|
minor
|
2020-04-30 20:22:07 +02:00 |
Miriam Baglioni
|
43053a286d
|
workflow pom with added blacklist module
|
2020-04-30 18:30:21 +02:00 |
Miriam Baglioni
|
0631fe548a
|
pom.xml
|
2020-04-30 18:29:46 +02:00 |
Miriam Baglioni
|
38ecfd5785
|
the wf with all the three steps for blacklisting relations
|
2020-04-30 18:28:46 +02:00 |
Miriam Baglioni
|
95433e1087
|
parameters for the preparation phase and blacklist phase
|
2020-04-30 18:28:13 +02:00 |
Miriam Baglioni
|
1070790c19
|
minor
|
2020-04-30 18:26:58 +02:00 |
Miriam Baglioni
|
b9d56b3ced
|
applies the actual removal of the relations
|
2020-04-30 18:26:25 +02:00 |
Miriam Baglioni
|
d6d6ebeae5
|
preparation step: creates the subset of the merges relations
|
2020-04-30 18:25:33 +02:00 |
Miriam Baglioni
|
13f30664ea
|
minor
|
2020-04-30 15:23:49 +02:00 |
Miriam Baglioni
|
276b95b7b3
|
add create file instruction
|
2020-04-30 15:05:17 +02:00 |
Miriam Baglioni
|
65a5d67b8b
|
minor modifications
|
2020-04-30 14:45:27 +02:00 |
Miriam Baglioni
|
418595fec2
|
removed the saveGraph parameter
|
2020-04-30 14:45:00 +02:00 |
Miriam Baglioni
|
ce8b1d0bc3
|
new workflow definition to be inserted in the provision pipeline
|
2020-04-30 14:38:54 +02:00 |
Miriam Baglioni
|
4b0bd91012
|
-
|
2020-04-30 12:45:28 +02:00 |
Miriam Baglioni
|
2349bfd8b8
|
changed the job test to remove the writeUpdate option
|
2020-04-30 11:43:33 +02:00 |
Miriam Baglioni
|
951517f9ec
|
new input parameters and workflow definition to be used in the provision pipeline
|
2020-04-30 11:32:50 +02:00 |
Miriam Baglioni
|
026f297e49
|
removed the writeUpdate oprion
|
2020-04-30 11:31:59 +02:00 |
Miriam Baglioni
|
c89fe762b1
|
modified relation datasource organization
|
2020-04-30 11:17:03 +02:00 |
Miriam Baglioni
|
3abb76ff7a
|
merge with upstream
|
2020-04-30 11:15:54 +02:00 |
Michele Artini
|
eb9bd42970
|
fixed a problem with journals
|
2020-04-30 11:06:05 +02:00 |
Miriam Baglioni
|
638a3c465b
|
-
|
2020-04-30 11:05:17 +02:00 |
Michele Artini
|
a0a6109bbc
|
fixed a problem with journals
|
2020-04-30 11:03:46 +02:00 |
Miriam Baglioni
|
354f0162be
|
changes in the blacklist and workflow definition
|
2020-04-30 10:26:50 +02:00 |
Claudio Atzori
|
439c6255a2
|
cleanup
|
2020-04-29 19:09:07 +02:00 |
Claudio Atzori
|
77ac995770
|
cleaned up poms, added descriptions
|
2020-04-29 18:44:17 +02:00 |
Miriam Baglioni
|
3cffee74b9
|
merge with upstream
|
2020-04-29 18:25:29 +02:00 |
Miriam Baglioni
|
9ab46535e7
|
pom with the new blacklist module added
|
2020-04-29 18:17:15 +02:00 |
Miriam Baglioni
|
6a47e6191d
|
read from blacklist and write the result as relations on hdfs
|
2020-04-29 18:16:01 +02:00 |
Miriam Baglioni
|
869f576273
|
added hash map for relationship entityType id prefix, and relation inverse
|
2020-04-29 18:14:52 +02:00 |
Miriam Baglioni
|
b85ad7012a
|
reads the blacklist from the blacklist db and writes it as a set of relations on hdfs
|
2020-04-29 17:29:49 +02:00 |
Claudio Atzori
|
8fd81e863d
|
added default value for the external_stats_db_name
|
2020-04-29 15:36:24 +02:00 |
Claudio Atzori
|
c6f3ff4462
|
stats workflow content relocated into common package; added <global> property definitions in stats workflow.xml
|
2020-04-29 14:29:27 +02:00 |
miconis
|
e0d14fe4f8
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-04-29 13:02:53 +02:00 |
miconis
|
0352d3b0ba
|
entity dumps in dedup compressed
|
2020-04-29 13:02:34 +02:00 |
Michele Artini
|
c43b4c8962
|
formatting
|
2020-04-29 12:56:58 +02:00 |
Michele Artini
|
a5d7007005
|
Fix relations in migration
Fix pom.xml in dhp-stats-update
|
2020-04-29 12:05:41 +02:00 |
Miriam Baglioni
|
f7695e833c
|
resolved conflicts
|
2020-04-29 11:41:31 +02:00 |
Claudio Atzori
|
3616d0f88d
|
Merge pull request 'Adding the stats workflow to the dnet-hadoop hierarchy' (#6) from spyros/dnet-hadoop:master into master
Integrating stats update workflow.
|
2020-04-29 10:35:02 +02:00 |
Claudio Atzori
|
964972d29a
|
added data provision workflow definition WIP
|
2020-04-29 09:25:50 +02:00 |
miconis
|
62e467eb0c
|
assertion numbers updated to fit the new implementation of the pace-core
|
2020-04-28 11:46:23 +02:00 |
Claudio Atzori
|
6f5b899038
|
reformatted code according to the updated style descriptor
|
2020-04-28 11:23:29 +02:00 |
Claudio Atzori
|
ac25f2d8d1
|
integrated changes from master
|
2020-04-28 08:55:28 +02:00 |
Miriam Baglioni
|
2980e50edf
|
merge upstream
|
2020-04-27 15:06:48 +02:00 |
Claudio Atzori
|
a0bdbacdae
|
switched automatic code formatting plugin to net.revelc.code.formatter:formatter-maven-plugin
|
2020-04-27 14:52:31 +02:00 |
Claudio Atzori
|
7a3f8085f7
|
switched automatic code formatting plugin to net.revelc.code.formatter:formatter-maven-plugin
|
2020-04-27 14:45:40 +02:00 |
Michele Artini
|
1260d03eba
|
skip empty projects
|
2020-04-27 13:51:13 +02:00 |
Miriam Baglioni
|
df34a4ebcc
|
changed the configuration to add ignorecase option to each verb related to covid-19 community
|
2020-04-27 12:32:56 +02:00 |
Miriam Baglioni
|
7a59324ccf
|
changed the test to check for the new ignorecase option
|
2020-04-27 12:31:46 +02:00 |
Miriam Baglioni
|
986c97348d
|
added the ignorecase option to each selection verb
|
2020-04-27 12:31:05 +02:00 |
Miriam Baglioni
|
a303fc9f73
|
resources for testing propagation of result to comminuty from organization and from semrel
|
2020-04-27 11:14:16 +02:00 |
Miriam Baglioni
|
c093d764a3
|
-
|
2020-04-27 11:12:38 +02:00 |
Miriam Baglioni
|
c925e2be16
|
test for propagation of result to community from organization and result to community from semrel
|
2020-04-27 10:59:53 +02:00 |
Miriam Baglioni
|
ec7f166690
|
changed the bl because of changed of the examples for the re implementation of the propagation step
|
2020-04-27 10:58:41 +02:00 |
Miriam Baglioni
|
6135096ef1
|
refactoring
|
2020-04-27 10:57:50 +02:00 |
Miriam Baglioni
|
d30e710165
|
fixed duplicates action name in the workflow
|
2020-04-27 10:52:30 +02:00 |
Miriam Baglioni
|
f9ee343fc0
|
new parametrized workflow with preparation steps and new parameter input files
|
2020-04-27 10:48:31 +02:00 |
Miriam Baglioni
|
e2093644dc
|
changed in the workflow the directory where to store the preparedInfo and the graph genearated at this step
|
2020-04-27 10:46:44 +02:00 |
Miriam Baglioni
|
8a58bf2744
|
removed the writeUpdate option
|
2020-04-27 10:45:06 +02:00 |
Miriam Baglioni
|
5dccbe13db
|
merge with upstream
|
2020-04-27 10:43:59 +02:00 |
Miriam Baglioni
|
7b6505ec69
|
new resuorces for testing propagation of project to result after the re-implementation
|
2020-04-27 10:42:16 +02:00 |
Miriam Baglioni
|
1b0e0bd1b5
|
refactoring
|
2020-04-27 10:40:26 +02:00 |
Miriam Baglioni
|
e5a177f0a7
|
refactoring
|
2020-04-27 10:36:21 +02:00 |
Miriam Baglioni
|
e000754c92
|
refactoring
|
2020-04-27 10:34:03 +02:00 |
Miriam Baglioni
|
95a54d5460
|
removed the writeUpdate option. The update is available in the preparedInfo path
|
2020-04-27 10:30:32 +02:00 |
Miriam Baglioni
|
8802e4126b
|
re-implemented inverting the couple: from (projectId, relatedResultList) to (resultId, relatedProjectList)
|
2020-04-27 10:26:55 +02:00 |
Claudio Atzori
|
268462623a
|
refined definition of equals and hash methods for Oaf model classes, now based on entity identifier, while relations consider sourceid, targetid and relationship semantic; Factored out function to group Oaf objects in grouping operations; Raw graph creation procedure merges entities and relationships providing the same identity
|
2020-04-24 14:42:01 +02:00 |
Claudio Atzori
|
a3e480d1c9
|
implmented DispatchEntitiesApplication using spark2 datasets
|
2020-04-24 14:36:53 +02:00 |
Claudio Atzori
|
48157e0fc4
|
GraphHiveImporterJob moved in dedicate package
|
2020-04-24 14:32:28 +02:00 |
Miriam Baglioni
|
adcbf0e29a
|
refactoring
|
2020-04-24 10:47:43 +02:00 |
Claudio Atzori
|
278fc9d276
|
code formatting
|
2020-04-23 18:51:38 +02:00 |
miconis
|
5414236644
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2020-04-23 18:17:23 +02:00 |
miconis
|
8d258c85ff
|
spark dedup test fixed, sample for dataset and orp added, test implemented
|
2020-04-23 18:16:20 +02:00 |