Michele Artini
8ba94833bd
added an es prop
4 years ago
Claudio Atzori
6f11c0496e
fixed typo in module name dhp-worfklow-profiles -> dhp-workflow-profiles
4 years ago
Claudio Atzori
f680eb3e12
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
4 years ago
Claudio Atzori
985b360c31
fixed typo in module name dhp-worfklow-profiles -> dhp-workflow-profiles
4 years ago
Claudio Atzori
7fc27bfdd1
Merge pull request 'islookup_timeout' ( #30 ) from islookup_timeout into master
...
Thanks, Michele!
4 years ago
Michele Artini
3acd632123
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
4 years ago
Michele Artini
35e6e9c064
tests
4 years ago
Claudio Atzori
2c4196ab22
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop into islookup_timeout
4 years ago
Claudio Atzori
ee832f358e
Merge pull request 'stats_wf_extensions_and_corrections' ( #28 ) from spyros/dnet-hadoop:stats_wf_extensions_and_corrections into master
...
Thank you Guys! The update workflow will be made available to the beta & production orchestration systems under the HDFS path
```/lib/dnet/oa/graph/stats/oozie_app```
4 years ago
Antonis Lempesis
4ac8ebe427
correctly calculating the project duration
4 years ago
Antonis Lempesis
18d9464b52
creating shadow db only if it not exists...
4 years ago
Antonis Lempesis
e217d496ab
added the dest db...
4 years ago
Antonis Lempesis
b16bb68b9f
added the target db name...
4 years ago
Antonis Lempesis
1ee7eeedf3
added the source db name...
4 years ago
Antonis Lempesis
cecbbfa0fc
added missing tables and views: contexts, creation_date, funder
4 years ago
Antonis Lempesis
25b7a615f5
moved datasource_sources table creating in the datasource section
4 years ago
Antonis Lempesis
a8da4ab9c0
years in projects are now integers
4 years ago
Antonis Lempesis
c9cfc165d9
not using impala since the resulting tables are not visible
4 years ago
Antonis Lempesis
dd3d6a6e15
compute stats for the used and new impala tables
4 years ago
Antonis Lempesis
e6f50de6ef
Separated impala from hive steps
4 years ago
Antonis Lempesis
de49173420
fixed a typo in queries
4 years ago
391cf80fb8
Added peer-reviewed, green, gold tables and fields in result. Added shortcuts from result-country
4 years ago
68389d0125
Corrected the script used by the last step of the wf
4 years ago
ec52141f1a
changed refereed type from value to clssname
4 years ago
Spyros Zoupanos
63cd797aba
Comment out step 15 to make it work with the new schema of Claudio
4 years ago
Spyros Zoupanos
138c6ddffa
Insert statement to datasource table that takes into account the piwik_id of the openAIRE graph
4 years ago
Spyros Zoupanos
3630794cef
Fix to consider the relationships that have been 'virtually deleted' for project_results - defect #5607
4 years ago
Spyros Zoupanos
5546f29e63
Corrections on the shadow schema and the impala table stats calculation
4 years ago
Spyros Zoupanos
adf8a025d2
Adding more relations (Sources, Licences, Additional) and shadow schema as provided and discussed with Antonis Lempesis
4 years ago
Spyros Zoupanos
657a40536b
Corrections by Spyros: Scipt cleanup, corrections and re-arrangement
4 years ago
477fa6234d
Script re-organisation and adding table invalidations needed for impala
4 years ago
Claudio Atzori
56bbfdc65d
introduced parameter 'numParitions', driving the hive DB table data partitioning. Currently specified only for table 'project'
4 years ago
Sandro La Bruzzo
9ab594ccf6
fixed test
4 years ago
Claudio Atzori
ebf60020ac
map results as OPRs in case of missing //CobjCategory/@type and the vocabulary dnet:result_typologies doesn't resolve the super type
4 years ago
Claudio Atzori
32f5e466e3
imports cleanup
4 years ago
Claudio Atzori
54ac583923
code formatting
4 years ago
Claudio Atzori
124e7ce19c
in case of missing attribute //dr:CobjCategory/@type the resulttype is derived by looking up the vocabulary dnet:result_typologies with the 1st instance type available
4 years ago
Claudio Atzori
050dda223d
Merge pull request 'removed duplicated fields' ( #25 ) from unique_field_in_lists into master
...
Looks good as a temporary workaround. I agree the model could seamlessly make the distinct operation by using HashSets instead of Linked (or Array) Lists.
The task to update the model in such a way is added on #9#issuecomment-1583
Thanks!
4 years ago
Claudio Atzori
e0c4cf6f7b
added parameter to drive the graph merge strategy: priority (BETA|PROD)
4 years ago
Claudio Atzori
94ccdb4852
Merge branch 'master' into merge_graph
4 years ago
Claudio Atzori
0937c9998f
Merge branch 'deduptesting'
4 years ago
Claudio Atzori
105176105c
updated dnet-pace-core dependency to version 4.0.4 to include the latest clustering function
4 years ago
Claudio Atzori
de72b1c859
cleanup
4 years ago
Michele Artini
331a3cbdd0
fixed originalId
4 years ago
Michele Artini
c59c5369b1
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
4 years ago
Michele Artini
346a1d2b5a
update eventId generator
4 years ago
Sandro La Bruzzo
9116d75b3e
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
4 years ago
Miriam Baglioni
47c7122773
changed priority from beta to production
4 years ago
Michele Artini
442f30930c
removed duplicated fields
4 years ago
Claudio Atzori
1781609508
code formatting
4 years ago
antleb
Giorgos Alexiou
