86f4f63daf
Updates to steps related to transfer data to impala cluster
...
1. Remove external table definitions in stats_ext
2. Fix the issue where some views are not created.
3. Added two workflow parameters for copying also the usage stats dbs
2023-05-18 09:33:05 +03:00
Claudio Atzori
909729a2fc
[dedup] tweaking num partitions, minor changes
2023-05-17 10:16:22 +02:00
Ilias Kanellos
38020e242a
Merge branch '8172_impact_indicators_workflow' of https://code-repo.d4science.org/D-Net/dnet-hadoop into 8172_impact_indicators_workflow
2023-05-16 17:34:53 +03:00
Ilias Kanellos
3d69f33c84
Fix selection of columns in graph creation
2023-05-16 17:34:42 +03:00
Ilias Kanellos
3c38f7ba6f
Fix selection of columns in graph creation
2023-05-16 17:32:53 +03:00
8ef718c363
Fix workflow application path
2023-05-16 16:28:48 +03:00
26328e2a0d
Move job.properties
2023-05-16 14:39:53 +03:00
4eec3e7052
Add jobTracker, nameNode && spark2Lib as global params in oozie wf
2023-05-15 22:28:48 +03:00
b83135c252
Add missing kill nodes in workflow.xml
2023-05-15 19:55:35 +03:00
45f2aa0867
Move end node ... at the end in workflow.xml
2023-05-15 17:52:20 +03:00
Claudio Atzori
e309688711
Merge pull request 'fix APC affiliation links' ( #294 ) from apc_affiliation into beta
...
Reviewed-on: #294
2023-05-15 15:47:57 +02:00
Claudio Atzori
8acad52a0c
Merge branch 'beta' into apc_affiliation
2023-05-15 15:47:33 +02:00
Claudio Atzori
8a463cc3e8
fixed organization id created when mapping APC affiliations. Factored out ROR constants in dhp-common
2023-05-15 15:44:46 +02:00
12a57e1f58
Resolve conflicts
2023-05-15 16:20:11 +03:00
82e2a96f51
Resolve conflicts
2023-05-15 15:53:12 +03:00
b8e8c959fe
Update workflow.xml && job.properties
2023-05-15 15:50:23 +03:00
Ilias Kanellos
4a905932a3
Spark properties from job.properties
2023-05-15 15:24:22 +03:00
Claudio Atzori
0c314d5e09
Merge pull request 'Update copyDataToImpalaCluster.sh' ( #293 ) from antonis.lempesis/dnet-hadoop:beta into beta
...
Reviewed-on: #293
2023-05-15 12:05:54 +02:00
07818131ef
Update documentation
2023-05-15 13:04:44 +03:00
b3f9633205
Update copyDataToImpalaCluster.sh
...
Added option --user to impala-shell command
2023-05-15 12:51:44 +03:00
Miriam Baglioni
021321ae06
Merge pull request 'removed the inverse of the Citing relation' ( #292 ) from citeOnly into beta
...
Reviewed-on: #292
2023-05-15 11:37:39 +02:00
Miriam Baglioni
78b07400c0
changed test classes
2023-05-15 11:37:08 +02:00
Miriam Baglioni
86fe886c1a
removed the inverse of the Citing relation
2023-05-15 11:20:51 +02:00
Ilias Kanellos
1788ac2d4d
Correct filtering for MAG records
2023-05-12 12:55:43 +03:00
Miriam Baglioni
12cd179d2d
Merge pull request 'Update copyDataToImpalaCluster.sh' ( #291 ) from antonis.lempesis/dnet-hadoop:beta into beta
...
Reviewed-on: #291
2023-05-12 11:36:34 +02:00
00d0d162b6
Update copyDataToImpalaCluster.sh
...
Added a temporary folder to copy the files to avoid permission issues
2023-05-12 12:31:13 +03:00
Ilias Kanellos
5ddbb4ad10
Spark properties no longer hardcoded
2023-05-11 15:36:47 +03:00
Ilias Kanellos
3de35fd6a3
Produce 5 classes of ranking scores
2023-05-11 14:42:25 +03:00
Miriam Baglioni
8c05f49665
moved the version as it was before the change
2023-05-09 10:48:34 +02:00
Miriam Baglioni
99ac5bab46
added check to avoid NPE when checking the organization country
2023-05-04 19:38:39 +02:00
Claudio Atzori
0704e186f6
Merge pull request 'Stats wf executed on hive only' ( #283 ) from antonis.lempesis/dnet-hadoop:beta into beta
...
Reviewed-on: #283
2023-05-02 14:05:12 +02:00
Claudio Atzori
cd80b200ee
Merge pull request 'Affiliation links from APC' ( #290 ) from apc_affiliation into beta
...
Reviewed-on: #290
2023-05-02 12:00:04 +02:00
Claudio Atzori
d8882c4481
extended mapping applied to datacite records to produce affiliations using the ROR ids. Inc ase of APCs it includes the amount and the currently in the relation
2023-05-02 11:56:51 +02:00
Claudio Atzori
d02916ef82
code formatting
2023-05-02 11:05:37 +02:00
Claudio Atzori
f653640cd9
Merge pull request 'Bulk Tagging single step' ( #289 ) from bulkTagRefactor into beta
...
Reviewed-on: #289
2023-05-02 10:54:14 +02:00
c3d58e58e1
Bug fixes
2023-05-02 11:54:07 +03:00
Claudio Atzori
abd7ca0c18
Merge branch 'beta' into bulkTagRefactor
2023-05-02 10:50:01 +02:00
Claudio Atzori
de36c7b083
Merge pull request 'Enrichment - result to community through organization' ( #255 ) from organizationToRepresentative into beta
...
Reviewed-on: #255
2023-05-02 10:47:07 +02:00
Claudio Atzori
45f625d14f
Merge branch 'beta' into organizationToRepresentative
2023-05-02 10:46:55 +02:00
Claudio Atzori
cdd33f7445
Merge pull request 'graph cleaning refactoring' ( #282 ) from graph_cleaning_refactoring into beta
...
Reviewed-on: #282
2023-05-02 10:40:02 +02:00
Claudio Atzori
de11edca98
Merge branch 'beta' into organizationToRepresentative
2023-05-02 09:59:41 +02:00
Claudio Atzori
851f664bd9
Merge branch 'beta' into graph_cleaning_refactoring
2023-05-02 09:55:40 +02:00
e57ecdaf98
Update step20-createMonitorDB.sql
...
Add University of Manitoba
2023-04-30 17:52:23 +03:00
Ilias Kanellos
90332439ad
Remove deletion of synonym folder
2023-04-28 13:45:19 +03:00
Ilias Kanellos
a98da54896
Merge branch '8172_impact_indicators_workflow' of https://code-repo.d4science.org/D-Net/dnet-hadoop into 8172_impact_indicators_workflow
2023-04-28 13:23:49 +03:00
Ilias Kanellos
09485fbee3
Fixed unicode bug. Workflow ends after first script
2023-04-28 13:09:13 +03:00
614cc1089b
Add separate forder for results && project actionsets
2023-04-27 12:37:15 +03:00
815a4ddbba
Add actionset creation for project bip indicators in workflow
2023-04-26 20:40:06 +03:00
ee04cf92bf
Add actionsets for project impact indicators
2023-04-26 20:23:46 +03:00
fdb5d2b39f
Bug fixes
2023-04-23 18:29:00 +03:00
dimitrispie
Claudio Atzori
Ilias Kanellos
Serafeim Chatzopoulos
Miriam Baglioni