dimitrispie
2324670714
Split Monitor DBs-Interdisciplinarity indicators
...
- Split DBs Monitor for faster rendering of visualizations
- Add interdisciplinarity indicators from result_fos
2023-06-02 13:34:16 +03:00
dimitrispie
86f4f63daf
Updates to steps related to transfer data to impala cluster
...
1. Remove external table definitions in stats_ext
2. Fix the issue where some views are not created.
3. Added two workflow parameters for copying also the usage stats dbs
2023-05-18 09:33:05 +03:00
dimitrispie
1547611246
Merge branch 'beta' into hive
2023-02-22 16:57:12 +02:00
dimitrispie
032a401cbf
Bug fixes
2023-02-20 09:29:20 +02:00
dimitrispie
595192d510
Bug fix
2023-02-14 16:24:08 +02:00
dimitrispie
3400133c2f
Bug fix
2023-02-13 09:44:00 +02:00
dimitrispie
7b78b15c81
Changes for copying to Impala Cluster
2023-02-13 09:27:00 +02:00
dimitrispie
d71f5672d3
Add monitor post step
2023-02-09 13:44:14 +02:00
dimitrispie
3ba11d64a1
Changes 07022023
2023-02-07 12:53:51 +02:00
dimitrispie
2dc6d47270
Changes 06022023
2023-02-06 13:18:53 +02:00
dimitrispie
dcb958e146
Changes to execute the stats wf only in hive
2023-01-04 11:39:01 +02:00
dimitrispie
592013d5dd
Added more steps in decision node
2022-12-23 09:43:16 +02:00
dimitrispie
6449ff4207
1. Added a decision node to enables the workflow to make a selection on the execution path to follow
...
2. Added new organization
3. Added 5 new tables from Eurostast
2022-12-22 10:18:21 +02:00
Antonis Lempesis
c8309fe18e
addded command line params to allow hive actions to run
2022-12-21 12:41:33 +02:00
Antonis Lempesis
028873cc51
added new hive opts
2022-12-21 12:41:33 +02:00
Antonis Lempesis
2754c3dd62
moving data to impala cluster and creating shadow databases there
2022-12-21 12:41:29 +02:00
Antonis Lempesis
421d55265d
created hive action for observatory queries
2021-09-21 03:07:58 +03:00
Antonis Lempesis
8fc89ae822
moved context table creation before indicators
2021-09-13 14:33:23 +03:00
Antonis Lempesis
f13cca7e83
moved dependencies of indicators before them...
2021-09-08 23:07:58 +03:00
Antonis Lempesis
1250ae197f
using new indicators for the definition of peerreviewed, gold, and green
2021-09-08 14:08:43 +03:00
Antonis Lempesis
018c4eb52c
copied latest changes from old fork: indicators+monitor institutions
2021-06-28 23:46:52 +03:00
Claudio Atzori
83722ebc47
pull #111 replied on stable_ids
2021-05-28 14:11:46 +02:00
Claudio Atzori
a0837ac357
[Stats update] integrating PR#100 for testing D-Net/dnet-hadoop#100
2021-03-29 15:59:58 +02:00
Antonis Lempesis
3681afbe04
typo
2021-02-19 03:04:27 +02:00
Antonis Lempesis
33c85d4e66
moved stats computation in impala instead of hive
2021-02-18 17:23:34 +02:00
Antonis Lempesis
b8e96c8ae7
moved cache update to the end
2021-02-18 16:42:22 +02:00
Antonis Lempesis
bcbfc052b1
fixed last errors in step 21
2021-02-18 16:32:54 +02:00
Antonis Lempesis
10a29a4b9a
fixes in monitor step
2021-02-18 15:05:59 +02:00
Antonis Lempesis
cd1b794409
added the monitor db wf
2021-02-17 02:11:55 +02:00
Antonis Lempesis
be5969a8c2
Changed typo in script names
2020-12-22 13:33:32 +02:00
Antonis Lempesis
2a074c3b2b
Changed typo in script names
2020-12-18 18:40:48 +02:00
Antonis Lempesis
7cb113e088
added the new parameter (stats_tool_api_url) in the workflow parameters
2020-12-04 13:04:25 +02:00
Antonis Lempesis
413afcfed5
finished first implementation of wf
2020-12-02 15:57:17 +02:00
Antonis Lempesis
815d6b25d9
added last step to update cache
2020-11-30 00:48:10 +02:00
Antonis Lempesis
01a6e03989
starting from first step...
2020-11-17 23:26:47 +02:00
Antonis Lempesis
99ebaee347
fixed #5913
2020-11-11 16:56:46 +02:00
Antonis Lempesis
e603fa5847
fixes in db creation
2020-11-10 17:11:12 +02:00
Antonis Lempesis
c9cfc165d9
not using impala since the resulting tables are not visible
2020-07-24 19:50:40 +03:00
Antonis Lempesis
e6f50de6ef
Separated impala from hive steps
2020-07-24 19:50:40 +03:00
Antonis Lempesis
de49173420
fixed a typo in queries
2020-07-24 19:50:40 +03:00
antleb
391cf80fb8
Added peer-reviewed, green, gold tables and fields in result. Added shortcuts from result-country
2020-07-24 19:50:40 +03:00
antleb
68389d0125
Corrected the script used by the last step of the wf
2020-07-24 19:50:40 +03:00
Spyros Zoupanos
5546f29e63
Corrections on the shadow schema and the impala table stats calculation
2020-07-24 19:50:40 +03:00
Spyros Zoupanos
adf8a025d2
Adding more relations (Sources, Licences, Additional) and shadow schema as provided and discussed with Antonis Lempesis
2020-07-24 19:50:40 +03:00
Spyros Zoupanos
657a40536b
Corrections by Spyros: Scipt cleanup, corrections and re-arrangement
2020-07-24 19:50:40 +03:00
Giorgos Alexiou
477fa6234d
Script re-organisation and adding table invalidations needed for impala
2020-07-24 19:50:40 +03:00
Spyros Zoupanos
3576dd186b
Adding hive timeout as workflow parameter
2020-06-05 22:29:54 +03:00
Claudio Atzori
077ccd8743
stats wf properties cleanup
2020-05-04 11:41:46 +02:00
Claudio Atzori
8fd81e863d
added default value for the external_stats_db_name
2020-04-29 15:36:24 +02:00
Claudio Atzori
c6f3ff4462
stats workflow content relocated into common package; added <global> property definitions in stats workflow.xml
2020-04-29 14:29:27 +02:00