dimitrispie
|
c3d58e58e1
|
Bug fixes
|
2023-05-02 11:54:07 +03:00 |
dimitrispie
|
e57ecdaf98
|
Update step20-createMonitorDB.sql
Add University of Manitoba
|
2023-04-30 17:52:23 +03:00 |
dimitrispie
|
5247cb7115
|
Bug fix
|
2023-04-19 11:11:19 +03:00 |
dimitrispie
|
25dafccc24
|
Merge branch 'hive' into beta
|
2023-04-12 11:36:59 +03:00 |
dimitrispie
|
c85de8fa1f
|
-Added Technological University Dublin
-Added project_organization_contribution table
-Add Delft University of Technology
|
2023-04-07 09:22:59 +03:00 |
dimitrispie
|
9b41dff33c
|
Update step20-createMonitorDB.sql
Added Delft University of Technology
|
2023-04-07 09:21:38 +03:00 |
dimitrispie
|
91e18ac7f4
|
Added project_organization_contribution table
|
2023-04-06 10:53:11 +03:00 |
dimitrispie
|
9e1335df4c
|
-Added Technological University Dublin
-Added project_organization_contribution table
|
2023-04-04 13:22:40 +03:00 |
dimitrispie
|
fad7fa4af8
|
Added Technological University Dublin
|
2023-03-22 09:44:00 +02:00 |
dimitrispie
|
43b23a9bf3
|
Update step20-createMonitorDB.sql
Added Technological University Dublin
|
2023-03-15 09:57:12 +02:00 |
dimitrispie
|
1547611246
|
Merge branch 'beta' into hive
|
2023-02-22 16:57:12 +02:00 |
Claudio Atzori
|
c20c1c9159
|
Merge pull request 'Added 4 institutions:' (#261) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #261
|
2023-02-22 09:53:45 +01:00 |
dimitrispie
|
032a401cbf
|
Bug fixes
|
2023-02-20 09:29:20 +02:00 |
dimitrispie
|
595192d510
|
Bug fix
|
2023-02-14 16:24:08 +02:00 |
dimitrispie
|
f3aaff3688
|
Remove duplicate orgs
|
2023-02-14 09:48:36 +02:00 |
dimitrispie
|
3400133c2f
|
Bug fix
|
2023-02-13 09:44:00 +02:00 |
dimitrispie
|
935db0ab25
|
Added organizations for Monitor
|
2023-02-13 09:29:09 +02:00 |
dimitrispie
|
7b78b15c81
|
Changes for copying to Impala Cluster
|
2023-02-13 09:27:00 +02:00 |
dimitrispie
|
d71f5672d3
|
Add monitor post step
|
2023-02-09 13:44:14 +02:00 |
dimitrispie
|
35ba8bb328
|
Bug fixes
|
2023-02-09 12:57:57 +02:00 |
dimitrispie
|
3ba11d64a1
|
Changes 07022023
|
2023-02-07 12:53:51 +02:00 |
dimitrispie
|
98c34263ed
|
Update step20-createMonitorDB.sql
Add University of Cape Town organization
|
2023-02-07 08:14:48 +02:00 |
dimitrispie
|
2dc6d47270
|
Changes 06022023
|
2023-02-06 13:18:53 +02:00 |
dimitrispie
|
973d78a4d6
|
Update step15_5.sql
Added unpaywalls open access colors
|
2023-02-02 08:03:54 +02:00 |
dimitrispie
|
cf58e4a5e4
|
Added Arts et Métiers ParisTech
|
2023-01-25 16:03:16 +02:00 |
dimitrispie
|
db7d625ba9
|
Addedd Arts et Métiers ParisTech organization
|
2023-01-25 12:22:21 +02:00 |
dimitrispie
|
686580a220
|
- New Monitor DB workflow
- New Organization added
|
2023-01-12 11:18:03 +02:00 |
dimitrispie
|
dcb958e146
|
Changes to execute the stats wf only in hive
|
2023-01-04 11:39:01 +02:00 |
dimitrispie
|
592013d5dd
|
Added more steps in decision node
|
2022-12-23 09:43:16 +02:00 |
dimitrispie
|
2a4bf32d4c
|
Merge branch 'hive' of https://code-repo.d4science.org/antonis.lempesis/dnet-hadoop into hive
# Conflicts:
# dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step10.sql
# dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step13.sql
# dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step14.sql
# dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step16_1-definitions.sql
# dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step7.sql
|
2022-12-22 10:22:46 +02:00 |
dimitrispie
|
6449ff4207
|
1. Added a decision node to enables the workflow to make a selection on the execution path to follow
2. Added new organization
3. Added 5 new tables from Eurostast
|
2022-12-22 10:18:21 +02:00 |
Antonis Lempesis
|
c8309fe18e
|
addded command line params to allow hive actions to run
|
2022-12-21 12:41:33 +02:00 |
Antonis Lempesis
|
028873cc51
|
added new hive opts
|
2022-12-21 12:41:33 +02:00 |
Antonis Lempesis
|
1ddea4f442
|
removed 'stored as parquet' from views..
|
2022-12-21 12:41:33 +02:00 |
Antonis Lempesis
|
2754c3dd62
|
moving data to impala cluster and creating shadow databases there
|
2022-12-21 12:41:29 +02:00 |
Antonis Lempesis
|
778a1a724f
|
finished migration to hive only
|
2022-12-21 12:41:25 +02:00 |
Antonis Lempesis
|
e84dd5fe26
|
first
|
2022-12-21 12:41:23 +02:00 |
dimitrispie
|
2a52a42169
|
Added 4 institutions:
-University of Modena and Reggio Emilia
-Bilkent University
-Saints Cyril and Methodius University of Skopje
-University of Milan
|
2022-12-06 10:10:21 +02:00 |
Claudio Atzori
|
ddf065756f
|
Merge pull request 'Two organizations are added for monitor' (#258) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #258
|
2022-12-02 14:45:27 +01:00 |
dimitrispie
|
992fc5b628
|
Added McMaster University Institution
|
2022-11-03 11:02:18 +02:00 |
dimitrispie
|
7fda05e380
|
Added Autonomous University of Barcelona
|
2022-11-01 13:59:40 +02:00 |
Claudio Atzori
|
22873c9172
|
Merge pull request 'Added fields: totalcost, fundedamount, currency, in project table' (#257) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #257
|
2022-10-31 13:49:27 +01:00 |
dimitrispie
|
7861c472e0
|
Hive memory parameters
|
2022-10-28 19:00:32 +03:00 |
dimitrispie
|
5df9c63963
|
Added fields: totalcost, fundedamount, currency, in project table
|
2022-10-27 16:44:26 +03:00 |
Claudio Atzori
|
89f7007080
|
Merge pull request '[stats wf] misc changes' (#254) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #254
|
2022-10-03 10:32:05 +02:00 |
dimitrispie
|
2c0c3f1806
|
Cast amount to float for table result_apcs
|
2022-09-28 19:33:24 +03:00 |
dimitrispie
|
bdc46e3eaa
|
Remove denormalization of results to fix downloads numbers in monitor
|
2022-09-28 14:59:08 +03:00 |
dimitrispie
|
2ebb1459a9
|
Fixed type in no_downloads
|
2022-09-28 14:36:57 +03:00 |
Claudio Atzori
|
3b60642ef9
|
Merge pull request 'Synchronize indicators in stats-db with monitor-db' (#249) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #249
|
2022-09-27 14:37:33 +02:00 |
dimitrispie
|
dcd85f8cd7
|
- Synchronize indicators in stats-db with monitor-db
- added new openorg id for Nanyang Technological University
- changed openorg id for University of Helsinki #8088 ticket
|
2022-09-22 13:33:07 +03:00 |
Claudio Atzori
|
465e941214
|
Merge pull request '[stats wf] Changes to indicators tables' (#244) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #244
|
2022-09-16 10:13:58 +02:00 |
dimitrispie
|
3bf3127251
|
Changes to monitor and indicator scripts
|
2022-09-14 16:36:19 +03:00 |
dimitrispie
|
71b069ca90
|
Changes to indicator and monitor scripts
|
2022-09-09 13:15:58 +03:00 |
dimitrispie
|
2b5f8c9c9a
|
comment out duplicate table creation
|
2022-09-06 12:27:53 +03:00 |
Claudio Atzori
|
84598c7535
|
Merge pull request 'restored some collab indicators' (#240) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #240
|
2022-08-05 15:50:39 +02:00 |
Antonis Lempesis
|
fcef5294e2
|
restored some collab indicators
|
2022-08-05 13:45:01 +03:00 |
Claudio Atzori
|
c1f2ffc53d
|
Merge pull request 'commenting out the collab indicators because they still fail' (#237) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #237
|
2022-08-05 11:57:36 +02:00 |
Antonis Lempesis
|
227e10f4b3
|
commenting out the collab indicators because they still fail
|
2022-08-05 12:54:36 +03:00 |
Claudio Atzori
|
efd96e7e66
|
Merge pull request 'fixed the datasourceOrganization relations' (#233) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #233
|
2022-08-03 12:25:05 +02:00 |
Antonis Lempesis
|
8b0407d8ec
|
fixed the datasourceOrganization relations
|
2022-08-03 12:26:59 +03:00 |
Claudio Atzori
|
27681cf6bf
|
Merge pull request '[stats wf] latest version of indicators + added FOS classification' (#232) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #232
|
2022-08-02 12:57:15 +02:00 |
Antonis Lempesis
|
1778d40c40
|
latest version of indicators
|
2022-08-02 13:39:34 +03:00 |
Antonis Lempesis
|
6fc9ef53f6
|
addded command line params to allow hive actions to run
|
2022-07-29 16:36:20 +03:00 |
Antonis Lempesis
|
9886fe87ec
|
- Added FOS classification
- Added extra orgs in monitor
- Fixed result-project and organization-project tables
|
2022-07-29 16:34:50 +03:00 |
Miriam Baglioni
|
b229c6e7af
|
Merge pull request 'beta' (#218) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #218
|
2022-06-10 11:03:48 +02:00 |
Antonis Lempesis
|
ab18c9daa9
|
Merge branch 'beta' of https://code-repo.d4science.org/antonis.lempesis/dnet-hadoop into beta
|
2022-06-09 15:48:21 +03:00 |
Antonis Lempesis
|
574492c659
|
removed double result_apc table creation from monitor
|
2022-06-09 15:48:13 +03:00 |
Antonis Lempesis
|
db088cc69c
|
fixed *_organization tables
|
2022-06-07 04:04:28 +03:00 |
Claudio Atzori
|
5c2949a864
|
Merge pull request '[stats wf] added open citations & more orgs in monitor, removed collab indicator' (#213) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #213
|
2022-05-20 11:38:43 +02:00 |
Antonis Lempesis
|
3fc9efeab6
|
fixed typo, addded open citations and apcs in monitor
|
2022-05-13 14:28:13 +03:00 |
Antonis Lempesis
|
23334479bb
|
removed yet another collab, added more orgs in monitor
|
2022-05-11 13:05:52 +03:00 |
Antonis Lempesis
|
61b4c19e65
|
restored indi_result_org_country_collab, removed indi_result_org_collab
|
2022-05-06 12:52:10 +03:00 |
Antonis Lempesis
|
cfbbcaf7c4
|
commented out indi_result_org_country_collab
|
2022-05-06 12:49:36 +03:00 |
Antonis Lempesis
|
0353f93d54
|
added new hive opts
|
2022-04-29 12:49:27 +03:00 |
Antonis Lempesis
|
b7cd2c6ca1
|
added open citations
|
2022-04-20 14:46:55 +03:00 |
Claudio Atzori
|
4eff7856f5
|
Merge pull request '[stats-wf] computing stats in each step' (#210) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #210
|
2022-04-08 14:21:01 +02:00 |
Claudio Atzori
|
c26222623f
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 13:32:22 +02:00 |
Claudio Atzori
|
86585a6b27
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 13:32:19 +02:00 |
Claudio Atzori
|
ad85d88eaf
|
[maven-release-plugin] rollback the release of dhp-1.2.4
|
2022-04-07 13:28:35 +02:00 |
Claudio Atzori
|
598e11dfd7
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 13:27:02 +02:00 |
Claudio Atzori
|
db3d9877a5
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 13:26:58 +02:00 |
Claudio Atzori
|
3bba6d6e38
|
[maven-release-plugin] rollback the release of dhp-1.2.4
|
2022-04-07 12:23:17 +02:00 |
Claudio Atzori
|
2ac2d928bd
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 12:18:47 +02:00 |
Claudio Atzori
|
85bc722ff4
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 12:18:43 +02:00 |
Claudio Atzori
|
bc05b6168a
|
[maven-release-plugin] rollback the release of dhp-1.2.4
|
2022-04-07 11:49:06 +02:00 |
Claudio Atzori
|
505420fd61
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 11:34:06 +02:00 |
Claudio Atzori
|
66e718981e
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 11:34:02 +02:00 |
Antonis Lempesis
|
c442c91f89
|
computing stats in each step
|
2022-04-06 12:40:02 +03:00 |
Antonis Lempesis
|
7112806a73
|
views cannot be stored as parquet...
|
2022-03-29 16:37:29 +03:00 |
Antonis Lempesis
|
fff0b3cc19
|
added apcs in monitor db
|
2022-03-29 14:15:31 +03:00 |
Antonis Lempesis
|
ee24f3eb2c
|
views cannot be stored as parquet...
|
2022-03-29 13:47:48 +03:00 |
Antonis Lempesis
|
d8503cd191
|
added moooar organizations
|
2022-03-24 14:02:36 +02:00 |
Antonis Lempesis
|
62f91b0869
|
cleanup
|
2022-03-22 16:17:49 +02:00 |
Antonis Lempesis
|
2e8394ecf8
|
creating aaall tables as parquet
|
2022-03-22 16:16:08 +02:00 |
Antonis Lempesis
|
dcfbeb8142
|
yet more typos
|
2022-03-21 12:36:03 +02:00 |
Antonis Lempesis
|
ad78e505da
|
yet another fix
|
2022-03-03 12:28:12 +02:00 |
Antonis Lempesis
|
efeeebfee1
|
fixed query after the change in the indicator table
|
2022-03-02 13:29:25 +02:00 |
Antonis Lempesis
|
3b92a2ab9c
|
added the rest of spring 6 in monitor db
|
2022-02-23 12:05:57 +02:00 |
Antonis Lempesis
|
87c91f70a2
|
added sprint 6 indicators to monitor db
|
2022-02-22 14:41:48 +02:00 |
dimitrispie
|
58c59f46eb
|
Added Sprint 6
|
2022-02-17 10:21:09 +02:00 |
Antonis Lempesis
|
5772f92dba
|
merged beta chnages in hive branch
|
2022-02-15 13:24:51 +02:00 |
Antonis Lempesis
|
393a4ee956
|
fixed yet another typo...
|
2022-02-15 12:56:50 +02:00 |
Antonis Lempesis
|
5f762cbd09
|
fixed yet another typo
|
2022-02-07 12:09:12 +02:00 |
Antonis Lempesis
|
ae633c566b
|
fixed the result_result table
|
2022-02-04 15:04:19 +02:00 |
Antonis Lempesis
|
c2b44530a3
|
typo...
|
2022-02-03 13:44:07 +02:00 |
Antonis Lempesis
|
dbd2646d59
|
fixed the result_result creation for monitor
|
2022-02-03 12:37:10 +02:00 |
Antonis Lempesis
|
81ee654271
|
added result_result relations
|
2021-12-23 15:46:17 +02:00 |
Antonis Lempesis
|
7551e52e95
|
fixed a typo
|
2021-12-23 15:33:53 +02:00 |
Antonis Lempesis
|
16539d7360
|
added usage stats
|
2021-12-22 02:54:42 +02:00 |
Antonis Lempesis
|
3edd661608
|
fixed column names
|
2021-12-21 22:55:04 +02:00 |
Antonis Lempesis
|
a4c0cbb98c
|
fixed typos in indicators. Added extra views in monitor
|
2021-12-21 15:54:38 +02:00 |
Antonis Lempesis
|
58996972d9
|
added first indicator of sprint 5
|
2021-12-21 03:35:04 +02:00 |
dimitrispie
|
c1cdec09a9
|
Sprint 5 and other changes
|
2021-12-20 19:23:57 +02:00 |
Antonis Lempesis
|
ddd34087c2
|
removed 'stored as parquet' from views..
|
2021-12-13 23:05:00 +02:00 |
Antonis Lempesis
|
915f758c82
|
moving data to impala cluster and creating shadow databases there
|
2021-12-13 16:26:14 +02:00 |
Antonis Lempesis
|
d05210ba99
|
finished migration to hive only
|
2021-11-30 19:01:48 +02:00 |
dimitrispie
|
09fc2afdca
|
Added indi_funder_country_collab
Kept only indi_pub_has_cc_licence
|
2021-11-26 16:13:10 +02:00 |
Antonis Lempesis
|
0b4163ee0b
|
added sprint3,4, removed 2, chaos
|
2021-11-26 15:58:01 +02:00 |
Antonis Lempesis
|
12749a0a77
|
first
|
2021-11-26 15:40:40 +02:00 |
dimitrispie
|
29f69f2f89
|
Sprint 4
|
2021-11-26 15:22:04 +02:00 |
Antonis Lempesis
|
cb3adb90f4
|
Merge branch 'beta' into beta
|
2021-11-17 14:33:45 +01:00 |
Antonis Lempesis
|
c283406829
|
added Universidad Polytecnica de Madrid
|
2021-11-17 15:33:00 +02:00 |
Antonis Lempesis
|
26f086dd64
|
removed the too restrctive clause. will discuss again
|
2021-11-11 12:57:19 +02:00 |
Antonis Lempesis
|
91354c6068
|
- fetching all context related results
- storing tables as parquet
|
2021-11-08 15:15:46 +02:00 |
Claudio Atzori
|
7fa49f6956
|
Merge pull request 'removed hardcoded reference' (#154) from antonis.lempesis/dnet-hadoop:beta into beta
Reviewed-on: #154
|
2021-11-02 09:11:30 +01:00 |
Antonis Lempesis
|
f78afb5ef9
|
removed hardcoded reference
|
2021-11-01 15:42:29 +02:00 |
Claudio Atzori
|
4f8970f8ed
|
[stats] reducing the step22 wait time
|
2021-10-20 14:14:53 +02:00 |
Antonis Lempesis
|
241dcf6df1
|
Merge branch 'beta' into beta
|
2021-10-19 23:54:21 +02:00 |
Antonis Lempesis
|
41ecb1eb61
|
invalidating medatadata before context thingies
|
2021-10-15 13:42:55 +03:00 |
Antonis Lempesis
|
4b7c8dff2d
|
fetching affiliated results for 4 orgs in monitor. fixed affiliated orgs in stats db
|
2021-10-14 18:53:35 +03:00 |
Claudio Atzori
|
b292e4a700
|
[stats wf] added extra logging in the context data retrieval phase
|
2021-10-13 17:31:53 +02:00 |
dimitrispie
|
3f25d2efb2
|
Merge branch 'beta' of https://code-repo.d4science.org/antonis.lempesis/dnet-hadoop into beta
|
2021-10-01 16:03:48 +03:00 |
dimitrispie
|
13687fd887
|
Sprint 3 indicators update
|
2021-10-01 16:02:02 +03:00 |
Antonis Lempesis
|
a1e1cf32d7
|
fixed an impala error
|
2021-09-24 12:57:24 +03:00 |
Antonis Lempesis
|
f358cabb2b
|
fixed typo
|
2021-09-22 21:50:37 +03:00 |
Antonis Lempesis
|
421d55265d
|
created hive action for observatory queries
|
2021-09-21 03:07:58 +03:00 |
Antonis Lempesis
|
8b681dcf1b
|
attempt to make the observatory wf run in hive
|
2021-09-18 00:35:14 +03:00 |
Antonis Lempesis
|
2943287d10
|
fixed the definition of cc_licence, part II
|
2021-09-16 15:59:06 +03:00 |
Antonis Lempesis
|
dd2329849f
|
fixed the definition of cc_licence
|
2021-09-16 13:50:34 +03:00 |
Antonis Lempesis
|
de9bf3a161
|
added cc_licences and abstracts in observatory db
|
2021-09-14 01:29:08 +03:00 |
Antonis Lempesis
|
9b1936701c
|
fixed yet another typo
|
2021-09-13 21:07:44 +03:00 |
Antonis Lempesis
|
8fc89ae822
|
moved context table creation before indicators
|
2021-09-13 14:33:23 +03:00 |
Antonis Lempesis
|
461bf90ca6
|
fixed the gold_oa definition
|
2021-09-13 11:10:30 +03:00 |
Antonis Lempesis
|
43852bac0e
|
creating other::other concept for all contexts
|
2021-09-13 01:36:41 +03:00 |
Antonis Lempesis
|
f13cca7e83
|
moved dependencies of indicators before them...
|
2021-09-08 23:07:58 +03:00 |
Antonis Lempesis
|
c6ada217a1
|
fixed typo
|
2021-09-08 22:34:59 +03:00 |
Antonis Lempesis
|
1250ae197f
|
using new indicators for the definition of peerreviewed, gold, and green
|
2021-09-08 14:08:43 +03:00 |
Antonis Lempesis
|
ccee451dde
|
added indicators of sprint 2 in monitor db
|
2021-09-07 23:17:13 +03:00 |
Antonis Lempesis
|
117c3d5c67
|
fixed a typo
|
2021-08-02 12:15:58 +03:00 |
Antonis Lempesis
|
26af0320d0
|
added the sprint 2 indicators in monitor db
|
2021-07-30 00:31:33 +03:00 |