gCubeSystem
/
accounting-dashboard-harvester-se-plugin
16
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
120 Commits 2 Branches 14 Tags 414 KiB
Commit Graph

93 Commits

Author SHA1 Message Date
Luca Frosini 4913a3fe03 Added the possibility of having partial harvesting 2019-10-21 16:56:44 +02:00
Luca Frosini 9a252fde8e Fixed test 2019-09-23 12:04:12 +02:00
Luca Frosini 6833e9fe3b improved code 2019-09-20 14:39:59 +02:00
Luca Frosini b9c72cd133 Fixed dependency 2019-09-19 14:27:55 +02:00
Luca Frosini 58e035998f Removed CKAN utils library 2019-09-18 16:03:40 +02:00
Luca Frosini 8af5665a91 Refs #17128: Remove HomeLibrary Usage from 
accounting-dashboard-harvester-se-plugin

Task-Url: https://support.d4science.org/issues/17128
 2019-07-18 11:22:52 +02:00
Luca Frosini 2977a684fd Migrating from HomeLibrary to StorageHub refs #17128 2019-07-15 17:45:30 +02:00
Luca Frosini 1a8c8f864a Removed tests 2019-07-15 16:29:22 +02:00
Luca Frosini 55b38ec587 VO file indication is not needed anymore 2019-07-15 15:37:48 +02:00
Luca Frosini 841c30c296 Finalising import 2019-07-15 15:29:38 +02:00
Massimiliano Assante 7c58db8e8b git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179285 82a268e6-3cf1-43bd-a215-b396298e98cf 2019-05-06 15:34:14 +00:00
Luca Frosini 372605d175 Updating haverster 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179261 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-03 14:30:24 +00:00
Luca Frosini 353154a2ed Updating haverster 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179260 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-03 14:29:56 +00:00
Luca Frosini 6f86f199d6 Fixed TagMe Harvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179259 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-03 14:22:20 +00:00
Luca Frosini 354bc153ce Updating haverster 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179244 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-03 12:22:11 +00:00
Luca Frosini e63e4a65f1 Fixed method on tag me 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179231 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-02 15:39:35 +00:00
Luca Frosini 1a3ef7a31d Merged from branch after release 4.13.1 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@177276 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-02-26 13:53:32 +00:00
Luca Frosini a0f5cc0331 Fixed dependencies 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@174562 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-12-05 08:52:19 +00:00
Luca Frosini 3923486937 Added test to create missing records 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@174108 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-11-12 15:30:20 +00:00
Massimiliano Assante 1ba2850a7c Fix for Bug #12454 BigG Analytics importer for accounting-dashboard wrong interpretation of ACE VRE accesses 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@171687 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-09-26 14:58:07 +00:00
Luca Frosini 03aa4b0c42 Commented tests 
Refs #11760: Refactor accounting-dashboard-harvester-se-plugin to support new DB schema 

Task-Url: https://support.d4science.org/issues/11760

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@171296 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-09-04 09:20:11 +00:00
Luca Frosini 95bf87332f Commented test 
Refs #11760: Refactor accounting-dashboard-harvester-se-plugin to support new DB schema 

Task-Url: https://support.d4science.org/issues/11760

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@171295 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-09-04 09:14:40 +00:00
Luca Frosini 99b12cfe07 Fixes #11760: Refactor accounting-dashboard-harvester-se-plugin to support new DB schema 
Task-Url: https://support.d4science.org/issues/11760

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@171294 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-09-04 09:13:16 +00:00
Luca Frosini 456f8b31b7 Added logs and fixed tests 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169689 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-07-05 16:25:05 +00:00
Luca Frosini c6e7ad3077 Fixed test to harvest old data for VREAccessesHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169298 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-18 13:44:10 +00:00
Luca Frosini 87849c04b8 Added test to harvest old data for VREAccessesHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169295 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-18 13:15:12 +00:00
Luca Frosini 3ba8504488 Corrected comment typo 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169294 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-18 12:42:59 +00:00
Massimiliano Assante 6281106d7a fixed bug in counting page accesses 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169293 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-18 12:30:14 +00:00
Luca Frosini 85ea36fb00 commented tests to avoid unwanted launch 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169228 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 16:19:10 +00:00
Luca Frosini 0f5311cb91 Added test to harvest old data 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169227 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 16:17:22 +00:00
Luca Frosini 56d97e228b Added logic to exclude harvesting in some cases 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169213 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 14:22:19 +00:00
Luca Frosini 13c74dc4f9 Added logic to exclude harvesting in some cases 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169212 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 14:19:25 +00:00
Luca Frosini 248bb98b88 Integrated VREAccessesHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169205 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 13:30:46 +00:00
Luca Frosini 4b4009d99b Integrated VREAccessesHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169204 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 13:30:07 +00:00
Massimiliano Assante adab0bea74 revised heuristic counting VRE accesses from page paths 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169200 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 13:15:08 +00:00
Luca Frosini 8335d11956 Added tests 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169193 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 12:50:42 +00:00
Massimiliano Assante 412e1e9264 modified condtion to match vre accesses better 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169192 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 12:35:55 +00:00
Luca Frosini 6d0c7ec0f4 Integrated VREAccessesHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169191 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 12:34:11 +00:00
Massimiliano Assante af5e7cb8c8 added VRE Access Harvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169190 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 10:46:50 +00:00
Luca Frosini 3d53d85865 Added logic to harvesting methods invocation by using Service Usage Records before 31/12/2017 and using JobUSageRecords from 01/01/2018 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169181 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-13 17:40:02 +00:00
Luca Frosini f27b583949 Added logic to avoid harvesting of VRE users for VO and logic to harvest VRE Accesses 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169180 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-13 17:39:01 +00:00
Luca Frosini 3630de9582 Added thread local variable 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169136 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-13 12:43:36 +00:00
Luca Frosini f0f981d197 Fixed launch 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167870 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-04 14:35:32 +00:00
Luca Frosini 3a69d5e50c Added auto creation of contexts 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167822 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-29 11:05:55 +00:00
Luca Frosini 618f18e331 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167820 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-29 09:08:24 +00:00
Luca Frosini 67ae816424 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167798 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 14:53:00 +00:00
Luca Frosini 0d0aaf1b21 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167797 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 14:39:13 +00:00
Luca Frosini b4aa929d55 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167795 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 14:33:57 +00:00
Luca Frosini b91e31853c git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167794 82a268e6-3cf1-43bd-a215-b396298e98cf 2018-05-28 14:22:44 +00:00
Luca Frosini b5ac35f453 git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167793 82a268e6-3cf1-43bd-a215-b396298e98cf 2018-05-28 14:03:31 +00:00