gCubeSystem
/
accounting-dashboard-harvester-se-plugin
15
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
121 Commits 2 Branches 16 Tags 422 KiB
Commit Graph

71 Commits

Author SHA1 Message Date
Luca Frosini 9f6b745c4d Fixed Plugin Name 2019-12-03 16:46:46 +01:00
Luca Frosini 4913a3fe03 Added the possibility of having partial harvesting 2019-10-21 16:56:44 +02:00
Luca Frosini 6833e9fe3b improved code 2019-09-20 14:39:59 +02:00
Luca Frosini 58e035998f Removed CKAN utils library 2019-09-18 16:03:40 +02:00
Luca Frosini 2977a684fd Migrating from HomeLibrary to StorageHub refs #17128 2019-07-15 17:45:30 +02:00
Massimiliano Assante 7c58db8e8b git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179285 82a268e6-3cf1-43bd-a215-b396298e98cf 2019-05-06 15:34:14 +00:00
Luca Frosini 6f86f199d6 Fixed TagMe Harvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179259 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-03 14:22:20 +00:00
Luca Frosini 354bc153ce Updating haverster 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179244 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-03 12:22:11 +00:00
Luca Frosini e63e4a65f1 Fixed method on tag me 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@179231 82a268e6-3cf1-43bd-a215-b396298e98cf
 2019-05-02 15:39:35 +00:00
Luca Frosini 3923486937 Added test to create missing records 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@174108 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-11-12 15:30:20 +00:00
Massimiliano Assante 1ba2850a7c Fix for Bug #12454 BigG Analytics importer for accounting-dashboard wrong interpretation of ACE VRE accesses 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@171687 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-09-26 14:58:07 +00:00
Luca Frosini 99b12cfe07 Fixes #11760: Refactor accounting-dashboard-harvester-se-plugin to support new DB schema 
Task-Url: https://support.d4science.org/issues/11760

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@171294 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-09-04 09:13:16 +00:00
Luca Frosini 456f8b31b7 Added logs and fixed tests 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169689 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-07-05 16:25:05 +00:00
Luca Frosini 3ba8504488 Corrected comment typo 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169294 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-18 12:42:59 +00:00
Massimiliano Assante 6281106d7a fixed bug in counting page accesses 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169293 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-18 12:30:14 +00:00
Luca Frosini 0f5311cb91 Added test to harvest old data 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169227 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 16:17:22 +00:00
Luca Frosini 56d97e228b Added logic to exclude harvesting in some cases 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169213 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 14:22:19 +00:00
Luca Frosini 13c74dc4f9 Added logic to exclude harvesting in some cases 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169212 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 14:19:25 +00:00
Luca Frosini 248bb98b88 Integrated VREAccessesHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169205 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 13:30:46 +00:00
Massimiliano Assante adab0bea74 revised heuristic counting VRE accesses from page paths 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169200 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 13:15:08 +00:00
Luca Frosini 8335d11956 Added tests 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169193 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 12:50:42 +00:00
Massimiliano Assante 412e1e9264 modified condtion to match vre accesses better 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169192 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 12:35:55 +00:00
Luca Frosini 6d0c7ec0f4 Integrated VREAccessesHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169191 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 12:34:11 +00:00
Massimiliano Assante af5e7cb8c8 added VRE Access Harvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169190 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-14 10:46:50 +00:00
Luca Frosini 3d53d85865 Added logic to harvesting methods invocation by using Service Usage Records before 31/12/2017 and using JobUSageRecords from 01/01/2018 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169181 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-13 17:40:02 +00:00
Luca Frosini f27b583949 Added logic to avoid harvesting of VRE users for VO and logic to harvest VRE Accesses 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169180 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-13 17:39:01 +00:00
Luca Frosini 3630de9582 Added thread local variable 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@169136 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-13 12:43:36 +00:00
Luca Frosini f0f981d197 Fixed launch 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167870 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-06-04 14:35:32 +00:00
Luca Frosini 3a69d5e50c Added auto creation of contexts 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167822 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-29 11:05:55 +00:00
Luca Frosini 618f18e331 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167820 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-29 09:08:24 +00:00
Luca Frosini 67ae816424 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167798 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 14:53:00 +00:00
Luca Frosini b4aa929d55 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167795 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 14:33:57 +00:00
Luca Frosini b91e31853c git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167794 82a268e6-3cf1-43bd-a215-b396298e98cf 2018-05-28 14:22:44 +00:00
Luca Frosini 6e3ca2befe Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167792 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 13:34:19 +00:00
Luca Frosini 9f40118fa7 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167791 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 13:22:59 +00:00
Luca Frosini 3c6b05d00b Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167788 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-28 10:00:15 +00:00
Francesco Mangiacrapa 19e396d374 Ultimated fixing by Francesco 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167722 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-24 15:13:16 +00:00
Francesco Mangiacrapa cb8e1a62a5 Fixed DataMethodDownloadHarvester 
git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167721 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-24 15:10:45 +00:00
Francesco Mangiacrapa 09a8f2d2dc Updated by Francesco: 
- new Resource Catalogue Harvester created.

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167720 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-24 10:49:53 +00:00
Luca Frosini 1971f14809 Changed Method invocations from using ServiceUsageRecord to JobUsageRecords 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167658 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-21 16:22:17 +00:00
Luca Frosini dd9f5b8553 Changed Method invocations from using ServiceUsageRecord to JobUsageRecords 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167657 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-21 16:22:08 +00:00
Luca Frosini 72f29300c9 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167646 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-21 14:23:14 +00:00
Luca Frosini 559251fe2f Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167645 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-21 13:56:52 +00:00
Luca Frosini b15ba30b1d Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167644 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-21 13:52:19 +00:00
Luca Frosini 95284de142 git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167643 82a268e6-3cf1-43bd-a215-b396298e98cf 2018-05-21 13:37:57 +00:00
Luca Frosini 9aea29c6e1 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167642 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-21 12:49:01 +00:00
Luca Frosini 0de8fd20a6 git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167641 82a268e6-3cf1-43bd-a215-b396298e98cf 2018-05-21 12:15:09 +00:00
Luca Frosini 302d29a607 git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167640 82a268e6-3cf1-43bd-a215-b396298e98cf 2018-05-21 12:06:27 +00:00
Luca Frosini 5a84a04fbd git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167639 82a268e6-3cf1-43bd-a215-b396298e98cf 2018-05-21 12:05:15 +00:00
Luca Frosini 1ef7cee987 Code redesign 
Refs #11756: Refactor DataHArvesterPlugin to support scheduled execution from smart-executor 

Task-Url: https://support.d4science.org/issues/11756

git-svn-id: https://svn.d4science.research-infrastructures.eu/gcube/trunk/accounting/accounting-dashboard-harvester-se-plugin@167636 82a268e6-3cf1-43bd-a215-b396298e98cf
 2018-05-21 10:51:17 +00:00