Merge pull request #26 from smotornyuk/use-service-account
Use google service account instead of temporary tokens
This commit is contained in:
commit
302aa3116f
36
README.rst
36
README.rst
|
@ -51,6 +51,7 @@ Installation
|
||||||
::
|
::
|
||||||
|
|
||||||
$ pip install -e git+https://github.com/ckan/ckanext-googleanalytics.git#egg=ckanext-googleanalytics
|
$ pip install -e git+https://github.com/ckan/ckanext-googleanalytics.git#egg=ckanext-googleanalytics
|
||||||
|
$ pip install -r ckanext-googleanalytics/requirements.txt
|
||||||
|
|
||||||
2. Edit your development.ini (or similar) to provide these necessary parameters:
|
2. Edit your development.ini (or similar) to provide these necessary parameters:
|
||||||
|
|
||||||
|
@ -122,8 +123,6 @@ See `Googles' documentation<https://support.google.com/analytics/answer/1034342?
|
||||||
Setting Up Statistics Retrieval from Google Analytics
|
Setting Up Statistics Retrieval from Google Analytics
|
||||||
-----------------------------------------------------
|
-----------------------------------------------------
|
||||||
|
|
||||||
*CKAN 1.x only*
|
|
||||||
|
|
||||||
1. Run the following command from ``src/ckanext-googleanalytics`` to
|
1. Run the following command from ``src/ckanext-googleanalytics`` to
|
||||||
set up the required database tables (of course, altering the
|
set up the required database tables (of course, altering the
|
||||||
``--config`` option to point to your site config file)::
|
``--config`` option to point to your site config file)::
|
||||||
|
@ -146,10 +145,11 @@ Setting Up Statistics Retrieval from Google Analytics
|
||||||
6. Import Google stats by running the following command from
|
6. Import Google stats by running the following command from
|
||||||
``src/ckanext-googleanalytics``::
|
``src/ckanext-googleanalytics``::
|
||||||
|
|
||||||
paster loadanalytics token.dat --config=../ckan/development.ini
|
paster loadanalytics credentials.json --config=../ckan/development.ini
|
||||||
|
|
||||||
(Of course, pointing config at your specific site config and token.dat at the
|
(Of course, pointing config at your specific site config and credentials.json at the
|
||||||
oauth file generated from the authorization step)
|
key file obtained from the authorization step)
|
||||||
|
Ignore warning `ImportError: file_cache is unavailable when using oauth2client >= 4.0.0`
|
||||||
|
|
||||||
7. Look at some stats within CKAN
|
7. Look at some stats within CKAN
|
||||||
|
|
||||||
|
@ -168,35 +168,19 @@ Setting Up Statistics Retrieval from Google Analytics
|
||||||
Authorization
|
Authorization
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
*CKAN 1.x only*
|
Before ckanext-googleanalytics can retrieve statistics from Google Analytics, you need to set up the OAUTH details which you can do by following the `instructions <https://developers.google.com/analytics/devguides/reporting/core/v3/quickstart/service-py>`_ the outcome of which will be a file with authentication key. These steps are below for convenience:
|
||||||
|
|
||||||
Before ckanext-googleanalytics can retrieve statistics from Google Analytics, you need to set up the OAUTH details which you can do by following the `instructions <https://developers.google.com/analytics/resources/tutorials/hello-analytics-api>`_ the outcome of which will be a file called credentials.json which should look like credentials.json.template with the relevant fields completed. These steps are below for convenience:
|
|
||||||
|
|
||||||
1. Visit the `Google APIs Console <https://code.google.com/apis/console>`_
|
1. Visit the `Google APIs Console <https://code.google.com/apis/console>`_
|
||||||
|
|
||||||
2. Sign-in and create a project or use an existing project.
|
2. Sign-in and create a project or use an existing project.
|
||||||
|
|
||||||
3. In the `Services pane <https://code.google.com/apis/console#:services>`_ , activate Analytics API for your project. If prompted, read and accept the terms of service.
|
3. In the `Service accounts pane <https://console.developers.google.com/iam-admin/serviceaccounts>`_ choose your project and create new account. During creation check "Furnish a new private key" -> JSON type. Write down "Service account ID"(looks like email) - it will be used later.
|
||||||
|
|
||||||
4. Go to the `API Access pane <https://code.google.com/apis/console/#:access>`_
|
4. Save downloaded file - it will be used by `loadanalytics` command(referenced as <credentials.json>)
|
||||||
|
|
||||||
5. Click Create an OAuth 2.0 client ID....
|
5. Go to `GoogleAnalytics console <https://analytics.google.com/analytics/web/#management>`_ and chose ADMIN tab.
|
||||||
|
|
||||||
6. Fill out the Branding Information fields and click Next.
|
|
||||||
|
|
||||||
7. In Client ID Settings, set Application type to Installed application.
|
|
||||||
|
|
||||||
8. Click Create client ID
|
|
||||||
|
|
||||||
9. The details you need below are Client ID, Client secret, and Redirect URIs
|
|
||||||
|
|
||||||
|
|
||||||
Once you have set up your credentials.json file you can generate an oauth token file by using the
|
|
||||||
following command, which will store your oauth token in a file called token.dat once you have finished
|
|
||||||
giving permission in the browser::
|
|
||||||
|
|
||||||
$ paster getauthtoken --config=../ckan/development.ini
|
|
||||||
|
|
||||||
|
6. Find "User management" button in corresponding column. Add service account using Service account ID(email) generated in 3rd step and grant "Read" role to it.
|
||||||
|
|
||||||
|
|
||||||
Testing
|
Testing
|
||||||
|
|
|
@ -18,32 +18,6 @@ RESOURCE_URL_REGEX = re.compile('/dataset/[a-z0-9-_]+/resource/([a-z0-9-_]+)')
|
||||||
DATASET_EDIT_REGEX = re.compile('/dataset/edit/([a-z0-9-_]+)')
|
DATASET_EDIT_REGEX = re.compile('/dataset/edit/([a-z0-9-_]+)')
|
||||||
|
|
||||||
|
|
||||||
class GetAuthToken(CkanCommand):
|
|
||||||
""" Get's the Google auth token
|
|
||||||
|
|
||||||
Usage: paster getauthtoken <credentials_file>
|
|
||||||
|
|
||||||
Where <credentials_file> is the file name containing the details
|
|
||||||
for the service (obtained from https://code.google.com/apis/console).
|
|
||||||
By default this is set to credentials.json
|
|
||||||
"""
|
|
||||||
summary = __doc__.split('\n')[0]
|
|
||||||
usage = __doc__
|
|
||||||
max_args = 1
|
|
||||||
min_args = 0
|
|
||||||
|
|
||||||
def command(self):
|
|
||||||
"""
|
|
||||||
In this case we don't want a valid service, but rather just to
|
|
||||||
force the user through the auth flow. We allow this to complete to
|
|
||||||
act as a form of verification instead of just getting the token and
|
|
||||||
assuming it is correct.
|
|
||||||
"""
|
|
||||||
from ga_auth import init_service
|
|
||||||
init_service('token.dat',
|
|
||||||
self.args[0] if self.args else 'credentials.json')
|
|
||||||
|
|
||||||
|
|
||||||
class InitDB(CkanCommand):
|
class InitDB(CkanCommand):
|
||||||
"""Initialise the local stats database tables
|
"""Initialise the local stats database tables
|
||||||
"""
|
"""
|
||||||
|
@ -65,8 +39,8 @@ class LoadAnalytics(CkanCommand):
|
||||||
in a local database
|
in a local database
|
||||||
|
|
||||||
Options:
|
Options:
|
||||||
<token_file> internal [date] use ckan internal tracking tables
|
<credentials_file> internal [date] use ckan internal tracking tables
|
||||||
token_file specifies the OAUTH token file
|
credentials_file specifies the OAUTH credentials file
|
||||||
date specifies start date for retrieving
|
date specifies start date for retrieving
|
||||||
analytics data YYYY-MM-DD format
|
analytics data YYYY-MM-DD format
|
||||||
"""
|
"""
|
||||||
|
@ -252,11 +226,9 @@ class LoadAnalytics(CkanCommand):
|
||||||
raise Exception('Cannot find the token file %s' % self.args[0])
|
raise Exception('Cannot find the token file %s' % self.args[0])
|
||||||
|
|
||||||
try:
|
try:
|
||||||
self.service = init_service(self.args[0], None)
|
self.service = init_service(self.args[0])
|
||||||
except TypeError:
|
except TypeError as e:
|
||||||
print ('Have you correctly run the getauthtoken task and '
|
raise Exception('Unable to create a service: {0}'.format(e))
|
||||||
'specified the correct file here')
|
|
||||||
raise Exception('Unable to create a service')
|
|
||||||
self.profile_id = get_profile_id(self.service)
|
self.profile_id = get_profile_id(self.service)
|
||||||
|
|
||||||
if len(self.args) > 1:
|
if len(self.args) > 1:
|
||||||
|
|
|
@ -1,31 +1,24 @@
|
||||||
import os
|
|
||||||
import httplib2
|
import httplib2
|
||||||
from apiclient.discovery import build
|
from apiclient.discovery import build
|
||||||
from oauth2client.client import flow_from_clientsecrets
|
from oauth2client.service_account import ServiceAccountCredentials
|
||||||
from oauth2client.file import Storage
|
|
||||||
from oauth2client.tools import run
|
|
||||||
|
|
||||||
from pylons import config
|
from pylons import config
|
||||||
|
|
||||||
|
|
||||||
def _prepare_credentials(token_filename, credentials_filename):
|
def _prepare_credentials(credentials_filename):
|
||||||
"""
|
"""
|
||||||
Either returns the user's oauth credentials or uses the credentials
|
Either returns the user's oauth credentials or uses the credentials
|
||||||
file to generate a token (by forcing the user to login in the browser)
|
file to generate a token (by forcing the user to login in the browser)
|
||||||
"""
|
"""
|
||||||
storage = Storage(token_filename)
|
scope = ['https://www.googleapis.com/auth/analytics.readonly']
|
||||||
credentials = storage.get()
|
credentials = ServiceAccountCredentials.from_json_keyfile_name(
|
||||||
|
credentials_filename,
|
||||||
if credentials is None or credentials.invalid:
|
scopes=scope
|
||||||
flow = flow_from_clientsecrets(credentials_filename,
|
)
|
||||||
scope='https://www.googleapis.com/auth/analytics.readonly',
|
|
||||||
message="Can't find the credentials file")
|
|
||||||
credentials = run(flow, storage)
|
|
||||||
|
|
||||||
return credentials
|
return credentials
|
||||||
|
|
||||||
|
|
||||||
def init_service(token_file, credentials_file):
|
def init_service(credentials_file):
|
||||||
"""
|
"""
|
||||||
Given a file containing the user's oauth token (and another with
|
Given a file containing the user's oauth token (and another with
|
||||||
credentials in case we need to generate the token) will return a
|
credentials in case we need to generate the token) will return a
|
||||||
|
@ -33,7 +26,7 @@ def init_service(token_file, credentials_file):
|
||||||
"""
|
"""
|
||||||
http = httplib2.Http()
|
http = httplib2.Http()
|
||||||
|
|
||||||
credentials = _prepare_credentials(token_file, credentials_file)
|
credentials = _prepare_credentials(credentials_file)
|
||||||
http = credentials.authorize(http) # authorize the http object
|
http = credentials.authorize(http) # authorize the http object
|
||||||
|
|
||||||
return build('analytics', 'v3', http=http)
|
return build('analytics', 'v3', http=http)
|
||||||
|
@ -58,7 +51,9 @@ def get_profile_id(service):
|
||||||
if acc.get('name') == accountName:
|
if acc.get('name') == accountName:
|
||||||
accountId = acc.get('id')
|
accountId = acc.get('id')
|
||||||
|
|
||||||
webproperties = service.management().webproperties().list(accountId=accountId).execute()
|
# TODO: check, whether next line is doing something useful.
|
||||||
|
webproperties = service.management().webproperties().list(
|
||||||
|
accountId=accountId).execute()
|
||||||
|
|
||||||
profiles = service.management().profiles().list(
|
profiles = service.management().profiles().list(
|
||||||
accountId=accountId, webPropertyId=webPropertyId).execute()
|
accountId=accountId, webPropertyId=webPropertyId).execute()
|
||||||
|
|
|
@ -1,10 +0,0 @@
|
||||||
{
|
|
||||||
"installed": {
|
|
||||||
"client_id": "",
|
|
||||||
"client_secret": "",
|
|
||||||
"redirect_uris": [""],
|
|
||||||
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
|
|
||||||
"token_uri": "https://accounts.google.com/o/oauth2/token"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
|
@ -0,0 +1,3 @@
|
||||||
|
gdata>=2.0.0
|
||||||
|
google-api-python-client>=1.6.1
|
||||||
|
pyOpenSSL>=16.2.0
|
4
setup.py
4
setup.py
|
@ -20,8 +20,7 @@ setup(
|
||||||
include_package_data=True,
|
include_package_data=True,
|
||||||
zip_safe=False,
|
zip_safe=False,
|
||||||
install_requires=[
|
install_requires=[
|
||||||
'gdata',
|
|
||||||
'google-api-python-client'
|
|
||||||
],
|
],
|
||||||
entry_points=\
|
entry_points=\
|
||||||
"""
|
"""
|
||||||
|
@ -32,6 +31,5 @@ setup(
|
||||||
[paste.paster_command]
|
[paste.paster_command]
|
||||||
loadanalytics = ckanext.googleanalytics.commands:LoadAnalytics
|
loadanalytics = ckanext.googleanalytics.commands:LoadAnalytics
|
||||||
initdb = ckanext.googleanalytics.commands:InitDB
|
initdb = ckanext.googleanalytics.commands:InitDB
|
||||||
getauthtoken = ckanext.googleanalytics.commands:GetAuthToken
|
|
||||||
""",
|
""",
|
||||||
)
|
)
|
||||||
|
|
Loading…
Reference in New Issue