CatalogueRevamping
/
docker-ckan
14
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update ckanext-xloader

This commit is contained in:
mjanez 2023-03-22 17:03:33 +00:00 committed by GitHub
parent 5fb6a14fb5
commit e3ab392736
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 18 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.env
View File

@ -58,7 +58,7 @@ CKAN__RESOURCE_PROXY__MAX_FILE_SIZE=50048576
CKAN__RESOURCE_PROXY__CHUNK_SIZE=4096 CKAN__RESOURCE_PROXY__CHUNK_SIZE=4096
## Default timeout for fetching proxied items ## Default timeout for fetching proxied items
CKAN__RESOURCE_PROXY__TIMEOUT=10 CKAN__RESOURCE_PROXY__TIMEOUT=10
CKAN__VIEWS__DEFAULT_VIEWS="image_view text_view recline_view harvest ckan_harvester wmts_view geojson_view geo_view shp_view pdf_view" CKAN__VIEWS__DEFAULT_VIEWS="image_view text_view recline_view recline_grid_view recline_graph_view wmts_view webpage_view geojson_view geo_view shp_view pdf_view"
# Solr # Solr
SOLR_IMAGE_VERSION=2.9-solr8-spatial SOLR_IMAGE_VERSION=2.9-solr8-spatial

2
README.md
View File

@ -55,7 +55,7 @@ Information about extensions installed in the `main` image. More info described
| Core | [CKAN](https://github.com/mjanez/ckan-docker) | 2.9.8 | Completed | ✔️ | ✔️ | Stable installation for version 2.9.8 (Production & Dev images) via Docker Compose based on [official images](https://github.com/ckan/ckan-docker-base)). Initial configuration, basic customisation and operation guide. | | Core | [CKAN](https://github.com/mjanez/ckan-docker) | 2.9.8 | Completed | ✔️ | ✔️ | Stable installation for version 2.9.8 (Production & Dev images) via Docker Compose based on [official images](https://github.com/ckan/ckan-docker-base)). Initial configuration, basic customisation and operation guide. |
| Core + | [Datastore](https://github.com/mjanez/ckan-docker) | 2.9.8 | Completed | ✔️ | ✔️ | Stable installation (Production & Dev images) via Docker Compose. | | Core + | [Datastore](https://github.com/mjanez/ckan-docker) | 2.9.8 | Completed | ✔️ | ✔️ | Stable installation (Production & Dev images) via Docker Compose. |
| Core + | [~~Datapusher~~](https://github.com/mjanez/ckan-docker) | 0.0.19 | Deprecated | ❌ | ❌ | Updated to [xloader](https://github.com/ckan/ckanext-xloader), an express Loader - quickly load data into DataStore. | | Core + | [~~Datapusher~~](https://github.com/mjanez/ckan-docker) | 0.0.19 | Deprecated | ❌ | ❌ | Updated to [xloader](https://github.com/ckan/ckanext-xloader), an express Loader - quickly load data into DataStore. |
| Extension | [ckanext-xloader](https://github.com/ckan/ckanext-xloader) | 0.12.2 | Completed | ✔️ | ❌ | Stable installation, a replacement for DataPusher because it offers ten times the speed and more robustness | | Extension | [ckanext-xloader](https://github.com/ckan/ckanext-xloader) | 0.12.2 | Completed | ✔️ | ✔️ | Stable installation, a replacement for DataPusher because it offers ten times the speed and more robustness |
| Extension | [ckanext-harvest](https://github.com/ckan/ckanext-harvest) | 1.5.0 | Completed | ✔️ | ✔️ | Stable installation, necessary for the implementation of the Collector ([ogc_ckan](#recollector-ckan)) | | Extension | [ckanext-harvest](https://github.com/ckan/ckanext-harvest) | 1.5.0 | Completed | ✔️ | ✔️ | Stable installation, necessary for the implementation of the Collector ([ogc_ckan](#recollector-ckan)) |
| Extension | [ckanext-geoview](https://github.com/ckan/ckanext-geoview) | 0.0.20 | Completed | ✔️ | ✔️ | Stable installation. | | Extension | [ckanext-geoview](https://github.com/ckan/ckanext-geoview) | 0.0.20 | Completed | ✔️ | ✔️ | Stable installation. |
| Extension | [ckanext-spatial](https://github.com/ckan/ckanext-spatial) | 2.0.0 | Completed | ✔️ | ✔️ | Stable installation, necessary for the implementation of the Collector ([ogc_ckan](#recollector-ckan)) | | Extension | [ckanext-spatial](https://github.com/ckan/ckanext-spatial) | 2.0.0 | Completed | ✔️ | ✔️ | Stable installation, necessary for the implementation of the Collector ([ogc_ckan](#recollector-ckan)) |

3
ckan/docker-entrypoint.d/setup_xloader.sh
View File

@ -9,3 +9,6 @@ ckan config-tool $CKAN_INI \
# Create ckanext-xloader API_TOKEN # Create ckanext-xloader API_TOKEN
echo "Set up ckanext.xloader.api_token in the CKAN config file" echo "Set up ckanext.xloader.api_token in the CKAN config file"
ckan config-tool $CKAN_INI "ckanext.xloader.api_token = $(ckan -c $CKAN_INI user token add ckan_admin xloader | tail -n 1 | tr -d '\t')" ckan config-tool $CKAN_INI "ckanext.xloader.api_token = $(ckan -c $CKAN_INI user token add ckan_admin xloader | tail -n 1 | tr -d '\t')"
# Setup worker
ckan config-tool $CKAN_INI jobs worker

24
samples/.env.example
View File

@ -3,7 +3,6 @@ NGINX_CONTAINER_NAME=nginx
REDIS_CONTAINER_NAME=redis REDIS_CONTAINER_NAME=redis
POSTGRESQL_CONTAINER_NAME=db POSTGRESQL_CONTAINER_NAME=db
SOLR_CONTAINER_NAME=solr SOLR_CONTAINER_NAME=solr
DATAPUSHER_CONTAINER_NAME=datapusher
CKAN_CONTAINER_NAME=ckan CKAN_CONTAINER_NAME=ckan
WORKER_CONTAINER_NAME=ckan-worker WORKER_CONTAINER_NAME=ckan-worker
@ -28,16 +27,15 @@ TEST_CKAN_DATASTORE_WRITE_URL=postgresql://ckan:ckan@db/datastore_test
TEST_CKAN_DATASTORE_READ_URL=postgresql://datastore_ro:datastore@db/datastore_test TEST_CKAN_DATASTORE_READ_URL=postgresql://datastore_ro:datastore@db/datastore_test
# CKAN core # CKAN core
CKAN_VERSION=2.10.0 CKAN_VERSION=2.9.8
CKAN_SITE_ID=default CKAN_SITE_ID=default
CKAN_SITE_URL=http://localhost:5000 CKAN_SITE_URL=http://localhost:5000
CKAN_PORT=5000 CKAN_PORT=5000
CKAN_PORT_HOST=5000 CKAN_PORT_HOST=5000
CKAN___BEAKER__SESSION__SECRET=39ASDK913r4jas0 CKAN___BEAKER__SESSION__SECRET=CHANGE_ME
# See https://docs.ckan.org/en/latest/maintaining/configuration.html#api-token-settings # See https://docs.ckan.org/en/latest/maintaining/configuration.html#api-token-settings
CKAN___API_TOKEN__JWT__ENCODE__SECRET=string:WqVhDAXsF7E9FvzVnNqdsKpDT9wZbiu6hzxyMijv4FN3B CKAN___API_TOKEN__JWT__ENCODE__SECRET=string:CHANGE_ME
CKAN___API_TOKEN__JWT__DECODE__SECRET=string:WqVhDAXsF7E9FvzVnNqdsKpDT9wZbiu6hzxyMijv4FN3B CKAN___API_TOKEN__JWT__DECODE__SECRET=string:CHANGE_ME
CKAN___DATAPUSHER__API_TOKEN = api_token
CKAN_SYSADMIN_NAME=ckan_admin CKAN_SYSADMIN_NAME=ckan_admin
CKAN_SYSADMIN_PASSWORD=test1234 CKAN_SYSADMIN_PASSWORD=test1234
CKAN_SYSADMIN_EMAIL=your_email@example.com CKAN_SYSADMIN_EMAIL=your_email@example.com
@ -60,10 +58,10 @@ CKAN__RESOURCE_PROXY__MAX_FILE_SIZE=50048576
CKAN__RESOURCE_PROXY__CHUNK_SIZE=4096 CKAN__RESOURCE_PROXY__CHUNK_SIZE=4096
## Default timeout for fetching proxied items ## Default timeout for fetching proxied items
CKAN__RESOURCE_PROXY__TIMEOUT=10 CKAN__RESOURCE_PROXY__TIMEOUT=10
CKAN__VIEWS__DEFAULT_VIEWS="image_view text_view recline_view harvest ckan_harvester wmts_view geojson_view geo_view shp_view pdf_view" CKAN__VIEWS__DEFAULT_VIEWS="image_view text_view recline_view recline_grid_view recline_graph_view wmts_view webpage_view geojson_view geo_view shp_view pdf_view"
# Solr # Solr
SOLR_IMAGE_VERSION=2.10-spatial SOLR_IMAGE_VERSION=2.9-solr8-spatial
CKAN_SOLR_URL=http://solr:8983/solr/ckan CKAN_SOLR_URL=http://solr:8983/solr/ckan
TEST_CKAN_SOLR_URL=http://solr:8983/solr/ckan TEST_CKAN_SOLR_URL=http://solr:8983/solr/ckan
@ -72,7 +70,7 @@ REDIS_VERSION=6
CKAN_REDIS_URL=redis://redis:6379/1 CKAN_REDIS_URL=redis://redis:6379/1
TEST_CKAN_REDIS_URL=redis://redis:6379/1 TEST_CKAN_REDIS_URL=redis://redis:6379/1
# Datapusher # Datapusher - Deprecated by ckanext-xloader
DATAPUSHER_VERSION=0.0.19 DATAPUSHER_VERSION=0.0.19
CKAN_DATAPUSHER_URL=http://datapusher:8800 CKAN_DATAPUSHER_URL=http://datapusher:8800
CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://ckan:5000 CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://ckan:5000
@ -84,13 +82,17 @@ NGINX_PORT=80
NGINX_SSLPORT=443 NGINX_SSLPORT=443
# Extensions # Extensions
CKAN__PLUGINS="envvars stats text_view image_view recline_view resourcedictionary datastore datapusher harvest ckan_harvester spatial_metadata spatial_query spatial_harvest_metadata_api csw_harvester waf_harvester doc_harvester resource_proxy geo_view geojson_view wmts_view shp_view dcat dcat_rdf_harvester dcat_json_harvester dcat_json_interface scheming_datasets scheming_groups scheming_organizations pdf_view pages" CKAN__PLUGINS="envvars stats text_view image_view recline_view resourcedictionary datastore xloader harvest ckan_harvester spatial_metadata spatial_query spatial_harvest_metadata_api csw_harvester waf_harvester doc_harvester resource_proxy geo_view geojson_view wmts_view shp_view dcat dcat_rdf_harvester dcat_json_harvester dcat_json_interface scheming_datasets scheming_groups scheming_organizations pdf_view pages"
CKAN__HARVEST__MQ__TYPE=redis CKAN__HARVEST__MQ__TYPE=redis
CKAN__HARVEST__MQ__HOSTNAME=redis CKAN__HARVEST__MQ__HOSTNAME=redis
CKAN__HARVEST__MQ__PORT=6379 CKAN__HARVEST__MQ__PORT=6379
CKAN__HARVEST__MQ__REDIS_DB=1 CKAN__HARVEST__MQ__REDIS_DB=1
# ckanext-xloader
CKANEXT__XLOADER__API_TOKEN=api_token
CKANEXT__XLOADER__JOBS__DB_URI=postgresql://ckan:ckan@db/ckan
# ckanext-dcat # ckanext-dcat
CKANEXT__DCAT__BASE_URI=https://localhost:5000 CKANEXT__DCAT__BASE_URI=https://localhost:5000
CKANEXT__DCAT__RDF_PROFILES="euro_dcat_ap_2 euro_dcat_ap" CKANEXT__DCAT__RDF_PROFILES="euro_dcat_ap_2 euro_dcat_ap"
@ -114,6 +116,6 @@ CKANEXT__GEOVIEW__OL_VIEWER__FORMATS="wms wfs geojson gml kml"
CKANEXT__GEOVIEW__SHP_VIEWER__SRID=3857 CKANEXT__GEOVIEW__SHP_VIEWER__SRID=3857
CKANEXT__GEOVIEW__SHP_VIEWER__ENCODING=UTF-8 CKANEXT__GEOVIEW__SHP_VIEWER__ENCODING=UTF-8
#ckanext-sparql #TODO: ckanext-sparql - Not implemented yet
CKANEXT__SPARQL__ENDPOINT_URL=http://dbpedia.org/sparql CKANEXT__SPARQL__ENDPOINT_URL=http://dbpedia.org/sparql
CKANEXT__SPARQL__HIDE_ENDPOINT_URL=False CKANEXT__SPARQL__HIDE_ENDPOINT_URL=False