5461bebb62
Merge branch 'master' into 63-extend-spatial-harvesters
2014-05-13 17:54:44 +01:00
fa4161af87
[ #70 ] Fix deletion of harvested CSW records
...
The object id was pushed to the list returned by gather_stage before
being saved on the db, so None was added, causing an exception in the
Redis queue
2014-05-13 17:53:18 +01:00
b25a01029a
Merge branch 'master' into 63-extend-spatial-harvesters
...
Conflicts:
ckanext/spatial/harvesters/base.py
2014-05-13 15:33:04 +01:00
ab241d2530
Pass defer_commit in context on get_site_user calls
...
See ckan/ckan#1714 . Until that is fixed properly, the `defer_commit`
flag avoids some `DetachedInstanceErrors` happening during the
harvesting.
2014-05-13 15:30:34 +01:00
57b7e51e5a
Merge branch '69-reindex-dataset-no-object-change' into 63-extend-spatial-harvesters
2014-04-30 18:02:15 +01:00
e979d08e77
[ #69 ] Reindex dataset if harvest object did not change
...
We replace the old harvest object with the new one, and if we don't
reindex the reference to the old harvest object will remain in the
dataset dict
2014-04-30 18:01:42 +01:00
e9be08d8f1
Merge branch 'master' into 63-extend-spatial-harvesters
2014-04-28 18:41:38 +01:00
e19ed8100f
[ #62 ] Fix xpath for Alternate Title
2014-04-28 17:25:17 +01:00
211f3e4a95
[ #63 ] Tweak docs
2014-03-19 13:02:03 +00:00
f960733727
Merge branch 'master' into 63-extend-spatial-harvesters
2014-03-19 12:48:49 +00:00
0513e360e9
[ #63 ] Add previous_object check
...
In rare cases (eg if there was a previous error of two objects sharing
a guid) we can have a "changed" state and no previous_object
2014-03-19 12:46:01 +00:00
119c0fd40c
[ #63 ] Add user to delete context to avoid exception
2014-03-19 12:45:49 +00:00
bbe4125257
[ #65 ] Add individualName to ResponsibleParty
2014-03-19 11:59:08 +00:00
3b1b857efd
Merge branch 'master' into 63-extend-spatial-harvesters
2014-03-14 14:12:30 +00:00
26763218bc
[ #62 ] Fix spatial-resolution parsing
2014-03-14 14:12:05 +00:00
13f03878e2
Merge branch 'master' into 63-extend-spatial-harvesters
2014-03-14 12:36:36 +00:00
1fa72f4c16
[ #64 ] Fix validation exception when resource-type was missing
2014-03-14 12:36:01 +00:00
badd723259
[ #63 ] Add new ISpatialHarvest interface
...
Two extension points: ``get_package_dict`` and ``transform_to_iso``,
with the same expected behaviour as the old hooks meant to be overriden.
For ``get_package_dict`` we now pass, apart from the generated
package_dict, the parsed iso_values and the harvest object.
Updated docs and added autodocs.
2014-03-14 11:30:26 +00:00
60828520ba
Merge branch '55_harvest_cql_master' of https://github.com/geosolutions-it/ckanext-spatial into geosolutions-it-55_harvest_cql_master
2014-03-14 10:29:47 +00:00
4551efdddb
[ #62 ] Relax multiplicities
2014-03-14 10:19:57 +00:00
8b4d23a315
[ #62 ] Add new fields to the ISO parser
...
* Metadata standard name / version
* Unique resource identifier (reenabled and fixed)
* Presentation form
* Purpose
* Maintenance note
* Access constraints
* Distributor
* Usage
* Aggregation info
2014-03-11 17:05:08 +00:00
ee77800111
[ #62 ] Clean up keywords parsing
...
Keywords were extracted via 3 different elements
`keyword-inspire-theme`, `keyword-controlled-other` and
`keyword-free-text`. The latter didn't actually do anything and the
second duplicated xpaths from the first and added a non-standard one.
A new `keywords` key has been added which contains all keyword objects,
including type. This is not used to modify the `tags` key right now.
2014-03-11 14:09:55 +00:00
239b8313e9
[ #62 ] Add links to organization parties
2014-03-11 14:07:01 +00:00
93e07fe987
[ #61 ] Add media types for GeoJSON and GML
2014-03-07 16:26:50 +00:00
9116a6fd1f
[ #55 ] Allow CSW harvesters to define CQL filters (2nd try)
2014-03-02 23:12:54 +01:00
f544f41806
[ #57 ] Add default_tags and default_extras handling
2014-02-25 14:55:20 +01:00
e3c35e9927
Merge branch 'tomkralidis-master'
2014-01-24 18:15:52 +00:00
15461acff7
[ #52 ] Add class for link removed on 2.2
2014-01-09 17:49:18 +00:00
c352fbdadb
[ #52 ] Show dataset map correctly on all 2.x versions
2014-01-09 17:46:45 +00:00
3f91a4411b
Update csw.py
2013-12-26 18:18:15 -05:00
af695826e8
add functionality to set CSW keywords from CKAN tag counts
2013-11-18 16:32:23 -05:00
dac18b69b1
Revert accidental deletion from ffc5121c
2013-10-23 13:02:22 +01:00
ffc5121c6d
Fix csw client to properly use getrecords2
2013-10-16 13:58:14 +01:00
1fdf0cde4c
s/owslib.csw.getrecords/owslib.csw.getrecords2/g
2013-10-14 09:18:42 -04:00
45ebb5a13e
[ #37 ] Update attributions
2013-10-10 17:56:23 +01:00
2f28c8c5c7
[ #37 ] Add support for MapBox tiles and others
...
The common map function has been extended to allow configuring different
options for the base layer. There are default presets for MapQuest and
MapBox, but any XYZ layer can be added configuration options. The
options are passed to the templates via a helper function and to the js
modules via data-module attributes. Full docs included.
2013-10-06 23:22:04 +01:00
8a48ceb0e7
Merge branch 'master' of git://github.com/FuhuXia/ckanext-spatial into FuhuXia-master
2013-10-04 15:45:44 +01:00
a19010d8e5
for progress, use gemini_values not iso_values
2013-09-10 10:57:15 -04:00
f7f1a72df8
Remove progress field from tests
2013-09-10 11:08:06 +01:00
8ffc392f40
Merge branch 'master' of git://github.com/FuhuXia/ckanext-spatial into FuhuXia-master
2013-09-10 11:00:15 +01:00
c1fe37647f
change progress multiplicity to *
2013-09-09 14:47:14 -04:00
028d26e6e0
[ #37 ] Common function for initializing maps
...
This avoids duplication, mkaes all map consistent and makes easier
future support for other base maps.
2013-08-30 17:10:11 +01:00
b371aee381
[ #36 ] Fix spatial query widget for CKAN 2.1
2013-08-30 16:26:02 +01:00
a9414e755d
add progress into the iso values
2013-08-29 00:05:24 -04:00
3f784de737
Merge branch 'master' into 24-pycsw-integration-command
...
Conflicts:
README.rst
2013-08-20 14:35:53 +01:00
c6e29ee25f
[ #35 ] Ignore auth when using site_user
2013-08-14 12:23:00 +01:00
5f56877eba
Merge branch '29-new-preview-interface'
2013-08-08 17:52:05 +01:00
063a1b8530
[ #26 ] Fix broken markers on debug mode
2013-08-08 17:33:56 +01:00
19ea9b1753
[ #32 ] Do not set height to avoid scrollbars if container is less than 600px
2013-08-08 17:31:21 +01:00
28042d2b6f
[ #28 ] Fix function name
2013-07-29 17:25:04 +01:00
14048a153c
[ #31 ] Check if sort parameter exists
2013-07-29 17:23:21 +01:00
f0aa5076a0
[ #29 ] Fix missing variable
2013-07-19 23:04:49 +02:00
f4ea4da3e0
[ #29 ] Use new precedence feature for previews.
2013-07-19 12:35:35 +02:00
44e4ae380a
[ #24 ] Document CKAN-pycsw integration
...
Ad overview of how it works, how to set it up and deploy it in a
production server.
2013-06-21 13:21:30 +01:00
67b0941b3b
[ #24 ] Add clear command to empty the pycsw table
2013-06-20 16:45:37 +01:00
7b65813cba
[ #24 ] Add command for pycsw integration
...
Based on the work by @kindly on https://github.com/kindly/ckan_pycsw_import ,
adds a command to the extension that allows to export easily the spatial
datasets to pycsw. This will only work for harvested ISO datasets for the time
being.
The command can be run both via paster (in the context of a CKAN install) or
independently (eg a separate server), as it uses the API to get the datasets.
TODO docs
2013-06-20 16:44:56 +01:00
d8ed7c03cd
Merge branch '21-geojson-preview'
2013-06-14 17:56:26 +01:00
e30b662a36
[ #21 ] Make popup table match the rest of CKAN tables
2013-06-14 17:53:38 +01:00
f8dd4c02bd
[ #21 ] Fix i18n calls
...
All the 'self' on the 'showError' function should be 'this', otherwise you
will refer to the window. Also 'this.i18n' assumes an 'i18n' key in the
module options. Looks like you are trying to pass some variables to the
string template, note the sprintf-like syntax.
2013-06-14 17:39:58 +01:00
a4b8ddb7db
Merge branch 'master' of github.com:okfn/ckanext-spatial
2013-06-14 16:01:49 +01:00
9eec88b320
[ #15 ] Check CKAN version when using the Solr Search backend
2013-06-14 15:59:45 +01:00
fe8c469c12
[ #21 ] show object in popup, fix layout
2013-06-14 13:23:06 +02:00
6eb180c02a
[ #21 ] Add properties information to popup, lighter style
2013-06-13 23:24:01 +02:00
64d1846e90
[ #21 ] Add basic GeoJSON preview and fix set up of wms preview
2013-06-13 22:51:01 +02:00
add78d5931
allow csw to fetch differen output schema
2013-05-18 18:28:25 +01:00
27521221d6
[ #20 ] Flag datasets created via the spatial harvesters via a generic extra
2013-05-15 16:58:12 +01:00
8e81d1bd69
[ #19 ] Extract thumbnail from ISO documents
2013-05-15 16:41:36 +01:00
b4a7cf2289
[ #15 ] Reenable the Solr backend on master
...
It can be used against CKAN core master (eventually 2.1)
2013-05-14 14:34:10 +01:00
ce8747198f
Merge branch 'release-v2.0'
...
Conflicts:
README.rst
ckanext/spatial/commands/validation.py
ckanext/spatial/controllers/api.py
ckanext/spatial/harvesters/gemini.py
ckanext/spatial/plugin.py
ckanext/spatial/tests/lib/test_spatial.py
ckanext/spatial/tests/model/test_harvested_metadata.py
ckanext/spatial/tests/test_harvest.py
2013-05-14 14:02:28 +01:00
7919a19f7c
Revert "[ #15 ] Add suport for Solr spatial indexing and querying"
...
Reverting #15 as CKAN 2.0 does not include the necessary changes.
This reverts commit c7a9cc660f
2013-05-13 18:55:32 +01:00
83d903f84f
Revert "[ #15 ] Ensure that bounding boxes are defined counter-clockwise"
...
Reverting #15 as CKAN 2.0 does not include the necessary changes.
This reverts commit fede0b0831
2013-05-13 18:55:27 +01:00
5e35ac0fcb
Revert "[ #15 ] Read search backend config option once only"
...
Reverting #15 as CKAN 2.0 does not include the necessary changes.
This reverts commit d11693f076
2013-05-13 18:55:12 +01:00
17796c4ad0
Revert "[ #15 ] Add a new Solr search backend based on boost function"
...
Reverting #15 as CKAN 2.0 does not include the necessary changes.
This reverts commit bddfdf6fba
2013-05-13 18:53:09 +01:00
72127fa151
Revert "[ #15 ] Tweak doc to remove paper"
...
Reverting #15 as CKAN 2.0 does not include the necessary changes.
This reverts commit cf22017b7d
2013-05-13 18:51:44 +01:00
6f3c699311
[ #7 ] Remove old genshi spatial query widget
2013-05-13 17:13:19 +01:00
7e10eb9c16
[ #7 ] Allow to customize the default extent for the spatial query widget
2013-05-13 16:59:49 +01:00
af253d0f89
[ #7 ] Spatial search widget for 2.0
...
Adds a new map widget to the 2.0 search templates. It is shown initially
in the sidebar but it is expanded when the user needs to draw an area.
It uses Leaflet and Leaflet.draw.
2013-05-13 16:14:48 +01:00
45f4f4da57
[ #16 ] Ignore time zones when parsing harvest object modified date
...
Otherwise you get this exception when the date on the document has time
zone information, as dates as stored without it on the database:
TypeError: can't compare offset-naive and offset-aware datetimes
2013-04-25 17:13:03 +01:00
a4c86d26e3
[ #15 ] Better checks following 27d4d5
2013-04-12 16:44:52 +01:00
27d4d5b2c9
make spatial field optional
2013-04-12 16:39:38 +01:00
51a2b20501
Merge branch '15-solr-based-spatial-search' into release-v2.0
2013-04-12 10:56:21 +01:00
822ddbb1b5
[ #8 ] Don't add object id to error so it can be aggregated
2013-04-12 10:54:02 +01:00
cf22017b7d
[ #15 ] Tweak doc to remove paper
2013-04-12 10:06:50 +01:00
bddfdf6fba
[ #15 ] Add a new Solr search backend based on boost function
...
This backend does not use the spatial field on Solr, but indexes the
area and coordinates of the dataset's extent. On query time, a boost
function is used to calculate a ratio between the query area and the
dataset extent, which will give a value of 0 for areas that don't
overlap and 1 for a perfect fit. The same function is used both for
sorting and filtering.
2013-04-11 19:29:41 +01:00
d11693f076
[ #15 ] Read search backend config option once only
2013-04-11 17:13:53 +01:00
970dfd1b68
Merge branch 'release-v2.0' into 15-solr-based-spatial-search
2013-04-11 12:51:08 +01:00
d1594b3790
do not use kwarg for unicode errors
2013-04-09 12:06:38 +01:00
65e056d519
Merge branch 'release-v2.0' into 15-solr-based-spatial-search
2013-04-03 12:14:20 +01:00
ff25ff2f2b
[ #8 ] Abort import stage if get_package_dict returns nothing
2013-04-02 18:40:03 +01:00
9760546d13
[ #8 ] Add extra xpath for getting resources
...
According to the ISO spec gmd:MD_DigitalTransferOptions can be accessed
via a gmd:MD_Distributer tag
2013-03-27 17:39:51 +00:00
0c98e6ec4c
[ #8 ] Minor fix in single doc harvester
2013-03-27 17:38:42 +00:00
fede0b0831
[ #15 ] Ensure that bounding boxes are defined counter-clockwise
...
To return correct results on a spatial query, rectangle geometries must
be defined in counter-clockwise order [1]. This changeset adds a small
sanity check to before_index when we are dealing with a Polygon geometry
that has 5 coordinate pairs. Shapely is used to generate a LinearRing
from the polygon coordinates and check if they are ccw. If not, they are
reordered and a new polygon is generated so the WKT sent to Solr is
properly ordered.
The GeoJSON template used for extents in the base spatial harvester has
been also updated to define the coordinates counter-clockwise.
[1]
http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4#JTS_.2BAC8_WKT_.2BAC8_Polygon_notes
2013-03-23 19:28:31 +00:00
40967385b0
[ #8 ] Fix typo in WMS format detection
2013-03-21 17:53:38 +00:00
0e0b5a2cc2
[ #8 ] Fix bug that prevented setting a default resource name
2013-03-21 14:34:42 +00:00
627c4c58e0
[ #8 ] Fix bug that prevented setting a default resource name
2013-03-21 14:34:23 +00:00
c7a9cc660f
[ #15 ] Add suport for Solr spatial indexing and querying
...
When the 'ckanext.spatial.search_backend' config option is set to
'solr', the extension will index geometries stored in the 'spatial'
extra on the spatial field of the Solr index (named 'spatial_geom').
This is done on the 'before_index' extension point.
Also, when doing a query, if the same config option is in place, the
necessary fq parameter will be set to pass the spatial query to Solr.
2013-03-20 16:54:55 +00:00
a727aa815b
Add helper functions useful to format extras coming from the spatial harvesters
2013-03-18 15:59:20 +00:00
a7fc19768b
[ #8 ] Remove print commands from WAF harvester
2013-03-14 17:45:34 +00:00
eb201e1759
[ #8 ] Waf harvester: improve exception and return empty list if no records
2013-03-14 17:35:51 +00:00
0aafffc8dc
[ #8 ] Capture exceptions during request in WAF harvester
2013-03-14 14:56:42 +00:00
amercader
etj
Tom Kralidis
fxia
Dominik Moritz
kindly
Ian Ward