Compare commits

...

50 Commits

Author SHA1 Message Date
Mario de Frutos
557f67c51f Merge pull request #96 from CartoDB/block_google_users_to_use_isolines
Block google services users to use isolines functions
2016-02-26 10:48:20 +01:00
Mario de Frutos
d27b2a2b78 Block google services users to use isolines functions
Due some service restrictions, google services users couldn't use
the HERE services which includes the isolines functions.
2016-02-26 10:42:05 +01:00
Carla
926655fdf9 Update README.md 2016-02-25 14:30:19 +01:00
Mario de Frutos
599f58fce0 Update README.md 2016-02-24 12:13:57 +01:00
Mario de Frutos
06a850efb0 Merge pull request #95 from CartoDB/ethervoid-patch-1
README updates
2016-02-24 12:02:41 +01:00
Mario de Frutos
a69f767417 README updates 2016-02-24 12:02:28 +01:00
Mario de Frutos
b0f4d5db88 Merge pull request #94 from CartoDB/update_readme
add some local steps
2016-02-24 12:00:33 +01:00
javi
0ab255eb93 missing confi 2016-02-24 11:58:46 +01:00
javi
6c9d59d7c0 add some local steps 2016-02-24 11:54:55 +01:00
Mario de Frutos
826a3e65c0 Added isolines integration tests 2016-02-24 09:54:00 +01:00
Mario de Frutos
448f47589a Renamed from routing to isolines helper 2016-02-22 10:28:19 +01:00
Mario de Frutos
c4d682a5a5 Merge pull request #89 from CartoDB/routing_quota_check
Quota check for the routing feature
2016-02-22 10:26:18 +01:00
Mario de Frutos
e764b9036d Quota check for the routing feature
We have refactor all the quota logic and extracted to a new QuotaChecker
class in order to have it ready to create a factory when needed

Added the logic for the routing quota check
2016-02-18 15:40:55 +01:00
Carla
c8e2c7b0b3 Merge pull request #88 from CartoDB/document_new_service
Document how to add a new service
2016-02-15 18:57:25 +01:00
Carla
41681a7ef1 Fix syntax error 2016-02-15 14:21:43 +01:00
Carla
6a37e9a550 Update internal_doc.md 2016-02-15 12:55:25 +01:00
Carla Iriberri
634ae4b96f Add project doc 2016-02-15 12:48:14 +01:00
Carla
983440086d remove routing 2016-02-12 14:53:48 +01:00
Mario de Frutos
32c146336d Change the url to production in here routing functions 2016-02-12 14:52:00 +01:00
Mario de Frutos
93f003a672 Fixed error in the imports for the version 0.3.0 2016-02-12 12:55:12 +01:00
Mario de Frutos
a90f8cac1c Changed the source point for isolines functions 2016-02-12 12:45:12 +01:00
Mario de Frutos
1cf766faaa Update reference.md 2016-02-12 11:49:28 +01:00
Mario de Frutos
fbc5a1bbf8 Changed the example queries to return all the columns 2016-02-12 11:48:56 +01:00
Mario de Frutos
f44ab1fc4c Merge pull request #85 from CartoDB/routing_isolines_functions
Routing isolines functions
2016-02-12 11:25:38 +01:00
Mario de Frutos
2896cbb287 Fixed more typos in the reference docs 2016-02-12 11:22:29 +01:00
Mario de Frutos
bde183e9c1 Fixed some typos in the reference docs 2016-02-12 11:16:35 +01:00
Mario de Frutos
3889904afe Added isolines functions to the reference doc 2016-02-12 11:13:04 +01:00
Mario de Frutos
3a4130282e Limit the mode types accepted 2016-02-12 10:01:42 +01:00
Mario de Frutos
d9ca7911e4 Options parameter in the isolines functions has an empty array by defaul 2016-02-11 18:23:23 +01:00
Mario de Frutos
8d18e50cc5 Fixed error in the return part of the isochrone function 2016-02-11 11:35:47 +01:00
Mario de Frutos
0ca20196c0 Client with isodistance and isochrone feature working 2016-02-11 11:10:12 +01:00
Mario de Frutos
f36a345db2 Updated python library version with routing part 2016-02-10 22:26:53 +01:00
Mario de Frutos
5c1b320555 Routing server side working properly 2016-02-10 22:26:32 +01:00
Carla Iriberri
d0f7ac4d49 Fix conflict 2016-02-10 20:00:48 +01:00
Carla Iriberri
d4a2184ce0 Multiple fixes 2016-02-10 19:59:00 +01:00
Carla
42d0812383 Fix indents 2016-02-10 17:58:48 +01:00
Carla Iriberri
e996d3796a Comment quota code 2016-02-10 17:42:11 +01:00
Carla Iriberri
b97ce043c3 Avoid conflict in gitignore file 2016-02-10 17:19:16 +01:00
Carla Iriberri
bd3f6678c1 Upload server functions for routing 2016-02-10 17:17:53 +01:00
Mario de Frutos
54622d5c81 Added the routing config and metrics 2016-02-10 17:01:41 +01:00
Mario de Frutos
75a915019f Helper to transform to Postgis multipolygon 2016-02-10 16:01:03 +01:00
Mario de Frutos
13cfee2d4c Fixed tests for the heremaps geocoder 2016-02-10 15:24:25 +01:00
Mario de Frutos
3b0f92d5e7 Routing isolines python library 2016-02-10 15:11:45 +01:00
Carla
9029e38216 Add 0.2.0. in gitignore 2016-02-09 16:09:41 +01:00
Carla Iriberri
b8c7015994 Add files for client side routing
This commit adds the new function interfaces and contains the frozen 2.0
version file as well as the upgrade/downgrade files for the previous
version according to the new function additions.
2016-02-09 15:56:50 +01:00
Mario de Frutos
05cd8f5c5c Dataservices client 0.2.0 scaffold 2016-02-09 10:51:19 +01:00
Mario de Frutos
910381e4a6 Dataservices server 0.4.0 scaffold 2016-02-09 10:51:12 +01:00
Mario de Frutos
3f6381ba78 Merge pull request #83 from CartoDB/readme_patch_1
Change from geocoder to dataservices api
2016-02-09 10:24:00 +01:00
Mario de Frutos
48e00329f3 Change from geocoder to dataservices api 2016-02-09 10:23:25 +01:00
Mario de Frutos
7544b3c4ab Freeze 0.3.0 server file 2016-02-09 10:20:38 +01:00
93 changed files with 5018 additions and 132 deletions

View File

@@ -1,21 +1,77 @@
# geocoder-api
The CartoDB Geocoder SQL API (server and client FTM)
# Data Services API
The CartoDB Data Services SQL API (server and client FTM)
### Deploy instructions
Steps to deploy a new Geocoder API version :
Steps to deploy a new Data Services API version :
- Deploy new version of geocoder API to all servers
- Update the server user using: ALTER EXTENSION cdb_geocoder_server UPDATE TO '<CURRENT_VERSION>';
- Deploy new version of dataservices API to all servers
- Update the server user using: ALTER EXTENSION cdb_dataservices_server UPDATE TO '<CURRENT_VERSION>';
- Update the python dependencies if needed: **cartodb_geocoder** and **heremaps**
- Add the needed config in the `cdb_conf` table:
- `redis_metadata_config` and `redis_metrics_conf`
- `{"sentinel_host": "localhost", "sentinel_port": 26739, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}`
- `{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}`
- `heremaps_conf`
- `{"app_id": "APP_ID", "app_code": "APP_CODE"}`
- Deploy the client to all the servers with the new version
- Deploy the editor with the new geocoder api version changed (https://github.com/CartoDB/cartodb/blob/master/app/models/user/db_service.rb#L18)
- Deploy the editor with the new dataservices api version changed (https://github.com/CartoDB/cartodb/blob/master/app/models/user/db_service.rb#L18)
- Execute the rails task to update first the CartoDB team organizaton to test in production
- `RAILS_ENV=production bundle exec rake cartodb:db:configure_geocoder_extension_for_organizations['team']`
- Check if all works perfectly for our team. If so, execute the rake tasks to update all the users and organizations:
- `RAILS_ENV=production bundle exec rake cartodb:db:configure_geocoder_extension_for_organizations['', true]`
- `RAILS_ENV=production bundle exec rake cartodb:db:configure_geocoder_extension_for_non_org_users['', true]`
- Freeze the generated SQL file for the version. Eg. cdb_dataservices_server--0.0.1.sql
### Local install instructions
- install data services extension
```
git clone git@github.com:CartoDB/data-services.git
data-services/geocoder/extension
sudo make install
```
- install server and client extensions
```
cd client && sudo make install
cd server && sudo make install
```
- install python library
```
cd server/lib/python/cartodb_services && python setup.py install
```
- install extensions in user database
```
create extension cdb_geocoder;
create extension plproxy;
create extension cdb_dataservices_server;
create extension cdb_dataservices_client;
```
- add configuration for different services in user database
```
# select CDB_Conf_SetConf('redis_metadata_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}');
# select CDB_Conf_SetConf('redis_metrics_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}');
# select CDB_Conf_SetConf('heremaps_conf', '{"app_id": "APP_ID", "app_code": "APP_CODE"}');
# select CDB_Conf_SetConf('user_config', '{"is_organization": false, "entity_name": "<YOUR_USERNAME>"}')
```
- configure plproxy to point to the same user database (you could do in a different one)
```
select CDB_Conf_SetConf('geocoder_server_config', '{ "connection_str": "host=localhost port=5432 dbname=cartodb_dev_user_accf0647-d942-4e37-b129-8287c117e687_db user=postgres"}');
```
- configure the search path in order to be able to execute the functions without use the schema:
```
alter role "<USER_ROLE>" set search_path='"$user", public, cartodb, cdb_dataservices_client';
```

1
client/.gitignore vendored
View File

@@ -6,3 +6,4 @@ regression.out
90_grant_execute.sql
cdb_geocoder_client--0.0.1.sql
cdb_geocoder_client--0.1.0.sql
cdb_geocoder_client--0.2.0.sql

View File

@@ -12,8 +12,11 @@ NEW_EXTENSION_ARTIFACT = $(EXTENSION)--$(EXTVERSION).sql
# @see http://www.postgresql.org/docs/current/static/extend-pgxs.html
DATA = $(NEW_EXTENSION_ARTIFACT) \
cdb_dataservices_client--0.0.1.sql \
cdb_dataservices_client--0.1.0.sql \
cdb_dataservices_client--0.1.0--0.0.1.sql \
cdb_dataservices_client--0.0.1--0.1.0.sql
cdb_dataservices_client--0.0.1--0.1.0.sql \
cdb_dataservices_client--0.2.0--0.1.0.sql \
cdb_dataservices_client--0.1.0--0.2.0.sql
REGRESS = $(notdir $(basename $(wildcard test/$(EXTVERSION)/sql/*test.sql)))

View File

@@ -0,0 +1,60 @@
CREATE TYPE cdb_dataservices_client.isoline AS (
center geometry(Geometry,4326),
data_range integer,
the_geom geometry(Multipolygon,4326)
);
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_isodistance (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
DECLARE
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
RETURN QUERY
SELECT * FROM cdb_dataservices_client._cdb_isodistance(username, orgname, source, mode, range, options);
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_isodistance (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT * FROM cdb_dataservices_server.cdb_isodistance (username, organization_name, source, mode, range, options);
$$ LANGUAGE plproxy;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_isodistance(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser;
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_isochrone (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
DECLARE
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
RETURN QUERY
SELECT * FROM cdb_dataservices_client._cdb_isochrone(username, orgname, source, mode, range, options);
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_isochrone (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT * FROM cdb_dataservices_server.cdb_isochrone (username, organization_name, source, mode, range, options);
$$ LANGUAGE plproxy;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_isochrone(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser;

View File

@@ -0,0 +1,7 @@
DROP FUNCTION IF EXISTS cdb_dataservices_client.cdb_isochrone (geometry(Geometry, 4326), text, integer[], text[]);
DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_isochrone (text, text, geometry(Geometry, 4326), text, integer[], text[]);
DROP FUNCTION IF EXISTS cdb_dataservices_client.cdb_isodistance (geometry(Geometry, 4326), text, integer[], text[]);
DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_isodistance (text, text, geometry(Geometry, 4326), text, integer[], text[]);
DROP TYPE IF EXISTS cdb_dataservices_client.isoline;

View File

@@ -0,0 +1,513 @@
--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION cdb_dataservices_client" to load this file. \quit
--
-- Geocoder server connection config
--
-- The purpose of this function is provide to the PL/Proxy functions
-- the connection string needed to connect with the server
CREATE OR REPLACE FUNCTION cdb_dataservices_client._server_conn_str()
RETURNS text AS $$
DECLARE
db_connection_str text;
BEGIN
SELECT cartodb.cdb_conf_getconf('geocoder_server_config')->'connection_str' INTO db_connection_str;
SELECT trim(both '"' FROM db_connection_str) INTO db_connection_str;
RETURN db_connection_str;
END;
$$ LANGUAGE 'plpgsql';
CREATE TYPE cdb_dataservices_client._entity_config AS (
username text,
organization_name text
);
--
-- Get entity config function
--
-- The purpose of this function is to retrieve the username and organization name from
-- a) schema where he/her is the owner in case is an organization user
-- b) entity_name from the cdb_conf database in case is a non organization user
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_entity_config()
RETURNS record AS $$
DECLARE
result cdb_dataservices_client._entity_config;
is_organization boolean;
username text;
organization_name text;
BEGIN
SELECT cartodb.cdb_conf_getconf('user_config')->'is_organization' INTO is_organization;
IF is_organization IS NULL THEN
RAISE EXCEPTION 'User must have user configuration in the config table';
ELSIF is_organization = TRUE THEN
SELECT nspname
FROM pg_namespace s
LEFT JOIN pg_roles r ON s.nspowner = r.oid
WHERE r.rolname = session_user INTO username;
SELECT cartodb.cdb_conf_getconf('user_config')->>'entity_name' INTO organization_name;
ELSE
SELECT cartodb.cdb_conf_getconf('user_config')->>'entity_name' INTO username;
organization_name = NULL;
END IF;
result.username = username;
result.organization_name = organization_name;
RETURN result;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
CREATE TYPE cdb_dataservices_client.isoline AS (
center geometry(Geometry,4326),
data_range integer,
the_geom geometry(Multipolygon,4326)
);--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_admin0_polygon (country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_admin0_polygon(username, orgname, country_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon (admin1_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon (admin1_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name, country_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point (city_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point (city_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, country_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point (city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, admin1_name, country_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_polygon (postal_code text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_postalcode_polygon(username, orgname, postal_code, country_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_point (postal_code text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_postalcode_point(username, orgname, postal_code, country_name) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_ipaddress_point (ip_address text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_ipaddress_point(username, orgname, ip_address) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_street_point (searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country) INTO ret;
RETURN ret;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_isodistance (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
DECLARE
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
RETURN QUERY
SELECT * FROM cdb_dataservices_client._cdb_isodistance(username, orgname, source, mode, range, options);
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
--
-- Public geocoder API function
--
-- These are the only ones with permissions to publicuser role
-- and should also be the only ones with SECURITY DEFINER
CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_isochrone (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
DECLARE
username text;
orgname text;
BEGIN
IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN
RAISE EXCEPTION 'The api_key must be provided';
END IF;
SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text);
-- JSON value stored "" is taken as literal
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
RETURN QUERY
SELECT * FROM cdb_dataservices_client._cdb_isochrone(username, orgname, source, mode, range, options);
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_admin0_polygon (username text, organization_name text, country_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_admin0_polygon (username, organization_name, country_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_admin1_polygon (username text, organization_name text, admin1_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon (username, organization_name, admin1_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_admin1_polygon (username text, organization_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon (username, organization_name, admin1_name, country_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_namedplace_point (username text, organization_name text, city_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point (username, organization_name, city_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_namedplace_point (username text, organization_name text, city_name text, country_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point (username, organization_name, city_name, country_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_namedplace_point (username text, organization_name text, city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point (username, organization_name, city_name, admin1_name, country_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_postalcode_polygon (username text, organization_name text, postal_code text, country_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_postalcode_polygon (username, organization_name, postal_code, country_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_postalcode_point (username text, organization_name text, postal_code text, country_name text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_postalcode_point (username, organization_name, postal_code, country_name);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_ipaddress_point (username text, organization_name text, ip_address text)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_ipaddress_point (username, organization_name, ip_address);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_street_point (username text, organization_name text, searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL)
RETURNS Geometry AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT cdb_dataservices_server.cdb_geocode_street_point (username, organization_name, searchtext, city, state_province, country);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_isodistance (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT * FROM cdb_dataservices_server.cdb_isodistance (username, organization_name, source, mode, range, options);
$$ LANGUAGE plproxy;
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_isochrone (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[])
RETURNS SETOF cdb_dataservices_client.isoline AS $$
CONNECT cdb_dataservices_client._server_conn_str();
SELECT * FROM cdb_dataservices_server.cdb_isochrone (username, organization_name, source, mode, range, options);
$$ LANGUAGE plproxy;
-- Make sure by default there are no permissions for publicuser
-- NOTE: this happens at extension creation time, as part of an implicit transaction.
REVOKE ALL PRIVILEGES ON SCHEMA cdb_dataservices_client FROM PUBLIC, publicuser CASCADE;
-- Grant permissions on the schema to publicuser (but just the schema)
GRANT USAGE ON SCHEMA cdb_dataservices_client TO publicuser;
-- Revoke execute permissions on all functions in the schema by default
REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_client FROM PUBLIC, publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_admin0_polygon(country_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon(admin1_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon(admin1_name text, country_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point(city_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point(city_name text, country_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_polygon(postal_code text, country_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_point(postal_code text, country_name text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_ipaddress_point(ip_address text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_isodistance(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser;
GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_isochrone(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser;

View File

@@ -1,5 +1,5 @@
comment = 'CartoDB dataservices client API extension'
default_version = '0.1.0'
default_version = '0.2.0'
requires = 'plproxy, cartodb'
superuser = true
schema = cdb_dataservices_client

View File

@@ -0,0 +1,79 @@
---
- name: cdb_geocode_admin0_polygon
return_type: Geometry
params:
- { name: country_name, type: text }
- name: cdb_geocode_admin1_polygon
return_type: Geometry
params:
- { name: admin1_name, type: text }
- name: cdb_geocode_admin1_polygon
return_type: Geometry
params:
- { name: admin1_name, type: text }
- { name: country_name, type: text }
- name: cdb_geocode_namedplace_point
return_type: Geometry
params:
- { name: city_name, type: text}
- name: cdb_geocode_namedplace_point
return_type: Geometry
params:
- { name: city_name, type: text}
- { name: country_name, type: text}
- name: cdb_geocode_namedplace_point
return_type: Geometry
params:
- { name: city_name, type: text}
- { name: admin1_name, type: text}
- { name: country_name, type: text}
- name: cdb_geocode_postalcode_polygon
return_type: Geometry
params:
- { name: postal_code, type: text}
- { name: country_name, type: text}
- name: cdb_geocode_postalcode_point
return_type: Geometry
params:
- { name: postal_code, type: text}
- { name: country_name, type: text}
- name: cdb_geocode_ipaddress_point
return_type: Geometry
params:
- { name: ip_address, type: text}
- name: cdb_geocode_street_point
return_type: Geometry
params:
- { name: searchtext, type: text}
- { name: city, type: text, default: 'NULL'}
- { name: state_province, type: text, default: 'NULL'}
- { name: country, type: text, default: 'NULL'}
- name: cdb_isodistance
return_type: SETOF cdb_dataservices_client.isoline
multi_row: true
params:
- { name: source, type: "geometry(Geometry, 4326)" }
- { name: mode, type: text }
- { name: range, type: "integer[]" }
- { name: options, type: "text[]", default: 'ARRAY[]::text[]' }
- name: cdb_isochrone
return_type: SETOF cdb_dataservices_client.isoline
multi_row: true
params:
- { name: source, type: "geometry(Geometry, 4326)" }
- { name: mode, type: text }
- { name: range, type: "integer[]" }
- { name: options, type: "text[]", default: 'ARRAY[]::text[]' }

View File

@@ -28,6 +28,10 @@ class SqlTemplateRenderer
@function_signature['return_type']
end
def multi_row
@function_signature['multi_row']
end
def user_config_key
@function_signature['user_config_key']
end

View File

@@ -7,7 +7,7 @@
CREATE OR REPLACE FUNCTION <%= DATASERVICES_CLIENT_SCHEMA %>.<%= name %> (<%= params_with_type_and_default %>)
RETURNS <%= return_type %> AS $$
DECLARE
ret <%= return_type %>;
<% if not multi_row %>ret <%= return_type %>;<% end %>
username text;
orgname text;
BEGIN
@@ -19,8 +19,13 @@ BEGIN
IF username IS NULL OR username = '' OR username = '""' THEN
RAISE EXCEPTION 'Username is a mandatory argument, check it out';
END IF;
SELECT <%= DATASERVICES_CLIENT_SCHEMA %>._<%= name %>(username, orgname, <%= params %>) INTO ret;
RETURN ret;
<% if multi_row %>
RETURN QUERY
SELECT * FROM <%= DATASERVICES_CLIENT_SCHEMA %>._<%= name %>(username, orgname, <%= params %>);
<% else %>
SELECT <%= DATASERVICES_CLIENT_SCHEMA %>._<%= name %>(username, orgname, <%= params %>) INTO ret;
RETURN ret;
<% end %>
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;

View File

@@ -1,6 +1,10 @@
CREATE OR REPLACE FUNCTION <%= DATASERVICES_CLIENT_SCHEMA %>._<%= name %> (username text, organization_name text, <%= params_with_type_and_default %>)
RETURNS <%= return_type %> AS $$
CONNECT <%= DATASERVICES_CLIENT_SCHEMA %>._server_conn_str();
<% if multi_row %>
SELECT * FROM <%= DATASERVICES_SERVER_SCHEMA %>.<%= name %> (username, organization_name, <%= params %>);
<% else %>
SELECT <%= DATASERVICES_SERVER_SCHEMA %>.<%= name %> (username, organization_name, <%= params %>);
<% end %>
$$ LANGUAGE plproxy;

View File

@@ -0,0 +1,3 @@
--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION cdb_dataservices_client" to load this file. \quit

View File

@@ -0,0 +1,16 @@
--
-- Geocoder server connection config
--
-- The purpose of this function is provide to the PL/Proxy functions
-- the connection string needed to connect with the server
CREATE OR REPLACE FUNCTION cdb_dataservices_client._server_conn_str()
RETURNS text AS $$
DECLARE
db_connection_str text;
BEGIN
SELECT cartodb.cdb_conf_getconf('geocoder_server_config')->'connection_str' INTO db_connection_str;
SELECT trim(both '"' FROM db_connection_str) INTO db_connection_str;
RETURN db_connection_str;
END;
$$ LANGUAGE 'plpgsql';

View File

@@ -0,0 +1,37 @@
CREATE TYPE cdb_dataservices_client._entity_config AS (
username text,
organization_name text
);
--
-- Get entity config function
--
-- The purpose of this function is to retrieve the username and organization name from
-- a) schema where he/her is the owner in case is an organization user
-- b) entity_name from the cdb_conf database in case is a non organization user
CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_entity_config()
RETURNS record AS $$
DECLARE
result cdb_dataservices_client._entity_config;
is_organization boolean;
username text;
organization_name text;
BEGIN
SELECT cartodb.cdb_conf_getconf('user_config')->'is_organization' INTO is_organization;
IF is_organization IS NULL THEN
RAISE EXCEPTION 'User must have user configuration in the config table';
ELSIF is_organization = TRUE THEN
SELECT nspname
FROM pg_namespace s
LEFT JOIN pg_roles r ON s.nspowner = r.oid
WHERE r.rolname = session_user INTO username;
SELECT cartodb.cdb_conf_getconf('user_config')->>'entity_name' INTO organization_name;
ELSE
SELECT cartodb.cdb_conf_getconf('user_config')->>'entity_name' INTO username;
organization_name = NULL;
END IF;
result.username = username;
result.organization_name = organization_name;
RETURN result;
END;
$$ LANGUAGE 'plpgsql' SECURITY DEFINER;

View File

@@ -0,0 +1,5 @@
CREATE TYPE cdb_dataservices_client.isoline AS (
center geometry(Geometry,4326),
data_range integer,
the_geom geometry(Multipolygon,4326)
);

View File

@@ -0,0 +1,9 @@
-- Make sure by default there are no permissions for publicuser
-- NOTE: this happens at extension creation time, as part of an implicit transaction.
REVOKE ALL PRIVILEGES ON SCHEMA cdb_dataservices_client FROM PUBLIC, publicuser CASCADE;
-- Grant permissions on the schema to publicuser (but just the schema)
GRANT USAGE ON SCHEMA cdb_dataservices_client TO publicuser;
-- Revoke execute permissions on all functions in the schema by default
REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_client FROM PUBLIC, publicuser;

View File

@@ -0,0 +1,29 @@
-- Install dependencies
CREATE EXTENSION postgis;
CREATE EXTENSION schema_triggers;
CREATE EXTENSION plpythonu;
CREATE EXTENSION cartodb;
CREATE EXTENSION plproxy;
-- Install the extension
CREATE EXTENSION cdb_dataservices_client;
-- Mock the server connection to point to this very test db
SELECT cartodb.cdb_conf_setconf('geocoder_server_config', '{"connection_str": "dbname=contrib_regression host=127.0.0.1 user=postgres"}');
cdb_conf_setconf
------------------
(1 row)
-- Mock the user configuration
SELECT cartodb.cdb_conf_setconf('user_config', '{"is_organization": false, "entity_name": "test_user"}');
cdb_conf_setconf
------------------
(1 row)
-- Mock the server schema
CREATE SCHEMA cdb_dataservices_server;
-- Create a test user to check permissions
DROP ROLE IF EXISTS test_regular_user;
CREATE ROLE test_regular_user;
GRANT publicuser TO test_regular_user;
ALTER ROLE test_regular_user SET search_path TO public,cartodb,cdb_dataservices_client;

View File

@@ -0,0 +1,20 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server function
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin0_polygon(username text, orgname text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_admin0_polygon invoked with params (%, %, %)', username, orgname, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_admin0_polygon('Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_admin0_polygon(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_admin0_polygon invoked with params (test_user, <NULL>, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_admin0_polygon(username, orgname, country_name)"
PL/pgSQL function cdb_geocode_admin0_polygon(text) line 16 at SQL statement
cdb_geocode_admin0_polygon
----------------------------
(1 row)

View File

@@ -0,0 +1,36 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (%, %, %)', username, orgname, admin1_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (%, %, %, %)', username, orgname, admin1_name, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_admin1_polygon('California');
NOTICE: cdb_dataservices_client._cdb_geocode_admin1_polygon(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (test_user, <NULL>, California)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name)"
PL/pgSQL function cdb_geocode_admin1_polygon(text) line 16 at SQL statement
cdb_geocode_admin1_polygon
----------------------------
(1 row)
SELECT cdb_geocode_admin1_polygon('California', 'United States');
NOTICE: cdb_dataservices_client._cdb_geocode_admin1_polygon(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (test_user, <NULL>, California, United States)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name, country_name)"
PL/pgSQL function cdb_geocode_admin1_polygon(text,text) line 16 at SQL statement
cdb_geocode_admin1_polygon
----------------------------
(1 row)

View File

@@ -0,0 +1,52 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (%, %, %)', username, orgname, city_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (%, %, %, %)', username, orgname, city_name, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (%, %, %, %, %)', username, orgname, city_name, admin1_name, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_namedplace_point('Elx');
NOTICE: cdb_dataservices_client._cdb_geocode_namedplace_point(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (test_user, <NULL>, Elx)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name)"
PL/pgSQL function cdb_geocode_namedplace_point(text) line 16 at SQL statement
cdb_geocode_namedplace_point
------------------------------
(1 row)
SELECT cdb_geocode_namedplace_point('Elx', 'Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_namedplace_point(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (test_user, <NULL>, Elx, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, country_name)"
PL/pgSQL function cdb_geocode_namedplace_point(text,text) line 16 at SQL statement
cdb_geocode_namedplace_point
------------------------------
(1 row)
SELECT cdb_geocode_namedplace_point('Elx', 'Valencia', 'Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_namedplace_point(5): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (test_user, <NULL>, Elx, Valencia, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, admin1_name, country_name)"
PL/pgSQL function cdb_geocode_namedplace_point(text,text,text) line 16 at SQL statement
cdb_geocode_namedplace_point
------------------------------
(1 row)

View File

@@ -0,0 +1,36 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, postal_code text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_postalcode_polygon invoked with params (%, %, %, %)', username, orgname, postal_code, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, postal_code text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_postalcode_point invoked with params (%, %, %, %)', username, orgname, postal_code, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_postalcode_polygon('03204', 'Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_postalcode_polygon(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_postalcode_polygon invoked with params (test_user, <NULL>, 03204, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_postalcode_polygon(username, orgname, postal_code, country_name)"
PL/pgSQL function cdb_geocode_postalcode_polygon(text,text) line 16 at SQL statement
cdb_geocode_postalcode_polygon
--------------------------------
(1 row)
SELECT cdb_geocode_postalcode_point('03204', 'Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_postalcode_point(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_postalcode_point invoked with params (test_user, <NULL>, 03204, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_postalcode_point(username, orgname, postal_code, country_name)"
PL/pgSQL function cdb_geocode_postalcode_point(text,text) line 16 at SQL statement
cdb_geocode_postalcode_point
------------------------------
(1 row)

View File

@@ -0,0 +1,20 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_ipaddress_point(username text, orgname text, ip_address text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_ipaddress_point invoked with params (%, %, %)', username, orgname, ip_address;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_ipaddress_point('8.8.8.8');
NOTICE: cdb_dataservices_client._cdb_geocode_ipaddress_point(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_ipaddress_point invoked with params (test_user, <NULL>, 8.8.8.8)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_ipaddress_point(username, orgname, ip_address)"
PL/pgSQL function cdb_geocode_ipaddress_point(text) line 16 at SQL statement
cdb_geocode_ipaddress_point
-----------------------------
(1 row)

View File

@@ -0,0 +1,101 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_street_point (username text, orgname text, searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (%, %, %, %, %, %)', username, orgname, searchtext, city, state_province, country;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_street_point('One street, 1');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, 1, <NULL>, <NULL>, <NULL>)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, <NULL>, <NULL>)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city', 'state');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, state, <NULL>)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city', 'state', 'country');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, state, country)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city', NULL, 'country');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, <NULL>, country)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street, 1');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, 1, <NULL>, <NULL>, <NULL>)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, <NULL>, <NULL>)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city', 'state');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, state, <NULL>)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city', 'state', 'country');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, state, country)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
SELECT cdb_geocode_street_point('One street', 'city', NULL, 'country');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, One street, city, <NULL>, country)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)

View File

@@ -0,0 +1,117 @@
-- Use regular user role
SET ROLE test_regular_user;
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Exercise the public function
-- it is public, it shall work
SELECT cdb_geocode_admin0_polygon('Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_admin0_polygon(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_admin0_polygon invoked with params (test_user, <NULL>, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_admin0_polygon(username, orgname, country_name)"
PL/pgSQL function cdb_geocode_admin0_polygon(text) line 16 at SQL statement
cdb_geocode_admin0_polygon
----------------------------
(1 row)
SELECT cdb_geocode_admin1_polygon('California');
NOTICE: cdb_dataservices_client._cdb_geocode_admin1_polygon(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (test_user, <NULL>, California)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name)"
PL/pgSQL function cdb_geocode_admin1_polygon(text) line 16 at SQL statement
cdb_geocode_admin1_polygon
----------------------------
(1 row)
SELECT cdb_geocode_admin1_polygon('California', 'United States');
NOTICE: cdb_dataservices_client._cdb_geocode_admin1_polygon(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (test_user, <NULL>, California, United States)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name, country_name)"
PL/pgSQL function cdb_geocode_admin1_polygon(text,text) line 16 at SQL statement
cdb_geocode_admin1_polygon
----------------------------
(1 row)
SELECT cdb_geocode_namedplace_point('Elx');
NOTICE: cdb_dataservices_client._cdb_geocode_namedplace_point(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (test_user, <NULL>, Elx)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name)"
PL/pgSQL function cdb_geocode_namedplace_point(text) line 16 at SQL statement
cdb_geocode_namedplace_point
------------------------------
(1 row)
SELECT cdb_geocode_namedplace_point('Elx', 'Valencia');
NOTICE: cdb_dataservices_client._cdb_geocode_namedplace_point(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (test_user, <NULL>, Elx, Valencia)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, country_name)"
PL/pgSQL function cdb_geocode_namedplace_point(text,text) line 16 at SQL statement
cdb_geocode_namedplace_point
------------------------------
(1 row)
SELECT cdb_geocode_namedplace_point('Elx', 'Valencia', 'Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_namedplace_point(5): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (test_user, <NULL>, Elx, Valencia, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, admin1_name, country_name)"
PL/pgSQL function cdb_geocode_namedplace_point(text,text,text) line 16 at SQL statement
cdb_geocode_namedplace_point
------------------------------
(1 row)
SELECT cdb_geocode_postalcode_polygon('03204', 'Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_postalcode_polygon(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_postalcode_polygon invoked with params (test_user, <NULL>, 03204, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_postalcode_polygon(username, orgname, postal_code, country_name)"
PL/pgSQL function cdb_geocode_postalcode_polygon(text,text) line 16 at SQL statement
cdb_geocode_postalcode_polygon
--------------------------------
(1 row)
SELECT cdb_geocode_postalcode_point('03204', 'Spain');
NOTICE: cdb_dataservices_client._cdb_geocode_postalcode_point(4): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_postalcode_point invoked with params (test_user, <NULL>, 03204, Spain)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_postalcode_point(username, orgname, postal_code, country_name)"
PL/pgSQL function cdb_geocode_postalcode_point(text,text) line 16 at SQL statement
cdb_geocode_postalcode_point
------------------------------
(1 row)
SELECT cdb_geocode_ipaddress_point('8.8.8.8');
NOTICE: cdb_dataservices_client._cdb_geocode_ipaddress_point(3): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_ipaddress_point invoked with params (test_user, <NULL>, 8.8.8.8)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_ipaddress_point(username, orgname, ip_address)"
PL/pgSQL function cdb_geocode_ipaddress_point(text) line 16 at SQL statement
cdb_geocode_ipaddress_point
-----------------------------
(1 row)
SELECT cdb_geocode_street_point('one street, 1');
NOTICE: cdb_dataservices_client._cdb_geocode_street_point(6): [contrib_regression] REMOTE NOTICE: cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (test_user, <NULL>, one street, 1, <NULL>, <NULL>, <NULL>)
CONTEXT: SQL statement "SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country)"
PL/pgSQL function cdb_geocode_street_point(text,text,text,text) line 16 at SQL statement
cdb_geocode_street_point
--------------------------
(1 row)
-- Check the regular user has no permissions on private functions
SELECT _cdb_geocode_admin0_polygon('evil_user', 'evil_orgname', 'Hell');
ERROR: permission denied for function _cdb_geocode_admin0_polygon
SELECT _cdb_geocode_admin1_polygon('evil_user', 'evil_orgname', 'Hell');
ERROR: permission denied for function _cdb_geocode_admin1_polygon
SELECT _cdb_geocode_admin1_polygon('evil_user', 'evil_orgname', 'Sheol', 'Hell');
ERROR: permission denied for function _cdb_geocode_admin1_polygon
SELECT _cdb_geocode_namedplace_point('evil_user', 'evil_orgname', 'Sheol');
ERROR: permission denied for function _cdb_geocode_namedplace_point
SELECT _cdb_geocode_namedplace_point('evil_user', 'evil_orgname', 'Sheol', 'Hell');
ERROR: permission denied for function _cdb_geocode_namedplace_point
SELECT _cdb_geocode_namedplace_point('evil_user', 'evil_orgname', 'Sheol', 'Hell', 'Ugly world');
ERROR: permission denied for function _cdb_geocode_namedplace_point
SELECT _cdb_geocode_postalcode_polygon('evil_user', 'evil_orgname', '66666', 'Hell');
ERROR: permission denied for function _cdb_geocode_postalcode_polygon
SELECT _cdb_geocode_postalcode_point('evil_user', 'evil_orgname', '66666', 'Hell');
ERROR: permission denied for function _cdb_geocode_postalcode_point
SELECT _cdb_geocode_ipaddress_point('evil_user', 'evil_orgname', '8.8.8.8');
ERROR: permission denied for function _cdb_geocode_ipaddress_point
SELECT _cdb_geocode_street_point('evil_user', 'evil_orgname', 'one street, 1');
ERROR: permission denied for function _cdb_geocode_street_point

View File

@@ -0,0 +1,23 @@
-- Install dependencies
CREATE EXTENSION postgis;
CREATE EXTENSION schema_triggers;
CREATE EXTENSION plpythonu;
CREATE EXTENSION cartodb;
CREATE EXTENSION plproxy;
-- Install the extension
CREATE EXTENSION cdb_dataservices_client;
-- Mock the server connection to point to this very test db
SELECT cartodb.cdb_conf_setconf('geocoder_server_config', '{"connection_str": "dbname=contrib_regression host=127.0.0.1 user=postgres"}');
-- Mock the user configuration
SELECT cartodb.cdb_conf_setconf('user_config', '{"is_organization": false, "entity_name": "test_user"}');
-- Mock the server schema
CREATE SCHEMA cdb_dataservices_server;
-- Create a test user to check permissions
DROP ROLE IF EXISTS test_regular_user;
CREATE ROLE test_regular_user;
GRANT publicuser TO test_regular_user;
ALTER ROLE test_regular_user SET search_path TO public,cartodb,cdb_dataservices_client;

View File

@@ -0,0 +1,15 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server function
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin0_polygon(username text, orgname text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_admin0_polygon invoked with params (%, %, %)', username, orgname, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_admin0_polygon('Spain');

View File

@@ -0,0 +1,24 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (%, %, %)', username, orgname, admin1_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_admin1_polygon invoked with params (%, %, %, %)', username, orgname, admin1_name, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_admin1_polygon('California');
SELECT cdb_geocode_admin1_polygon('California', 'United States');

View File

@@ -0,0 +1,33 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (%, %, %)', username, orgname, city_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (%, %, %, %)', username, orgname, city_name, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_namedplace_point invoked with params (%, %, %, %, %)', username, orgname, city_name, admin1_name, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_namedplace_point('Elx');
SELECT cdb_geocode_namedplace_point('Elx', 'Spain');
SELECT cdb_geocode_namedplace_point('Elx', 'Valencia', 'Spain');

View File

@@ -0,0 +1,23 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, postal_code text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_postalcode_polygon invoked with params (%, %, %, %)', username, orgname, postal_code, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, postal_code text, country_name text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_postalcode_point invoked with params (%, %, %, %)', username, orgname, postal_code, country_name;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_postalcode_polygon('03204', 'Spain');
SELECT cdb_geocode_postalcode_point('03204', 'Spain');

View File

@@ -0,0 +1,15 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_ipaddress_point(username text, orgname text, ip_address text)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_ipaddress_point invoked with params (%, %, %)', username, orgname, ip_address;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_ipaddress_point('8.8.8.8');

View File

@@ -0,0 +1,24 @@
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Mock the server functions
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_street_point (username text, orgname text, searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL)
RETURNS Geometry AS $$
BEGIN
RAISE NOTICE 'cdb_dataservices_server.cdb_geocode_geocoder_street_point invoked with params (%, %, %, %, %, %)', username, orgname, searchtext, city, state_province, country;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
-- Exercise the public and the proxied function
SELECT cdb_geocode_street_point('One street, 1');
SELECT cdb_geocode_street_point('One street', 'city');
SELECT cdb_geocode_street_point('One street', 'city', 'state');
SELECT cdb_geocode_street_point('One street', 'city', 'state', 'country');
SELECT cdb_geocode_street_point('One street', 'city', NULL, 'country');
SELECT cdb_geocode_street_point('One street, 1');
SELECT cdb_geocode_street_point('One street', 'city');
SELECT cdb_geocode_street_point('One street', 'city', 'state');
SELECT cdb_geocode_street_point('One street', 'city', 'state', 'country');
SELECT cdb_geocode_street_point('One street', 'city', NULL, 'country');

View File

@@ -0,0 +1,30 @@
-- Use regular user role
SET ROLE test_regular_user;
-- Add to the search path the schema
SET search_path TO public,cartodb,cdb_dataservices_client;
-- Exercise the public function
-- it is public, it shall work
SELECT cdb_geocode_admin0_polygon('Spain');
SELECT cdb_geocode_admin1_polygon('California');
SELECT cdb_geocode_admin1_polygon('California', 'United States');
SELECT cdb_geocode_namedplace_point('Elx');
SELECT cdb_geocode_namedplace_point('Elx', 'Valencia');
SELECT cdb_geocode_namedplace_point('Elx', 'Valencia', 'Spain');
SELECT cdb_geocode_postalcode_polygon('03204', 'Spain');
SELECT cdb_geocode_postalcode_point('03204', 'Spain');
SELECT cdb_geocode_ipaddress_point('8.8.8.8');
SELECT cdb_geocode_street_point('one street, 1');
-- Check the regular user has no permissions on private functions
SELECT _cdb_geocode_admin0_polygon('evil_user', 'evil_orgname', 'Hell');
SELECT _cdb_geocode_admin1_polygon('evil_user', 'evil_orgname', 'Hell');
SELECT _cdb_geocode_admin1_polygon('evil_user', 'evil_orgname', 'Sheol', 'Hell');
SELECT _cdb_geocode_namedplace_point('evil_user', 'evil_orgname', 'Sheol');
SELECT _cdb_geocode_namedplace_point('evil_user', 'evil_orgname', 'Sheol', 'Hell');
SELECT _cdb_geocode_namedplace_point('evil_user', 'evil_orgname', 'Sheol', 'Hell', 'Ugly world');
SELECT _cdb_geocode_postalcode_polygon('evil_user', 'evil_orgname', '66666', 'Hell');
SELECT _cdb_geocode_postalcode_point('evil_user', 'evil_orgname', '66666', 'Hell');
SELECT _cdb_geocode_ipaddress_point('evil_user', 'evil_orgname', '8.8.8.8');
SELECT _cdb_geocode_street_point('evil_user', 'evil_orgname', 'one street, 1');

View File

@@ -0,0 +1,69 @@
# Data Services API Internal documentation
* [Existent services](#existent-services)
* [How to add a new service](#how-to-add-a-new-service)
## Existent services
Available at [cartodb_services](https://github.com/CartoDB/geocoder-api/tree/master/server/lib/python/cartodb_services/cartodb_services).
* Google
* [Geocoding](https://github.com/CartoDB/geocoder-api/blob/983440086d3fabf03aedc66f53dcf4c4a8cb2323/server/lib/python/cartodb_services/cartodb_services/google/geocoder.py)
* Here
* [Geocoding](https://github.com/CartoDB/geocoder-api/blob/983440086d3fabf03aedc66f53dcf4c4a8cb2323/server/lib/python/cartodb_services/cartodb_services/here/geocoder.py)
* [Routing](https://github.com/CartoDB/geocoder-api/blob/983440086d3fabf03aedc66f53dcf4c4a8cb2323/server/lib/python/cartodb_services/cartodb_services/here/routing.py)
## How to add a new service
These are the steps that need to be followed when creating a new service in the API or updating an existent one.
### Creating a new service function or editing an existent one
In this scenario, both client and server sides require to be edited/created.
* **Update the interface file** with the function addition or update
* Interfaces are stored in `client/renderer/interfaces`
* Interface YAML filenames follow the client versioning schema with the format `interface-x.y.z.yaml`.
* Update the renderer templates or script, *if applicable*
* Renderer templates are stored in `client/renderer/templates`
* The Renderer script (`client/renderer/sql-template-renderer`) generates SQL from the defined interfaces
* Generate a **new subfolder version** for `sql` and `test` folders to define the new functions and tests
* TODO: Use symlinks to avoid file duplication between versions that don't update them
* The `client/sql` folder contents are generated from the interfaces in the first step
* Add or upgrade your SQL server functions
* Create tests for the client and server functions -- at least, to check that those are created
* Generate the **upgrade and downgrade files** for the extension for both client and server
* Update the control files and the Makefiles to generate the complete SQL file for the new created version
* These new version files (`cdb_dataservices_client--X.Y.Z.sql and cdb_dataservices_server--X.Y.X.sql`) must be pushed and frozen. You can add these to the `.gitignore` file.
* Update the public docs! ;-)
### Updating an existing server side function
With no changes in client side.
#### Extension
* Generate a **new subfolder version** for `sql` and `test` folders to define the new functions and tests
* **TODO:** Use symlinks to avoid file duplication between versions that don't update them
* **Add or upgrade your SQL server functions**
* For example, if a new street geocoder service is implemented, it will require a change in the main function (`cdb_dataservices_server.cdb_geocode_street_point`) and generate a new `cdb_dataservices_server._cdb_newservice_geocode_street_point`
* Generate the upgrade and downgrade files for the extension for the client
#### Python
* Add, if needed, [new configuration elements](https://github.com/CartoDB/geocoder-api/blob/983440086d3fabf03aedc66f53dcf4c4a8cb2323/server/lib/python/cartodb_services/cartodb_services/metrics/config.py#L100)
* Add the new **functionality** into the provider folder. If the provider is new, create a new folder(`server/lib/python/cartodb_services/cartodb_services/{provider_name}` and add the service (`geocoder.py`)
* Check the `__init__.py` files to follow the existent [import conventions](https://github.com/CartoDB/geocoder-api/blob/983440086d3fabf03aedc66f53dcf4c4a8cb2323/server/lib/python/cartodb_services/cartodb_services/metrics/__init__.py)
* Add a **new metric**, if needed, at the [corresponding service](https://github.com/CartoDB/geocoder-api/blob/983440086d3fabf03aedc66f53dcf4c4a8cb2323/server/lib/python/cartodb_services/cartodb_services/metrics/quota.py#L37-L60)
* **Update the package version** in [setup.py](https://github.com/CartoDB/geocoder-api/blob/983440086d3fabf03aedc66f53dcf4c4a8cb2323/server/lib/python/cartodb_services/setup.py)

View File

@@ -4,3 +4,4 @@ regression.out
cdb_geocoder_server--0.0.1.sql
cdb_geocoder_server--0.1.0.sql
cdb_geocoder_server--0.2.0.sql
cdb_geocoder_server--0.3.0.sql

View File

@@ -14,12 +14,15 @@ DATA = $(NEW_EXTENSION_ARTIFACT) \
cdb_dataservices_server--0.0.1.sql \
cdb_dataservices_server--0.1.0.sql \
cdb_dataservices_server--0.2.0.sql \
cdb_dataservices_server--0.3.0.sql \
cdb_dataservices_server--0.1.0--0.0.1.sql \
cdb_dataservices_server--0.0.1--0.1.0.sql \
cdb_dataservices_server--0.2.0--0.1.0.sql \
cdb_dataservices_server--0.1.0--0.2.0.sql \
cdb_dataservices_server--0.2.0--0.3.0.sql \
cdb_dataservices_server--0.3.0--0.2.0.sql
cdb_dataservices_server--0.3.0--0.2.0.sql \
cdb_dataservices_server--0.3.0--0.4.0.sql \
cdb_dataservices_server--0.4.0--0.3.0.sql
REGRESS = $(notdir $(basename $(wildcard test/$(EXTVERSION)/sql/*test.sql)))
TEST_DIR = test/$(EXTVERSION)

View File

@@ -337,4 +337,4 @@ RETURNS Geometry AS $$
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
$$ LANGUAGE plpythonu;

View File

@@ -0,0 +1,124 @@
-- Get the Redis configuration from the _conf table --
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_isolines_routing_config(username text, orgname text)
RETURNS boolean AS $$
cache_key = "user_isolines_routing_config_{0}".format(username)
if cache_key in GD:
return False
else:
import json
from cartodb_services.metrics import IsolinesRoutingConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
heremaps_conf_json = plpy.execute("SELECT cartodb.CDB_Conf_GetConf('heremaps_conf') as heremaps_conf", 1)[0]['heremaps_conf']
if not heremaps_conf_json:
heremaps_app_id = None
heremaps_app_code = None
else:
heremaps_conf = json.loads(heremaps_conf_json)
heremaps_app_id = heremaps_conf['app_id']
heremaps_app_code = heremaps_conf['app_code']
routing_config = IsolinesRoutingConfig(redis_conn, username, orgname, heremaps_app_id, heremaps_app_code)
# --Think about the security concerns with this kind of global cache, it should be only available
# --for this user session but...
GD[cache_key] = routing_config
return True
$$ LANGUAGE plpythonu SECURITY DEFINER;
CREATE TYPE cdb_dataservices_server.isoline AS (center geometry(Geometry,4326), data_range integer, the_geom geometry(Multipolygon,4326));
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_routing_isolines(username TEXT, orgname TEXT, type TEXT, source geometry(Geometry, 4326), mode TEXT, data_range integer[], options text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
import json
from cartodb_services.here import HereMapsRoutingIsoline
from cartodb_services.metrics import QuotaService
from cartodb_services.here.types import geo_polyline_to_multipolygon
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)]
# -- Check the quota
quota_service = QuotaService(user_isolines_routing_config, redis_conn)
if not quota_service.check_user_quota():
plpy.error('You have reach the limit of your quota')
try:
client = HereMapsRoutingIsoline(user_isolines_routing_config.heremaps_app_id, user_isolines_routing_config.heremaps_app_code, base_url = HereMapsRoutingIsoline.PRODUCTION_ROUTING_BASE_URL)
if source:
lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat']
lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon']
source_str = 'geo!%f,%f' % (lat, lon)
else:
source_str = None
if type == 'isodistance':
resp = client.calculate_isodistance(source_str, mode, data_range, options)
elif type == 'isochrone':
resp = client.calculate_isochrone(source_str, mode, data_range, options)
if resp:
result = []
for isoline in resp:
data_range_n = isoline['range']
polyline = isoline['geom']
multipolygon = geo_polyline_to_multipolygon(polyline)
result.append([source, data_range_n, multipolygon])
quota_service.increment_success_geocoder_use()
quota_service.increment_isolines_service_use(len(resp))
return result
else:
quota_service.increment_empty_geocoder_use()
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to obtain isodistances: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu SECURITY DEFINER;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isodistance(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)]
type = 'isodistance'
if user_isolines_config.google_services_user:
plpy.error('This service is not available for google service users.')
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"])
result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options])
isolines = []
for element in result:
isoline = element['isoline']
isoline = isoline.translate(None, "()").split(',')
isolines.append(isoline)
return isolines
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isochrone(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)]
type = 'isochrone'
if user_isolines_config.google_services_user:
plpy.error('This service is not available for google service users.')
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"])
result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options])
isolines = []
for element in result:
isoline = element['isoline']
isoline = isoline.translate(None, "()").split(',')
isolines.append(isoline)
return isolines
$$ LANGUAGE plpythonu;

View File

@@ -66,7 +66,7 @@ RETURNS boolean AS $$
return False
else:
import json
from cartodb_services.quota import GeocoderConfig
from cartodb_services.metrics import GeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
heremaps_conf_json = plpy.execute("SELECT cartodb.CDB_Conf_GetConf('heremaps_conf') as heremaps_conf", 1)[0]['heremaps_conf']
@@ -105,7 +105,7 @@ $$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
from cartodb_services.here import HereMapsGeocoder
from cartodb_services.quota import QuotaService
from cartodb_services.metrics import QuotaService
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]
@@ -140,7 +140,7 @@ $$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_google_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
from cartodb_services.google import GoogleMapsGeocoder
from cartodb_services.quota import QuotaService
from cartodb_services.metrics import QuotaService
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]

View File

@@ -0,0 +1,5 @@
DROP FUNCTION IF EXISTS cdb_dataservices_server.cdb_isochrone(TEXT, TEXT, geometry(Geometry, 4326), TEXT, integer[], text[]);
DROP FUNCTION IF EXISTS cdb_dataservices_server.cdb_isodistance(TEXT, TEXT, geometry(Geometry, 4326), TEXT, integer[], text[]);
DROP FUNCTION IF EXISTS cdb_dataservices_server._cdb_here_routing_isolines(TEXT, TEXT, TEXT, geometry(Geometry, 4326), TEXT, integer[], text[]);
DROP FUNCTION IF EXISTS cdb_dataservices_server._get_isolines_routing_config(text, text);
DROP TYPE IF EXISTS cdb_dataservices_server.isoline;

View File

@@ -0,0 +1,917 @@
--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION cdb_dataservices_server" to load this file. \quit
CREATE TYPE cdb_dataservices_server._redis_conf_params AS (
sentinel_host text,
sentinel_port int,
sentinel_master_id text,
redis_db text,
timeout float
);
-- Get the Redis configuration from the _conf table --
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_redis_conf_v2(config_key text)
RETURNS cdb_dataservices_server._redis_conf_params AS $$
conf_query = "SELECT cartodb.CDB_Conf_GetConf('{0}') as conf".format(config_key)
conf = plpy.execute(conf_query)[0]['conf']
if conf is None:
plpy.error("There is no redis configuration defined")
else:
import json
params = json.loads(conf)
return {
"sentinel_host": params['sentinel_host'],
"sentinel_port": params['sentinel_port'],
"sentinel_master_id": params['sentinel_master_id'],
"timeout": params['timeout'],
"redis_db": params['redis_db']
}
$$ LANGUAGE plpythonu;
-- Get the connection to redis from cache or create a new one
CREATE OR REPLACE FUNCTION cdb_dataservices_server._connect_to_redis(user_id text)
RETURNS boolean AS $$
cache_key = "redis_connection_{0}".format(user_id)
if cache_key in GD:
return False
else:
from cartodb_services.tools import RedisConnection
metadata_config_params = plpy.execute("""select c.sentinel_host, c.sentinel_port,
c.sentinel_master_id, c.timeout, c.redis_db
from cdb_dataservices_server._get_redis_conf_v2('redis_metadata_config') c;""")[0]
metrics_config_params = plpy.execute("""select c.sentinel_host, c.sentinel_port,
c.sentinel_master_id, c.timeout, c.redis_db
from cdb_dataservices_server._get_redis_conf_v2('redis_metrics_config') c;""")[0]
redis_metadata_connection = RedisConnection(metadata_config_params['sentinel_host'],
metadata_config_params['sentinel_port'],
metadata_config_params['sentinel_master_id'],
timeout=metadata_config_params['timeout'],
redis_db=metadata_config_params['redis_db']).redis_connection()
redis_metrics_connection = RedisConnection(metrics_config_params['sentinel_host'],
metrics_config_params['sentinel_port'],
metrics_config_params['sentinel_master_id'],
timeout=metrics_config_params['timeout'],
redis_db=metrics_config_params['redis_db']).redis_connection()
GD[cache_key] = {
'redis_metadata_connection': redis_metadata_connection,
'redis_metrics_connection': redis_metrics_connection,
}
return True
$$ LANGUAGE plpythonu SECURITY DEFINER;
-- Get the Redis configuration from the _conf table --
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_geocoder_config(username text, orgname text)
RETURNS boolean AS $$
cache_key = "user_geocoder_config_{0}".format(username)
if cache_key in GD:
return False
else:
import json
from cartodb_services.metrics import GeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
heremaps_conf_json = plpy.execute("SELECT cartodb.CDB_Conf_GetConf('heremaps_conf') as heremaps_conf", 1)[0]['heremaps_conf']
if not heremaps_conf_json:
heremaps_app_id = None
heremaps_app_code = None
else:
heremaps_conf = json.loads(heremaps_conf_json)
heremaps_app_id = heremaps_conf['app_id']
heremaps_app_code = heremaps_conf['app_code']
geocoder_config = GeocoderConfig(redis_conn, username, orgname, heremaps_app_id, heremaps_app_code)
# --Think about the security concerns with this kind of global cache, it should be only available
# --for this user session but...
GD[cache_key] = geocoder_config
return True
$$ LANGUAGE plpythonu SECURITY DEFINER;
-- Get the Redis configuration from the _conf table --
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_isolines_routing_config(username text, orgname text)
RETURNS boolean AS $$
cache_key = "user_isolines_routing_config_{0}".format(username)
if cache_key in GD:
return False
else:
import json
from cartodb_services.metrics import IsolinesRoutingConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
heremaps_conf_json = plpy.execute("SELECT cartodb.CDB_Conf_GetConf('heremaps_conf') as heremaps_conf", 1)[0]['heremaps_conf']
if not heremaps_conf_json:
heremaps_app_id = None
heremaps_app_code = None
else:
heremaps_conf = json.loads(heremaps_conf_json)
heremaps_app_id = heremaps_conf['app_id']
heremaps_app_code = heremaps_conf['app_code']
isolines_routing_config = IsolinesRoutingConfig(redis_conn, username, orgname, heremaps_app_id, heremaps_app_code)
# --Think about the security concerns with this kind of global cache, it should be only available
# --for this user session but...
GD[cache_key] = isolines_routing_config
return True
$$ LANGUAGE plpythonu SECURITY DEFINER;
-- Geocodes a street address given a searchtext and a state and/or country
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]
if user_geocoder_config.heremaps_geocoder:
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"])
return plpy.execute(here_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point']
elif user_geocoder_config.google_geocoder:
google_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_google_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"])
return plpy.execute(google_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point']
else:
plpy.error('Requested geocoder is not available')
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
from cartodb_services.here import HereMapsGeocoder
from cartodb_services.metrics import QuotaService
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]
# -- Check the quota
quota_service = QuotaService(user_geocoder_config, redis_conn)
if not quota_service.check_user_quota():
plpy.error('You have reach the limit of your quota')
try:
geocoder = HereMapsGeocoder(user_geocoder_config.heremaps_app_id, user_geocoder_config.heremaps_app_code)
coordinates = geocoder.geocode(searchtext=searchtext, city=city, state=state_province, country=country)
if coordinates:
quota_service.increment_success_geocoder_use()
plan = plpy.prepare("SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326); ", ["double precision", "double precision"])
point = plpy.execute(plan, [coordinates[0], coordinates[1]], 1)[0]
return point['st_setsrid']
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using here maps geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_google_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
from cartodb_services.google import GoogleMapsGeocoder
from cartodb_services.metrics import QuotaService
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
geocoder = GoogleMapsGeocoder(user_geocoder_config.google_client_id, user_geocoder_config.google_api_key)
coordinates = geocoder.geocode(searchtext=searchtext, city=city, state=state_province, country=country)
if coordinates:
quota_service.increment_success_geocoder_use()
plan = plpy.prepare("SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326); ", ["double precision", "double precision"])
point = plpy.execute(plan, [coordinates[0], coordinates[1]], 1)[0]
return point['st_setsrid']
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using google maps geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin0_polygon(username text, orgname text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin0_polygon(trim($1)) AS mypolygon", ["text"])
rv = plpy.execute(plan, [country_name], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin0_polygon(country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT n.the_geom as geom INTO ret
FROM (SELECT q, lower(regexp_replace(q, '[^a-zA-Z\u00C0-\u00ff]+', '', 'g'))::text x
FROM (SELECT country_name q) g) d
LEFT OUTER JOIN admin0_synonyms s ON name_ = d.x
LEFT OUTER JOIN ne_admin0_v3 n ON s.adm0_a3 = n.adm0_a3 GROUP BY d.q, n.the_geom, s.adm0_a3;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_admin1_polygon(admin1_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin1_polygon(trim($1)) AS mypolygon", ["text"])
rv = plpy.execute(plan, [admin1_name], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
---- cdb_geocode_admin1_polygon(admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin1_polygon(trim($1), trim($2)) AS mypolygon", ["text", "text"])
rv = plpy.execute(plan, [admin1_name, country_name], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
---- cdb_geocode_admin1_polygon(admin1_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin1_polygon(admin1_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT q, (
SELECT the_geom
FROM global_province_polygons
WHERE d.c = ANY (synonyms)
ORDER BY frequency DESC LIMIT 1
) geom
FROM (
SELECT
trim(replace(lower(admin1_name),'.',' ')) c, admin1_name q
) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_admin1_polygon(admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin1_polygon(admin1_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
WITH p AS (SELECT r.c, r.q, (SELECT iso3 FROM country_decoder WHERE lower(country_name) = ANY (synonyms)) i FROM (SELECT trim(replace(lower(admin1_name),'.',' ')) c, country_name q) r)
SELECT
geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_province_polygons
WHERE p.c = ANY (synonyms)
AND iso3 = p.i
ORDER BY frequency DESC LIMIT 1
) geom
FROM p) n;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_namedplace_point(city_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1)) AS mypoint", ["text"])
rv = plpy.execute(plan, [city_name], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
---- cdb_geocode_namedplace_point(city_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1), trim($2)) AS mypoint", ["text", "text"])
rv = plpy.execute(plan, [city_name, country_name], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
---- cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1), trim($2), trim($3)) AS mypoint", ["text", "text", "text"])
rv = plpy.execute(plan, [city_name, admin1_name, country_name], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
---- cdb_geocode_namedplace_point(city_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
WITH best AS (SELECT s AS q, (SELECT the_geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) ORDER BY population DESC LIMIT 1) AS geom FROM (SELECT city_name as s) p),
next AS (SELECT p.s AS q, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM (SELECT city_name as s) p WHERE p.s NOT IN (SELECT q FROM best WHERE geom IS NOT NULL))
SELECT q, geom, TRUE AS success FROM best WHERE geom IS NOT NULL
UNION ALL
SELECT q, geom, CASE WHEN geom IS NULL THEN FALSE ELSE TRUE END AS success FROM next
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_namedplace_point(city_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
WITH p AS (SELECT r.s, r.c, (SELECT iso2 FROM country_decoder WHERE lower(r.c) = ANY (synonyms)) i FROM (SELECT city_name AS s, country_name::text AS c) r),
best AS (SELECT p.s AS q, p.c AS c, (SELECT gp.the_geom AS geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) AND gp.iso2 = p.i ORDER BY population DESC LIMIT 1) AS geom FROM p),
next AS (SELECT p.s AS q, p.c AS c, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND gp.iso2 = p.i AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM p WHERE p.s NOT IN (SELECT q FROM best WHERE c = p.c AND geom IS NOT NULL))
SELECT geom FROM best WHERE geom IS NOT NULL
UNION ALL
SELECT geom FROM next
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
WITH inputcountry AS (
SELECT iso2 as isoTwo FROM country_decoder WHERE lower(country_name) = ANY (synonyms) LIMIT 1
),
p AS (
SELECT r.s, r.a1, (SELECT admin1 FROM admin1_decoder, inputcountry WHERE lower(r.a1) = ANY (synonyms) AND admin1_decoder.iso2 = inputcountry.isoTwo LIMIT 1) i FROM (SELECT city_name AS s, admin1_name::text AS a1) r),
best AS (SELECT p.s AS q, p.a1 as a1, (SELECT gp.the_geom AS geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) AND gp.admin1 = p.i ORDER BY population DESC LIMIT 1) AS geom FROM p),
next AS (SELECT p.s AS q, p.a1 AS a1, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND ga.admin1 = p.i AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM p WHERE p.s NOT IN (SELECT q FROM best WHERE geom IS NOT NULL))
SELECT geom FROM best WHERE geom IS NOT NULL
UNION ALL
SELECT geom FROM next
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, code text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_point(trim($1)) AS mypoint", ["text"])
rv = plpy.execute(plan, [code], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, code text, country text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_point(trim($1), trim($2)) AS mypoint", ["TEXT", "TEXT"])
rv = plpy.execute(plan, [code, country], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, code text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_polygon(trim($1)) AS mypolygon", ["text"])
rv = plpy.execute(plan, [code], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, code text, country text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_polygon(trim($1), trim($2)) AS mypolygon", ["TEXT", "TEXT"])
rv = plpy.execute(plan, [code, country], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_point(code text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_points
WHERE postal_code = upper(d.q)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_point(code text, country text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_points
WHERE postal_code = upper(d.q)
AND iso3 = (
SELECT iso3 FROM country_decoder WHERE
lower(country) = ANY (synonyms) LIMIT 1
)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_polygon(code text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_polygons
WHERE postal_code = upper(d.q)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_polygon(code text, country text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_polygons
WHERE postal_code = upper(d.q)
AND iso3 = (
SELECT iso3 FROM country_decoder WHERE
lower(country) = ANY (synonyms) LIMIT 1
)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_ipaddress_point(username text, orgname text, ip text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_ipaddress_point(trim($1)) AS mypoint", ["TEXT"])
rv = plpy.execute(plan, [ip], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_ipaddress_point(ip text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
new_ip INET;
BEGIN
BEGIN
IF family(ip::inet) = 6 THEN
new_ip := ip::inet;
ELSE
new_ip := ('::ffff:' || ip)::inet;
END IF;
EXCEPTION WHEN OTHERS THEN
SELECT NULL as geom INTO ret;
RETURN ret;
END;
WITH
ips AS (SELECT ip s, new_ip net),
matches AS (SELECT s, (SELECT the_geom FROM ip_address_locations WHERE network_start_ip <= ips.net ORDER BY network_start_ip DESC LIMIT 1) geom FROM ips)
SELECT geom INTO ret
FROM matches;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE TYPE cdb_dataservices_server.isoline AS (center geometry(Geometry,4326), data_range integer, the_geom geometry(Multipolygon,4326));
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_routing_isolines(username TEXT, orgname TEXT, type TEXT, source geometry(Geometry, 4326), mode TEXT, data_range integer[], options text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
import json
from cartodb_services.here import HereMapsRoutingIsoline
from cartodb_services.metrics import QuotaService
from cartodb_services.here.types import geo_polyline_to_multipolygon
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)]
# -- Check the quota
quota_service = QuotaService(user_isolines_routing_config, redis_conn)
if not quota_service.check_user_quota():
plpy.error('You have reach the limit of your quota')
try:
client = HereMapsRoutingIsoline(user_isolines_routing_config.heremaps_app_id, user_isolines_routing_config.heremaps_app_code, base_url = HereMapsRoutingIsoline.PRODUCTION_ROUTING_BASE_URL)
if source:
lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat']
lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon']
source_str = 'geo!%f,%f' % (lat, lon)
else:
source_str = None
if type == 'isodistance':
resp = client.calculate_isodistance(source_str, mode, data_range, options)
elif type == 'isochrone':
resp = client.calculate_isochrone(source_str, mode, data_range, options)
if resp:
result = []
for isoline in resp:
data_range_n = isoline['range']
polyline = isoline['geom']
multipolygon = geo_polyline_to_multipolygon(polyline)
result.append([source, data_range_n, multipolygon])
quota_service.increment_success_geocoder_use()
quota_service.increment_isolines_service_use(len(resp))
return result
else:
quota_service.increment_empty_geocoder_use()
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to obtain isodistances using here maps geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu SECURITY DEFINER;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isodistance(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)]
type = 'isodistance'
if user_isolines_config.google_services_user:
plpy.error('This service is not available for google service users.')
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"])
result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options])
isolines = []
for element in result:
isoline = element['isoline']
isoline = isoline.translate(None, "()").split(',')
isolines.append(isoline)
return isolines
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isochrone(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)]
type = 'isochrone'
if user_isolines_config.google_services_user:
plpy.error('This service is not available for google service users.')
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"])
result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options])
isolines = []
for element in result:
isoline = element['isoline']
isoline = isoline.translate(None, "()").split(',')
isolines.append(isoline)
return isolines
$$ LANGUAGE plpythonu;
DO $$
BEGIN
IF NOT EXISTS (
SELECT *
FROM pg_catalog.pg_user
WHERE usename = 'geocoder_api') THEN
CREATE USER geocoder_api;
END IF;
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_server TO geocoder_api;
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO geocoder_api;
GRANT USAGE ON SCHEMA cdb_dataservices_server TO geocoder_api;
GRANT USAGE ON SCHEMA public TO geocoder_api;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO geocoder_api;
END$$;

View File

@@ -1,5 +1,5 @@
comment = 'CartoDB dataservices server extension'
default_version = '0.3.0'
default_version = '0.4.0'
requires = 'plpythonu, postgis, cdb_geocoder'
superuser = true
schema = cdb_dataservices_server

View File

@@ -0,0 +1,3 @@
--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION cdb_dataservices_server" to load this file. \quit

View File

@@ -0,0 +1,57 @@
CREATE TYPE cdb_dataservices_server._redis_conf_params AS (
sentinel_host text,
sentinel_port int,
sentinel_master_id text,
redis_db text,
timeout float
);
-- Get the Redis configuration from the _conf table --
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_redis_conf_v2(config_key text)
RETURNS cdb_dataservices_server._redis_conf_params AS $$
conf_query = "SELECT cartodb.CDB_Conf_GetConf('{0}') as conf".format(config_key)
conf = plpy.execute(conf_query)[0]['conf']
if conf is None:
plpy.error("There is no redis configuration defined")
else:
import json
params = json.loads(conf)
return {
"sentinel_host": params['sentinel_host'],
"sentinel_port": params['sentinel_port'],
"sentinel_master_id": params['sentinel_master_id'],
"timeout": params['timeout'],
"redis_db": params['redis_db']
}
$$ LANGUAGE plpythonu;
-- Get the connection to redis from cache or create a new one
CREATE OR REPLACE FUNCTION cdb_dataservices_server._connect_to_redis(user_id text)
RETURNS boolean AS $$
cache_key = "redis_connection_{0}".format(user_id)
if cache_key in GD:
return False
else:
from cartodb_services.tools import RedisConnection
metadata_config_params = plpy.execute("""select c.sentinel_host, c.sentinel_port,
c.sentinel_master_id, c.timeout, c.redis_db
from cdb_dataservices_server._get_redis_conf_v2('redis_metadata_config') c;""")[0]
metrics_config_params = plpy.execute("""select c.sentinel_host, c.sentinel_port,
c.sentinel_master_id, c.timeout, c.redis_db
from cdb_dataservices_server._get_redis_conf_v2('redis_metrics_config') c;""")[0]
redis_metadata_connection = RedisConnection(metadata_config_params['sentinel_host'],
metadata_config_params['sentinel_port'],
metadata_config_params['sentinel_master_id'],
timeout=metadata_config_params['timeout'],
redis_db=metadata_config_params['redis_db']).redis_connection()
redis_metrics_connection = RedisConnection(metrics_config_params['sentinel_host'],
metrics_config_params['sentinel_port'],
metrics_config_params['sentinel_master_id'],
timeout=metrics_config_params['timeout'],
redis_db=metrics_config_params['redis_db']).redis_connection()
GD[cache_key] = {
'redis_metadata_connection': redis_metadata_connection,
'redis_metrics_connection': redis_metrics_connection,
}
return True
$$ LANGUAGE plpythonu SECURITY DEFINER;

View File

@@ -0,0 +1,51 @@
-- Get the Redis configuration from the _conf table --
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_geocoder_config(username text, orgname text)
RETURNS boolean AS $$
cache_key = "user_geocoder_config_{0}".format(username)
if cache_key in GD:
return False
else:
import json
from cartodb_services.metrics import GeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
heremaps_conf_json = plpy.execute("SELECT cartodb.CDB_Conf_GetConf('heremaps_conf') as heremaps_conf", 1)[0]['heremaps_conf']
if not heremaps_conf_json:
heremaps_app_id = None
heremaps_app_code = None
else:
heremaps_conf = json.loads(heremaps_conf_json)
heremaps_app_id = heremaps_conf['app_id']
heremaps_app_code = heremaps_conf['app_code']
geocoder_config = GeocoderConfig(redis_conn, username, orgname, heremaps_app_id, heremaps_app_code)
# --Think about the security concerns with this kind of global cache, it should be only available
# --for this user session but...
GD[cache_key] = geocoder_config
return True
$$ LANGUAGE plpythonu SECURITY DEFINER;
-- Get the Redis configuration from the _conf table --
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_isolines_routing_config(username text, orgname text)
RETURNS boolean AS $$
cache_key = "user_isolines_routing_config_{0}".format(username)
if cache_key in GD:
return False
else:
import json
from cartodb_services.metrics import IsolinesRoutingConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
heremaps_conf_json = plpy.execute("SELECT cartodb.CDB_Conf_GetConf('heremaps_conf') as heremaps_conf", 1)[0]['heremaps_conf']
if not heremaps_conf_json:
heremaps_app_id = None
heremaps_app_code = None
else:
heremaps_conf = json.loads(heremaps_conf_json)
heremaps_app_id = heremaps_conf['app_id']
heremaps_app_code = heremaps_conf['app_code']
isolines_routing_config = IsolinesRoutingConfig(redis_conn, username, orgname, heremaps_app_id, heremaps_app_code)
# --Think about the security concerns with this kind of global cache, it should be only available
# --for this user session but...
GD[cache_key] = isolines_routing_config
return True
$$ LANGUAGE plpythonu SECURITY DEFINER;

View File

@@ -0,0 +1,84 @@
-- Geocodes a street address given a searchtext and a state and/or country
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]
if user_geocoder_config.heremaps_geocoder:
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"])
return plpy.execute(here_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point']
elif user_geocoder_config.google_geocoder:
google_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_google_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"])
return plpy.execute(google_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point']
else:
plpy.error('Requested geocoder is not available')
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
from cartodb_services.here import HereMapsGeocoder
from cartodb_services.metrics import QuotaService
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]
# -- Check the quota
quota_service = QuotaService(user_geocoder_config, redis_conn)
if not quota_service.check_user_quota():
plpy.error('You have reach the limit of your quota')
try:
geocoder = HereMapsGeocoder(user_geocoder_config.heremaps_app_id, user_geocoder_config.heremaps_app_code)
coordinates = geocoder.geocode(searchtext=searchtext, city=city, state=state_province, country=country)
if coordinates:
quota_service.increment_success_geocoder_use()
plan = plpy.prepare("SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326); ", ["double precision", "double precision"])
point = plpy.execute(plan, [coordinates[0], coordinates[1]], 1)[0]
return point['st_setsrid']
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using here maps geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_google_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL)
RETURNS Geometry AS $$
from cartodb_services.google import GoogleMapsGeocoder
from cartodb_services.metrics import QuotaService
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = GD["user_geocoder_config_{0}".format(username)]
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
geocoder = GoogleMapsGeocoder(user_geocoder_config.google_client_id, user_geocoder_config.google_api_key)
coordinates = geocoder.geocode(searchtext=searchtext, city=city, state=state_province, country=country)
if coordinates:
quota_service.increment_success_geocoder_use()
plan = plpy.prepare("SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326); ", ["double precision", "double precision"])
point = plpy.execute(plan, [coordinates[0], coordinates[1]], 1)[0]
return point['st_setsrid']
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using google maps geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;

View File

@@ -0,0 +1,50 @@
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin0_polygon(username text, orgname text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin0_polygon(trim($1)) AS mypolygon", ["text"])
rv = plpy.execute(plan, [country_name], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin0_polygon(country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT n.the_geom as geom INTO ret
FROM (SELECT q, lower(regexp_replace(q, '[^a-zA-Z\u00C0-\u00ff]+', '', 'g'))::text x
FROM (SELECT country_name q) g) d
LEFT OUTER JOIN admin0_synonyms s ON name_ = d.x
LEFT OUTER JOIN ne_admin0_v3 n ON s.adm0_a3 = n.adm0_a3 GROUP BY d.q, n.the_geom, s.adm0_a3;
RETURN ret;
END
$$ LANGUAGE plpgsql;

View File

@@ -0,0 +1,117 @@
---- cdb_geocode_admin1_polygon(admin1_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin1_polygon(trim($1)) AS mypolygon", ["text"])
rv = plpy.execute(plan, [admin1_name], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
---- cdb_geocode_admin1_polygon(admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin1_polygon(trim($1), trim($2)) AS mypolygon", ["text", "text"])
rv = plpy.execute(plan, [admin1_name, country_name], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
---- cdb_geocode_admin1_polygon(admin1_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin1_polygon(admin1_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT q, (
SELECT the_geom
FROM global_province_polygons
WHERE d.c = ANY (synonyms)
ORDER BY frequency DESC LIMIT 1
) geom
FROM (
SELECT
trim(replace(lower(admin1_name),'.',' ')) c, admin1_name q
) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_admin1_polygon(admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin1_polygon(admin1_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
WITH p AS (SELECT r.c, r.q, (SELECT iso3 FROM country_decoder WHERE lower(country_name) = ANY (synonyms)) i FROM (SELECT trim(replace(lower(admin1_name),'.',' ')) c, country_name q) r)
SELECT
geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_province_polygons
WHERE p.c = ANY (synonyms)
AND iso3 = p.i
ORDER BY frequency DESC LIMIT 1
) geom
FROM p) n;
RETURN ret;
END
$$ LANGUAGE plpgsql;

View File

@@ -0,0 +1,164 @@
---- cdb_geocode_namedplace_point(city_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1)) AS mypoint", ["text"])
rv = plpy.execute(plan, [city_name], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
---- cdb_geocode_namedplace_point(city_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1), trim($2)) AS mypoint", ["text", "text"])
rv = plpy.execute(plan, [city_name, country_name], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
---- cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1), trim($2), trim($3)) AS mypoint", ["text", "text", "text"])
rv = plpy.execute(plan, [city_name, admin1_name, country_name], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
---- cdb_geocode_namedplace_point(city_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
WITH best AS (SELECT s AS q, (SELECT the_geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) ORDER BY population DESC LIMIT 1) AS geom FROM (SELECT city_name as s) p),
next AS (SELECT p.s AS q, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM (SELECT city_name as s) p WHERE p.s NOT IN (SELECT q FROM best WHERE geom IS NOT NULL))
SELECT q, geom, TRUE AS success FROM best WHERE geom IS NOT NULL
UNION ALL
SELECT q, geom, CASE WHEN geom IS NULL THEN FALSE ELSE TRUE END AS success FROM next
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_namedplace_point(city_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
WITH p AS (SELECT r.s, r.c, (SELECT iso2 FROM country_decoder WHERE lower(r.c) = ANY (synonyms)) i FROM (SELECT city_name AS s, country_name::text AS c) r),
best AS (SELECT p.s AS q, p.c AS c, (SELECT gp.the_geom AS geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) AND gp.iso2 = p.i ORDER BY population DESC LIMIT 1) AS geom FROM p),
next AS (SELECT p.s AS q, p.c AS c, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND gp.iso2 = p.i AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM p WHERE p.s NOT IN (SELECT q FROM best WHERE c = p.c AND geom IS NOT NULL))
SELECT geom FROM best WHERE geom IS NOT NULL
UNION ALL
SELECT geom FROM next
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
---- cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text)
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
WITH inputcountry AS (
SELECT iso2 as isoTwo FROM country_decoder WHERE lower(country_name) = ANY (synonyms) LIMIT 1
),
p AS (
SELECT r.s, r.a1, (SELECT admin1 FROM admin1_decoder, inputcountry WHERE lower(r.a1) = ANY (synonyms) AND admin1_decoder.iso2 = inputcountry.isoTwo LIMIT 1) i FROM (SELECT city_name AS s, admin1_name::text AS a1) r),
best AS (SELECT p.s AS q, p.a1 as a1, (SELECT gp.the_geom AS geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) AND gp.admin1 = p.i ORDER BY population DESC LIMIT 1) AS geom FROM p),
next AS (SELECT p.s AS q, p.a1 AS a1, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND ga.admin1 = p.i AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM p WHERE p.s NOT IN (SELECT q FROM best WHERE geom IS NOT NULL))
SELECT geom FROM best WHERE geom IS NOT NULL
UNION ALL
SELECT geom FROM next
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;

View File

@@ -0,0 +1,219 @@
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, code text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_point(trim($1)) AS mypoint", ["text"])
rv = plpy.execute(plan, [code], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, code text, country text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_point(trim($1), trim($2)) AS mypoint", ["TEXT", "TEXT"])
rv = plpy.execute(plan, [code, country], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, code text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_polygon(trim($1)) AS mypolygon", ["text"])
rv = plpy.execute(plan, [code], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, code text, country text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_polygon(trim($1), trim($2)) AS mypolygon", ["TEXT", "TEXT"])
rv = plpy.execute(plan, [code, country], 1)
result = rv[0]["mypolygon"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_point(code text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_points
WHERE postal_code = upper(d.q)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_point(code text, country text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_points
WHERE postal_code = upper(d.q)
AND iso3 = (
SELECT iso3 FROM country_decoder WHERE
lower(country) = ANY (synonyms) LIMIT 1
)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_polygon(code text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_polygons
WHERE postal_code = upper(d.q)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_polygon(code text, country text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
BEGIN
SELECT geom INTO ret
FROM (
SELECT
q, (
SELECT the_geom
FROM global_postal_code_polygons
WHERE postal_code = upper(d.q)
AND iso3 = (
SELECT iso3 FROM country_decoder WHERE
lower(country) = ANY (synonyms) LIMIT 1
)
LIMIT 1
) geom
FROM (SELECT code q) d
) v;
RETURN ret;
END
$$ LANGUAGE plpgsql;

View File

@@ -0,0 +1,61 @@
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_ipaddress_point(username text, orgname text, ip text)
RETURNS Geometry AS $$
from cartodb_services.metrics import QuotaService
from cartodb_services.metrics import InternalGeocoderConfig
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_geocoder_config = InternalGeocoderConfig(redis_conn, username, orgname)
quota_service = QuotaService(user_geocoder_config, redis_conn)
try:
plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_ipaddress_point(trim($1)) AS mypoint", ["TEXT"])
rv = plpy.execute(plan, [ip], 1)
result = rv[0]["mypoint"]
if result:
quota_service.increment_success_geocoder_use()
return result
else:
quota_service.increment_empty_geocoder_use()
return None
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu;
--------------------------------------------------------------------------------
-- Implementation of the server extension
-- Note: these functions depend on the cdb_geocoder extension
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_ipaddress_point(ip text)
RETURNS Geometry AS $$
DECLARE
ret Geometry;
new_ip INET;
BEGIN
BEGIN
IF family(ip::inet) = 6 THEN
new_ip := ip::inet;
ELSE
new_ip := ('::ffff:' || ip)::inet;
END IF;
EXCEPTION WHEN OTHERS THEN
SELECT NULL as geom INTO ret;
RETURN ret;
END;
WITH
ips AS (SELECT ip s, new_ip net),
matches AS (SELECT s, (SELECT the_geom FROM ip_address_locations WHERE network_start_ip <= ips.net ORDER BY network_start_ip DESC LIMIT 1) geom FROM ips)
SELECT geom INTO ret
FROM matches;
RETURN ret;
END
$$ LANGUAGE plpgsql;

View File

@@ -0,0 +1,54 @@
CREATE TYPE cdb_dataservices_server.isoline AS (center geometry(Geometry,4326), data_range integer, the_geom geometry(Multipolygon,4326));
CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_routing_isolines(username TEXT, orgname TEXT, type TEXT, source geometry(Geometry, 4326), mode TEXT, data_range integer[], options text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
import json
from cartodb_services.here import HereMapsRoutingIsoline
from cartodb_services.metrics import QuotaService
from cartodb_services.here.types import geo_polyline_to_multipolygon
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)]
# -- Check the quota
quota_service = QuotaService(user_isolines_routing_config, redis_conn)
if not quota_service.check_user_quota():
plpy.error('You have reach the limit of your quota')
try:
client = HereMapsRoutingIsoline(user_isolines_routing_config.heremaps_app_id, user_isolines_routing_config.heremaps_app_code, base_url = HereMapsRoutingIsoline.PRODUCTION_ROUTING_BASE_URL)
if source:
lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat']
lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon']
source_str = 'geo!%f,%f' % (lat, lon)
else:
source_str = None
if type == 'isodistance':
resp = client.calculate_isodistance(source_str, mode, data_range, options)
elif type == 'isochrone':
resp = client.calculate_isochrone(source_str, mode, data_range, options)
if resp:
result = []
for isoline in resp:
data_range_n = isoline['range']
polyline = isoline['geom']
multipolygon = geo_polyline_to_multipolygon(polyline)
result.append([source, data_range_n, multipolygon])
quota_service.increment_success_geocoder_use()
quota_service.increment_isolines_service_use(len(resp))
return result
else:
quota_service.increment_empty_geocoder_use()
except BaseException as e:
import sys, traceback
type_, value_, traceback_ = sys.exc_info()
quota_service.increment_failed_geocoder_use()
error_msg = 'There was an error trying to obtain isodistances using here maps geocoder: {0}'.format(e)
plpy.notice(traceback.format_tb(traceback_))
plpy.error(error_msg)
finally:
quota_service.increment_total_geocoder_use()
$$ LANGUAGE plpythonu SECURITY DEFINER;

View File

@@ -0,0 +1,21 @@
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isodistance(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)]
type = 'isodistance'
if user_isolines_config.google_services_user:
plpy.error('This service is not available for google service users.')
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"])
result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options])
isolines = []
for element in result:
isoline = element['isoline']
isoline = isoline.translate(None, "()").split(',')
isolines.append(isoline)
return isolines
$$ LANGUAGE plpythonu;

View File

@@ -0,0 +1,21 @@
CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isochrone(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[])
RETURNS SETOF cdb_dataservices_server.isoline AS $$
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)]
type = 'isochrone'
if user_isolines_config.google_services_user:
plpy.error('This service is not available for google service users.')
here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"])
result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options])
isolines = []
for element in result:
isoline = element['isoline']
isoline = isoline.translate(None, "()").split(',')
isolines.append(isoline)
return isolines
$$ LANGUAGE plpythonu;

View File

@@ -0,0 +1,15 @@
DO $$
BEGIN
IF NOT EXISTS (
SELECT *
FROM pg_catalog.pg_user
WHERE usename = 'geocoder_api') THEN
CREATE USER geocoder_api;
END IF;
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_server TO geocoder_api;
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO geocoder_api;
GRANT USAGE ON SCHEMA cdb_dataservices_server TO geocoder_api;
GRANT USAGE ON SCHEMA public TO geocoder_api;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO geocoder_api;
END$$;

View File

@@ -0,0 +1,36 @@
-- Install dependencies
CREATE EXTENSION postgis;
CREATE EXTENSION schema_triggers;
CREATE EXTENSION plpythonu;
CREATE EXTENSION cartodb;
CREATE EXTENSION cdb_geocoder;
-- Install the extension
CREATE EXTENSION cdb_dataservices_server;
-- Mock the redis server connection to point to this very test db
SELECT cartodb.cdb_conf_setconf('redis_metrics_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}');
cdb_conf_setconf
------------------
(1 row)
SELECT cartodb.cdb_conf_setconf('redis_metadata_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}');
cdb_conf_setconf
------------------
(1 row)
-- Mock the varnish invalidation function
-- (used by cdb_geocoder tests)
CREATE OR REPLACE FUNCTION public.cdb_invalidate_varnish(table_name text) RETURNS void AS $$
BEGIN
RETURN;
END
$$
LANGUAGE plpgsql;
-- Set user quota
SELECT cartodb.CDB_SetUserQuotaInBytes(0);
cdb_setuserquotainbytes
-------------------------
0
(1 row)

View File

@@ -0,0 +1,12 @@
-- Check for namedplaces signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_street_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text, text, text');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,47 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_admin0_polygon('test_user', 'test_orgname', 'Spain');
cdb_geocode_admin0_polygon
----------------------------
(1 row)
-- Insert some dummy synonym
INSERT INTO admin0_synonyms (name, adm0_a3) VALUES ('Spain', 'ESP');
-- Insert some dummy geometry to return
INSERT INTO ne_admin0_v3 (adm0_a3, the_geom) VALUES('ESP', ST_GeomFromText(
'POLYGON((-71.1031880899493 42.3152774590236,
-71.1031627617667 42.3152960829043,
-71.102923838298 42.3149156848307,
-71.1031880899493 42.3152774590236))',4326)
);
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_admin0_polygon('test_user', 'test_orgname', 'Spain');
cdb_geocode_admin0_polygon
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
0103000020E61000000100000004000000D0EA37A29AC651C00FD603035B284540FEFCFB379AC651C0C0503E9F5B284540FFDDDD4D96C651C033AC3B284F284540D0EA37A29AC651C00FD603035B284540
(1 row)
-- Check for admin0 signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_admin0_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_admin0_polygon'
AND oidvectortypes(p.proargtypes) = 'text');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,81 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California');
cdb_geocode_admin1_polygon
----------------------------
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California', 'United States');
cdb_geocode_admin1_polygon
----------------------------
(1 row)
-- Insert dummy data into country decoder table
INSERT INTO country_decoder (synonyms, iso3) VALUES (Array['united states'], 'USA');
-- Insert some dummy data and geometry to return
INSERT INTO global_province_polygons (synonyms, iso3, the_geom) VALUES (Array['california'], 'USA', ST_GeomFromText(
'POLYGON((-71.1031880899493 42.3152774590236,
-71.1031627617667 42.3152960829043,
-71.102923838298 42.3149156848307,
-71.1031880899493 42.3152774590236))',4326)
);
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California');
cdb_geocode_admin1_polygon
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
0103000020E61000000100000004000000D0EA37A29AC651C00FD603035B284540FEFCFB379AC651C0C0503E9F5B284540FFDDDD4D96C651C033AC3B284F284540D0EA37A29AC651C00FD603035B284540
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California', 'United States');
cdb_geocode_admin1_polygon
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
0103000020E61000000100000004000000D0EA37A29AC651C00FD603035B284540FEFCFB379AC651C0C0503E9F5B284540FFDDDD4D96C651C033AC3B284F284540D0EA37A29AC651C00FD603035B284540
(1 row)
-- Check for admin1 signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,136 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx');
cdb_geocode_namedplace_point
------------------------------
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Spain');
cdb_geocode_namedplace_point
------------------------------
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Valencia', 'Spain');
cdb_geocode_namedplace_point
------------------------------
(1 row)
-- Insert dummy data into points table
INSERT INTO global_cities_points_limited (geoname_id, name, iso2, admin1, admin2, population, lowername, the_geom) VALUES (3128760, 'Elche', 'ES', 'Valencia', 'AL', 34534, 'elche', ST_GeomFromText(
'POINT(0.6983 39.26787)',4326)
);
-- Insert dummy data into alternates table
INSERT INTO global_cities_alternates_limited (geoname_id, name, preferred, lowername, admin1_geonameid, iso2, admin1, the_geom) VALUES (3128760, 'Elx', true, 'elx', '000000', 'ES', 'Valencia', ST_GeomFromText(
'POINT(0.6983 39.26787)',4326)
);
-- Insert dummy data into country decoder table
INSERT INTO country_decoder (synonyms, iso2) VALUES (Array['spain'], 'ES');
-- Insert dummy data into admin1 decoder table
INSERT INTO admin1_decoder (admin1, synonyms, iso2) VALUES ('Valencia', Array['valencia', 'Valencia'], 'ES');
-- This should return the point inserted above
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx');
cdb_geocode_namedplace_point
----------------------------------------------------
0101000020E6100000637FD93D7958E63F2ECA6C9049A24340
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elche');
cdb_geocode_namedplace_point
----------------------------------------------------
0101000020E6100000637FD93D7958E63F2ECA6C9049A24340
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Spain');
cdb_geocode_namedplace_point
----------------------------------------------------
0101000020E6100000637FD93D7958E63F2ECA6C9049A24340
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elche', 'Spain');
cdb_geocode_namedplace_point
----------------------------------------------------
0101000020E6100000637FD93D7958E63F2ECA6C9049A24340
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Valencia', 'Spain');
cdb_geocode_namedplace_point
----------------------------------------------------
0101000020E6100000637FD93D7958E63F2ECA6C9049A24340
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elche', 'valencia', 'Spain');
cdb_geocode_namedplace_point
----------------------------------------------------
0101000020E6100000637FD93D7958E63F2ECA6C9049A24340
(1 row)
-- Check for namedplaces signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,163 @@
-- Make sure dbs are clean
DELETE FROM global_postal_code_points;
DELETE FROM global_postal_code_polygons;
DELETE FROM country_decoder;
DELETE FROM available_services;
DELETE FROM admin0_synonyms;
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_postalcode_point('test_user', 'test_org', '03204');
cdb_geocode_postalcode_point
------------------------------
(1 row)
-- Insert dummy data into ip_address_locations
INSERT INTO global_postal_code_points (the_geom, iso3, postal_code, postal_code_num) VALUES (
'0101000020E61000000000000000E040408036B47414764840',
'ESP',
'03204',
3204
);
INSERT INTO global_postal_code_polygons (the_geom, iso3, postal_code, postal_code_num) VALUES (
'0106000020E610000001000000010300000001000000040000000000000000E000C01F383D7839B740400000000000E000C0AA3C0EDE220F3B4000000000004812404FB7FCCD04893D400000000000E000C01F383D7839B74040',
'ESP',
'03204',
3204
);
INSERT INTO country_decoder (iso3, synonyms) VALUES (
'ESP',
Array['spain', 'Spain', 'ESP']
);
INSERT INTO available_services (adm0_a3, admin0, postal_code_points, postal_code_polygons) VALUES (
'ESP',
't',
't',
't'
);
INSERT INTO admin0_synonyms (adm0_a3, name, name_, rank) VALUES (
'ESP',
'Spain',
'spain',
3
);
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_postalcode_point('test_user', 'test_org', '03204');
cdb_geocode_postalcode_point
----------------------------------------------------
0101000020E61000000000000000E040408036B47414764840
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_postalcode_point('test_user', 'test_org', '03204', 'spain');
cdb_geocode_postalcode_point
----------------------------------------------------
0101000020E61000000000000000E040408036B47414764840
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_postalcode_polygon('test_user', 'test_org', '03204');
cdb_geocode_postalcode_polygon
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0106000020E610000001000000010300000001000000040000000000000000E000C01F383D7839B740400000000000E000C0AA3C0EDE220F3B4000000000004812404FB7FCCD04893D400000000000E000C01F383D7839B74040
(1 row)
SELECT cdb_dataservices_server.cdb_geocode_postalcode_polygon('test_user', 'test_org', '03204', 'spain');
cdb_geocode_postalcode_polygon
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0106000020E610000001000000010300000001000000040000000000000000E000C01F383D7839B740400000000000E000C0AA3C0EDE220F3B4000000000004812404FB7FCCD04893D400000000000E000C01F383D7839B74040
(1 row)
-- Clean dbs
DELETE FROM global_postal_code_points;
DELETE FROM global_postal_code_polygons;
DELETE FROM country_decoder;
DELETE FROM available_services;
DELETE FROM admin0_synonyms;
-- Check for namedplaces signatures (point and polygon)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,40 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_ipaddress_point('test_user', 'test_orgname', '0.0.0.0');
cdb_geocode_ipaddress_point
-----------------------------
(1 row)
-- Insert dummy data into ip_address_locations
INSERT INTO ip_address_locations VALUES ('::ffff:0.0.0.0'::inet, (ST_SetSRID(ST_MakePoint('40.40', '3.71'), 4326)));
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_ipaddress_point('test_user', 'test_orgname', '0.0.0.0');
cdb_geocode_ipaddress_point
----------------------------------------------------
0101000020E61000003333333333334440AE47E17A14AE0D40
(1 row)
-- Check for namedplaces signatures (point and polygon)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_ipaddress_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
exists
--------
t
(1 row)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_ipaddress_point'
AND oidvectortypes(p.proargtypes) = 'text');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,12 @@
-- Check for isodistance signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_isodistance'
AND oidvectortypes(p.proargtypes) = 'text, text, geometry, text, integer[], text[]');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,12 @@
-- Check for isochrone signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_isochrone'
AND oidvectortypes(p.proargtypes) = 'text, text, geometry, text, integer[], text[]');
exists
--------
t
(1 row)

View File

@@ -0,0 +1,5 @@
REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_server FROM geocoder_api;
REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA public FROM geocoder_api;
REVOKE USAGE ON SCHEMA cdb_dataservices_server FROM geocoder_api;
REVOKE USAGE ON SCHEMA public FROM geocoder_api;
REVOKE SELECT ON ALL TABLES IN SCHEMA public FROM geocoder_api;

View File

@@ -0,0 +1,25 @@
-- Install dependencies
CREATE EXTENSION postgis;
CREATE EXTENSION schema_triggers;
CREATE EXTENSION plpythonu;
CREATE EXTENSION cartodb;
CREATE EXTENSION cdb_geocoder;
-- Install the extension
CREATE EXTENSION cdb_dataservices_server;
-- Mock the redis server connection to point to this very test db
SELECT cartodb.cdb_conf_setconf('redis_metrics_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}');
SELECT cartodb.cdb_conf_setconf('redis_metadata_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}');
-- Mock the varnish invalidation function
-- (used by cdb_geocoder tests)
CREATE OR REPLACE FUNCTION public.cdb_invalidate_varnish(table_name text) RETURNS void AS $$
BEGIN
RETURN;
END
$$
LANGUAGE plpgsql;
-- Set user quota
SELECT cartodb.CDB_SetUserQuotaInBytes(0);

View File

@@ -0,0 +1,7 @@
-- Check for namedplaces signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_street_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text, text, text');

View File

@@ -0,0 +1,32 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_admin0_polygon('test_user', 'test_orgname', 'Spain');
-- Insert some dummy synonym
INSERT INTO admin0_synonyms (name, adm0_a3) VALUES ('Spain', 'ESP');
-- Insert some dummy geometry to return
INSERT INTO ne_admin0_v3 (adm0_a3, the_geom) VALUES('ESP', ST_GeomFromText(
'POLYGON((-71.1031880899493 42.3152774590236,
-71.1031627617667 42.3152960829043,
-71.102923838298 42.3149156848307,
-71.1031880899493 42.3152774590236))',4326)
);
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_admin0_polygon('test_user', 'test_orgname', 'Spain');
-- Check for admin0 signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_admin0_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_admin0_polygon'
AND oidvectortypes(p.proargtypes) = 'text');

View File

@@ -0,0 +1,48 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California');
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California', 'United States');
-- Insert dummy data into country decoder table
INSERT INTO country_decoder (synonyms, iso3) VALUES (Array['united states'], 'USA');
-- Insert some dummy data and geometry to return
INSERT INTO global_province_polygons (synonyms, iso3, the_geom) VALUES (Array['california'], 'USA', ST_GeomFromText(
'POLYGON((-71.1031880899493 42.3152774590236,
-71.1031627617667 42.3152960829043,
-71.102923838298 42.3149156848307,
-71.1031880899493 42.3152774590236))',4326)
);
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California');
SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon('test_user', 'test_orgname', 'California', 'United States');
-- Check for admin1 signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_admin1_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text');

View File

@@ -0,0 +1,72 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx');
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Spain');
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Valencia', 'Spain');
-- Insert dummy data into points table
INSERT INTO global_cities_points_limited (geoname_id, name, iso2, admin1, admin2, population, lowername, the_geom) VALUES (3128760, 'Elche', 'ES', 'Valencia', 'AL', 34534, 'elche', ST_GeomFromText(
'POINT(0.6983 39.26787)',4326)
);
-- Insert dummy data into alternates table
INSERT INTO global_cities_alternates_limited (geoname_id, name, preferred, lowername, admin1_geonameid, iso2, admin1, the_geom) VALUES (3128760, 'Elx', true, 'elx', '000000', 'ES', 'Valencia', ST_GeomFromText(
'POINT(0.6983 39.26787)',4326)
);
-- Insert dummy data into country decoder table
INSERT INTO country_decoder (synonyms, iso2) VALUES (Array['spain'], 'ES');
-- Insert dummy data into admin1 decoder table
INSERT INTO admin1_decoder (admin1, synonyms, iso2) VALUES ('Valencia', Array['valencia', 'Valencia'], 'ES');
-- This should return the point inserted above
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx');
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elche');
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Spain');
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elche', 'Spain');
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elx', 'Valencia', 'Spain');
SELECT cdb_dataservices_server.cdb_geocode_namedplace_point('test_user', 'test_orgname', 'Elche', 'valencia', 'Spain');
-- Check for namedplaces signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_namedplace_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');

View File

@@ -0,0 +1,117 @@
-- Make sure dbs are clean
DELETE FROM global_postal_code_points;
DELETE FROM global_postal_code_polygons;
DELETE FROM country_decoder;
DELETE FROM available_services;
DELETE FROM admin0_synonyms;
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_postalcode_point('test_user', 'test_org', '03204');
-- Insert dummy data into ip_address_locations
INSERT INTO global_postal_code_points (the_geom, iso3, postal_code, postal_code_num) VALUES (
'0101000020E61000000000000000E040408036B47414764840',
'ESP',
'03204',
3204
);
INSERT INTO global_postal_code_polygons (the_geom, iso3, postal_code, postal_code_num) VALUES (
'0106000020E610000001000000010300000001000000040000000000000000E000C01F383D7839B740400000000000E000C0AA3C0EDE220F3B4000000000004812404FB7FCCD04893D400000000000E000C01F383D7839B74040',
'ESP',
'03204',
3204
);
INSERT INTO country_decoder (iso3, synonyms) VALUES (
'ESP',
Array['spain', 'Spain', 'ESP']
);
INSERT INTO available_services (adm0_a3, admin0, postal_code_points, postal_code_polygons) VALUES (
'ESP',
't',
't',
't'
);
INSERT INTO admin0_synonyms (adm0_a3, name, name_, rank) VALUES (
'ESP',
'Spain',
'spain',
3
);
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_postalcode_point('test_user', 'test_org', '03204');
SELECT cdb_dataservices_server.cdb_geocode_postalcode_point('test_user', 'test_org', '03204', 'spain');
SELECT cdb_dataservices_server.cdb_geocode_postalcode_polygon('test_user', 'test_org', '03204');
SELECT cdb_dataservices_server.cdb_geocode_postalcode_polygon('test_user', 'test_org', '03204', 'spain');
-- Clean dbs
DELETE FROM global_postal_code_points;
DELETE FROM global_postal_code_polygons;
DELETE FROM country_decoder;
DELETE FROM available_services;
DELETE FROM admin0_synonyms;
-- Check for namedplaces signatures (point and polygon)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_point'
AND oidvectortypes(p.proargtypes) = 'text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_postalcode_polygon'
AND oidvectortypes(p.proargtypes) = 'text, text');

View File

@@ -0,0 +1,24 @@
-- Check that the public function is callable, even with no data
-- It should return NULL
SELECT cdb_dataservices_server.cdb_geocode_ipaddress_point('test_user', 'test_orgname', '0.0.0.0');
-- Insert dummy data into ip_address_locations
INSERT INTO ip_address_locations VALUES ('::ffff:0.0.0.0'::inet, (ST_SetSRID(ST_MakePoint('40.40', '3.71'), 4326)));
-- This should return the polygon inserted above
SELECT cdb_dataservices_server.cdb_geocode_ipaddress_point('test_user', 'test_orgname', '0.0.0.0');
-- Check for namedplaces signatures (point and polygon)
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_geocode_ipaddress_point'
AND oidvectortypes(p.proargtypes) = 'text, text, text');
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = '_cdb_geocode_ipaddress_point'
AND oidvectortypes(p.proargtypes) = 'text');

View File

@@ -0,0 +1,7 @@
-- Check for isodistance signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_isodistance'
AND oidvectortypes(p.proargtypes) = 'text, text, geometry, text, integer[], text[]');

View File

@@ -0,0 +1,7 @@
-- Check for isochrone signatures
SELECT exists(SELECT *
FROM pg_proc p
INNER JOIN pg_namespace ns ON (p.pronamespace = ns.oid)
WHERE ns.nspname = 'cdb_dataservices_server'
AND proname = 'cdb_isochrone'
AND oidvectortypes(p.proargtypes) = 'text, text, geometry, text, integer[], text[]');

View File

@@ -0,0 +1,5 @@
REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_server FROM geocoder_api;
REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA public FROM geocoder_api;
REVOKE USAGE ON SCHEMA cdb_dataservices_server FROM geocoder_api;
REVOKE USAGE ON SCHEMA public FROM geocoder_api;
REVOKE SELECT ON ALL TABLES IN SCHEMA public FROM geocoder_api;

View File

@@ -1 +1,2 @@
from geocoder import HereMapsGeocoder
from routing import HereMapsRoutingIsoline

View File

@@ -11,6 +11,14 @@ class BadGeocodingParams(Exception):
return repr('Bad geocoding params: ' + json.dumps(self.value))
class WrongParams(Exception):
def __init__(self, value):
self.value = value
def __str__(self):
return repr('Wrong parameters passed: ' + json.dumps(self.value))
class NoGeocodingParams(Exception):
def __str__(self):
return repr('No params for geocoding specified')

View File

@@ -0,0 +1,119 @@
import requests
import json
from exceptions import WrongParams
class HereMapsRoutingIsoline:
'A Here Maps Routing wrapper for python'
PRODUCTION_ROUTING_BASE_URL = 'https://isoline.route.api.here.com'
STAGING_ROUTING_BASE_URL = 'https://isoline.route.cit.api.here.com'
ISOLINE_PATH = '/routing/7.2/calculateisoline.json'
ACCEPTED_MODES = {
"walk": "pedestrian",
"car": "car"
}
OPTIONAL_PARAMS = [
'departure',
'arrival',
'singlecomponent',
'resolution',
'maxpoints',
'quality'
]
def __init__(self, app_id, app_code, base_url=PRODUCTION_ROUTING_BASE_URL):
self._app_id = app_id
self._app_code = app_code
self._url = "{0}{1}".format(base_url, self.ISOLINE_PATH)
def calculate_isodistance(self, source, mode, data_range, options=[]):
return self.__calculate_isolines(source, mode, data_range, 'distance',
options)
def calculate_isochrone(self, source, mode, data_range, options=[]):
return self.__calculate_isolines(source, mode, data_range, 'time',
options)
def __calculate_isolines(self, source, mode, data_range, range_type,
options=[]):
parsed_options = self.__parse_options(options)
source_param = self.__parse_source_param(source, parsed_options)
mode_param = self.__parse_mode_param(mode, parsed_options)
request_params = self.__parse_request_parameters(source_param,
mode_param,
data_range,
range_type,
parsed_options)
response = requests.get(self._url, params=request_params)
if response.status_code == requests.codes.ok:
return self.__parse_isolines_response(response.text)
else:
response.raise_for_status()
def __parse_options(self, options):
return dict(option.split('=') for option in options)
def __parse_request_parameters(self, source, mode, data_range, range_type,
options):
filtered_options = {k: v for k, v in options.iteritems()
if k.lower() in self.OPTIONAL_PARAMS}
filtered_options.update(source)
filtered_options.update(mode)
filtered_options.update({'range': ",".join(map(str, data_range))})
filtered_options.update({'rangetype': range_type})
filtered_options.update({'app_id': self._app_id})
filtered_options.update({'app_code': self._app_code})
return filtered_options
def __parse_isolines_response(self, response):
parsed_response = json.loads(response)
isolines_response = parsed_response['response']['isoline']
isolines = []
for isoline in isolines_response:
isolines.append({'range': isoline['range'],
'geom': isoline['component'][0]['shape']})
return isolines
def __parse_source_param(self, source, options):
key = 'start'
if 'is_destination' in options and options['is_destination']:
key = 'destination'
return {key: source}
def __parse_mode_param(self, mode, options):
if mode in self.ACCEPTED_MODES:
mode_source = self.ACCEPTED_MODES[mode]
else:
raise WrongParams("{0} is not an accepted mode type".format(mode))
if 'mode_type' in options:
mode_type = options['mode_type']
else:
mode_type = 'shortest'
if 'mode_traffic' in options:
mode_traffic = "traffic:{0}".format(options['mode_traffic'])
else:
mode_traffic = None
if 'mode_feature' in options and 'mode_feature_weight' in options:
mode_feature = "{0}:{1}".format(options['mode_feature'],
options['mode_feature_weight'])
else:
mode_feature = None
mode_param = "{0};{1}".format(mode_type, mode_source)
if mode_traffic:
mode_param = "{0};{1}".format(mode_param, mode_traffic)
if mode_feature:
mode_param = "{0};{1}".format(mode_param, mode_feature)
return {'mode': mode_param}

View File

@@ -0,0 +1,16 @@
# Helper to deal with type conversion between HERE and PostGIS
import plpy
def geo_polyline_to_multipolygon(polyline):
"""Convert a HERE polyline shape to a PostGIS multipolygon"""
coordinates = []
for point in polyline:
lat, lon = point.split(',')
coordinates.append("%s %s" % (lon, lat))
wkt_coordinates = ','.join(coordinates)
sql = "SELECT ST_MPolyFromText('MULTIPOLYGON((({0})))', 4326) as geom".format(wkt_coordinates)
geometry = plpy.execute(sql, 1)[0]['geom']
return geometry

View File

@@ -1,3 +1,3 @@
from config import GeocoderConfig, InternalGeocoderConfig, ConfigException
from config import GeocoderConfig, IsolinesRoutingConfig, InternalGeocoderConfig, ConfigException
from quota import QuotaService
from user import UserMetricsService

View File

@@ -28,6 +28,93 @@ class ServiceConfig(object):
return self._orgname
class IsolinesRoutingConfig(ServiceConfig):
ROUTING_CONFIG_KEYS = ['here_isolines_quota', 'soft_here_isolines_limit',
'period_end_date', 'username', 'orgname',
'heremaps_app_id', 'heremaps_app_code', 'geocoder_type']
NOKIA_APP_ID_KEY = 'heremaps_app_id'
NOKIA_APP_CODE_KEY = 'heremaps_app_code'
QUOTA_KEY = 'here_isolines_quota'
SOFT_LIMIT_KEY = 'soft_here_isolines_limit'
USERNAME_KEY = 'username'
ORGNAME_KEY = 'orgname'
PERIOD_END_DATE = 'period_end_date'
GEOCODER_TYPE_KEY = 'geocoder_type'
GOOGLE_GEOCODER = 'google'
def __init__(self, redis_connection, username, orgname=None,
heremaps_app_id=None, heremaps_app_code=None):
super(IsolinesRoutingConfig, self).__init__(redis_connection, username,
orgname)
config = self.__get_user_config(username, orgname, heremaps_app_id,
heremaps_app_code)
filtered_config = {key: config[key] for key in self.ROUTING_CONFIG_KEYS if key in config.keys()}
self.__parse_config(filtered_config)
def __get_user_config(self, username, orgname=None, heremaps_app_id=None,
heremaps_app_code=None):
user_config = self._redis_connection.hgetall(
"rails:users:{0}".format(username))
if not user_config:
raise ConfigException("""There is no user config available. Please check your configuration.'""")
else:
user_config[self.NOKIA_APP_ID_KEY] = heremaps_app_id
user_config[self.NOKIA_APP_CODE_KEY] = heremaps_app_code
if orgname:
self.__get_organization_config(orgname, user_config)
return user_config
def __get_organization_config(self, orgname, user_config):
org_config = self._redis_connection.hgetall(
"rails:orgs:{0}".format(orgname))
if not org_config:
raise ConfigException("""There is no organization config available. Please check your configuration.'""")
else:
user_config[self.QUOTA_KEY] = org_config[self.QUOTA_KEY]
user_config[self.PERIOD_END_DATE] = org_config[self.PERIOD_END_DATE]
def __parse_config(self, filtered_config):
self._geocoder_type = filtered_config[self.GEOCODER_TYPE_KEY].lower()
self._isolines_quota = float(filtered_config[self.QUOTA_KEY])
self._period_end_date = date_parse(filtered_config[self.PERIOD_END_DATE])
if filtered_config[self.SOFT_LIMIT_KEY].lower() == 'true':
self._soft_isolines_limit = True
else:
self._soft_isolines_limit = False
self._heremaps_app_id = filtered_config[self.NOKIA_APP_ID_KEY]
self._heremaps_app_code = filtered_config[self.NOKIA_APP_CODE_KEY]
@property
def service_type(self):
return 'here_isolines'
@property
def isolines_quota(self):
return self._isolines_quota
@property
def soft_isolines_limit(self):
return self._soft_isolines_limit
@property
def period_end_date(self):
return self._period_end_date
@property
def heremaps_app_id(self):
return self._heremaps_app_id
@property
def heremaps_app_code(self):
return self._heremaps_app_code
@property
def google_services_user(self):
return self._geocoder_type == self.GOOGLE_GEOCODER
class InternalGeocoderConfig(ServiceConfig):
def __init__(self, redis_connection, username, orgname=None):

View File

@@ -1,49 +1,97 @@
from user import UserMetricsService
from datetime import date
import re
class QuotaService:
""" Class to manage all the quota operation for
the Geocoder SQL API Extension """
def __init__(self, user_geocoder_config, redis_connection):
self._user_geocoder_config = user_geocoder_config
def __init__(self, user_service_config, redis_connection):
self._user_service_config = user_service_config
# TODO First step to extract to a factory if needed in the future
self._quota_checker = QuotaChecker(user_service_config,
redis_connection)
self._user_service = UserMetricsService(
self._user_geocoder_config, redis_connection)
self._user_service_config, redis_connection)
def check_user_quota(self):
""" Check if the current user quota surpasses the current quota """
# We don't have quota check for google geocoder
if self._user_geocoder_config.google_geocoder:
return True
return self._quota_checker.check()
user_quota = self._user_geocoder_config.geocoding_quota
today = date.today()
service_type = self._user_geocoder_config.service_type
current_used = self._user_service.used_quota(service_type, today)
soft_geocoding_limit = self._user_geocoder_config.soft_geocoding_limit
if soft_geocoding_limit or current_used <= user_quota:
return True
else:
return False
# TODO
# We are going to change this class to be the generic one and
# create specific for routing and geocoding services but because
# this implies change all the extension functions, we are going to
# make the change in a minor release
def increment_success_geocoder_use(self, amount=1):
self._user_service.increment_service_use(
self._user_geocoder_config.service_type, "success_responses",
self._user_service_config.service_type, "success_responses",
amount=amount)
def increment_empty_geocoder_use(self, amount=1):
self._user_service.increment_service_use(
self._user_geocoder_config.service_type, "empty_responses",
self._user_service_config.service_type, "empty_responses",
amount=amount)
def increment_failed_geocoder_use(self, amount=1):
self._user_service.increment_service_use(
self._user_geocoder_config.service_type, "fail_responses",
self._user_service_config.service_type, "fail_responses",
amount=amount)
def increment_total_geocoder_use(self, amount=1):
self._user_service.increment_service_use(
self._user_geocoder_config.service_type, "total_requests",
self._user_service_config.service_type, "total_requests",
amount=amount)
def increment_isolines_service_use(self, amount=1):
self._user_service.increment_service_use(
self._user_service_config.service_type, "isolines_generated",
amount=amount)
class QuotaChecker:
def __init__(self, user_service_config, redis_connection):
self._user_service_config = user_service_config
self._user_service = UserMetricsService(
self._user_service_config, redis_connection)
def check(self):
""" Check if the current user quota surpasses the current quota """
if re.match('geocoder_*',
self._user_service_config.service_type) is not None:
return self.__check_geocoder_quota()
elif re.match('here_isolines',
self._user_service_config.service_type) is not None:
return self.__check_isolines_quota()
else:
return False
def __check_geocoder_quota(self):
# We don't have quota check for google geocoder
if self._user_service_config.google_geocoder:
return True
user_quota = self._user_service_config.geocoding_quota
today = date.today()
service_type = self._user_service_config.service_type
current_used = self._user_service.used_quota(service_type, today)
soft_geocoding_limit = self._user_service_config.soft_geocoding_limit
if soft_geocoding_limit or (user_quota > 0 and current_used <= user_quota):
return True
else:
return False
def __check_isolines_quota(self):
user_quota = self._user_service_config.isolines_quota
today = date.today()
service_type = self._user_service_config.service_type
current_used = self._user_service.used_quota(service_type, today)
soft_isolines_limit = self._user_service_config.soft_isolines_limit
if soft_isolines_limit or (user_quota > 0 and current_used <= user_quota):
return True
else:
return False

View File

@@ -6,16 +6,8 @@ class UserMetricsService:
""" Class to manage all the user info """
SERVICE_GEOCODER_NOKIA = 'geocoder_here'
SERVICE_GEOCODER_GOOGLE = 'geocoder_google'
SERVICE_GEOCODER_CACHE = 'geocoder_cache'
GEOCODING_QUOTA_KEY = "geocoding_quota"
GEOCODING_SOFT_LIMIT_KEY = "soft_geocoder_limit"
REDIS_CONNECTION_KEY = "redis_connection"
REDIS_CONNECTION_HOST = "redis_host"
REDIS_CONNECTION_PORT = "redis_port"
REDIS_CONNECTION_DB = "redis_db"
SERVICE_HERE_ISOLINES = 'here_isolines'
def __init__(self, user_geocoder_config, redis_connection):
self._user_geocoder_config = user_geocoder_config
@@ -24,6 +16,12 @@ class UserMetricsService:
self._orgname = user_geocoder_config.organization
def used_quota(self, service_type, date):
if service_type == self.SERVICE_HERE_ISOLINES:
return self.__used_isolines_quota(service_type, date)
else:
return self.__used_geocoding_quota(service_type, date)
def __used_geocoding_quota(self, service_type, date):
""" Recover the used quota for the user in the current month """
date_from, date_to = self.__current_billing_cycle()
current_use = 0
@@ -42,6 +40,20 @@ class UserMetricsService:
return current_use
def __used_isolines_quota(self, service_type, date):
date_from, date_to = self.__current_billing_cycle()
current_use = 0
isolines_generated = self.get_metrics(service_type,
'isolines_generated', date_from,
date_to)
empty_responses = self.get_metrics(service_type,
'empty_responses', date_from,
date_to)
current_use += (isolines_generated + empty_responses)
return current_use
def increment_service_use(self, service_type, metric, date=date.today(), amount=1):
""" Increment the services uses in monthly and daily basis"""
self.__increment_user_uses(service_type, metric, date, amount)

View File

@@ -10,7 +10,7 @@ from setuptools import setup, find_packages
setup(
name='cartodb_services',
version='0.1.0',
version='0.2.0',
description='CartoDB Services API Python Library',

View File

@@ -23,68 +23,81 @@ class HereMapsGeocoderTestCase(unittest.TestCase):
}
}"""
GOOD_RESPONSE = """{
"Response": {
"MetaInfo": {
"Timestamp": "2015-11-04T16:30:32.187+0000"
},
"View": [{
"_type": "SearchResultsViewType",
"ViewId": 0,
"Result": {
"Relevance": 0.89,
"MatchLevel": "street",
"MatchQuality": {
"City": 1.0,
"Street": [1.0]
},
"Location": {
"LocationId": "NT_yyKB4r3mCWAX4voWgxPcuA",
"LocationType": "address",
"DisplayPosition": {
"Latitude": 40.43433,
"Longitude": -3.70126
},
"NavigationPosition": [{
"Latitude": 40.43433,
"Longitude": -3.70126
}],
"MapView": {
"TopLeft": {
"Latitude": 40.43493,
"Longitude": -3.70404
},
"BottomRight": {
"Latitude": 40.43373,
"Longitude": -3.69873
}
},
"Address": {
"Label": "Calle de Eloy Gonzalo, Madrid, Espana",
"Country": "ESP",
"State": "Comunidad de Madrid",
"County": "Madrid",
"City": "Madrid",
"District": "Trafalgar",
"Street": "Calle de Eloy Gonzalo",
"AdditionalData": [{
"value": "Espana",
"key": "CountryName"
},
{
"value": "Comunidad de Madrid",
"key": "StateName"
},
{
"value": "Madrid",
"key": "CountyName"
}]
}
GOOD_RESPONSE = unicode("""{
"Response": {
"MetaInfo": {
"Timestamp": "2016-02-10T14:17:33.792+0000",
"NextPageInformation": "2"
},
"View": [
{
"_type": "SearchResultsViewType",
"ViewId": 0,
"Result": [
{
"Relevance": 1,
"MatchLevel": "houseNumber",
"MatchQuality": {
"Street": [
1
],
"HouseNumber": 1
},
"MatchType": "pointAddress",
"Location": {
"LocationId": "NT_CKopMSB9JnBYAO11CMOrxB_zUD",
"LocationType": "address",
"DisplayPosition": {
"Latitude": 37.70246,
"Longitude": -5.2794
},
"NavigationPosition": [
{
"Latitude": 37.7024199,
"Longitude": -5.27939
}
],
"MapView": {
"TopLeft": {
"Latitude": 37.7035842,
"Longitude": -5.2808208
},
"BottomRight": {
"Latitude": 37.7013358,
"Longitude": -5.2779792
}
},
"Address": {
"Label": "Calle Amor de Dios, 35, 14700 Palma del Río (Córdoba), España",
"Country": "ESP",
"State": "Andalucía",
"County": "Córdoba",
"City": "Palma del Río",
"Street": "Calle Amor de Dios",
"HouseNumber": "35",
"PostalCode": "14700",
"AdditionalData": [
{
"value": "España",
"key": "CountryName"
},
{
"value": "Andalucía",
"key": "StateName"
},
{
"value": "Córdoba",
"key": "CountyName"
}
]
}
}
}]
}
}"""
}
]
}
]
}
}""", 'utf-8')
MALFORMED_RESPONSE = """{"manolo": "escobar"}"""
@@ -95,12 +108,12 @@ class HereMapsGeocoderTestCase(unittest.TestCase):
req_mock.register_uri('GET', HereMapsGeocoder.PRODUCTION_GEOCODE_JSON_URL,
text=self.GOOD_RESPONSE)
response = self.geocoder.geocode(
searchtext='Calle Eloy Gonzalo 27',
city='Madrid',
searchtext='Calle amor de dios',
city='Cordoba',
country='España')
self.assertEqual(response[0], -3.70126)
self.assertEqual(response[1], 40.43433)
self.assertEqual(response[0], -5.2794)
self.assertEqual(response[1], 37.70246)
def test_geocode_address_with_invalid_params(self, req_mock):
req_mock.register_uri('GET', HereMapsGeocoder.PRODUCTION_GEOCODE_JSON_URL,
@@ -127,6 +140,6 @@ class HereMapsGeocoderTestCase(unittest.TestCase):
text=self.MALFORMED_RESPONSE)
with self.assertRaises(MalformedResult):
self.geocoder.geocode(
searchtext='Calle Eloy Gonzalo 27',
city='Madrid',
searchtext='Calle amor de dios',
city='Cordoba',
country='España')

View File

@@ -0,0 +1,214 @@
#!/usr/local/bin/python
# -*- coding: utf-8 -*-
import unittest
import requests_mock
from urlparse import urlparse, parse_qs
from cartodb_services.here import HereMapsRoutingIsoline
from cartodb_services.here.exceptions import BadGeocodingParams
from cartodb_services.here.exceptions import NoGeocodingParams
from cartodb_services.here.exceptions import MalformedResult
requests_mock.Mocker.TEST_PREFIX = 'test_'
@requests_mock.Mocker()
class HereMapsRoutingIsolineTestCase(unittest.TestCase):
EMPTY_RESPONSE = """{
"response": {
"metaInfo": {
"timestamp": "2016-02-10T10:42:21Z",
"mapVersion": "8.30.61.107",
"moduleVersion": "7.2.65.0-1222",
"interfaceVersion": "2.6.20"
},
"center": {
"latitude": 33,
"longitude": 0.9999999
},
"isoline": [
{
"range": 1000,
"component": [
{
"id": 0,
"shape": []
}
]
}
],
"start": {
"linkId": "+1025046831",
"mappedPosition": {
"latitude": 32.968725,
"longitude": 0.9993629
},
"originalPosition": {
"latitude": 33,
"longitude": 0.9999999
}
}
}
}"""
ERROR_RESPONSE = """{
"_type": "ns2:RoutingServiceErrorType",
"type": "ApplicationError",
"subtype": "InitIsolineSearchFailed",
"details": "Error is NGEO_ERROR_UNKNOWN",
"additionalData": [
{
"key": "error_code",
"value": "NGEO_ERROR_UNKNOWN"
}
],
"metaInfo": {
"timestamp": "2016-02-10T10:39:35Z",
"mapVersion": "8.30.61.107",
"moduleVersion": "7.2.65.0-1222",
"interfaceVersion": "2.6.20"
}
}"""
GOOD_RESPONSE = """{
"response": {
"metaInfo": {
"timestamp": "2016-02-10T10:42:21Z",
"mapVersion": "8.30.61.107",
"moduleVersion": "7.2.65.0-1222",
"interfaceVersion": "2.6.20"
},
"center": {
"latitude": 33,
"longitude": 0.9999999
},
"isoline": [
{
"range": 1000,
"component": [
{
"id": 0,
"shape": [
"32.9699707,0.9462833",
"32.9699707,0.9458542",
"32.9699707,0.9462833"
]
}
]
}, {
"range": 2000,
"component": [
{
"id": 0,
"shape": [
"32.9699707,0.9462833",
"32.9699707,0.9750366",
"32.9699707,0.9462833"
]
}
]
}
],
"start": {
"linkId": "+1025046831",
"mappedPosition": {
"latitude": 32.968725,
"longitude": 0.9993629
},
"originalPosition": {
"latitude": 33,
"longitude": 0.9999999
}
}
}
}"""
MALFORMED_RESPONSE = """{"manolo": "escobar"}"""
def setUp(self):
self.routing = HereMapsRoutingIsoline(None, None)
self.isoline_url = "{0}{1}".format(HereMapsRoutingIsoline.PRODUCTION_ROUTING_BASE_URL,
HereMapsRoutingIsoline.ISOLINE_PATH)
def test_calculate_isodistance_with_valid_params(self, req_mock):
print self.isoline_url
url = "{0}?start=geo%2133.0%2C1.0&mode=shortest%3Bcar".format(self.isoline_url)
req_mock.register_uri('GET', url, text=self.GOOD_RESPONSE)
response = self.routing.calculate_isodistance('geo!33.0,1.0', 'car',
['1000', '2000'])
self.assertEqual(len(response), 2)
self.assertEqual(response[0]['range'], 1000)
self.assertEqual(response[1]['range'], 2000)
self.assertEqual(response[0]['geom'], [u'32.9699707,0.9462833',
u'32.9699707,0.9458542',
u'32.9699707,0.9462833'])
self.assertEqual(response[1]['geom'], [u'32.9699707,0.9462833',
u'32.9699707,0.9750366',
u'32.9699707,0.9462833'])
def test_calculate_isochrone_with_valid_params(self, req_mock):
print self.isoline_url
url = "{0}?start=geo%2133.0%2C1.0&mode=shortest%3Bcar".format(self.isoline_url)
req_mock.register_uri('GET', url, text=self.GOOD_RESPONSE)
response = self.routing.calculate_isochrone('geo!33.0,1.0', 'car',
['1000', '2000'])
self.assertEqual(len(response), 2)
self.assertEqual(response[0]['range'], 1000)
self.assertEqual(response[1]['range'], 2000)
self.assertEqual(response[0]['geom'], [u'32.9699707,0.9462833',
u'32.9699707,0.9458542',
u'32.9699707,0.9462833'])
self.assertEqual(response[1]['geom'], [u'32.9699707,0.9462833',
u'32.9699707,0.9750366',
u'32.9699707,0.9462833'])
def test_calculate_isolines_empty_response(self, req_mock):
url = "{0}?start=geo%2133.0%2C1.0&mode=shortest%3Bcar".format(
self.isoline_url)
req_mock.register_uri('GET', url, text=self.EMPTY_RESPONSE)
response = self.routing.calculate_isochrone('geo!33.0,1.0', 'car',
['1000', '2000'])
self.assertEqual(len(response), 1)
self.assertEqual(response[0]['range'], 1000)
self.assertEqual(response[0]['geom'], [])
def test_non_listed_parameters_filter_works_properly(self, req_mock):
url = "{0}?start=geo%2133.0%2C1.0&mode=shortest%3Bcar".format(
self.isoline_url)
req_mock.register_uri('GET', url, text=self.GOOD_RESPONSE)
response = self.routing.calculate_isochrone('geo!33.0,1.0', 'car',
['1000', '2000'],
['singlecomponent=true',
'resolution=3',
'maxpoints=1000',
'quality=2',
'false_option=true'])
parsed_url = urlparse(req_mock.request_history[0].url)
url_params = parse_qs(parsed_url.query)
self.assertEqual(len(url_params), 8)
self.assertEqual('false_option' in url_params, False)
def test_mode_parameters_works_properly(self, req_mock):
req_mock.register_uri('GET', requests_mock.ANY,
text=self.GOOD_RESPONSE)
response = self.routing.calculate_isochrone('geo!33.0,1.0', 'car',
['1000', '2000'],
['mode_type=fastest',
'mode_feature=motorway',
'mode_feature_weight=-1',
'mode_traffic=false'])
parsed_url = urlparse(req_mock.request_history[0].url)
url_params = parse_qs(parsed_url.query)
self.assertEqual(url_params['mode'][0],
'fastest;car;traffic:false;motorway:-1')
def test_source_parameters_works_properly(self, req_mock):
req_mock.register_uri('GET', requests_mock.ANY,
text=self.GOOD_RESPONSE)
response = self.routing.calculate_isochrone('geo!33.0,1.0', 'car',
['1000', '2000'],
['is_destination=true'])
parsed_url = urlparse(req_mock.request_history[0].url)
url_params = parse_qs(parsed_url.query)
self.assertEqual(url_params['destination'][0], 'geo!33.0,1.0')

View File

@@ -29,4 +29,4 @@ class IntegrationTestHelper:
raise Exception(json.loads(query_response.text)['error'])
query_response_data = json.loads(query_response.text)
return query_response_data['rows'][0]['geometry']
return query_response_data['rows'][0]

View File

@@ -16,16 +16,15 @@ class TestAdmin0Functions(TestCase):
def test_if_select_with_admin0_is_ok(self):
query = "SELECT cdb_geocode_admin0_polygon(country) as geometry " \
"FROM {0} LIMIT 1&api_key={1}".format(
self.env_variables['table_name'],
self.env_variables['api_key'])
"FROM {0} LIMIT 1&api_key={1}".format(
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_admin0_without_api_key_raise_error(self):
query = "SELECT cdb_geocode_admin0_polygon(country) as geometry " \
"FROM {0} LIMIT 1".format(
self.env_variables['table_name'])
"FROM {0} LIMIT 1".format(self.env_variables['table_name'])
try:
IntegrationTestHelper.execute_query(self.sql_api_url, query)
except Exception as e:

View File

@@ -20,7 +20,7 @@ class TestAdmin1Functions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_admin1_with_country_is_ok(self):
query = "SELECT cdb_geocode_admin1_polygon(province,country)" \
@@ -28,7 +28,7 @@ class TestAdmin1Functions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_admin1_without_api_key_raise_error(self):
query = "SELECT cdb_geocode_admin1_polygon(province) as geometry " \

View File

@@ -20,7 +20,7 @@ class TestPostalcodeFunctions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_ipaddress_without_api_key_raise_error(self):
query = "SELECT cdb_geocode_ipaddress_point(ip) " \

View File

@@ -0,0 +1,45 @@
from unittest import TestCase
from nose.tools import assert_raises
from nose.tools import assert_not_equal, assert_equal
from ..helpers.integration_test_helper import IntegrationTestHelper
class TestIsolinesFunctions(TestCase):
def setUp(self):
self.env_variables = IntegrationTestHelper.get_environment_variables()
self.sql_api_url = "https://{0}.{1}/api/v2/sql".format(
self.env_variables['username'],
self.env_variables['host'],
self.env_variables['api_key']
)
def test_if_select_with_isochrones_is_ok(self):
query = "SELECT * FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, " \
"'car', ARRAY[300]::integer[]);&api_key={0}".format(
self.env_variables['api_key'])
isolines = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(isolines['the_geom'], None)
def test_if_select_with_isochrones_without_api_key_raise_error(self):
query = "SELECT * FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, " \
"'car', ARRAY[300]::integer[]);"
try:
IntegrationTestHelper.execute_query(self.sql_api_url, query)
except Exception as e:
assert_equal(e.message[0], "The api_key must be provided")
def test_if_select_with_isodistance_is_ok(self):
query = "SELECT * FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, " \
"'car', ARRAY[300]::integer[]);&api_key={0}".format(
self.env_variables['api_key'])
isolines = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(isolines['the_geom'], None)
def test_if_select_with_isodistance_without_api_key_raise_error(self):
query = "SELECT * FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, " \
"'car', ARRAY[300]::integer[]);"
try:
IntegrationTestHelper.execute_query(self.sql_api_url, query)
except Exception as e:
assert_equal(e.message[0], "The api_key must be provided")

View File

@@ -20,7 +20,7 @@ class TestNameplaceFunctions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_namedplace_city_country_is_ok(self):
query = "SELECT cdb_geocode_namedplace_point(city,country) " \
@@ -28,7 +28,7 @@ class TestNameplaceFunctions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_namedplace_city_province_country_is_ok(self):
query = "SELECT cdb_geocode_namedplace_point(city,province,country) " \
@@ -36,7 +36,7 @@ class TestNameplaceFunctions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_namedplace_without_api_key_raise_error(self):
query = "SELECT cdb_geocode_namedplace_point(city) as geometry " \

View File

@@ -20,7 +20,7 @@ class TestPostalcodeFunctions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_postalcode_point_is_ok(self):
query = "SELECT cdb_geocode_postalcode_point(postalcode, country) " \
@@ -28,7 +28,7 @@ class TestPostalcodeFunctions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_postalcode_without_api_key_raise_error(self):
query = "SELECT cdb_geocode_postalcode_polygon(postalcode, country) " \

View File

@@ -20,7 +20,7 @@ class TestStreetFunctions(TestCase):
self.env_variables['table_name'],
self.env_variables['api_key'])
geometry = IntegrationTestHelper.execute_query(self.sql_api_url, query)
assert_not_equal(geometry, None)
assert_not_equal(geometry['geometry'], None)
def test_if_select_with_street_without_api_key_raise_error(self):
query = "SELECT cdb_geocode_street_point(street) " \