Merge pull request #173 from CartoDB/development

On 4xx we have to return an empty response instead of an exception

README.md

@@ -30,6 +30,14 @@ Steps to deploy a new Data Services API version :
     cd data-services/geocoder/extension
     sudo make install
     ```
+    
+- install observatory extension 
+
+    ```
+    git clone git@github.com:CartoDB/observatory-extension.git
+    cd observatory
+    sudo make install
+    ```
 
 - install server and client extensions
     
@@ -49,6 +57,7 @@ Steps to deploy a new Data Services API version :
     ```
     create extension cdb_geocoder;
     create extension plproxy;
+    create extension observatory;
     create extension cdb_dataservices_server;
     create extension cdb_dataservices_client;
     ```
@@ -65,9 +74,9 @@ Steps to deploy a new Data Services API version :
       SELECT CDB_Conf_SetConf('redis_metadata_config', '{"redis_host": "localhost", "redis_port": 26379, "sentinel_master_id": "", "timeout": 0.1, "redis_db": 5}');
       SELECT CDB_Conf_SetConf('redis_metrics_config', '{"redis_host": "localhost", "redis_port": 6379, "sentinel_master_id": "", "timeout": 0.1, "redis_db": 5}');
   
-    SELECT CDB_Conf_SetConf('heremaps_conf', '{"app_id": "APP_ID", "app_code": "APP_CODE", "geocoder_cost_per_hit": "COST_PER_HIT"}');
+    SELECT CDB_Conf_SetConf('heremaps_conf', '{"geocoder": {"app_id": "here_geocoder_app_id", "app_code": "here_geocoder_app_code", "geocoder_cost_per_hit": "1"}, "isolines" : {"app_id": "here_isolines_app_id", "app_code": "here_geocoder_app_code"}}');
     SELECT CDB_Conf_SetConf('user_config', '{"is_organization": false, "entity_name": "<YOUR_USERNAME>"}')
-    SELECT CDB_Conf_SetConf('mapzen_conf', '{"routing_app_key": "ROUTING_API_KEY", "geocoder_app_key": "GEOCODER_API_KEY"}');
+    SELECT CDB_Conf_SetConf('mapzen_conf', '{"routing": {"api_key": "valhalla_app_key", "monthly_quota": 999999}, "geocoder": {"api_key": "search_app_key", "monthly_quota": 999999}}');
     SELECT CDB_Conf_SetConf('logger_con', '{"geocoder_log_path": "/tmp/geocodings.log"}')
     ```
 

client/cdb_dataservices_client--0.4.0--0.3.0.sql

@@ -1,7 +1,7 @@
 --DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
 -- Complain if script is sourced in psql, rather than via CREATE EXTENSION
 \echo Use "ALTER EXTENSION cdb_dataservices_client UPDATE TO '0.3.0'" to load this file. \quit
-DROP FUNCTION IF EXISTS cdb_dataservices_client._obs_get_demographic_snapshot (text, text, geometry(Geometry, 4326), text, text)
-DROP FUNCTION IF EXISTS cdb_dataservices_client._obs_get_segment_snapshot (text, text, geometry(Geometry, 4326), text)
-DROP FUNCTION IF EXISTS cdb_dataservices_client.obs_get_demographic_snapshot (geometry(Geometry, 4326), text, text)
-DROP FUNCTION IF EXISTS cdb_dataservices_client.obs_get_segment_snapshot (geometry(Geometry, 4326), text)
+DROP FUNCTION IF EXISTS cdb_dataservices_client._obs_get_demographic_snapshot (text, text, geometry(Geometry, 4326), text, text);
+DROP FUNCTION IF EXISTS cdb_dataservices_client._obs_get_segment_snapshot (text, text, geometry(Geometry, 4326), text);
+DROP FUNCTION IF EXISTS cdb_dataservices_client.obs_get_demographic_snapshot (geometry(Geometry, 4326), text, text);
+DROP FUNCTION IF EXISTS cdb_dataservices_client.obs_get_segment_snapshot (geometry(Geometry, 4326), text);

doc/API.md

@@ -2,14 +2,10 @@
 
 The CartoDB Data Services API offers a set of location based services that can be used programatically to empower your geospatial applications.
 
-The geocoder functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CartoDB SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses.
-
-The isoline functions provide a way to generate isolines in terms of distance and time, by means of the available isodistance and isochrone functions useful for Trade Areas Analysis.
-
 ## Documentation
 
-* [Quickstart](quickstart.md)
-* [General Concepts](general_concepts.md)
+* [Overview](overview.md)
 * [Geocoding Functions](geocoding_functions.md)
 * [Isoline Functions](isoline_functions.md)
+* [Routing Functions](routing_functions.md)
 * [Quota Information](quota_information.md)

doc/general_concepts.md

@@ -1,33 +0,0 @@
-# General Concepts
-
-The Data Services API offers geocoding and isoline services on top of the CartoDB SQL API by means of a set of functions. Each one of these functions is oriented to one kind of operation and returns the corresponding geometry (a `polygon` or a `point`), according to the input information.
-
-The Data Services API decouples the geocoding and isoline services from the CartoDB Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically through authenticated requests.
-
-The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/).
-
-The Geocoder functions can return different types of geometries (points or polygons) as result of different geocoding processes. The CartoDB platform does not support multigeometry layers or datasets, therefore the final users of this Data Services API must check that they are using consistent geometry types inside a table to avoid further conflicts in the map visualization.
-
-## Authentication
-
-All requests performed to the CartoDB Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/cartodb-platform/sql-api/authentication/) guide.
-
-## Errors
-
-Errors are described in the response of the request. An example is as follows:
-
-```json
-{
-  error: [
-    "The api_key must be provided"
-  ]
-}
-```
-
-Since the Data Services API is used on top of the CartoDB SQL API, you can refer to the [Making calls to the SQL API](/cartodb-platform/sql-api/making-calls/) documentation for help debugging your SQL errors.
-
-If the requested information is not in the CartoDB geocoding database, or if CartoDB is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result.
-
-## Limits
-
-Usage of the Data Services API is subject to the CartoDB SQL API limits, stated in our [Terms of Service](https://cartodb.com/terms/#excessive).

doc/geocoding_functions.md

@@ -1,14 +1,38 @@
 # Geocoding Functions
 
+The [geocoder](https://cartodb.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CartoDB SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. CartoDB provides functions for several different categories of geocoding through the Data Services API.
+
+_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._
+
+Here is an example of how to geocode a single country:
+
+```bash
+https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={api_key}
+```
+
+In order to geocode an existent CartoDB dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Data Services API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset:
+
+```bash
+https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_
+```
+
+Notice that you can make use of Postgres or PostGIS functions in your Data Services API requests, as the result is a geometry that can be handled by the system. For example, suppose you need to retrieve the centroid of a specific country, you can wrap the resulting geometry from the geocoder functions inside the PostGIS `ST_Centroid` function:
+
+```bash
+https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={api_key}
+```
+
+
 The following geocoding functions are available, grouped by categories.
 
+## Country Geocoder
 
-## Country geocoder
-
-This function provides a country geocoding service. It recognizes the names of the different countries from different synonyms, such as their English name, their endonym, or their ISO2 or ISO3 codes.
+This function geocodes your data into country border geometries. It recognizes the names of the different countries either by different synonyms (such as their English name or their endonym), or by ISO (ISO2 or ISO3) codes.
 
 ### cdb_geocode_admin0_polygon(_country_name text_)
 
+Geocodes the text name of a country into a country_name geometry, displayed as polygon data.
+
 #### Arguments
 
 Name | Type | Description
@@ -21,25 +45,26 @@ Geometry (polygon, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_admin0_polygon('France')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon({country_column})
 ```
 
+##### Insert a geocoded row into a table
 
-## Level-1 Administrative regions geocoder
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin0_polygon('France')
+```
 
-The following functions provide a geocoding service for administrative regions of level 1 (or NUTS-1) such as states for the United States, *départements* in France or autonomous communities in Spain.
+## Level-1 Administrative Regions Geocoder
+
+This function geocodes your data into polygon geometries for [Level 1](https://en.wikipedia.org/wiki/Table_of_administrative_divisions_by_country), or [NUTS-1](https://en.wikipedia.org/wiki/NUTS_1_statistical_regions_of_England), administrative divisions (or units) of countries. For example, a "state" in the United States, "départements" in France, or an autonomous community in Spain.
 
 ### cdb_geocode_admin1_polygon(_admin1_name text_)
 
+Geocodes the name of the province/state into a Level-1 administrative region, displayed as a polygon geometry.
+
 #### Arguments
 
 Name | Type | Description
@@ -52,20 +77,23 @@ Geometry (polygon, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_admin1_polygon('Alicante')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column})
 ```
 
+##### Insert a geocoded row into a table
+
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin1_polygon('Alicante')
+```
+
+
 ### cdb_geocode_admin1_polygon(_admin1_name text, country_name text_)
 
+Geocodes the name of the province/state for a specified country into a Level-1 administrative region, displayed as a polygon geometry.
+
 #### Arguments
 
 Name | Type | Description
@@ -79,25 +107,26 @@ Geometry (polygon, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_admin1_polygon('Alicante', 'Spain')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column}, {country_column})
 ```
+##### Insert a geocoded row into a table
+
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin1_polygon('Alicante', 'Spain')
+```
 
 
-## City geocoder
+## City Geocoder
 
-The following functions provide a city geocoder service. It is recommended to use the more specific geocoding function -- the one that requires more parameters — in order for the result to be as accurate as possible when several cities share their name. If there are duplicate results for a city name, the city name with the highest population will be returned.
+This function geocodes your data into point geometries for names of cities. It is recommended to use geocoding functions that require more defined parameters — this returns more accurate results when several cities have the same name. _If there are duplicate results for a city name, the city name with the highest population will be returned._
 
 ### cdb_geocode_namedplace_point(_city_name text_)
 
+Geocodes the text name of a city into a named place geometry, displayed as point data.
+
 #### Arguments
 
 Name | Type | Description
@@ -112,18 +141,23 @@ Geometry (point, EPSG 4326) or null
 
 ##### Select
 
-```bash
-SELECT cdb_geocode_namedplace_point('Barcelona')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column})
 ```
 
+##### Insert a geocoded row into a table
+
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('Barcelona')
+```
+
+
 ### cdb_geocode_namedplace_point(_city_name text, country_name text_)
 
+Geocodes the text name of a city for a specified country into a named place point geometry.
+
 #### Arguments
 
 Name | Type | Description
@@ -137,20 +171,22 @@ Geometry (point, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_namedplace_point('Barcelona', 'Spain')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, 'Spain')
 ```
 
+##### Insert a geocoded row into a table
+
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('Barcelona', 'Spain')
+```
+
+
 ### cdb_geocode_namedplace_point(_city_name text, admin1_name text, country_name text_)
 
+Geocodes your data into a named place point geometry, containing the text name of a city, for a specified province/state and country. This is recommended for the most accurate geocoding of city data. 
 #### Arguments
 
 Name | Type | Description
@@ -165,26 +201,28 @@ Geometry (point, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_namedplace_point('New York', 'New York', 'USA')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, {province_column}, 'USA')
 ```
 
-## Postal codes geocoder
+##### Insert a geocoded row into a table
 
-The following functions provide a postal code geocoding service that can be used to obtain points or polygon results. The postal code polygon geocoder covers the United States, France, Australia and Canada; a request for a different country will return an empty response.
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('New York', 'New York', 'USA')
+```
+
+## Postal Code Geocoder
+
+This function geocodes your data into point, or polygon, geometries for postal codes. The postal code polygon geocoder covers the United States, France, Australia and Canada; a request for a different country will return an empty response.
 
 **Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://docs.cartodb.com/faqs/datasets-and-data/#why-does-cartodb-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details.
 
 ### cdb_geocode_postalcode_polygon(_postal_code text, country_name text_)
 
+Goecodes the postal code for a specified country into a **polygon** geometry.
+
 #### Arguments
 
 Name | Type | Description
@@ -198,20 +236,22 @@ Geometry (polygon, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_postalcode_polygon('11211', 'USA')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_polygon({postal_code_column}, 'USA')
 ```
 
+##### Insert a geocoded row into a table
+
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_postalcode_polygon('11211', 'USA')
+```
+
 ### cdb_geocode_postalcode_point(_code text, country_name text_)
 
+Goecodes the postal code for a specified country into a **point** geometry.
+
 #### Arguments
 
 Name | Type | Description
@@ -225,24 +265,27 @@ Geometry (point, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_postalcode_point('11211', 'USA')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_point({postal_code_column}, 'USA')
 ```
 
-## IP addresses geocoder
+##### Insert a geocoded row into a table
 
-This function provides an IP address geocoding service, for both IPv4 and IPv6 addresses.
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_postalcode_point('11211', 'USA')
+```
+
+
+## IP Addresses Geocoder
+
+This function geocodes your data into point geometries for IP addresses. This is useful if you are analyzing location based data, based on a set of user's IP addresses.
 
 ### cdb_geocode_ipaddress_point(_ip_address text_)
 
+Geocodes a postal code from a specified country into an IP address, displayed as a point geometry.
+
 #### Arguments
 
 Name | Type | Description
@@ -256,33 +299,36 @@ Geometry (point, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_ipaddress_point('102.23.34.1')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_ipaddress_point('102.23.34.1')
 ```
-## Street-level geocoder
 
-This function provides a street-level geocoding service. This service uses the street level geocoder defined for the user.
+##### Insert a geocoded row into a table
 
-**This service is subject to quota limitations, and extra fees may apply**. Please view our [terms and conditions](https://cartodb.com/terms/) and check out the [Quota information section](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) for details and recommendations related with quota usage.
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_ipaddress_point('102.23.34.1')
+```
+
+## Street-Level Geocoder
+
+This function geocodes your data into a point geometry for a street address. CartoDB uses several different service providers for street-level geocoding, depending on your platform. If you access CartoDB on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future.
+
+**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) for details and recommendations about quota consumption.
 
 ### cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_)
 
+Geocodes a complete address into a single street geometry, displayed as point data.
+
 #### Arguments
 
 Name | Type | Description
---- | --- | ---
-`searchtext` | `text` | searchtext contains free-form text containing address elements. You can specify the searchtext parameter by itself, or you can specify it with other parameters to narrow your search. For example, you can specify the state or country parameters, along with a free-form address in the searchtext field.
-`city` | `text` | (Optional) Name of the city
-`state` | `text` | (Optional) Name of the state
-`country` | `text` | (Optional) Name of the country
+--- | --- | --- | ---
+`searchtext` | `text` | searchtext contains free-form text containing address elements. You can specify the searchtext parameter by itself, or with other parameters, to narrow your search. For example, you can specify the state or country parameters, along with a free-form address in the searchtext field.
+`city` | `text` | (Optional) Name of the city.
+`state` | `text` | (Optional) Name of the state.
+`country` | `text` | (Optional) Name of the country.
 
 #### Returns
 
@@ -290,17 +336,14 @@ Geometry (point, EPSG 4326) or null
 
 #### Example
 
-##### Select
-
-```bash
-SELECT cdb_geocode_street_point('651 Lombard Street, San Francisco, California, United States')
-SELECT cdb_geocode_street_point('651 Lombard Street', 'San Francisco')
-SELECT cdb_geocode_street_point('651 Lombard Street', 'San Francisco', 'California')
-SELECT cdb_geocode_street_point('651 Lombard Street', 'San Francisco', 'California', 'United States')
-```
-
-##### Update
+##### Update the geometry of a table to geocode it
 
 ```bash
 UPDATE {tablename} SET the_geom = cdb_geocode_street_point({street_name_column})
 ```
+
+##### Insert a geocoded row into a table
+
+```bash
+INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_street_point('651 Lombard Street', 'San Francisco', 'California', 'United States')
+```

doc/isoline_functions.md

@@ -1,10 +1,20 @@
 # Isoline Functions
 
-The following functions provide an isolines generator service based on time or distance. This service uses the isolines service defined for the user (currently, only the [HERE](https://developer.here.com/coverage-info) isolines service is available).
+[Isolines](https://cartodb.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through CartoDB are available by requesting a single function in the Data Services API.
 
-**This service is subject to quota limitations, and extra fees may apply**. Please view our [terms and conditions](https://cartodb.com/terms/) and check out the [Quota information section](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) for details and recommendations related with quota usage.
+_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._
 
-### cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_)
+You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing and inserts them into a table.
+
+```bash
+https://{username}.cartodb.com/api/v2/sql?q=INSERT INTO {table} (the_geom) SELECT  the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 600, 900]::integer[])&api_key={api_key}
+```
+
+The following functions provide an isoline generator service, based on time or distance. This service uses the isolines service defined for your account. The default service limits the usage of displayed polygons represented on top of [HERE](https://developer.here.com/coverage-info) maps.
+
+## cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_)
+
+Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of distance (in meters).
 
 #### Arguments
 
@@ -26,20 +36,16 @@ Name | Type | Description
 
 #### Examples
 
-##### Select the results of the isodistance function
+##### Calculate and insert isodistance polygons from a point into another table
 
 ```bash
-SELECT * FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[1000,2000]::integer[]);
+INSERT INTO {table} (the_geom) SELECT  the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[])
 ```
 
-```bash
-SELECT * FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[1000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[]);
-```
-
-##### Select the geometric results of the isodistance function
+or equivalently:
 
 ```bash
-SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[1000]::integer[]);
+INSERT INTO {table} (the_geom) SELECT (cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[])).the_geom
 ```
 
 ##### Calculate and insert the generated isolines from `points_table` table to another table
@@ -48,7 +54,10 @@ SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk
 INSERT INTO {table} (the_geom) SELECT (cdb_isodistance(the_geom, 'walk', string_to_array(distance, ',')::integer[])).the_geom FROM {points_table}
 ```
 
-### cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_)
+
+## cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_)
+
+Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of time (in seconds).
 
 #### Arguments
 
@@ -63,23 +72,19 @@ Name | Type | Description | Accepted values
 
 #### Examples
 
-##### Select the results of the isochrone function
+##### Calculate and insert isochrone polygons from a point into another table
 
 ```bash
-SELECT * FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300,900,12000]::integer[]);
+INSERT INTO {table} (the_geom) SELECT  the_geom FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[])
 ```
 
+or equivalently:
+
 ```bash
-SELECT * FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300,900]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[]);
+INSERT INTO {table} (the_geom) SELECT (cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[])).the_geom
 ```
 
-##### Select the geometric results of the isochrone function
-
-```bash
-SELECT the_geom FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300]::integer[]);
-```
-
-##### Calculate and insert the generated isolines from `points_table` table to another table
+##### Calculate and insert the generated isolines from `points_table` table into another table
 
 ```bash
 INSERT INTO {table} (the_geom) SELECT (cdb_isochrone(the_geom, 'walk', string_to_array(time_distance, ',')::integer[])).the_geom FROM {points_table}

doc/overview.md

@@ -0,0 +1,49 @@
+# Overview
+
+By using CartoDB libraries and the SQL API, you can apply location data services to your maps with unique data services functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations.
+
+**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/#quota-information).
+
+_The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our CartoDB Platform._
+
+## Data Services Integration
+
+By using the SQL API to query the Data Services API functions, you can manage specific operations and the corresponding geometries (a `polygon` or a `point`), according to the input information.
+
+The Data Services API decouples the geocoding and isoline services from the CartoDB Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically, through authenticated requests.
+
+The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/).
+
+**Note:** The Data Services API [geocoding functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CartoDB Platform does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization.
+
+### Best Practices
+
+_Be mindful of the following usage notes when using the Data Services functions with the SQL API:_
+
+It is discouraged to use the SELECT operation with the Data Services API functions in your map layers, as these type of queries consume quota when rendering tiles for your live map views. It may also result in sync performance issues, due to executing multiple requests to the API each time your map is viewed. See details about [Quota Consumption](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/#quota-consumption).
+
+The Data Services API is **recommended** to be used with INSERT or UPDATE operations, for applying location data to your tables. While SELECT (retrieve) is standard for SQL API requests, be mindful of quota consumption and use INSERT (to insert a new record) or UPDATE (to update an existing record), for best practices.
+
+## Authentication
+
+All requests performed to the CartoDB Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/cartodb-platform/sql-api/authentication/) documentation.
+
+## Errors
+
+Errors are described in the response of the request. An example is as follows:
+
+```json
+{
+  error: [
+    "The api_key must be provided"
+  ]
+}
+```
+
+Since the Data Services API is used on top of the CartoDB SQL API, you can refer to the [Making calls to the SQL API](/cartodb-platform/sql-api/making-calls/) documentation for help debugging your SQL errors.
+
+If the requested information is not in the CartoDB geocoding database, or if CartoDB is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result.
+
+## Limits
+
+Usage of the Data Services API is subject to the CartoDB SQL API limits, stated in our [Terms of Service](https://cartodb.com/terms/#excessive).

doc/quickstart.md

@@ -1,26 +0,0 @@
-# Quickstart
-
-If you are using the set of APIs and libraries that CartoDB offers, and you are handling your data with the SQL API, you can make your data visible in your maps by geocoding the dataset programatically. The set of isoline functions allow you to calculate the area which can be reached by travelling a given distance or time, useful for geospatial analysis, such as Trade Area Analysis.
-
-Here is an example of how to geocode a single country:
-
-```bash
-https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={Your API key}
-```
-
-In order to geocode an existent CartoDB dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Geocoding API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset:
-
-```bash
-https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon({country_column})&api_key={Your API key}
-```
-You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing.
-
-```bash
-https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300,600,900]::integer[])&api_key={Your API key}
-```
-
-Notice that you can make use of Postgres or PostGIS functions in your Data Services API requests, as the result is a geometry that can be handled by the system. For example, suppose you need to retrieve the centroid of a specific country, you can wrap the resulting geometry from the geocoder functions inside the PostGIS `ST_Centroid` function:
-
-```bash
-https://{username}.cartodb.com/api/v2/sql?q=SELECT ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={Your API key}
-```

doc/quota_information.md

@@ -1,17 +1,14 @@
 # Quota Information
 
-**This Data Services API provides functions which are subject to quota limitations, and extra fees may apply**. Please check our [terms and conditions](https://cartodb.com/terms/).
+**Based on your account plan, some of the Data Services API functions are subject to quota limitations and extra fees may apply.** View our [terms and conditions](https://cartodb.com/terms/), or [contact us](mailto:sales@cartodb.com) for details about which functions require service credits to your account.
 
-The functions that require service credits to be used are as follows:
+## Quota Consumption
 
-* cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_); from [Geocoding functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/geocoding-functions/)
-* cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_); from [Isoline functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/isoline-functions/)
-* cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_); from [Isoline functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/isoline-functions/)
+Quota consumption is calculated based on the number of request made for each function. Be mindful of the following usage recommendations when using the Data Services API functions:
 
-## Quota consumption information and usage recommendations
+* One credit per function call will be consumed. The results are not cached. If the query is applied to a _N_ rows dataset, then _N_ credits are consumed
+* Avoid running dynamic queries to these functions in your maps. This can result in credit consumption per map view. 
 
-Be mindful of the following when using the abovementioned functions:
+  **Note:** Queries to the Data Services API, and any of its functions in your maps, may be forbidden in the future
 
-* One credit per function call will be consumed, and the results are not cached. If the query is applied to a _N_ rows dataset, then _N_ credits will be used.
-* You are discouraged from using dynamic queries to these functions in your maps. This can result in credits consumption per map view. **Note:** Queries to the Data Services API and any of its functions in your maps may be forbidden in the future.
-* You are advised to store results of these queries into your datasets and refresh them as needed, so that you can have finer control on your credits' usage.
+* It is advised to store results of these queries into your datasets, and refresh them as needed. This ensure more control of quota credits for your account

doc/routing_functions.md

@@ -0,0 +1,45 @@
+# Routing Functions
+
+Routing is the navigation from a defined start location to a defined end location. The calculated results are displayed as turn-by-turn directions on your map, based on the transportation mode that you specified. Routing services through CartoDB are available by requesting a single function in the Data Services API.
+
+### cdb_route_point_to_point(_origin geometry(Point), destination geometry(Point), mode text, [options text[], units text]_)
+
+#### Arguments
+
+Name | Type | Description | Accepted values
+--- | --- | --- | ---
+`origin` | `geometry(Point)` | Origin point, in 4326 projection, which defines the start location. |
+`destination` | `geometry(Point)` | Destination point, in 4326 projection, which defines the end location. |
+`mode` | `text` | Type of transport used to calculate the isolines. | `car`, `walk`, `bicycle` or `public_transport`
+`options` | `text[]` | (Optional) Multiple options to add more capabilities to the analysis. See [Optional routing parameters](#optional-routing-parameters) for details.
+`units` | `text` | Unit used to represent the length of the route. | `kilometers`, `miles`. By default is `kilometers`
+
+
+#### Returns
+
+Name | Type | Description
+--- | --- | ---
+`duration` | `integer` | Duration in seconds of the calculated route.
+`length` | `real` | Length in the defined unit in the `units` field. `kilometers` by default .
+`the_geom` | `geometry(LineString)` | LineString geometry of the calculated route in the 4326 projection.
+
+#### Examples
+
+##### Insert the values from the calculated route in your table
+
+```bash
+INSERT INTO <TABLE> (duration, length, the_geom) SELECT duration, length, shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car')
+```
+##### Update the geometry field with the calculated route shape
+
+```bash
+UPDATE <TABLE> SET the_geom = (SELECT shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car', ARRAY['mode_type=shortest']::text[]))
+```
+
+### Optional routing parameters
+
+The optional value parameters must be passed using the format: `option=value`.
+
+Name | Type | Description | Accepted values
+--- | --- | --- | ---
+`mode_type` | `text` | Type of route calculation | `shortest` (this option only applies to the car transport mode)

server/extension/Makefile

@@ -13,8 +13,8 @@ OLD_VERSIONS = $(wildcard old_versions/*.sql)
 # @see http://www.postgresql.org/docs/current/static/extend-pgxs.html
 DATA =  $(NEW_EXTENSION_ARTIFACT) \
   $(OLD_VERSIONS) \
-  cdb_dataservices_server--0.6.2--0.7.0.sql \
-  cdb_dataservices_server--0.7.0--0.6.2.sql
+  cdb_dataservices_server--0.7.3--0.7.4.sql \
+  cdb_dataservices_server--0.7.4--0.7.3.sql
 
 REGRESS = $(notdir $(basename $(wildcard test/sql/*test.sql)))
 TEST_DIR = test/

server/extension/cdb_dataservices_server--0.7.3--0.7.4.sql

@@ -0,0 +1,57 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.4'" to load this file. \quit
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_routing_isolines(username TEXT, orgname TEXT, type TEXT, source geometry(Geometry, 4326), mode TEXT, data_range integer[], options text[])
+RETURNS SETOF cdb_dataservices_server.isoline AS $$
+  import json
+  from cartodb_services.here import HereMapsRoutingIsoline
+  from cartodb_services.metrics import QuotaService
+  from cartodb_services.here.types import geo_polyline_to_multipolygon
+
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)]
+
+  # -- Check the quota
+  quota_service = QuotaService(user_isolines_routing_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+    client = HereMapsRoutingIsoline(user_isolines_routing_config.heremaps_app_id, user_isolines_routing_config.heremaps_app_code, base_url = HereMapsRoutingIsoline.PRODUCTION_ROUTING_BASE_URL)
+
+    if source:
+      lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat']
+      lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon']
+      source_str = 'geo!%f,%f' % (lat, lon)
+    else:
+      source_str = None
+
+    if type == 'isodistance':
+      resp = client.calculate_isodistance(source_str, mode, data_range, options)
+    elif type == 'isochrone':
+      resp = client.calculate_isochrone(source_str, mode, data_range, options)
+
+    if resp:
+      result = []
+      for isoline in resp:
+        data_range_n = isoline['range']
+        polyline = isoline['geom']
+        multipolygon = geo_polyline_to_multipolygon(polyline)
+        result.append([source, data_range_n, multipolygon])
+      quota_service.increment_success_service_use()
+      quota_service.increment_isolines_service_use(len(resp))
+      return result
+    else:
+      quota_service.increment_empty_service_use()
+      return []
+  except BaseException as e:
+    import sys, traceback
+    type_, value_, traceback_ = sys.exc_info()
+    quota_service.increment_failed_service_use()
+    error_msg = 'There was an error trying to obtain isodistances using here maps geocoder: {0}'.format(e)
+    plpy.notice(traceback.format_tb(traceback_))
+    plpy.error(error_msg)
+  finally:
+    quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu SECURITY DEFINER;

server/extension/cdb_dataservices_server--0.7.4--0.7.3.sql

@@ -0,0 +1,56 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.3'" to load this file. \quit
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_routing_isolines(username TEXT, orgname TEXT, type TEXT, source geometry(Geometry, 4326), mode TEXT, data_range integer[], options text[])
+RETURNS SETOF cdb_dataservices_server.isoline AS $$
+  import json
+  from cartodb_services.here import HereMapsRoutingIsoline
+  from cartodb_services.metrics import QuotaService
+  from cartodb_services.here.types import geo_polyline_to_multipolygon
+
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)]
+
+  # -- Check the quota
+  quota_service = QuotaService(user_isolines_routing_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+    client = HereMapsRoutingIsoline(user_isolines_routing_config.heremaps_app_id, user_isolines_routing_config.heremaps_app_code, base_url = HereMapsRoutingIsoline.PRODUCTION_ROUTING_BASE_URL)
+
+    if source:
+      lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat']
+      lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon']
+      source_str = 'geo!%f,%f' % (lat, lon)
+    else:
+      source_str = None
+
+    if type == 'isodistance':
+      resp = client.calculate_isodistance(source_str, mode, data_range, options)
+    elif type == 'isochrone':
+      resp = client.calculate_isochrone(source_str, mode, data_range, options)
+
+    if resp:
+      result = []
+      for isoline in resp:
+        data_range_n = isoline['range']
+        polyline = isoline['geom']
+        multipolygon = geo_polyline_to_multipolygon(polyline)
+        result.append([source, data_range_n, multipolygon])
+      quota_service.increment_success_service_use()
+      quota_service.increment_isolines_service_use(len(resp))
+      return result
+    else:
+      quota_service.increment_empty_service_use()
+  except BaseException as e:
+    import sys, traceback
+    type_, value_, traceback_ = sys.exc_info()
+    quota_service.increment_failed_service_use()
+    error_msg = 'There was an error trying to obtain isodistances using here maps geocoder: {0}'.format(e)
+    plpy.notice(traceback.format_tb(traceback_))
+    plpy.error(error_msg)
+  finally:
+    quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu SECURITY DEFINER;

server/extension/cdb_dataservices_server.control

@@ -1,5 +1,5 @@
 comment = 'CartoDB dataservices server extension'
-default_version = '0.7.0'
-requires = 'plpythonu, postgis, cdb_geocoder'
+default_version = '0.7.4'
+requires = 'plpythonu, plproxy, postgis, cdb_geocoder'
 superuser = true
 schema = cdb_dataservices_server

server/extension/old_versions/cdb_dataservices_server--0.7.0--0.7.1.sql

@@ -0,0 +1,94 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.1'" to load this file. \quit
+DROP FUNCTION IF EXISTS cdb_dataservices_server._get_data_observatory_config(text, text);
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_obs_snapshot_config(username text, orgname text)
+RETURNS boolean AS $$
+  cache_key = "user_obs_snapshot_config_{0}".format(username)
+  if cache_key in GD:
+    return False
+  else:
+    from cartodb_services.metrics import ObservatorySnapshotConfig
+    plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+    redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
+    obs_snapshot_config = ObservatorySnapshotConfig(redis_conn, plpy, username, orgname)
+    GD[cache_key] = obs_snapshot_config
+    return True
+$$ LANGUAGE plpythonu SECURITY DEFINER;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
+      result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
+      result = plpy.execute(obs_plan, [geom, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;

server/extension/old_versions/cdb_dataservices_server--0.7.1--0.7.0.sql

@@ -0,0 +1,94 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.0'" to load this file. \quit
+DROP FUNCTION IF EXISTS cdb_dataservices_server._get_obs_snapshot_config(text, text);
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_data_observatory_config(username text, orgname text)
+RETURNS boolean AS $$
+  cache_key = "user_data_observatory_config_{0}".format(username)
+  if cache_key in GD:
+    return False
+  else:
+    from cartodb_services.metrics import DataObservatoryConfig
+    plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+    redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
+    data_observatory_config = DataObservatoryConfig(redis_conn, plpy, username, orgname)
+    GD[cache_key] = data_observatory_config
+    return True
+$$ LANGUAGE plpythonu SECURITY DEFINER;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_data_observatory_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
+      result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_exception(type_, value_, traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_data_observatory_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
+      result = plpy.execute(obs_plan, [geom, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;

server/extension/old_versions/cdb_dataservices_server--0.7.1--0.7.2.sql

@@ -0,0 +1,115 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.2'" to load this file. \quit
+DROP FUNCTION IF EXISTS cdb_dataservices_server.obs_get_demographic_snapshot(text, text, geometry(Geometry, 4326), text, text);
+DROP FUNCTION IF EXISTS cdb_dataservices_server.obs_get_segment_snapshot(text, text, geometry(Geometry, 4326), text);
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_server_conn_str(
+  username TEXT,
+  orgname TEXT
+)
+RETURNS text AS $$
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  return user_obs_snapshot_config.connection_str
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_dataservices_server._obs_get_demographic_snapshot (username, orgname, geom, time_span, geometry_level);
+$$ LANGUAGE plproxy;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
+      result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_dataservices_server._obs_get_segment_snapshot (username, orgname, geom, geometry_level);
+$$ LANGUAGE plproxy;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
+      result = plpy.execute(obs_plan, [geom, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;

server/extension/old_versions/cdb_dataservices_server--0.7.2--0.7.1.sql

@@ -0,0 +1,84 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.1'" to load this file. \quit
+DROP FUNCTION IF EXISTS cdb_dataservices_server._obs_server_conn_str(text, text);
+DROP FUNCTION IF EXISTS cdb_dataservices_server.obs_get_demographic_snapshot(text, text, geometry(Geometry, 4326), text, text);
+DROP FUNCTION IF EXISTS cdb_dataservices_server.obs_get_segment_snapshot(text, text, geometry(Geometry, 4326), text);
+DROP FUNCTION IF EXISTS cdb_dataservices_server._obs_get_demographic_snapshot(text, text, geometry(Geometry, 4326), text, text);
+DROP FUNCTION IF EXISTS cdb_dataservices_server._obs_get_segment_snapshot(text, text, geometry(Geometry, 4326), text);
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
+      result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
+      result = plpy.execute(obs_plan, [geom, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;

server/extension/old_versions/cdb_dataservices_server--0.7.2--0.7.3.sql

@@ -0,0 +1,113 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.3'" to load this file. \quit
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_server_conn_str(
+  username TEXT,
+  orgname TEXT)
+RETURNS text AS $$
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  return user_obs_snapshot_config.connection_str
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetDemographicSnapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_observatory.OBS_GetDemographicSnapshot(geom, time_span, geometry_level);
+$$ LANGUAGE plproxy;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetDemographicSnapshot($1, $2, $3, $4, $5) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text", "text"])
+      result = plpy.execute(obs_plan, [username, orgname, geom, time_span, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetSegmentSnapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_observatory.OBS_GetSegmentSnapshot(geom, geometry_level);
+$$ LANGUAGE plproxy;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetSegmentSnapshot($1, $2, $3, $4) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text"])
+      result = plpy.execute(obs_plan, [username, orgname, geom, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;

server/extension/old_versions/cdb_dataservices_server--0.7.3--0.7.2.sql

@@ -0,0 +1,103 @@
+--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
+-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.2'" to load this file. \quit
+DROP FUNCTION IF EXISTS cdb_dataservices_server._OBS_GetDemographicSnapshot(text, text, geometry(Geometry, 4326), text, text);
+DROP FUNCTION IF EXISTS cdb_dataservices_server._OBS_GetSegmentSnapshot(text, text, geometry(Geometry, 4326), text);
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_dataservices_server._obs_get_demographic_snapshot (username, orgname, geom, time_span, geometry_level);
+$$ LANGUAGE plproxy;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_get_demographic_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
+      result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_dataservices_server._obs_get_segment_snapshot (username, orgname, geom, geometry_level);
+$$ LANGUAGE plproxy;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_get_segment_snapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  from cartodb_services.metrics import QuotaService
+  import json
+
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
+  if not quota_service.check_user_quota():
+    plpy.error('You have reached the limit of your quota')
+
+  try:
+      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
+      result = plpy.execute(obs_plan, [geom, geometry_level])
+      if result:
+        quota_service.increment_success_service_use()
+        return result[0]['snapshot']
+      else:
+        quota_service.increment_empty_service_use()
+        return None
+  except BaseException as e:
+      import sys, traceback
+      type_, value_, traceback_ = sys.exc_info()
+      quota_service.increment_failed_service_use()
+      error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
+      plpy.notice(traceback.format_tb(traceback_))
+      plpy.error(error_msg)
+  finally:
+      quota_service.increment_total_service_use()
+$$ LANGUAGE plpythonu;

server/extension/sql/110_data_observatory.sql

@@ -1,3 +1,32 @@
+--
+-- Observatory connection config
+--
+-- The purpose of this function is provide to the PL/Proxy functions
+-- the connection string needed to connect with the current production database
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_server_conn_str(
+  username TEXT,
+  orgname TEXT)
+RETURNS text AS $$
+  plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
+  redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
+
+  return user_obs_snapshot_config.connection_str
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetDemographicSnapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  time_span TEXT DEFAULT '2009 - 2013',
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_observatory.OBS_GetDemographicSnapshot(geom, time_span, geometry_level);
+$$ LANGUAGE plproxy;
+
 CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
   username TEXT,
   orgname TEXT,
@@ -10,16 +39,16 @@ RETURNS json AS $$
 
   plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
   redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
-  plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
-  user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
 
-  quota_service = QuotaService(user_data_observatory_config, redis_conn)
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
   if not quota_service.check_user_quota():
     plpy.error('You have reached the limit of your quota')
 
   try:
-      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
-      result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
+      obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetDemographicSnapshot($1, $2, $3, $4, $5) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text", "text"])
+      result = plpy.execute(obs_plan, [username, orgname, geom, time_span, geometry_level])
       if result:
         quota_service.increment_success_service_use()
         return result[0]['snapshot']
@@ -37,6 +66,16 @@ RETURNS json AS $$
       quota_service.increment_total_service_use()
 $$ LANGUAGE plpythonu;
 
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetSegmentSnapshot(
+  username TEXT,
+  orgname TEXT,
+  geom geometry(Geometry, 4326),
+  geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
+RETURNS json AS $$
+  CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname);
+  SELECT cdb_observatory.OBS_GetSegmentSnapshot(geom, geometry_level);
+$$ LANGUAGE plproxy;
+
 CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
   username TEXT,
   orgname TEXT,
@@ -48,16 +87,16 @@ RETURNS json AS $$
 
   plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
   redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
-  plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
-  user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
+  plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
+  user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
 
-  quota_service = QuotaService(user_data_observatory_config, redis_conn)
+  quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
   if not quota_service.check_user_quota():
     plpy.error('You have reached the limit of your quota')
 
   try:
-      obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
-      result = plpy.execute(obs_plan, [geom, geometry_level])
+      obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetSegmentSnapshot($1, $2, $3, $4) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text"])
+      result = plpy.execute(obs_plan, [username, orgname, geom, geometry_level])
       if result:
         quota_service.increment_success_service_use()
         return result[0]['snapshot']
@@ -73,4 +112,4 @@ RETURNS json AS $$
       plpy.error(error_msg)
   finally:
       quota_service.increment_total_service_use()
-$$ LANGUAGE plpythonu;
+$$ LANGUAGE plpythonu;

server/extension/sql/15_config_helper.sql

@@ -54,16 +54,16 @@ RETURNS boolean AS $$
     return True
 $$ LANGUAGE plpythonu SECURITY DEFINER;
 
-CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_data_observatory_config(username text, orgname text)
+CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_obs_snapshot_config(username text, orgname text)
 RETURNS boolean AS $$
-  cache_key = "user_data_observatory_config_{0}".format(username)
+  cache_key = "user_obs_snapshot_config_{0}".format(username)
   if cache_key in GD:
     return False
   else:
-    from cartodb_services.metrics import DataObservatoryConfig
+    from cartodb_services.metrics import ObservatorySnapshotConfig
     plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
     redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
-    data_observatory_config = DataObservatoryConfig(redis_conn, plpy, username, orgname)
-    GD[cache_key] = data_observatory_config
+    obs_snapshot_config = ObservatorySnapshotConfig(redis_conn, plpy, username, orgname)
+    GD[cache_key] = obs_snapshot_config
     return True
 $$ LANGUAGE plpythonu SECURITY DEFINER;

server/extension/sql/80_isolines_helper.sql

@@ -42,6 +42,7 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$
       return result
     else:
       quota_service.increment_empty_service_use()
+      return []
   except BaseException as e:
     import sys, traceback
     type_, value_, traceback_ = sys.exc_info()

server/extension/test/expected/00_install_test.out

@@ -2,6 +2,7 @@
 CREATE EXTENSION postgis;
 CREATE EXTENSION schema_triggers;
 CREATE EXTENSION plpythonu;
+CREATE EXTENSION plproxy;
 CREATE EXTENSION cartodb;
 CREATE EXTENSION cdb_geocoder;
 CREATE EXTENSION observatory VERSION 'dev';
@@ -38,7 +39,7 @@ SELECT cartodb.cdb_conf_setconf('logger_conf', '{"geocoder_log_path": "/dev/null
  
 (1 row)
 
-SELECT cartodb.cdb_conf_setconf('data_observatory_conf', '{"monthly_quota": 10000}');
+SELECT cartodb.cdb_conf_setconf('data_observatory_conf', '{"connection": {"whitelist": ["ethervoid"], "production": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api", "staging": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api"}, "monthly_quota": 100000}');
  cdb_conf_setconf 
 ------------------
  

server/extension/test/sql/00_install_test.sql

@@ -2,6 +2,7 @@
 CREATE EXTENSION postgis;
 CREATE EXTENSION schema_triggers;
 CREATE EXTENSION plpythonu;
+CREATE EXTENSION plproxy;
 CREATE EXTENSION cartodb;
 CREATE EXTENSION cdb_geocoder;
 CREATE EXTENSION observatory VERSION 'dev';
@@ -15,7 +16,7 @@ SELECT cartodb.cdb_conf_setconf('redis_metadata_config', '{"redis_host": "localh
 SELECT cartodb.cdb_conf_setconf('heremaps_conf', '{"geocoder": {"app_id": "dummy_id", "app_code": "dummy_code", "geocoder_cost_per_hit": 1}, "isolines": {"app_id": "dummy_id", "app_code": "dummy_code"}}');
 SELECT cartodb.cdb_conf_setconf('mapzen_conf', '{"routing": {"api_key": "routing_dummy_api_key", "monthly_quota": 1500000}, "geocoder": {"api_key": "geocoder_dummy_api_key", "monthly_quota": 1500000}}');
 SELECT cartodb.cdb_conf_setconf('logger_conf', '{"geocoder_log_path": "/dev/null"}');
-SELECT cartodb.cdb_conf_setconf('data_observatory_conf', '{"monthly_quota": 10000}');
+SELECT cartodb.cdb_conf_setconf('data_observatory_conf', '{"connection": {"whitelist": ["ethervoid"], "production": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api", "staging": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api"}, "monthly_quota": 100000}');
 
 -- Mock the varnish invalidation function
 -- (used by cdb_geocoder tests)

server/lib/python/cartodb_services/cartodb_services/here/geocoder.py

@@ -87,6 +87,8 @@ class HereMapsGeocoder:
         response = requests.get(self.host, params=request_params)
         if response.status_code == requests.codes.ok:
             return json.loads(response.text)
+        elif response.status_code == requests.codes.bad_request:
+            return []
         else:
             response.raise_for_status()
 

server/lib/python/cartodb_services/cartodb_services/here/routing.py

@@ -51,6 +51,8 @@ class HereMapsRoutingIsoline:
         response = requests.get(self._url, params=request_params)
         if response.status_code == requests.codes.ok:
             return self.__parse_isolines_response(response.text)
+        elif response.status_code == requests.codes.bad_request:
+            return []
         else:
             response.raise_for_status()
 

server/lib/python/cartodb_services/cartodb_services/metrics/__init__.py

@@ -1,3 +1,3 @@
-from config import GeocoderConfig, IsolinesRoutingConfig, InternalGeocoderConfig, RoutingConfig, ConfigException, DataObservatoryConfig
+from config import GeocoderConfig, IsolinesRoutingConfig, InternalGeocoderConfig, RoutingConfig, ConfigException, ObservatorySnapshotConfig
 from quota import QuotaService
 from user import UserMetricsService

server/lib/python/cartodb_services/cartodb_services/metrics/config.py

@@ -14,9 +14,10 @@ class ServiceConfig(object):
         self._redis_connection = redis_connection
         self._username = username
         self._orgname = orgname
-        self._db_config = ServicesDBConfig(db_conn)
+        self._db_config = ServicesDBConfig(db_conn, username, orgname)
         if redis_connection:
-            self._redis_config = ServicesRedisConfig(redis_connection).build(username, orgname)
+            self._redis_config = ServicesRedisConfig(redis_connection).build(
+                username, orgname)
         else:
             self._redis_config = None
 
@@ -32,19 +33,32 @@ class ServiceConfig(object):
     def organization(self):
         return self._orgname
 
-class DataObservatoryConfig(ServiceConfig):
 
+class ObservatorySnapshotConfig(ServiceConfig):
+
+    SOFT_LIMIT_KEY = 'soft_obs_snapshot_limit'
+    QUOTA_KEY = 'obs_snapshot_quota'
     PERIOD_END_DATE = 'period_end_date'
 
     def __init__(self, redis_connection, db_conn, username, orgname=None):
-        super(DataObservatoryConfig, self).__init__(redis_connection, db_conn,
+        super(ObservatorySnapshotConfig, self).__init__(redis_connection, db_conn,
                                             username, orgname)
-        self._monthly_quota = self._db_config.data_observatory_monthly_quota
         self._period_end_date = date_parse(self._redis_config[self.PERIOD_END_DATE])
+        if self.SOFT_LIMIT_KEY in self._redis_config and self._redis_config[self.SOFT_LIMIT_KEY].lower() == 'true':
+            self._soft_limit = True
+        else:
+            self._soft_limit = False
+        # Mixed config between db and redis. If we don't update all the users
+        # in redis, we could use the db value as default
+        if self.QUOTA_KEY in self._redis_config:
+            self._monthly_quota = float(self._redis_config[self.QUOTA_KEY])
+        else:
+            self._monthly_quota = float(self._db_config.data_observatory_monthly_quota)
+        self._connection_str = self._db_config.data_observatory_connection_str
 
     @property
     def service_type(self):
-        return 'data_observatory'
+        return 'obs_snapshot'
 
     @property
     def monthly_quota(self):
@@ -54,6 +68,13 @@ class DataObservatoryConfig(ServiceConfig):
     def period_end_date(self):
         return self._period_end_date
 
+    @property
+    def soft_limit(self):
+        return self._soft_limit
+
+    @property
+    def connection_str(self):
+        return self._connection_str
 
 class RoutingConfig(ServiceConfig):
 
@@ -200,28 +221,6 @@ class GeocoderConfig(ServiceConfig):
         self.__parse_config(filtered_config, self._db_config)
         self.__check_config(filtered_config)
 
-    def __get_user_config(self, username, orgname):
-        user_config = self._redis_connection.hgetall(
-            "rails:users:{0}".format(username))
-        if not user_config:
-            raise ConfigException("""There is no user config available. Please check your configuration.'""")
-        else:
-            if orgname:
-                self.__get_organization_config(orgname, user_config)
-
-            return user_config
-
-    def __get_organization_config(self, orgname, user_config):
-        org_config = self._redis_connection.hgetall(
-            "rails:orgs:{0}".format(orgname))
-        if not org_config:
-            raise ConfigException("""There is no organization config available. Please check your configuration.'""")
-        else:
-            user_config[self.QUOTA_KEY] = org_config[self.QUOTA_KEY]
-            user_config[self.PERIOD_END_DATE] = org_config[self.PERIOD_END_DATE]
-            user_config[self.GOOGLE_GEOCODER_CLIENT_ID] = org_config[self.GOOGLE_GEOCODER_CLIENT_ID]
-            user_config[self.GOOGLE_GEOCODER_API_KEY] = org_config[self.GOOGLE_GEOCODER_API_KEY]
-
     def __check_config(self, filtered_config):
         if filtered_config[self.GEOCODER_TYPE].lower() == self.NOKIA_GEOCODER:
             if not set(self.NOKIA_GEOCODER_REDIS_MANDATORY_KEYS).issubset(set(filtered_config.keys())) or \
@@ -329,8 +328,10 @@ class GeocoderConfig(ServiceConfig):
 
 class ServicesDBConfig:
 
-    def __init__(self, db_conn):
+    def __init__(self, db_conn, username, orgname):
         self._db_conn = db_conn
+        self._username = username
+        self._orgname = orgname
         return self._build()
 
     def _build(self):
@@ -370,6 +371,12 @@ class ServicesDBConfig:
         else:
             do_conf = json.loads(do_conf_json)
             self._data_observatory_monthly_quota = do_conf['monthly_quota']
+            if self._orgname and self._orgname in do_conf['connection']['whitelist']:
+                self._data_observatory_connection_str = do_conf['connection']['staging']
+            elif self._username in do_conf['connection']['whitelist']:
+                self._data_observatory_connection_str = do_conf['connection']['staging']
+            else:
+                self._data_observatory_connection_str = do_conf['connection']['production']
 
     def _get_logger_config(self):
         logger_conf_json = self._get_conf('logger_conf')
@@ -431,6 +438,10 @@ class ServicesDBConfig:
     def data_observatory_monthly_quota(self):
         return self._data_observatory_monthly_quota
 
+    @property
+    def data_observatory_connection_str(self):
+        return self._data_observatory_connection_str
+
 
 class ServicesRedisConfig:
 
@@ -438,24 +449,24 @@ class ServicesRedisConfig:
     GOOGLE_GEOCODER_CLIENT_ID = 'google_maps_client_id'
     QUOTA_KEY = 'geocoding_quota'
     ISOLINES_QUOTA_KEY = 'here_isolines_quota'
+    OBS_SNAPSHOT_QUOTA_KEY = 'obs_snapshot_quota'
     PERIOD_END_DATE = 'period_end_date'
 
     def __init__(self, redis_conn):
         self._redis_connection = redis_conn
 
-    def build(self, username, password):
-        return self.__get_user_config(username, password)
+    def build(self, username, orgname):
+        return self.__get_user_config(username, orgname)
 
     def __get_user_config(self, username, orgname):
         user_config = self._redis_connection.hgetall(
             "rails:users:{0}".format(username))
         if not user_config:
             raise ConfigException("""There is no user config available. Please check your configuration.'""")
-        else:
-            if orgname:
-                self.__get_organization_config(orgname, user_config)
+        elif orgname:
+            self.__get_organization_config(orgname, user_config)
 
-            return user_config
+        return user_config
 
     def __get_organization_config(self, orgname, user_config):
         org_config = self._redis_connection.hgetall(
@@ -465,6 +476,8 @@ class ServicesRedisConfig:
         else:
             user_config[self.QUOTA_KEY] = org_config[self.QUOTA_KEY]
             user_config[self.ISOLINES_QUOTA_KEY] = org_config[self.ISOLINES_QUOTA_KEY]
+            if self.OBS_SNAPSHOT_QUOTA_KEY in org_config:
+                user_config[self.OBS_SNAPSHOT_QUOTA_KEY] = org_config[self.OBS_SNAPSHOT_QUOTA_KEY]
             user_config[self.PERIOD_END_DATE] = org_config[self.PERIOD_END_DATE]
             user_config[self.GOOGLE_GEOCODER_CLIENT_ID] = org_config[self.GOOGLE_GEOCODER_CLIENT_ID]
             user_config[self.GOOGLE_GEOCODER_API_KEY] = org_config[self.GOOGLE_GEOCODER_API_KEY]

server/lib/python/cartodb_services/cartodb_services/metrics/quota.py

@@ -73,9 +73,9 @@ class QuotaChecker:
         elif re.match('routing_mapzen',
                       self._user_service_config.service_type) is not None:
             return self.__check_routing_quota()
-        elif re.match('data_observatory',
+        elif re.match('obs_snapshot',
                       self._user_service_config.service_type) is not None:
-            return self.__check_data_observatory_quota()
+            return self.__check_obs_snapshot_quota()
         else:
             return False
 
@@ -118,7 +118,7 @@ class QuotaChecker:
         else:
             return False
 
-    def __check_data_observatory_quota(self):
+    def __check_obs_snapshot_quota(self):
         user_quota = self._user_service_config.monthly_quota
         today = date.today()
         service_type = self._user_service_config.service_type

server/lib/python/cartodb_services/setup.py

@@ -10,7 +10,7 @@ from setuptools import setup, find_packages
 setup(
     name='cartodb_services',
 
-    version='0.5.0',
+    version='0.5.3',
 
     description='CartoDB Services API Python Library',
 

server/lib/python/cartodb_services/test/test_config.py

@@ -1,5 +1,5 @@
 import test_helper
-from cartodb_services.metrics import GeocoderConfig, ConfigException
+from cartodb_services.metrics import GeocoderConfig, ObservatorySnapshotConfig, ConfigException
 from unittest import TestCase
 from nose.tools import assert_raises
 from mockredis import MockRedis
@@ -32,10 +32,29 @@ class TestConfig(TestCase):
         assert geocoder_config.soft_geocoding_limit is False
         assert geocoder_config.period_end_date.date() == yesterday.date()
 
+    def test_should_return_config_for_obs_snapshot(self):
+        yesterday = datetime.today() - timedelta(days=1)
+        test_helper.build_redis_user_config(self.redis_conn, 'test_user',
+                                            do_quota=100, soft_do_limit=True,
+                                            end_date=yesterday)
+        do_config = ObservatorySnapshotConfig(self.redis_conn, self.plpy_mock,
+                                          'test_user')
+        assert do_config.monthly_quota == 100
+        assert do_config.soft_limit is True
+        assert do_config.period_end_date.date() == yesterday.date()
+
+    def test_should_return_db_quota_if_not_redis_quota_config_obs_snapshot(self):
+        yesterday = datetime.today() - timedelta(days=1)
+        test_helper.build_redis_user_config(self.redis_conn, 'test_user',
+                                            end_date=yesterday)
+        do_config = ObservatorySnapshotConfig(self.redis_conn, self.plpy_mock,
+                                          'test_user')
+        assert do_config.monthly_quota == 100000
+        assert do_config.soft_limit is False
+        assert do_config.period_end_date.date() == yesterday.date()
+
     def test_should_raise_exception_when_missing_parameters(self):
         plpy_mock = test_helper.build_plpy_mock(empty=True)
         test_helper.build_redis_user_config(self.redis_conn, 'test_user')
-        assert_raises(ConfigException,
-                      GeocoderConfig,
-                      self.redis_conn, plpy_mock, 'test_user',
-                      None)
+        assert_raises(ConfigException, GeocoderConfig, self.redis_conn,
+                      plpy_mock, 'test_user', None)

server/lib/python/cartodb_services/test/test_helper.py

@@ -4,6 +4,7 @@ from mock import Mock
 
 def build_redis_user_config(redis_conn, username, quota=100, soft_limit=False,
                             service="heremaps", isolines_quota=0,
+                            do_quota=None, soft_do_limit=None,
                             end_date=datetime.today()):
     user_redis_name = "rails:users:{0}".format(username)
     redis_conn.hset(user_redis_name, 'soft_geocoding_limit', soft_limit)
@@ -11,23 +12,31 @@ def build_redis_user_config(redis_conn, username, quota=100, soft_limit=False,
     redis_conn.hset(user_redis_name, 'here_isolines_quota', isolines_quota)
     redis_conn.hset(user_redis_name, 'geocoder_type', service)
     redis_conn.hset(user_redis_name, 'period_end_date', end_date)
+    if do_quota:
+        redis_conn.hset(user_redis_name, 'obs_snapshot_quota', do_quota)
+    if soft_do_limit:
+        redis_conn.hset(user_redis_name, 'soft_obs_snapshot_limit',
+                        soft_do_limit)
     redis_conn.hset(user_redis_name, 'google_maps_client_id', '')
     redis_conn.hset(user_redis_name, 'google_maps_api_key', '')
 
 
-def build_redis_org_config(redis_conn, orgname, quota=100, isolines_quota=0,
+def build_redis_org_config(redis_conn, orgname, quota=100, service="heremaps",
+                           isolines_quota=0, do_quota=None,
                            end_date=datetime.today()):
     org_redis_name = "rails:orgs:{0}".format(orgname)
     redis_conn.hset(org_redis_name, 'geocoding_quota', quota)
     redis_conn.hset(org_redis_name, 'here_isolines_quota', isolines_quota)
+    if do_quota:
+        redis_conn.hset(org_redis_name, 'obs_snapshot_quota', do_quota)
     redis_conn.hset(org_redis_name, 'period_end_date', end_date)
     redis_conn.hset(org_redis_name, 'google_maps_client_id', '')
     redis_conn.hset(org_redis_name, 'google_maps_api_key', '')
 
 
-def increment_geocoder_uses(redis_conn, username, orgname=None,
-                            date=date.today(), service='geocoder_here',
-                            metric='success_responses', amount=20):
+def increment_service_uses(redis_conn, username, orgname=None,
+                           date=date.today(), service='geocoder_here',
+                           metric='success_responses', amount=20):
     prefix = 'org' if orgname else 'user'
     entity_name = orgname if orgname else username
     yearmonth = date.strftime('%Y%m')
@@ -52,4 +61,4 @@ def _plpy_execute_side_effect(*args, **kwargs):
     elif args[0] == "SELECT cartodb.CDB_Conf_GetConf('logger_conf') as conf":
         return [{'conf': '{"geocoder_log_path": "/dev/null"}'}]
     elif args[0] == "SELECT cartodb.CDB_Conf_GetConf('data_observatory_conf') as conf":
-        return [{'conf': '{"monthly_quota": 100000}'}]
+        return [{'conf': '{"connection": {"whitelist": ["ethervoid"], "production": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api", "staging": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api"}, "monthly_quota": 100000}'}]

server/lib/python/cartodb_services/test/test_quota_service.py

@@ -1,7 +1,7 @@
 import test_helper
 from mockredis import MockRedis
 from cartodb_services.metrics import QuotaService
-from cartodb_services.metrics import GeocoderConfig, RoutingConfig, DataObservatoryConfig
+from cartodb_services.metrics import GeocoderConfig, RoutingConfig, ObservatorySnapshotConfig
 from unittest import TestCase
 from nose.tools import assert_raises
 from datetime import datetime, date
@@ -28,40 +28,40 @@ class TestQuotaService(TestCase):
 
     def test_should_return_true_if_user_quota_is_not_completely_used(self):
         qs = self.__build_geocoder_quota_service('test_user')
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user')
+        test_helper.increment_service_uses(self.redis_conn, 'test_user')
         assert qs.check_user_quota() is True
 
     def test_should_return_true_if_org_quota_is_not_completely_used(self):
         qs = self.__build_geocoder_quota_service('test_user',
                                                  orgname='test_org')
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
-                                            orgname='test_org')
+        test_helper.increment_service_uses(self.redis_conn, 'test_user',
+                                           orgname='test_org')
         assert qs.check_user_quota() is True
 
     def test_should_return_false_if_user_quota_is_surpassed(self):
         qs = self.__build_geocoder_quota_service('test_user')
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
-                                            amount=300)
+        test_helper.increment_service_uses(self.redis_conn, 'test_user',
+                                           amount=300)
         assert qs.check_user_quota() is False
 
     def test_should_return_false_if_org_quota_is_surpassed(self):
         qs = self.__build_geocoder_quota_service('test_user',
                                                  orgname='test_org')
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
-                                            orgname='test_org', amount=400)
+        test_helper.increment_service_uses(self.redis_conn, 'test_user',
+                                           orgname='test_org', amount=400)
         assert qs.check_user_quota() is False
 
     def test_should_return_true_if_user_quota_is_surpassed_but_soft_limit_is_enabled(self):
         qs = self.__build_geocoder_quota_service('test_user', soft_limit=True)
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
-                                            amount=300)
+        test_helper.increment_service_uses(self.redis_conn, 'test_user',
+                                           amount=300)
         assert qs.check_user_quota() is True
 
     def test_should_return_true_if_org_quota_is_surpassed_but_soft_limit_is_enabled(self):
         qs = self.__build_geocoder_quota_service('test_user',
                                                  orgname='test_org',
                                                  soft_limit=True)
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
+        test_helper.increment_service_uses(self.redis_conn, 'test_user',
                                             orgname='test_org', amount=400)
         assert qs.check_user_quota() is True
 
@@ -109,29 +109,34 @@ class TestQuotaService(TestCase):
         qs.increment_success_service_use(amount=1500000)
         assert qs.check_user_quota() is False
 
-    def test_should_check_user_data_observatory_quota_correctly(self):
-        qs = self.__build_data_observatory_quota_service('test_user')
+    def test_should_check_user_obs_snapshot_quota_correctly(self):
+        qs = self.__build_obs_snapshot_quota_service('test_user')
         qs.increment_success_service_use()
         assert qs.check_user_quota() is True
         qs.increment_success_service_use(amount=100000)
         assert qs.check_user_quota() is False
 
-    def test_should_check_org_data_observatory_quota_correctly(self):
-        qs = self.__build_data_observatory_quota_service('test_user', orgname='testorg')
+    def test_should_check_org_obs_snapshot_quota_correctly(self):
+        qs = self.__build_obs_snapshot_quota_service('test_user',
+                                                     orgname='testorg')
         qs.increment_success_service_use()
         assert qs.check_user_quota() is True
         qs.increment_success_service_use(amount=100000)
         assert qs.check_user_quota() is False
 
     def __prepare_quota_service(self, username, quota, service, orgname,
-                                soft_limit, end_date):
+                                soft_limit, do_quota, soft_do_limit, end_date):
         test_helper.build_redis_user_config(self.redis_conn, username,
                                             quota=quota, service=service,
                                             soft_limit=soft_limit,
+                                            soft_do_limit=soft_do_limit,
+                                            do_quota=do_quota,
                                             end_date=end_date)
         if orgname:
             test_helper.build_redis_org_config(self.redis_conn, orgname,
-                                               quota=quota, end_date=end_date)
+                                               quota=quota, service=service,
+                                               do_quota=do_quota,
+                                               end_date=end_date)
         self._plpy_mock = test_helper.build_plpy_mock()
 
     def __build_geocoder_quota_service(self, username, quota=100,
@@ -139,27 +144,27 @@ class TestQuotaService(TestCase):
                                        soft_limit=False,
                                        end_date=datetime.today()):
         self.__prepare_quota_service(username, quota, service, orgname,
-                                     soft_limit, end_date)
+                                     soft_limit, 0, False, end_date)
         geocoder_config = GeocoderConfig(self.redis_conn, self._plpy_mock,
                                          username, orgname)
         return QuotaService(geocoder_config, redis_connection=self.redis_conn)
 
-    def __build_routing_quota_service(self, username, quota=100,
-                                      service='routing_mapzen', orgname=None,
-                                      soft_limit=False,
-                                      end_date=datetime.today()):
+    def __build_routing_quota_service(self, username, service='routing_mapzen',
+                                      orgname=None, soft_limit=False,
+                                      quota=100, end_date=datetime.today()):
         self.__prepare_quota_service(username, quota, service, orgname,
-                                     soft_limit, end_date)
+                                     soft_limit, 0, False, end_date)
         routing_config = RoutingConfig(self.redis_conn, self._plpy_mock,
                                        username, orgname)
         return QuotaService(routing_config, redis_connection=self.redis_conn)
 
-    def __build_data_observatory_quota_service(self, username, quota=100,
-                                               service='data_observatory', orgname=None,
+    def __build_obs_snapshot_quota_service(self, username, quota=100,
+                                               service='obs_snapshot',
+                                               orgname=None,
                                                soft_limit=False,
                                                end_date=datetime.today()):
-        self.__prepare_quota_service(username, quota, service, orgname,
-                                     soft_limit, end_date)
-        do_config = DataObservatoryConfig(self.redis_conn, self._plpy_mock,
+        self.__prepare_quota_service(username, 0, service, orgname, False,
+                                     quota, soft_limit, end_date)
+        do_config = ObservatorySnapshotConfig(self.redis_conn, self._plpy_mock,
                                           username, orgname)
         return QuotaService(do_config, redis_connection=self.redis_conn)

server/lib/python/cartodb_services/test/test_user_service.py

@@ -18,15 +18,15 @@ class TestUserService(TestCase):
 
     def test_user_used_quota_for_a_day(self):
         us = self.__build_user_service('test_user')
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
-                                            amount=400)
+        test_helper.increment_service_uses(self.redis_conn, 'test_user',
+                                           amount=400)
         assert us.used_quota(self.NOKIA_GEOCODER, date.today()) == 400
 
     def test_org_used_quota_for_a_day(self):
         us = self.__build_user_service('test_user', orgname='test_org')
-        test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
-                                            orgname='test_org',
-                                            amount=400)
+        test_helper.increment_service_uses(self.redis_conn, 'test_user',
+                                           orgname='test_org',
+                                           amount=400)
         assert us.used_quota(self.NOKIA_GEOCODER, date.today()) == 400
 
     def test_user_not_amount_in_used_quota_for_month_should_be_0(self):

test/integration/test_data_observatory_functions.py

@@ -15,30 +15,24 @@ class TestDataObservatoryFunctions(TestCase):
         )
 
     def test_if_get_demographic_snapshot_is_ok(self):
-        query = "SELECT duration, length, shape as the_geom " \
-                "FROM cdb_do_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669))".format(
-                    self.env_variables['api_key'])
-        routing = IntegrationTestHelper.execute_query(self.sql_api_url, query)
-        assert_not_equal(routing['the_geom'], None)
+        query = "SELECT obs_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669)) as snapshot;&api_key={0}".format(self.env_variables['api_key'])
+        snapshot = IntegrationTestHelper.execute_query(self.sql_api_url, query)
+        assert_not_equal(snapshot['snapshot'], None)
 
     def test_if_get_demographic_snapshot_without_api_key_raise_error(self):
-        query = "SELECT duration, length, shape as the_geom " \
-                "FROM cdb_do_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669))"
+        query = "SELECT obs_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669));"
         try:
             IntegrationTestHelper.execute_query(self.sql_api_url, query)
         except Exception as e:
             assert_equal(e.message[0], "The api_key must be provided")
 
     def test_if_get_segment_snapshot_is_ok(self):
-        query = "SELECT duration, length, shape as the_geom " \
-                "FROM cdb_do_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669))".format(
-                    self.env_variables['api_key'])
-        routing = IntegrationTestHelper.execute_query(self.sql_api_url, query)
-        assert_not_equal(routing['the_geom'], None)
+        query = "SELECT obs_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669)) as snapshot;&api_key={0}".format(self.env_variables['api_key'])
+        snapshot = IntegrationTestHelper.execute_query(self.sql_api_url, query)
+        assert_not_equal(snapshot['snapshot'], None)
 
     def test_if_get_segment_snapshot_without_api_key_raise_error(self):
-        query = "SELECT duration, length, shape as the_geom " \
-                "FROM cdb_do_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669))"
+        query = "SELECT obs_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669));"
         try:
             IntegrationTestHelper.execute_query(self.sql_api_url, query)
         except Exception as e: