Compare commits
43 Commits
0.7.0-serv
...
0.7.1-serv
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ee13920a8e | ||
|
|
5efb81d809 | ||
|
|
4407ff630e | ||
|
|
a9c420ba7b | ||
|
|
976697c81e | ||
|
|
3de15de1c9 | ||
|
|
303c7c81fb | ||
|
|
b9757d2026 | ||
|
|
8aaaba3737 | ||
|
|
e73d6b84ff | ||
|
|
178651150c | ||
|
|
b0d614d032 | ||
|
|
b3e6d6731a | ||
|
|
403f0d2164 | ||
|
|
4c5c31cfea | ||
|
|
5ac9bd884f | ||
|
|
6b6c30ff17 | ||
|
|
3ed89c09d5 | ||
|
|
2ffee03e7b | ||
|
|
9fce6b558b | ||
|
|
6eb8df2955 | ||
|
|
1fbab0617f | ||
|
|
d18eb95575 | ||
|
|
3c1250ee72 | ||
|
|
7e8d666c8c | ||
|
|
f223ad9ca0 | ||
|
|
2c8cfa3032 | ||
|
|
0b53ede06c | ||
|
|
6bd854a395 | ||
|
|
30cb5869f1 | ||
|
|
187fe8f849 | ||
|
|
ee9e5aba38 | ||
|
|
6105f677af | ||
|
|
9e90ee3e1f | ||
|
|
ea24aa938f | ||
|
|
75d58dc836 | ||
|
|
652bd6abd6 | ||
|
|
dab46269d4 | ||
|
|
8eeef94eab | ||
|
|
14d02fd63f | ||
|
|
28b4b12ce4 | ||
|
|
dd2af74b03 | ||
|
|
30001b68a7 |
@@ -2,14 +2,10 @@
|
||||
|
||||
The CartoDB Data Services API offers a set of location based services that can be used programatically to empower your geospatial applications.
|
||||
|
||||
The geocoder functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CartoDB SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses.
|
||||
|
||||
The isoline functions provide a way to generate isolines in terms of distance and time, by means of the available isodistance and isochrone functions useful for Trade Areas Analysis.
|
||||
|
||||
## Documentation
|
||||
|
||||
* [Quickstart](quickstart.md)
|
||||
* [General Concepts](general_concepts.md)
|
||||
* [Overview](overview.md)
|
||||
* [Geocoding Functions](geocoding_functions.md)
|
||||
* [Isoline Functions](isoline_functions.md)
|
||||
* [Routing Functions](routing_functions.md)
|
||||
* [Quota Information](quota_information.md)
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
# General Concepts
|
||||
|
||||
The Data Services API offers geocoding and isoline services on top of the CartoDB SQL API by means of a set of functions. Each one of these functions is oriented to one kind of operation and returns the corresponding geometry (a `polygon` or a `point`), according to the input information.
|
||||
|
||||
The Data Services API decouples the geocoding and isoline services from the CartoDB Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically through authenticated requests.
|
||||
|
||||
The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/).
|
||||
|
||||
The Geocoder functions can return different types of geometries (points or polygons) as result of different geocoding processes. The CartoDB platform does not support multigeometry layers or datasets, therefore the final users of this Data Services API must check that they are using consistent geometry types inside a table to avoid further conflicts in the map visualization.
|
||||
|
||||
## Authentication
|
||||
|
||||
All requests performed to the CartoDB Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/cartodb-platform/sql-api/authentication/) guide.
|
||||
|
||||
## Errors
|
||||
|
||||
Errors are described in the response of the request. An example is as follows:
|
||||
|
||||
```json
|
||||
{
|
||||
error: [
|
||||
"The api_key must be provided"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Since the Data Services API is used on top of the CartoDB SQL API, you can refer to the [Making calls to the SQL API](/cartodb-platform/sql-api/making-calls/) documentation for help debugging your SQL errors.
|
||||
|
||||
If the requested information is not in the CartoDB geocoding database, or if CartoDB is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result.
|
||||
|
||||
## Limits
|
||||
|
||||
Usage of the Data Services API is subject to the CartoDB SQL API limits, stated in our [Terms of Service](https://cartodb.com/terms/#excessive).
|
||||
@@ -1,14 +1,31 @@
|
||||
# Geocoding Functions
|
||||
|
||||
The [geocoder](https://cartodb.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CartoDB SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. CartoDB provides functions for several different categories of geocoding through the Data Services API.
|
||||
|
||||
_**This service is subject to quota limitations, and extra fees may apply**. View the [Quota Information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) section for details, and recommendations, about to quota consumption._
|
||||
|
||||
Here is an example of how to geocode a single country:
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={api_key}
|
||||
```
|
||||
|
||||
In order to geocode an existent CartoDB dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Data Services API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset:
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_
|
||||
```
|
||||
|
||||
The following geocoding functions are available, grouped by categories.
|
||||
|
||||
## Country Geocoder
|
||||
|
||||
## Country geocoder
|
||||
|
||||
This function provides a country geocoding service. It recognizes the names of the different countries from different synonyms, such as their English name, their endonym, or their ISO2 or ISO3 codes.
|
||||
This function geocodes your data into country border geometries. It recognizes the names of the different countries either by different synonyms (such as their English name or their endonym), or by ISO (ISO2 or ISO3) codes.
|
||||
|
||||
### cdb_geocode_admin0_polygon(_country_name text_)
|
||||
|
||||
Geocodes the text name of a country into a country_name geometry, displayed as polygon data.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -34,12 +51,14 @@ UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon({country_column})
|
||||
```
|
||||
|
||||
|
||||
## Level-1 Administrative regions geocoder
|
||||
## Level-1 Administrative Regions Geocoder
|
||||
|
||||
The following functions provide a geocoding service for administrative regions of level 1 (or NUTS-1) such as states for the United States, *départements* in France or autonomous communities in Spain.
|
||||
This function geocodes your data into polygon geometries for [Level 1](https://en.wikipedia.org/wiki/Table_of_administrative_divisions_by_country), or [NUTS-1](https://en.wikipedia.org/wiki/NUTS_1_statistical_regions_of_England), administrative divisions (or units) of countries. For example, a "state" in the United States, "départements" in France, or an autonomous community in Spain.
|
||||
|
||||
### cdb_geocode_admin1_polygon(_admin1_name text_)
|
||||
|
||||
Geocodes the name of the province/state into a Level-1 administrative region, displayed as a polygon geometry.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -66,6 +85,8 @@ UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column})
|
||||
|
||||
### cdb_geocode_admin1_polygon(_admin1_name text, country_name text_)
|
||||
|
||||
Geocodes the name of the province/state for a specified country into a Level-1 administrative region, displayed as a polygon geometry.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -92,12 +113,14 @@ UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column},
|
||||
```
|
||||
|
||||
|
||||
## City geocoder
|
||||
## City Geocoder
|
||||
|
||||
The following functions provide a city geocoder service. It is recommended to use the more specific geocoding function -- the one that requires more parameters — in order for the result to be as accurate as possible when several cities share their name. If there are duplicate results for a city name, the city name with the highest population will be returned.
|
||||
This function geocodes your data into point geometries for names of cities. It is recommended to use geocoding functions that require more defined parameters — this returns more accurate results when several cities have the same name. _If there are duplicate results for a city name, the city name with the highest population will be returned._
|
||||
|
||||
### cdb_geocode_namedplace_point(_city_name text_)
|
||||
|
||||
Geocodes the text name of a city into a named place geometry, displayed as point data.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -124,6 +147,8 @@ UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column})
|
||||
|
||||
### cdb_geocode_namedplace_point(_city_name text, country_name text_)
|
||||
|
||||
Geocodes the text name of a city for a specified country into a named place point geometry.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -151,6 +176,7 @@ UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, 'S
|
||||
|
||||
### cdb_geocode_namedplace_point(_city_name text, admin1_name text, country_name text_)
|
||||
|
||||
Geocodes your data into a named place point geometry, containing the text name of a city, for a specified province/state and country. This is recommended for the most accurate geocoding of city data.
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -177,14 +203,16 @@ SELECT cdb_geocode_namedplace_point('New York', 'New York', 'USA')
|
||||
UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, {province_column}, 'USA')
|
||||
```
|
||||
|
||||
## Postal codes geocoder
|
||||
## Postal Code Geocoder
|
||||
|
||||
The following functions provide a postal code geocoding service that can be used to obtain points or polygon results. The postal code polygon geocoder covers the United States, France, Australia and Canada; a request for a different country will return an empty response.
|
||||
This function geocodes your data into point, or polygon, geometries for postal codes. The postal code polygon geocoder covers the United States, France, Australia and Canada; a request for a different country will return an empty response.
|
||||
|
||||
**Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://docs.cartodb.com/faqs/datasets-and-data/#why-does-cartodb-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details.
|
||||
|
||||
### cdb_geocode_postalcode_polygon(_postal_code text, country_name text_)
|
||||
|
||||
Goecodes the postal code for a specified country into a **polygon** geometry.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -212,6 +240,8 @@ UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_polygon({postal_code_co
|
||||
|
||||
### cdb_geocode_postalcode_point(_code text, country_name text_)
|
||||
|
||||
Goecodes the postal code for a specified country into a **point** geometry.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -237,12 +267,14 @@ SELECT cdb_geocode_postalcode_point('11211', 'USA')
|
||||
UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_point({postal_code_column}, 'USA')
|
||||
```
|
||||
|
||||
## IP addresses geocoder
|
||||
## IP Addresses Geocoder
|
||||
|
||||
This function provides an IP address geocoding service, for both IPv4 and IPv6 addresses.
|
||||
This function geocodes your data into point geometries for IP addresses. This is useful if you are analyzing location based data, based on a set of user's IP addresses.
|
||||
|
||||
### cdb_geocode_ipaddress_point(_ip_address text_)
|
||||
|
||||
Geocodes a postal code from a specified country into an IP address, displayed as a point geometry.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
@@ -267,22 +299,25 @@ SELECT cdb_geocode_ipaddress_point('102.23.34.1')
|
||||
```bash
|
||||
UPDATE {tablename} SET the_geom = cdb_geocode_ipaddress_point('102.23.34.1')
|
||||
```
|
||||
## Street-level geocoder
|
||||
|
||||
This function provides a street-level geocoding service. This service uses the street level geocoder defined for the user.
|
||||
## Street-Level Geocoder
|
||||
|
||||
**This service is subject to quota limitations, and extra fees may apply**. Please view our [terms and conditions](https://cartodb.com/terms/) and check out the [Quota information section](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) for details and recommendations related with quota usage.
|
||||
This function geocodes your data into a point geometry for a street address. CartoDB uses several different service providers for street-level geocoding, depending on your platform. If you access CartoDB on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future.
|
||||
|
||||
**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) for details and recommendations about quota consumption.
|
||||
|
||||
### cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_)
|
||||
|
||||
Geocodes a complete address into a single street geometry, displayed as point data.
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description
|
||||
--- | --- | ---
|
||||
`searchtext` | `text` | searchtext contains free-form text containing address elements. You can specify the searchtext parameter by itself, or you can specify it with other parameters to narrow your search. For example, you can specify the state or country parameters, along with a free-form address in the searchtext field.
|
||||
`city` | `text` | (Optional) Name of the city
|
||||
`state` | `text` | (Optional) Name of the state
|
||||
`country` | `text` | (Optional) Name of the country
|
||||
--- | --- | --- | ---
|
||||
`searchtext` | `text` | searchtext contains free-form text containing address elements. You can specify the searchtext parameter by itself, or with other parameters, to narrow your search. For example, you can specify the state or country parameters, along with a free-form address in the searchtext field.
|
||||
`city` | `text` | (Optional) Name of the city.
|
||||
`state` | `text` | (Optional) Name of the state.
|
||||
`country` | `text` | (Optional) Name of the country.
|
||||
|
||||
#### Returns
|
||||
|
||||
@@ -292,6 +327,8 @@ Geometry (point, EPSG 4326) or null
|
||||
|
||||
##### Select
|
||||
|
||||
Using SELECT for geocoding functions
|
||||
|
||||
```bash
|
||||
SELECT cdb_geocode_street_point('651 Lombard Street, San Francisco, California, United States')
|
||||
SELECT cdb_geocode_street_point('651 Lombard Street', 'San Francisco')
|
||||
|
||||
@@ -1,10 +1,27 @@
|
||||
# Isoline Functions
|
||||
|
||||
The following functions provide an isolines generator service based on time or distance. This service uses the isolines service defined for the user (currently, only the [HERE](https://developer.here.com/coverage-info) isolines service is available).
|
||||
[Isolines](https://cartodb.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through CartoDB are available by requesting a single function in the Data Services API.
|
||||
|
||||
**This service is subject to quota limitations, and extra fees may apply**. Please view our [terms and conditions](https://cartodb.com/terms/) and check out the [Quota information section](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) for details and recommendations related with quota usage.
|
||||
_**This service is subject to quota limitations, and extra fees may apply**. View the [Quota Information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) section for details, and recommendations, about to quota consumption._
|
||||
|
||||
### cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_)
|
||||
You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing.
|
||||
polygon({country_column})&api_key={api_key}
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300,600,900]::integer[])&api_key={api_key}
|
||||
```
|
||||
|
||||
Notice that you can make use of Postgres or PostGIS functions in your Data Services API requests, as the result is a geometry that can be handled by the system. For example, suppose you need to retrieve the centroid of a specific country, you can wrap the resulting geometry from the geocoder functions inside the PostGIS `ST_Centroid` function:
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={api_key}
|
||||
```
|
||||
|
||||
The following functions provide an isoline generator service, based on time or distance. This service uses the isolines service defined for your account. The default service limits the usage of displayed polygons represented on top of [HERE](https://developer.here.com/coverage-info) maps.
|
||||
|
||||
## cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_)
|
||||
|
||||
Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of distance (in meters).
|
||||
|
||||
#### Arguments
|
||||
|
||||
@@ -48,7 +65,9 @@ SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk
|
||||
INSERT INTO {table} (the_geom) SELECT (cdb_isodistance(the_geom, 'walk', string_to_array(distance, ',')::integer[])).the_geom FROM {points_table}
|
||||
```
|
||||
|
||||
### cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_)
|
||||
## cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_)
|
||||
|
||||
Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of time (in seconds).
|
||||
|
||||
#### Arguments
|
||||
|
||||
|
||||
41
doc/overview.md
Normal file
41
doc/overview.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Overview
|
||||
|
||||
By using CartoDB libraries and the SQL API, you can apply location data services to your maps with select functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. For example, you can transform an address to a geometry with [geocoding functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/geocoding-functions/#geocoding-functions). You can also use [isoline functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/isoline-functions/#isoline-functions) to calculate the travel distance for a defined area. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations.
|
||||
|
||||
**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/#quota-information).
|
||||
|
||||
_The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our CartoDB Platform._
|
||||
|
||||
## Data Services Integration
|
||||
|
||||
By using the SQL API to query the Data Services API functions, you can manage specific operations and the corresponding geometries (a `polygon` or a `point`), according to the input information.
|
||||
|
||||
The Data Services API decouples the geocoding and isoline services from the CartoDB Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically, through authenticated requests.
|
||||
|
||||
The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/).
|
||||
|
||||
**Note:** The Data Services API [geocoding functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CartoDB Platform does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization.
|
||||
|
||||
## Authentication
|
||||
|
||||
All requests performed to the CartoDB Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/cartodb-platform/sql-api/authentication/) documentation.
|
||||
|
||||
## Errors
|
||||
|
||||
Errors are described in the response of the request. An example is as follows:
|
||||
|
||||
```json
|
||||
{
|
||||
error: [
|
||||
"The api_key must be provided"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Since the Data Services API is used on top of the CartoDB SQL API, you can refer to the [Making calls to the SQL API](/cartodb-platform/sql-api/making-calls/) documentation for help debugging your SQL errors.
|
||||
|
||||
If the requested information is not in the CartoDB geocoding database, or if CartoDB is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result.
|
||||
|
||||
## Limits
|
||||
|
||||
Usage of the Data Services API is subject to the CartoDB SQL API limits, stated in our [Terms of Service](https://cartodb.com/terms/#excessive).
|
||||
@@ -1,26 +0,0 @@
|
||||
# Quickstart
|
||||
|
||||
If you are using the set of APIs and libraries that CartoDB offers, and you are handling your data with the SQL API, you can make your data visible in your maps by geocoding the dataset programatically. The set of isoline functions allow you to calculate the area which can be reached by travelling a given distance or time, useful for geospatial analysis, such as Trade Area Analysis.
|
||||
|
||||
Here is an example of how to geocode a single country:
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={Your API key}
|
||||
```
|
||||
|
||||
In order to geocode an existent CartoDB dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Geocoding API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset:
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon({country_column})&api_key={Your API key}
|
||||
```
|
||||
You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing.
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300,600,900]::integer[])&api_key={Your API key}
|
||||
```
|
||||
|
||||
Notice that you can make use of Postgres or PostGIS functions in your Data Services API requests, as the result is a geometry that can be handled by the system. For example, suppose you need to retrieve the centroid of a specific country, you can wrap the resulting geometry from the geocoder functions inside the PostGIS `ST_Centroid` function:
|
||||
|
||||
```bash
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={Your API key}
|
||||
```
|
||||
@@ -1,17 +1,11 @@
|
||||
# Quota Information
|
||||
|
||||
**This Data Services API provides functions which are subject to quota limitations, and extra fees may apply**. Please check our [terms and conditions](https://cartodb.com/terms/).
|
||||
**Based on your account plan, some of the Data Services API functions are subject to quota limitations and extra fees may apply.** View our [terms and conditions](https://cartodb.com/terms/), or [contact us](mailto:sales@cartodb.com) for details about which functions require service credits to your account.
|
||||
|
||||
The functions that require service credits to be used are as follows:
|
||||
## Quota Consumption
|
||||
|
||||
* cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_); from [Geocoding functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/geocoding-functions/)
|
||||
* cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_); from [Isoline functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/isoline-functions/)
|
||||
* cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_); from [Isoline functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/isoline-functions/)
|
||||
Quota consumption is calculated based on the number of request made for each function. Be mindful of the following usage recommendations when using the Data Services API functions:
|
||||
|
||||
## Quota consumption information and usage recommendations
|
||||
|
||||
Be mindful of the following when using the abovementioned functions:
|
||||
|
||||
* One credit per function call will be consumed, and the results are not cached. If the query is applied to a _N_ rows dataset, then _N_ credits will be used.
|
||||
* You are discouraged from using dynamic queries to these functions in your maps. This can result in credits consumption per map view. **Note:** Queries to the Data Services API and any of its functions in your maps may be forbidden in the future.
|
||||
* You are advised to store results of these queries into your datasets and refresh them as needed, so that you can have finer control on your credits' usage.
|
||||
* One credit per function call will be consumed. The results are not cached. If the query is applied to a _N_ rows dataset, then _N_ credits are consumed
|
||||
* Avoid running dynamic queries to these functions in your maps. This can result in credit consumption per map view. **Note:** Queries to the Data Services API, and any of its functions in your maps, may be forbidden in the future
|
||||
* It is advised to store results of these queries into your datasets, and refresh them as needed. This ensure more control of quota credits for your account
|
||||
|
||||
45
doc/routing_functions.md
Normal file
45
doc/routing_functions.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Routing Functions
|
||||
|
||||
Routing is the navigation from a defined start location to a defined end location. The calculated results are displayed as turn-by-turn directions on your map, based on the transportation mode that you specified. Routing services through CartoDB are available by requesting a single function in the Data Services API.
|
||||
|
||||
### cdb_route_point_to_point(_origin geometry(Point), destination geometry(Point), mode text, [options text[], units text]_)
|
||||
|
||||
#### Arguments
|
||||
|
||||
Name | Type | Description | Accepted values
|
||||
--- | --- | --- | ---
|
||||
`origin` | `geometry(Point)` | Origin point, in 4326 projection, which defines the start location. |
|
||||
`destination` | `geometry(Point)` | Destination point, in 4326 projection, which defines the end location. |
|
||||
`mode` | `text` | Type of transport used to calculate the isolines. | `car`, `walk`, `bicycle` or `public_transport`
|
||||
`options` | `text[]` | (Optional) Multiple options to add more capabilities to the analysis. See [Optional routing parameters](#optional-routing-parameters) for details.
|
||||
`units` | `text` | Unit used to represent the length of the route. | `kilometers`, `miles`. By default is `kilometers`
|
||||
|
||||
|
||||
#### Returns
|
||||
|
||||
Name | Type | Description
|
||||
--- | --- | ---
|
||||
`duration` | `integer` | Duration in seconds of the calculated route.
|
||||
`length` | `real` | Length in the defined unit in the `units` field. `kilometers` by default .
|
||||
`the_geom` | `geometry(LineString)` | LineString geometry of the calculated route in the 4326 projection.
|
||||
|
||||
#### Examples
|
||||
|
||||
##### Insert the values from the calculated route in your table
|
||||
|
||||
```bash
|
||||
INSERT INTO <TABLE> (duration, length, the_geom) SELECT duration, length, shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car')
|
||||
```
|
||||
##### Update the geometry field with the calculated route shape
|
||||
|
||||
```bash
|
||||
UPDATE <TABLE> SET the_geom = (SELECT shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car', ARRAY['mode_type=shortest']::text[]))
|
||||
```
|
||||
|
||||
### Optional routing parameters
|
||||
|
||||
The optional value parameters must be passed using the format: `option=value`.
|
||||
|
||||
Name | Type | Description | Accepted values
|
||||
--- | --- | --- | ---
|
||||
`mode_type` | `text` | Type of route calculation | `shortest` (this option only applies to the car transport mode)
|
||||
@@ -13,8 +13,8 @@ OLD_VERSIONS = $(wildcard old_versions/*.sql)
|
||||
# @see http://www.postgresql.org/docs/current/static/extend-pgxs.html
|
||||
DATA = $(NEW_EXTENSION_ARTIFACT) \
|
||||
$(OLD_VERSIONS) \
|
||||
cdb_dataservices_server--0.6.2--0.7.0.sql \
|
||||
cdb_dataservices_server--0.7.0--0.6.2.sql
|
||||
cdb_dataservices_server--0.7.0--0.7.1.sql \
|
||||
cdb_dataservices_server--0.7.1--0.7.0.sql
|
||||
|
||||
REGRESS = $(notdir $(basename $(wildcard test/sql/*test.sql)))
|
||||
TEST_DIR = test/
|
||||
|
||||
94
server/extension/cdb_dataservices_server--0.7.0--0.7.1.sql
Normal file
94
server/extension/cdb_dataservices_server--0.7.0--0.7.1.sql
Normal file
@@ -0,0 +1,94 @@
|
||||
--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
|
||||
-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
|
||||
\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.1'" to load this file. \quit
|
||||
DROP FUNCTION IF EXISTS cdb_dataservices_server._get_data_observatory_config(text, text);
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_obs_snapshot_config(username text, orgname text)
|
||||
RETURNS boolean AS $$
|
||||
cache_key = "user_obs_snapshot_config_{0}".format(username)
|
||||
if cache_key in GD:
|
||||
return False
|
||||
else:
|
||||
from cartodb_services.metrics import ObservatorySnapshotConfig
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
|
||||
obs_snapshot_config = ObservatorySnapshotConfig(redis_conn, plpy, username, orgname)
|
||||
GD[cache_key] = obs_snapshot_config
|
||||
return True
|
||||
$$ LANGUAGE plpythonu SECURITY DEFINER;
|
||||
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
|
||||
username TEXT,
|
||||
orgname TEXT,
|
||||
geom geometry(Geometry, 4326),
|
||||
time_span TEXT DEFAULT '2009 - 2013',
|
||||
geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
|
||||
RETURNS json AS $$
|
||||
from cartodb_services.metrics import QuotaService
|
||||
import json
|
||||
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
|
||||
|
||||
quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
|
||||
if not quota_service.check_user_quota():
|
||||
plpy.error('You have reached the limit of your quota')
|
||||
|
||||
try:
|
||||
obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
|
||||
result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
|
||||
if result:
|
||||
quota_service.increment_success_service_use()
|
||||
return result[0]['snapshot']
|
||||
else:
|
||||
quota_service.increment_empty_service_use()
|
||||
return None
|
||||
except BaseException as e:
|
||||
import sys, traceback
|
||||
type_, value_, traceback_ = sys.exc_info()
|
||||
quota_service.increment_failed_service_use()
|
||||
error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
|
||||
plpy.notice(traceback.format_tb(traceback_))
|
||||
plpy.error(error_msg)
|
||||
finally:
|
||||
quota_service.increment_total_service_use()
|
||||
$$ LANGUAGE plpythonu;
|
||||
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
|
||||
username TEXT,
|
||||
orgname TEXT,
|
||||
geom geometry(Geometry, 4326),
|
||||
geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
|
||||
RETURNS json AS $$
|
||||
from cartodb_services.metrics import QuotaService
|
||||
import json
|
||||
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
|
||||
|
||||
quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
|
||||
if not quota_service.check_user_quota():
|
||||
plpy.error('You have reached the limit of your quota')
|
||||
|
||||
try:
|
||||
obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
|
||||
result = plpy.execute(obs_plan, [geom, geometry_level])
|
||||
if result:
|
||||
quota_service.increment_success_service_use()
|
||||
return result[0]['snapshot']
|
||||
else:
|
||||
quota_service.increment_empty_service_use()
|
||||
return None
|
||||
except BaseException as e:
|
||||
import sys, traceback
|
||||
type_, value_, traceback_ = sys.exc_info()
|
||||
quota_service.increment_failed_service_use()
|
||||
error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
|
||||
plpy.notice(traceback.format_tb(traceback_))
|
||||
plpy.error(error_msg)
|
||||
finally:
|
||||
quota_service.increment_total_service_use()
|
||||
$$ LANGUAGE plpythonu;
|
||||
94
server/extension/cdb_dataservices_server--0.7.1--0.7.0.sql
Normal file
94
server/extension/cdb_dataservices_server--0.7.1--0.7.0.sql
Normal file
@@ -0,0 +1,94 @@
|
||||
--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES
|
||||
-- Complain if script is sourced in psql, rather than via CREATE EXTENSION
|
||||
\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.7.0'" to load this file. \quit
|
||||
DROP FUNCTION IF EXISTS cdb_dataservices_server._get_obs_snapshot_config(text, text);
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_data_observatory_config(username text, orgname text)
|
||||
RETURNS boolean AS $$
|
||||
cache_key = "user_data_observatory_config_{0}".format(username)
|
||||
if cache_key in GD:
|
||||
return False
|
||||
else:
|
||||
from cartodb_services.metrics import DataObservatoryConfig
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
|
||||
data_observatory_config = DataObservatoryConfig(redis_conn, plpy, username, orgname)
|
||||
GD[cache_key] = data_observatory_config
|
||||
return True
|
||||
$$ LANGUAGE plpythonu SECURITY DEFINER;
|
||||
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot(
|
||||
username TEXT,
|
||||
orgname TEXT,
|
||||
geom geometry(Geometry, 4326),
|
||||
time_span TEXT DEFAULT '2009 - 2013',
|
||||
geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
|
||||
RETURNS json AS $$
|
||||
from cartodb_services.metrics import QuotaService
|
||||
import json
|
||||
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
|
||||
|
||||
quota_service = QuotaService(user_data_observatory_config, redis_conn)
|
||||
if not quota_service.check_user_quota():
|
||||
plpy.error('You have reached the limit of your quota')
|
||||
|
||||
try:
|
||||
obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetDemographicSnapshot($1, $2, $3) as snapshot;", ["geometry(Geometry, 4326)", "text", "text"])
|
||||
result = plpy.execute(obs_plan, [geom, time_span, geometry_level])
|
||||
if result:
|
||||
quota_service.increment_success_service_use()
|
||||
return result[0]['snapshot']
|
||||
else:
|
||||
quota_service.increment_empty_service_use()
|
||||
return None
|
||||
except BaseException as e:
|
||||
import sys, traceback
|
||||
type_, value_, traceback_ = sys.exc_info()
|
||||
quota_service.increment_failed_service_use()
|
||||
error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e)
|
||||
plpy.notice(traceback.format_exception(type_, value_, traceback_))
|
||||
plpy.error(error_msg)
|
||||
finally:
|
||||
quota_service.increment_total_service_use()
|
||||
$$ LANGUAGE plpythonu;
|
||||
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot(
|
||||
username TEXT,
|
||||
orgname TEXT,
|
||||
geom geometry(Geometry, 4326),
|
||||
geometry_level TEXT DEFAULT '"us.census.tiger".block_group')
|
||||
RETURNS json AS $$
|
||||
from cartodb_services.metrics import QuotaService
|
||||
import json
|
||||
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
|
||||
|
||||
quota_service = QuotaService(user_data_observatory_config, redis_conn)
|
||||
if not quota_service.check_user_quota():
|
||||
plpy.error('You have reached the limit of your quota')
|
||||
|
||||
try:
|
||||
obs_plan = plpy.prepare("SELECT cdb_observatory.OBS_GetSegmentSnapshot($1, $2) as snapshot;", ["geometry(Geometry, 4326)", "text"])
|
||||
result = plpy.execute(obs_plan, [geom, geometry_level])
|
||||
if result:
|
||||
quota_service.increment_success_service_use()
|
||||
return result[0]['snapshot']
|
||||
else:
|
||||
quota_service.increment_empty_service_use()
|
||||
return None
|
||||
except BaseException as e:
|
||||
import sys, traceback
|
||||
type_, value_, traceback_ = sys.exc_info()
|
||||
quota_service.increment_failed_service_use()
|
||||
error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e)
|
||||
plpy.notice(traceback.format_tb(traceback_))
|
||||
plpy.error(error_msg)
|
||||
finally:
|
||||
quota_service.increment_total_service_use()
|
||||
$$ LANGUAGE plpythonu;
|
||||
1109
server/extension/cdb_dataservices_server--0.7.1.sql
Normal file
1109
server/extension/cdb_dataservices_server--0.7.1.sql
Normal file
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,5 @@
|
||||
comment = 'CartoDB dataservices server extension'
|
||||
default_version = '0.7.0'
|
||||
default_version = '0.7.1'
|
||||
requires = 'plpythonu, postgis, cdb_geocoder'
|
||||
superuser = true
|
||||
schema = cdb_dataservices_server
|
||||
|
||||
@@ -10,10 +10,10 @@ RETURNS json AS $$
|
||||
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
|
||||
|
||||
quota_service = QuotaService(user_data_observatory_config, redis_conn)
|
||||
quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
|
||||
if not quota_service.check_user_quota():
|
||||
plpy.error('You have reached the limit of your quota')
|
||||
|
||||
@@ -48,10 +48,10 @@ RETURNS json AS $$
|
||||
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection']
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_data_observatory_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_data_observatory_config = GD["user_data_observatory_config_{0}".format(username)]
|
||||
plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname)))
|
||||
user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)]
|
||||
|
||||
quota_service = QuotaService(user_data_observatory_config, redis_conn)
|
||||
quota_service = QuotaService(user_obs_snapshot_config, redis_conn)
|
||||
if not quota_service.check_user_quota():
|
||||
plpy.error('You have reached the limit of your quota')
|
||||
|
||||
|
||||
@@ -54,16 +54,16 @@ RETURNS boolean AS $$
|
||||
return True
|
||||
$$ LANGUAGE plpythonu SECURITY DEFINER;
|
||||
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_data_observatory_config(username text, orgname text)
|
||||
CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_obs_snapshot_config(username text, orgname text)
|
||||
RETURNS boolean AS $$
|
||||
cache_key = "user_data_observatory_config_{0}".format(username)
|
||||
cache_key = "user_obs_snapshot_config_{0}".format(username)
|
||||
if cache_key in GD:
|
||||
return False
|
||||
else:
|
||||
from cartodb_services.metrics import DataObservatoryConfig
|
||||
from cartodb_services.metrics import ObservatorySnapshotConfig
|
||||
plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username))
|
||||
redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection']
|
||||
data_observatory_config = DataObservatoryConfig(redis_conn, plpy, username, orgname)
|
||||
GD[cache_key] = data_observatory_config
|
||||
obs_snapshot_config = ObservatorySnapshotConfig(redis_conn, plpy, username, orgname)
|
||||
GD[cache_key] = obs_snapshot_config
|
||||
return True
|
||||
$$ LANGUAGE plpythonu SECURITY DEFINER;
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
from config import GeocoderConfig, IsolinesRoutingConfig, InternalGeocoderConfig, RoutingConfig, ConfigException, DataObservatoryConfig
|
||||
from config import GeocoderConfig, IsolinesRoutingConfig, InternalGeocoderConfig, RoutingConfig, ConfigException, ObservatorySnapshotConfig
|
||||
from quota import QuotaService
|
||||
from user import UserMetricsService
|
||||
|
||||
@@ -32,19 +32,31 @@ class ServiceConfig(object):
|
||||
def organization(self):
|
||||
return self._orgname
|
||||
|
||||
class DataObservatoryConfig(ServiceConfig):
|
||||
|
||||
class ObservatorySnapshotConfig(ServiceConfig):
|
||||
|
||||
SOFT_LIMIT_KEY = 'soft_obs_snapshot_limit'
|
||||
QUOTA_KEY = 'obs_snapshot_quota'
|
||||
PERIOD_END_DATE = 'period_end_date'
|
||||
|
||||
def __init__(self, redis_connection, db_conn, username, orgname=None):
|
||||
super(DataObservatoryConfig, self).__init__(redis_connection, db_conn,
|
||||
super(ObservatorySnapshotConfig, self).__init__(redis_connection, db_conn,
|
||||
username, orgname)
|
||||
self._monthly_quota = self._db_config.data_observatory_monthly_quota
|
||||
self._period_end_date = date_parse(self._redis_config[self.PERIOD_END_DATE])
|
||||
if self.SOFT_LIMIT_KEY in self._redis_config and self._redis_config[self.SOFT_LIMIT_KEY].lower() == 'true':
|
||||
self._soft_limit = True
|
||||
else:
|
||||
self._soft_limit = False
|
||||
# Mixed config between db and redis. If we don't update all the users
|
||||
# in redis, we could use the db value as default
|
||||
if self.QUOTA_KEY in self._redis_config:
|
||||
self._monthly_quota = float(self._redis_config[self.QUOTA_KEY])
|
||||
else:
|
||||
self._monthly_quota = float(self._db_config.data_observatory_monthly_quota)
|
||||
|
||||
@property
|
||||
def service_type(self):
|
||||
return 'data_observatory'
|
||||
return 'obs_snapshot'
|
||||
|
||||
@property
|
||||
def monthly_quota(self):
|
||||
@@ -54,6 +66,10 @@ class DataObservatoryConfig(ServiceConfig):
|
||||
def period_end_date(self):
|
||||
return self._period_end_date
|
||||
|
||||
@property
|
||||
def soft_limit(self):
|
||||
return self._soft_limit
|
||||
|
||||
|
||||
class RoutingConfig(ServiceConfig):
|
||||
|
||||
@@ -200,28 +216,6 @@ class GeocoderConfig(ServiceConfig):
|
||||
self.__parse_config(filtered_config, self._db_config)
|
||||
self.__check_config(filtered_config)
|
||||
|
||||
def __get_user_config(self, username, orgname):
|
||||
user_config = self._redis_connection.hgetall(
|
||||
"rails:users:{0}".format(username))
|
||||
if not user_config:
|
||||
raise ConfigException("""There is no user config available. Please check your configuration.'""")
|
||||
else:
|
||||
if orgname:
|
||||
self.__get_organization_config(orgname, user_config)
|
||||
|
||||
return user_config
|
||||
|
||||
def __get_organization_config(self, orgname, user_config):
|
||||
org_config = self._redis_connection.hgetall(
|
||||
"rails:orgs:{0}".format(orgname))
|
||||
if not org_config:
|
||||
raise ConfigException("""There is no organization config available. Please check your configuration.'""")
|
||||
else:
|
||||
user_config[self.QUOTA_KEY] = org_config[self.QUOTA_KEY]
|
||||
user_config[self.PERIOD_END_DATE] = org_config[self.PERIOD_END_DATE]
|
||||
user_config[self.GOOGLE_GEOCODER_CLIENT_ID] = org_config[self.GOOGLE_GEOCODER_CLIENT_ID]
|
||||
user_config[self.GOOGLE_GEOCODER_API_KEY] = org_config[self.GOOGLE_GEOCODER_API_KEY]
|
||||
|
||||
def __check_config(self, filtered_config):
|
||||
if filtered_config[self.GEOCODER_TYPE].lower() == self.NOKIA_GEOCODER:
|
||||
if not set(self.NOKIA_GEOCODER_REDIS_MANDATORY_KEYS).issubset(set(filtered_config.keys())) or \
|
||||
@@ -438,24 +432,24 @@ class ServicesRedisConfig:
|
||||
GOOGLE_GEOCODER_CLIENT_ID = 'google_maps_client_id'
|
||||
QUOTA_KEY = 'geocoding_quota'
|
||||
ISOLINES_QUOTA_KEY = 'here_isolines_quota'
|
||||
OBS_SNAPSHOT_QUOTA_KEY = 'obs_snapshot_quota'
|
||||
PERIOD_END_DATE = 'period_end_date'
|
||||
|
||||
def __init__(self, redis_conn):
|
||||
self._redis_connection = redis_conn
|
||||
|
||||
def build(self, username, password):
|
||||
return self.__get_user_config(username, password)
|
||||
def build(self, username, orgname):
|
||||
return self.__get_user_config(username, orgname)
|
||||
|
||||
def __get_user_config(self, username, orgname):
|
||||
user_config = self._redis_connection.hgetall(
|
||||
"rails:users:{0}".format(username))
|
||||
if not user_config:
|
||||
raise ConfigException("""There is no user config available. Please check your configuration.'""")
|
||||
else:
|
||||
if orgname:
|
||||
self.__get_organization_config(orgname, user_config)
|
||||
elif orgname:
|
||||
self.__get_organization_config(orgname, user_config)
|
||||
|
||||
return user_config
|
||||
return user_config
|
||||
|
||||
def __get_organization_config(self, orgname, user_config):
|
||||
org_config = self._redis_connection.hgetall(
|
||||
@@ -465,6 +459,8 @@ class ServicesRedisConfig:
|
||||
else:
|
||||
user_config[self.QUOTA_KEY] = org_config[self.QUOTA_KEY]
|
||||
user_config[self.ISOLINES_QUOTA_KEY] = org_config[self.ISOLINES_QUOTA_KEY]
|
||||
if self.OBS_SNAPSHOT_QUOTA_KEY in org_config:
|
||||
user_config[self.OBS_SNAPSHOT_QUOTA_KEY] = org_config[self.OBS_SNAPSHOT_QUOTA_KEY]
|
||||
user_config[self.PERIOD_END_DATE] = org_config[self.PERIOD_END_DATE]
|
||||
user_config[self.GOOGLE_GEOCODER_CLIENT_ID] = org_config[self.GOOGLE_GEOCODER_CLIENT_ID]
|
||||
user_config[self.GOOGLE_GEOCODER_API_KEY] = org_config[self.GOOGLE_GEOCODER_API_KEY]
|
||||
|
||||
@@ -73,9 +73,9 @@ class QuotaChecker:
|
||||
elif re.match('routing_mapzen',
|
||||
self._user_service_config.service_type) is not None:
|
||||
return self.__check_routing_quota()
|
||||
elif re.match('data_observatory',
|
||||
elif re.match('obs_snapshot',
|
||||
self._user_service_config.service_type) is not None:
|
||||
return self.__check_data_observatory_quota()
|
||||
return self.__check_obs_snapshot_quota()
|
||||
else:
|
||||
return False
|
||||
|
||||
@@ -118,7 +118,7 @@ class QuotaChecker:
|
||||
else:
|
||||
return False
|
||||
|
||||
def __check_data_observatory_quota(self):
|
||||
def __check_obs_snapshot_quota(self):
|
||||
user_quota = self._user_service_config.monthly_quota
|
||||
today = date.today()
|
||||
service_type = self._user_service_config.service_type
|
||||
|
||||
@@ -10,7 +10,7 @@ from setuptools import setup, find_packages
|
||||
setup(
|
||||
name='cartodb_services',
|
||||
|
||||
version='0.5.0',
|
||||
version='0.5.1',
|
||||
|
||||
description='CartoDB Services API Python Library',
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import test_helper
|
||||
from cartodb_services.metrics import GeocoderConfig, ConfigException
|
||||
from cartodb_services.metrics import GeocoderConfig, ObservatorySnapshotConfig, ConfigException
|
||||
from unittest import TestCase
|
||||
from nose.tools import assert_raises
|
||||
from mockredis import MockRedis
|
||||
@@ -32,10 +32,29 @@ class TestConfig(TestCase):
|
||||
assert geocoder_config.soft_geocoding_limit is False
|
||||
assert geocoder_config.period_end_date.date() == yesterday.date()
|
||||
|
||||
def test_should_return_config_for_obs_snapshot(self):
|
||||
yesterday = datetime.today() - timedelta(days=1)
|
||||
test_helper.build_redis_user_config(self.redis_conn, 'test_user',
|
||||
do_quota=100, soft_do_limit=True,
|
||||
end_date=yesterday)
|
||||
do_config = ObservatorySnapshotConfig(self.redis_conn, self.plpy_mock,
|
||||
'test_user')
|
||||
assert do_config.monthly_quota == 100
|
||||
assert do_config.soft_limit is True
|
||||
assert do_config.period_end_date.date() == yesterday.date()
|
||||
|
||||
def test_should_return_db_quota_if_not_redis_quota_config_obs_snapshot(self):
|
||||
yesterday = datetime.today() - timedelta(days=1)
|
||||
test_helper.build_redis_user_config(self.redis_conn, 'test_user',
|
||||
end_date=yesterday)
|
||||
do_config = ObservatorySnapshotConfig(self.redis_conn, self.plpy_mock,
|
||||
'test_user')
|
||||
assert do_config.monthly_quota == 100000
|
||||
assert do_config.soft_limit is False
|
||||
assert do_config.period_end_date.date() == yesterday.date()
|
||||
|
||||
def test_should_raise_exception_when_missing_parameters(self):
|
||||
plpy_mock = test_helper.build_plpy_mock(empty=True)
|
||||
test_helper.build_redis_user_config(self.redis_conn, 'test_user')
|
||||
assert_raises(ConfigException,
|
||||
GeocoderConfig,
|
||||
self.redis_conn, plpy_mock, 'test_user',
|
||||
None)
|
||||
assert_raises(ConfigException, GeocoderConfig, self.redis_conn,
|
||||
plpy_mock, 'test_user', None)
|
||||
|
||||
@@ -4,6 +4,7 @@ from mock import Mock
|
||||
|
||||
def build_redis_user_config(redis_conn, username, quota=100, soft_limit=False,
|
||||
service="heremaps", isolines_quota=0,
|
||||
do_quota=None, soft_do_limit=None,
|
||||
end_date=datetime.today()):
|
||||
user_redis_name = "rails:users:{0}".format(username)
|
||||
redis_conn.hset(user_redis_name, 'soft_geocoding_limit', soft_limit)
|
||||
@@ -11,23 +12,31 @@ def build_redis_user_config(redis_conn, username, quota=100, soft_limit=False,
|
||||
redis_conn.hset(user_redis_name, 'here_isolines_quota', isolines_quota)
|
||||
redis_conn.hset(user_redis_name, 'geocoder_type', service)
|
||||
redis_conn.hset(user_redis_name, 'period_end_date', end_date)
|
||||
if do_quota:
|
||||
redis_conn.hset(user_redis_name, 'obs_snapshot_quota', do_quota)
|
||||
if soft_do_limit:
|
||||
redis_conn.hset(user_redis_name, 'soft_obs_snapshot_limit',
|
||||
soft_do_limit)
|
||||
redis_conn.hset(user_redis_name, 'google_maps_client_id', '')
|
||||
redis_conn.hset(user_redis_name, 'google_maps_api_key', '')
|
||||
|
||||
|
||||
def build_redis_org_config(redis_conn, orgname, quota=100, isolines_quota=0,
|
||||
def build_redis_org_config(redis_conn, orgname, quota=100, service="heremaps",
|
||||
isolines_quota=0, do_quota=None,
|
||||
end_date=datetime.today()):
|
||||
org_redis_name = "rails:orgs:{0}".format(orgname)
|
||||
redis_conn.hset(org_redis_name, 'geocoding_quota', quota)
|
||||
redis_conn.hset(org_redis_name, 'here_isolines_quota', isolines_quota)
|
||||
if do_quota:
|
||||
redis_conn.hset(org_redis_name, 'obs_snapshot_quota', do_quota)
|
||||
redis_conn.hset(org_redis_name, 'period_end_date', end_date)
|
||||
redis_conn.hset(org_redis_name, 'google_maps_client_id', '')
|
||||
redis_conn.hset(org_redis_name, 'google_maps_api_key', '')
|
||||
|
||||
|
||||
def increment_geocoder_uses(redis_conn, username, orgname=None,
|
||||
date=date.today(), service='geocoder_here',
|
||||
metric='success_responses', amount=20):
|
||||
def increment_service_uses(redis_conn, username, orgname=None,
|
||||
date=date.today(), service='geocoder_here',
|
||||
metric='success_responses', amount=20):
|
||||
prefix = 'org' if orgname else 'user'
|
||||
entity_name = orgname if orgname else username
|
||||
yearmonth = date.strftime('%Y%m')
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import test_helper
|
||||
from mockredis import MockRedis
|
||||
from cartodb_services.metrics import QuotaService
|
||||
from cartodb_services.metrics import GeocoderConfig, RoutingConfig, DataObservatoryConfig
|
||||
from cartodb_services.metrics import GeocoderConfig, RoutingConfig, ObservatorySnapshotConfig
|
||||
from unittest import TestCase
|
||||
from nose.tools import assert_raises
|
||||
from datetime import datetime, date
|
||||
@@ -28,40 +28,40 @@ class TestQuotaService(TestCase):
|
||||
|
||||
def test_should_return_true_if_user_quota_is_not_completely_used(self):
|
||||
qs = self.__build_geocoder_quota_service('test_user')
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user')
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user')
|
||||
assert qs.check_user_quota() is True
|
||||
|
||||
def test_should_return_true_if_org_quota_is_not_completely_used(self):
|
||||
qs = self.__build_geocoder_quota_service('test_user',
|
||||
orgname='test_org')
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
|
||||
orgname='test_org')
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user',
|
||||
orgname='test_org')
|
||||
assert qs.check_user_quota() is True
|
||||
|
||||
def test_should_return_false_if_user_quota_is_surpassed(self):
|
||||
qs = self.__build_geocoder_quota_service('test_user')
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
|
||||
amount=300)
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user',
|
||||
amount=300)
|
||||
assert qs.check_user_quota() is False
|
||||
|
||||
def test_should_return_false_if_org_quota_is_surpassed(self):
|
||||
qs = self.__build_geocoder_quota_service('test_user',
|
||||
orgname='test_org')
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
|
||||
orgname='test_org', amount=400)
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user',
|
||||
orgname='test_org', amount=400)
|
||||
assert qs.check_user_quota() is False
|
||||
|
||||
def test_should_return_true_if_user_quota_is_surpassed_but_soft_limit_is_enabled(self):
|
||||
qs = self.__build_geocoder_quota_service('test_user', soft_limit=True)
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
|
||||
amount=300)
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user',
|
||||
amount=300)
|
||||
assert qs.check_user_quota() is True
|
||||
|
||||
def test_should_return_true_if_org_quota_is_surpassed_but_soft_limit_is_enabled(self):
|
||||
qs = self.__build_geocoder_quota_service('test_user',
|
||||
orgname='test_org',
|
||||
soft_limit=True)
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user',
|
||||
orgname='test_org', amount=400)
|
||||
assert qs.check_user_quota() is True
|
||||
|
||||
@@ -109,29 +109,34 @@ class TestQuotaService(TestCase):
|
||||
qs.increment_success_service_use(amount=1500000)
|
||||
assert qs.check_user_quota() is False
|
||||
|
||||
def test_should_check_user_data_observatory_quota_correctly(self):
|
||||
qs = self.__build_data_observatory_quota_service('test_user')
|
||||
def test_should_check_user_obs_snapshot_quota_correctly(self):
|
||||
qs = self.__build_obs_snapshot_quota_service('test_user')
|
||||
qs.increment_success_service_use()
|
||||
assert qs.check_user_quota() is True
|
||||
qs.increment_success_service_use(amount=100000)
|
||||
assert qs.check_user_quota() is False
|
||||
|
||||
def test_should_check_org_data_observatory_quota_correctly(self):
|
||||
qs = self.__build_data_observatory_quota_service('test_user', orgname='testorg')
|
||||
def test_should_check_org_obs_snapshot_quota_correctly(self):
|
||||
qs = self.__build_obs_snapshot_quota_service('test_user',
|
||||
orgname='testorg')
|
||||
qs.increment_success_service_use()
|
||||
assert qs.check_user_quota() is True
|
||||
qs.increment_success_service_use(amount=100000)
|
||||
assert qs.check_user_quota() is False
|
||||
|
||||
def __prepare_quota_service(self, username, quota, service, orgname,
|
||||
soft_limit, end_date):
|
||||
soft_limit, do_quota, soft_do_limit, end_date):
|
||||
test_helper.build_redis_user_config(self.redis_conn, username,
|
||||
quota=quota, service=service,
|
||||
soft_limit=soft_limit,
|
||||
soft_do_limit=soft_do_limit,
|
||||
do_quota=do_quota,
|
||||
end_date=end_date)
|
||||
if orgname:
|
||||
test_helper.build_redis_org_config(self.redis_conn, orgname,
|
||||
quota=quota, end_date=end_date)
|
||||
quota=quota, service=service,
|
||||
do_quota=do_quota,
|
||||
end_date=end_date)
|
||||
self._plpy_mock = test_helper.build_plpy_mock()
|
||||
|
||||
def __build_geocoder_quota_service(self, username, quota=100,
|
||||
@@ -139,27 +144,27 @@ class TestQuotaService(TestCase):
|
||||
soft_limit=False,
|
||||
end_date=datetime.today()):
|
||||
self.__prepare_quota_service(username, quota, service, orgname,
|
||||
soft_limit, end_date)
|
||||
soft_limit, 0, False, end_date)
|
||||
geocoder_config = GeocoderConfig(self.redis_conn, self._plpy_mock,
|
||||
username, orgname)
|
||||
return QuotaService(geocoder_config, redis_connection=self.redis_conn)
|
||||
|
||||
def __build_routing_quota_service(self, username, quota=100,
|
||||
service='routing_mapzen', orgname=None,
|
||||
soft_limit=False,
|
||||
end_date=datetime.today()):
|
||||
def __build_routing_quota_service(self, username, service='routing_mapzen',
|
||||
orgname=None, soft_limit=False,
|
||||
quota=100, end_date=datetime.today()):
|
||||
self.__prepare_quota_service(username, quota, service, orgname,
|
||||
soft_limit, end_date)
|
||||
soft_limit, 0, False, end_date)
|
||||
routing_config = RoutingConfig(self.redis_conn, self._plpy_mock,
|
||||
username, orgname)
|
||||
return QuotaService(routing_config, redis_connection=self.redis_conn)
|
||||
|
||||
def __build_data_observatory_quota_service(self, username, quota=100,
|
||||
service='data_observatory', orgname=None,
|
||||
def __build_obs_snapshot_quota_service(self, username, quota=100,
|
||||
service='obs_snapshot',
|
||||
orgname=None,
|
||||
soft_limit=False,
|
||||
end_date=datetime.today()):
|
||||
self.__prepare_quota_service(username, quota, service, orgname,
|
||||
soft_limit, end_date)
|
||||
do_config = DataObservatoryConfig(self.redis_conn, self._plpy_mock,
|
||||
self.__prepare_quota_service(username, 0, service, orgname, False,
|
||||
quota, soft_limit, end_date)
|
||||
do_config = ObservatorySnapshotConfig(self.redis_conn, self._plpy_mock,
|
||||
username, orgname)
|
||||
return QuotaService(do_config, redis_connection=self.redis_conn)
|
||||
|
||||
@@ -18,15 +18,15 @@ class TestUserService(TestCase):
|
||||
|
||||
def test_user_used_quota_for_a_day(self):
|
||||
us = self.__build_user_service('test_user')
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
|
||||
amount=400)
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user',
|
||||
amount=400)
|
||||
assert us.used_quota(self.NOKIA_GEOCODER, date.today()) == 400
|
||||
|
||||
def test_org_used_quota_for_a_day(self):
|
||||
us = self.__build_user_service('test_user', orgname='test_org')
|
||||
test_helper.increment_geocoder_uses(self.redis_conn, 'test_user',
|
||||
orgname='test_org',
|
||||
amount=400)
|
||||
test_helper.increment_service_uses(self.redis_conn, 'test_user',
|
||||
orgname='test_org',
|
||||
amount=400)
|
||||
assert us.used_quota(self.NOKIA_GEOCODER, date.today()) == 400
|
||||
|
||||
def test_user_not_amount_in_used_quota_for_month_should_be_0(self):
|
||||
|
||||
@@ -15,30 +15,24 @@ class TestDataObservatoryFunctions(TestCase):
|
||||
)
|
||||
|
||||
def test_if_get_demographic_snapshot_is_ok(self):
|
||||
query = "SELECT duration, length, shape as the_geom " \
|
||||
"FROM cdb_do_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669))".format(
|
||||
self.env_variables['api_key'])
|
||||
routing = IntegrationTestHelper.execute_query(self.sql_api_url, query)
|
||||
assert_not_equal(routing['the_geom'], None)
|
||||
query = "SELECT obs_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669)) as snapshot;&api_key={0}".format(self.env_variables['api_key'])
|
||||
snapshot = IntegrationTestHelper.execute_query(self.sql_api_url, query)
|
||||
assert_not_equal(snapshot['snapshot'], None)
|
||||
|
||||
def test_if_get_demographic_snapshot_without_api_key_raise_error(self):
|
||||
query = "SELECT duration, length, shape as the_geom " \
|
||||
"FROM cdb_do_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669))"
|
||||
query = "SELECT obs_get_demographic_snapshot(CDB_LatLng(40.704512, -73.936669));"
|
||||
try:
|
||||
IntegrationTestHelper.execute_query(self.sql_api_url, query)
|
||||
except Exception as e:
|
||||
assert_equal(e.message[0], "The api_key must be provided")
|
||||
|
||||
def test_if_get_segment_snapshot_is_ok(self):
|
||||
query = "SELECT duration, length, shape as the_geom " \
|
||||
"FROM cdb_do_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669))".format(
|
||||
self.env_variables['api_key'])
|
||||
routing = IntegrationTestHelper.execute_query(self.sql_api_url, query)
|
||||
assert_not_equal(routing['the_geom'], None)
|
||||
query = "SELECT obs_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669)) as snapshot;&api_key={0}".format(self.env_variables['api_key'])
|
||||
snapshot = IntegrationTestHelper.execute_query(self.sql_api_url, query)
|
||||
assert_not_equal(snapshot['snapshot'], None)
|
||||
|
||||
def test_if_get_segment_snapshot_without_api_key_raise_error(self):
|
||||
query = "SELECT duration, length, shape as the_geom " \
|
||||
"FROM cdb_do_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669))"
|
||||
query = "SELECT obs_get_segment_snapshot(CDB_LatLng(40.704512, -73.936669));"
|
||||
try:
|
||||
IntegrationTestHelper.execute_query(self.sql_api_url, query)
|
||||
except Exception as e:
|
||||
|
||||
Reference in New Issue
Block a user