From 607c8e82c96b991f5bb1889ff77b7e1657caf497 Mon Sep 17 00:00:00 2001 From: Guido Fioravantti Date: Wed, 5 Apr 2017 16:56:48 +0200 Subject: [PATCH 1/4] Update README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a2b74dd..0275417 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,9 @@ Steps to deploy a new Data Services API version : - install server and client extensions ``` - # in dataservices-api repo root path: + # in your workspace root path + git clone https://github.com/CartoDB/dataservices-api.git + cd dataservices-api cd client && sudo make install cd - cd server/extension && sudo make install From 160409c8c0c4372ff5b85c6cc02c6065a252ed28 Mon Sep 17 00:00:00 2001 From: Javier Goizueta Date: Wed, 12 Apr 2017 13:34:16 +0200 Subject: [PATCH 2/4] Fix the rate limits documentation The service name used to store the geocoding rate limits is geocoder, not geocoding. --- doc/rate_limits.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/doc/rate_limits.md b/doc/rate_limits.md index 333a8f3..78eafe8 100644 --- a/doc/rate_limits.md +++ b/doc/rate_limits.md @@ -83,7 +83,7 @@ The result is a JSON object with the configuration (`period` and `limit` attribu #### Example: ``` -SELECT cdb_dataservices_client.cdb_service_get_rate_limit('geocoding'); +SELECT cdb_dataservices_client.cdb_service_get_rate_limit('geocoder'); cdb_service_get_rate_limit --------------------------------- @@ -108,16 +108,18 @@ This functions doesn't return any value. #### Example -This will configure the geocoding service rate limit for user `myusername`, a non-organization user. +This will configure the geocoder service rate limit for user `myusername`, a non-organization user. The limit will be set at 1000 requests per day. Since the user doesn't belong to any organization, `NULL` will be passed to the organization argument; otherwise the name of the user's organization should be provided. +Note that the name of the geocoding services is `geocoder` and not `geocoding`. + ``` SELECT cdb_dataservices_client.cdb_service_set_user_rate_limit( 'myusername', NULL, - 'geocoding', + 'geocoder', '{"limit":1000,"period":86400}' ); @@ -140,7 +142,7 @@ This functions doesn't return any value. #### Example -This will configure the geocoding service rate limit for the `myorg` organization. +This will configure the geocoder service rate limit for the `myorg` organization. The limit will be set at 100 requests per hour. Note that even we're setting the default configuration for the whole organization, the name of a user of the organization must be provided for technical reasons. @@ -149,7 +151,7 @@ the name of a user of the organization must be provided for technical reasons. SELECT cdb_dataservices_client.cdb_service_set_org_rate_limit( 'myorgadmin', 'myorg', - 'geocoding', + 'geocoder', '{"limit":100,"period":3600}' ); @@ -172,7 +174,7 @@ This functions doesn't return any value. #### Example -This will configure the default geocoding service rate limit for all users +This will configure the default geocoder service rate limit for all users accesing the data-services server. The limit will be set at 10000 requests per month. Note that even we're setting the default configuration for the server, @@ -183,7 +185,7 @@ must be provided for technical reasons. SELECT cdb_dataservices_client.cdb_service_set_server_rate_limit( 'myorgadmin', 'myorg', - 'geocoding', + 'geocoder', '{"limit":10000,"period":108000}' ); From 6654b69212a7f18edc6e1655185d6f9554f7846f Mon Sep 17 00:00:00 2001 From: Guido Fioravantti Date: Wed, 5 Apr 2017 16:56:48 +0200 Subject: [PATCH 3/4] Update README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a2b74dd..0275417 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,9 @@ Steps to deploy a new Data Services API version : - install server and client extensions ``` - # in dataservices-api repo root path: + # in your workspace root path + git clone https://github.com/CartoDB/dataservices-api.git + cd dataservices-api cd client && sudo make install cd - cd server/extension && sudo make install From 63dbfa27b578cb8f20cde04c115abce44f130636 Mon Sep 17 00:00:00 2001 From: Mario de Frutos Date: Fri, 21 Apr 2017 11:12:38 +0200 Subject: [PATCH 4/4] Improve the install steps in the README --- README.md | 76 +++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 54 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 0275417..2bbb69f 100644 --- a/README.md +++ b/README.md @@ -23,22 +23,6 @@ Steps to deploy a new Data Services API version : ### Local install instructions -- install data services geocoder extension - - ``` - git clone https://github.com/CartoDB/data-services.git - cd data-services/geocoder/extension - sudo make install - ``` - -- install observatory extension - - ``` - git clone https://github.com/CartoDB/observatory-extension.git - cd observatory - sudo make install - ``` - - install server and client extensions ``` @@ -57,16 +41,64 @@ Steps to deploy a new Data Services API version : cd server/lib/python/cartodb_services && pip install -r requirements.txt && sudo pip install . --upgrade ``` -- install extensions in user database +- Create a database to hold all the server part and a user for it + + ```sql + CREATE DATABASE dataservices_db ENCODING = 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8'; + CREATE USER dataservices_user; + ``` + +- Install needed extensions in `dataservices_db` database ``` - create extension cdb_geocoder; - create extension plproxy; - create extension observatory; - create extension cdb_dataservices_server; - create extension cdb_dataservices_client; + psql -U postgres -d dataservices_db -c "BEGIN;CREATE EXTENSION IF NOT EXISTS plproxy; COMMIT" -e + psql -U postgres -d dataservices_db -c "BEGIN;CREATE EXTENSION IF NOT EXISTS cdb_dataservices_server; COMMIT" -e ``` +- [optional] install internal geocoder + - Make the extension available in postgres + ``` + git clone https://github.com/CartoDB/data-services.git + cd data-services/geocoder/extension + sudo make install + ``` + + - Make sure you have `wget` installed because is needed for the next step. + + - Go to `geocoder` folder and execute the `geocoder_dowload_dumps` script to download the internal geocoder data. + + - Once the data is downloaded, execute this command: + ```bash + geocoder_restore_dump postgres dataservices_db {DOWNLOADED_DUMPS_FOLDER}/*.sql + ``` + + - Now we have to make available the extension to be installed by postgres. Follow [this](https://github.com/CartoDB/data-services/tree/master/geocoder/extension) instructions. + + - Now install the extension with: + ``` + psql -U postgres -d dataservices_db -c "BEGIN;CREATE EXTENSION IF NOT EXISTS cdb_geocoder; COMMIT" -e + ``` + +- [optional] install data observatory extension + - Make the extension available in postgresql to be installed + ``` + git clone https://github.com/CartoDB/observatory-extension.git + cd observatory + sudo make install + ``` + - This extension needs data, dumps are not available so we're going to use the test fixtures to make it work. Execute: + ``` + psql -U postgres -d dataservices_db -f src/pg/test/fixtures/load_fixtures.sql + ``` + - Give permission to execute and select to the `dataservices_user` user: + ``` + psql -U postgres -d dataservices_db -c "BEGIN;CREATE EXTENSION IF NOT EXISTS observatoru; COMMIT" -e + psql -U postgres -d dataservices_db -c "BEGIN;GRANT SELECT ON ALL TABLES IN SCHEMA cdb_observatory TO dataservices_user; COMMIT" -e + psql -U postgres -d dataservices_db -c "BEGIN;GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_observatory TO dataservices_user; COMMIT" -e + psql -U postgres -d dataservices_db -c "BEGIN;GRANT SELECT ON ALL TABLES IN SCHEMA observatory TO dataservices_user; COMMIT" -e + psql -U postgres -d dataservices_db -c "BEGIN;GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA observatory TO dataservices_user; COMMIT" -e + ``` + ### Server configuration Configuration for the different services must be stored in the server database using `CDB_Conf_SetConf()`.