Compare commits

..

69 Commits
1.6.2 ... 1.5.2

Author SHA1 Message Date
Sandro Santilli
a660fdee12 Release 1.5.2 2013-12-05 14:05:41 +01:00
Sandro Santilli
e3b0813230 Fix use of old layergroups on mapnik upgrade (#97) 2013-12-05 14:02:16 +01:00
Sandro Santilli
8347dd4257 Set grainstore's GC run probability, for documentation purpose
It sets it to the current grainstore default, so nothing changes.
2013-12-05 11:11:13 +01:00
Sandro Santilli
dac89b30cb Allow requesting run_test.sh to prepare redis but not postgresql
Adds --nocreate-pg, --nocreate-redis, --nodrop-pg, --nodrop-redis
NOTE that dropping pg is still unimplemented
2013-12-05 11:07:12 +01:00
Sandro Santilli
5951608d2e Add backward-compatibility fix item in NEWS (#96) 2013-11-29 13:33:53 +01:00
Sandro Santilli
13c39f598b Prepare for 1.5.2 2013-11-29 13:32:28 +01:00
Sandro Santilli
6ca3135a3c Revert "fixed #91" -- the fix was for an unconfirmed bug
This reverts commit 9155724082.
See #38 for further action
2013-11-29 13:25:10 +01:00
Sandro Santilli
f582b04b23 Enable test for fetcing tiles of private tables using api_key
See #39 and #91
2013-11-29 13:20:21 +01:00
Sandro Santilli
8a1539af7f Add test for fetching tile of private table showing api_key
See #38 and #91
2013-11-29 13:14:43 +01:00
javi
9155724082 fixed #91 2013-11-29 12:46:45 +01:00
javi
75b0ab5c87 fixed #96 2013-11-29 12:46:25 +01:00
Sandro Santilli
046c570f5a Release 1.5.1 2013-11-28 18:52:11 +01:00
Sandro Santilli
f10d7cd93f Accept unused CartoCSS directives
Closes #93

An example unused CartoCSS directive is
"point-transform" without "point-file"
or "point-url". Unused means it has no effect.

It used to be accepted but regressed in release 1.5.0
2013-11-28 18:47:45 +01:00
Sandro Santilli
364c6e4910 Fix test for invalid font usage after Windshaft update (#90)
NOTE: the error is less friendly now, see
      http://github.com/mapbox/carto/issues/242
2013-11-28 18:41:45 +01:00
Sandro Santilli
c67199b3eb Survive presence of malformed CartoCSS in redis
Closes #94, enable relative testcase
2013-11-28 18:31:37 +01:00
Sandro Santilli
87a9e0ca79 Add package keywords 2013-11-28 11:17:00 +01:00
Sandro Santilli
87e18da45b Set test redis port to 6335 2013-11-27 12:16:43 +01:00
Sandro Santilli
1712829808 Add (pending) test for getting unrenderable stored styles (#94)
Required upgrading mocha tester to ~0.14.0
2013-11-27 09:47:33 +01:00
Sandro Santilli
f1f9f5f233 Notify travis builds on #cartodb @ freenode.irc 2013-11-26 18:11:49 +01:00
Sandro Santilli
19512b776f Use a variable to hold the name of test database 2013-11-26 11:27:38 +01:00
Sandro Santilli
82956e0b7c Reduce ppa and explicit package usage
Should fix travis builds despite package compatibilit bugs
(https://travis-ci.org/CartoDB/Windshaft-cartodb/builds/14314805)
2013-11-21 18:17:58 +01:00
Sandro Santilli
0a30a6cd56 Add note about new directives in the 1.5.0 section 2013-11-21 16:00:47 +01:00
Sandro Santilli
a9151b856c Improve documentation for postgres_auth_* configuration directives 2013-11-21 15:55:20 +01:00
Sandro Santilli
d864441f74 Prepare for 1.5.1 2013-11-19 15:45:33 +01:00
Sandro Santilli
6b39864a1b Release 1.5.0 2013-11-19 15:43:11 +01:00
Sandro Santilli
dcc75dbf5f Drop unneeded include from outermost app 2013-11-18 10:07:28 +01:00
Sandro Santilli
4c16087ea8 Update to cartodb-redis 0.1.0 2013-11-15 19:14:00 +01:00
Sandro Santilli
d61c373fad CartoDB redis interaction delegated to "cartodb-redis" module 2013-11-15 15:49:04 +01:00
Sandro Santilli
2e4526ce8e Require windshaft-0.14.3 to get 3 new bugfixes:
- Return CORS headers when creating layergroups via GET
 - Fix http status on database authentication error
 - Ensure bogus text-face-name error raises at layergroup creation
2013-11-14 16:26:05 +01:00
Sandro Santilli
f1045078e1 Remember per-environment ./configure parameters
This is to avoid breaking test.js configuration while switching
between branches.
2013-11-14 16:21:40 +01:00
Sandro Santilli
ab3e76ce36 Fix ticket reference 2013-11-13 11:24:07 +01:00
Sandro Santilli
88daefe353 Add support for reading user-specific database_password from redis
This commits adds support for CartoDB-2.5.0 model.
Closes #89.
Change is backward compatible.
2013-11-12 23:20:50 +01:00
Sandro Santilli
bcb6807306 Avoid caches during test for user-specific database_host 2013-11-12 22:38:42 +01:00
Sandro Santilli
2897c0c8bf Do not force ending dot in SQL-API hostname, for easier testing 2013-11-12 17:41:10 +01:00
Sandro Santilli
eb9dfed731 Improve tests robustness on failure 2013-11-12 17:29:57 +01:00
Sandro Santilli
6567ea5090 Add NEWS item about CartoDB-2.5.0+ user-specific database_host (#88) 2013-11-11 17:49:57 +01:00
Sandro Santilli
b428a7be9a Add test for redis-specifid database_host. Closes #88 2013-11-11 17:49:05 +01:00
Sandro Santilli
759ebadfc8 Read user's database_host from redis, when available (#88)
Still lacks a testcase
2013-11-11 17:23:10 +01:00
Sandro Santilli
69f15a5086 Indent fixes 2013-11-11 16:55:51 +01:00
Sandro Santilli
4b12295862 Tweak error messages on missing redis variables, update tests 2013-11-11 15:56:58 +01:00
Luis Bosque
0508203ba3 Add function to read database host from redis 2013-11-11 15:47:13 +01:00
Sandro Santilli
4eb3368c99 Style only change 2013-11-11 15:45:34 +01:00
Sandro Santilli
357702d61b Add support for specifying database connection passwords 2013-11-11 00:50:03 +01:00
Sandro Santilli
4baff6e018 Release 1.14.1 2013-11-08 12:43:47 +01:00
Sandro Santilli
5b4184c2db Fix support for exponential notation in CartoCSS filter values
Closes #87.
Includes testcase
2013-11-08 12:34:34 +01:00
Sandro Santilli
895aa3b977 Prepare for 1.4.1 2013-10-31 16:02:40 +01:00
Sandro Santilli
d9d2adf5d8 Add Support for Mapnik-2.2.0. Closes #78. 2013-10-31 15:54:15 +01:00
Sandro Santilli
9b18ad2637 Prepare for mapnik-2.2.0 support (#78)
- Tolerate change in CartoCSS error message between 0.9.3 and 0.9.5
- Expect default style to be different for mapnik-2.2.0+ target
2013-10-29 20:45:15 +01:00
Sandro Santilli
78ba279eb8 Prepare for 1.3.7 2013-10-11 10:06:38 +02:00
Sandro Santilli
cef82aedd8 Release 1.3.6, fixing support for node-0.8.9 2013-10-11 10:05:31 +02:00
Sandro Santilli
5ae1513eea Prepare for 1.3.6 2013-10-03 17:15:11 +02:00
Sandro Santilli
dc52bb8751 Release 1.3.5 2013-10-03 17:10:37 +02:00
Sandro Santilli
47f42e4031 Fix support for apostrophes in CartoCSS
Requires windshaft 0.13.7
Jira ref CDB-414
2013-10-03 17:03:13 +02:00
Sandro Santilli
66a77cd255 Do not let anonymous requests use authorized renderer caches
Puts dbuser in params, for correct use by Windshaft renderer cache.
Before this fix, and after commit 1c9f63c9, the renderer cache key
did not contain the db user.
2013-09-23 12:02:43 +02:00
Sandro Santilli
0301cef1bb tweak test description 2013-09-23 10:57:22 +02:00
Sandro Santilli
72b3f23f72 Add more profile slots 2013-09-19 14:34:03 +01:00
Sandro Santilli
8685ef640e Remove spaces from configuration input, to make editing easier :) 2013-09-18 14:37:22 +02:00
Sandro Santilli
2d36521f92 Make testsuite accept an installed mapnik version 2.1.0
See https://travis-ci.org/CartoDB/Windshaft-cartodb/builds/11286823
2013-09-12 18:37:25 +02:00
Sandro Santilli
ca4644f4ce Add travis widget, fix documented node dependency 2013-09-12 18:09:18 +02:00
Sandro Santilli
58a462ab95 Add travis configuration 2013-09-12 18:04:38 +02:00
Sandro Santilli
c2d4aace56 Read test redis port configuration from test.js env 2013-09-12 17:55:16 +02:00
Sandro Santilli
4d524e5969 Clean handling of redis connection failures in testcase 2013-09-12 17:48:35 +02:00
Sandro Santilli
8c74a39262 Fix error for invalid text-name in CartoCSS. Closes #81. 2013-09-12 17:32:10 +02:00
Sandro Santilli
743b5388a3 Add backward compatibility sqlapi configuration item in NEWS 2013-09-12 16:20:28 +02:00
Sandro Santilli
4144ad2c7a Only use sqlapi configuration "host" if "domain" is undefined
We'll consider an empty string domain as valid (it's actually used
for testsuite).
2013-09-12 16:19:01 +02:00
Javier Arce
fb4ef5f768 Sets the sqlapi domain. Fixes #82 2013-09-12 15:36:50 +02:00
Sandro Santilli
cbb85e5dd8 Read redis port from test.js environment when running tests 2013-09-12 10:17:02 +02:00
Sandro Santilli
56bfed5a0e Fix use of blank-prefixed "zoom" variable in CartoCSS 2013-09-09 11:58:51 +02:00
Luis Bosque
ff9af5f923 Target v1.3.5 2013-09-06 12:13:09 +02:00
29 changed files with 173 additions and 3708 deletions

View File

@@ -17,8 +17,10 @@ config/environments/test.js: config.status--test
check-local: config/environments/test.js
./run_tests.sh ${RUNTESTFLAGS} \
test/unit/cartodb/*.js \
test/acceptance/*.js
test/unit/cartodb/req2params.test.js \
test/acceptance/cache_validator.js \
test/acceptance/server.js \
test/acceptance/multilayer.js
check-submodules:
PATH="$$PATH:$(srcdir)/node_modules/.bin/"; \

35
NEWS.md
View File

@@ -1,38 +1,3 @@
1.6.2 -- 2014-01-23
-------------------
Bug fixes:
* Fix support for long (>64k chars) queries in layergroup creation (#111)
Enhancements:
* Enhance tools/show_style to accept an environment parameter and
print XML style now it is not in redis anymore (#110)
* Support CORS in template instanciation endpoint (#113)
1.6.1 -- 2014-01-15
-------------------
Bug fixes:
* Drop cache headers from error responses (#107)
* Localize external CartoCSS resources at renderer creation time (#108)
1.6.0 -- 2014-01-10
-------------------
New features:
* Add 'user_from_host' directive to generalize username extraction (#100)
* Implement signed template maps (#98)
Other changes:
* Update cartodb-redis dependency to "~0.3.0"
* Update redis-server dependency to "2.4.0+"
1.5.2 -- 2013-12-05
-------------------

View File

@@ -1,8 +1,7 @@
Windshaft-CartoDB
==================
[![Build Status](http://travis-ci.org/CartoDB/Windshaft-cartodb.png)]
(http://travis-ci.org/CartoDB/Windshaft-cartodb)
[![Build Status](https://travis-ci.org/CartoDB/Windshaft-cartodb.png)](http://travis-ci.org/CartoDB/Windshaft-cartodb)
This is the CartoDB map tiler. It extends Windshaft with some extra
functionality and custom filters for authentication
@@ -12,10 +11,8 @@ functionality and custom filters for authentication
* gets the default geometry type from the cartodb redis store
* allows tiles to be styled individually
* provides a link to varnish high speed cache
* provides a ``infowindow`` endpoint for windshaft (DEPRECATED)
* provides a ``map_metadata`` endpoint for windshaft (DEPRECATED)
* provides signed template maps API
(http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps)
* provides a infowindow endpoint for windshaft
* provides a ``map_metadata`` endpoint for windshaft
Requirements
------------
@@ -24,16 +21,13 @@ Requirements
- node-0.8.x+
- PostgreSQL-8.3+
- PostGIS-1.5.0+
- Redis 2.4.0+ (http://www.redis.io)
- Redis 2.2.0+ (http://www.redis.io)
- Mapnik 2.0 or 2.1
[for cache control]
- CartoDB-SQL-API 1.0.0+
- CartoDB 0.9.5+ (for ``CDB_QueryTables``)
- Varnish (http://www.varnish-cache.org)
[for running the testsuite]
- Imagemagick (http://www.imagemagick.org)
- Varnish (https://www.varnish-cache.org)
Configure
---------

7
app.js
View File

@@ -18,15 +18,14 @@ if (ENV != 'development' && ENV != 'production' && ENV != 'staging' ){
var _ = require('underscore')
, Step = require('step')
;
, CartodbWindshaft = require('./lib/cartodb/cartodb_windshaft');
// set environment specific variables
global.settings = require(__dirname + '/config/settings');
global.environment = require(__dirname + '/config/environments/' + ENV);
_.extend(global.settings, global.environment);
// Include cartodb_windshaft only _after_ the "global" variable is set
// See https://github.com/Vizzuality/Windshaft-cartodb/issues/28
var CartodbWindshaft = require('./lib/cartodb/cartodb_windshaft');
var Windshaft = require('windshaft');
var serverOptions = require('./lib/cartodb/server_options');
ws = CartodbWindshaft(serverOptions);

View File

@@ -2,9 +2,6 @@ var config = {
environment: 'development'
,port: 8181
,host: '127.0.0.1'
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
,user_from_host: '^(.*)\\.localhost'
// Maximum number of connections for one process
// 128 is a good value with a limit of 1024 open file descriptors
,maxConnections:128

View File

@@ -2,9 +2,6 @@ var config = {
environment: 'production'
,port: 8181
,host: '127.0.0.1'
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
,user_from_host: '^(.*)\\.cartodb\\.com$'
// Maximum number of connections for one process
// 128 is a good value with a limit of 1024 open file descriptors
,maxConnections:128

View File

@@ -2,9 +2,6 @@ var config = {
environment: 'production'
,port: 8181
,host: '127.0.0.1'
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
,user_from_host: '^(.*)\\.cartodb\\.com$'
// Maximum number of connections for one process
// 128 is a good value with a limit of 1024 open file descriptors
,maxConnections:128

View File

@@ -2,9 +2,6 @@ var config = {
environment: 'test'
,port: 8888
,host: '127.0.0.1'
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
,user_from_host: '(.*)'
// Maximum number of connections for one process
// 128 is a good value with a limit of 1024 open file descriptors
,maxConnections:128

View File

@@ -2,11 +2,6 @@
var _ = require('underscore')
, Step = require('step')
, Windshaft = require('windshaft')
, redisPool = new require('redis-mpool')(global.environment.redis)
// TODO: instanciate cartoData with redisPool
, cartoData = require('cartodb-redis')(global.environment.redis)
, SignedMaps = require('./signed_maps.js')
, TemplateMaps = require('./template_maps.js')
, Cache = require('./cache_validator');
var CartodbWindshaft = function(serverOptions) {
@@ -28,9 +23,6 @@ var CartodbWindshaft = function(serverOptions) {
callback(err, req);
}
serverOptions.signedMaps = new SignedMaps(redisPool);
var templateMaps = new TemplateMaps(redisPool, serverOptions.signedMaps);
// boot
var ws = new Windshaft.Server(serverOptions);
@@ -42,22 +34,6 @@ var CartodbWindshaft = function(serverOptions) {
return version;
}
// Override sendError to drop added cache headers (if any)
// See http://github.com/CartoDB/Windshaft-cartodb/issues/107
var ws_sendError = ws.sendError;
ws.sendError = function(res) {
// NOTE: the "res" object will have no _headers when
// faked by Windshaft, see
// http://github.com/CartoDB/Windshaft-cartodb/issues/109
//
if ( res._headers ) {
delete res._headers['cache-control'];
delete res._headers['last-modified'];
delete res._headers['x-cache-channel'];
}
ws_sendError.apply(this, arguments);
};
/**
* Helper to allow access to the layer to be used in the maps infowindow popup.
*/
@@ -119,381 +95,6 @@ var CartodbWindshaft = function(serverOptions) {
}
);
});
// ---- Template maps interface starts @{
ws.userByReq = function(req) {
return serverOptions.userByReq(req);
}
var template_baseurl = serverOptions.base_url_notable + '/template';
// Add a template
ws.post(template_baseurl, function(req, res) {
ws.doCORS(res);
var that = this;
var response = {};
var cdbuser = ws.userByReq(req);
Step(
function checkPerms(){
serverOptions.authorizedByAPIKey(req, this);
},
function addTemplate(err, authenticated) {
if ( err ) throw err;
if (authenticated !== 1) {
err = new Error("Only authenticated user can create templated maps");
err.http_status = 401;
throw err;
}
var next = this;
if ( ! req.headers['content-type'] || req.headers['content-type'].split(';')[0] != 'application/json' )
throw new Error('template POST data must be of type application/json');
var cfg = req.body;
templateMaps.addTemplate(cdbuser, cfg, this);
},
function prepareResponse(err, tpl_id){
if ( err ) throw err;
// NOTE: might omit "cdbuser" if == dbowner ...
return { template_id: cdbuser + '@' + tpl_id };
},
function finish(err, response){
if (err){
response = { error: ''+err };
var statusCode = 400;
if ( ! _.isUndefined(err.http_status) ) {
statusCode = err.http_status;
}
ws.sendError(res, response, statusCode, 'POST TEMPLATE', err.message);
} else {
res.send(response, 200);
}
}
);
});
// Update a template
ws.put(template_baseurl + '/:template_id', function(req, res) {
ws.doCORS(res);
var that = this;
var response = {};
var cdbuser = ws.userByReq(req);
var template;
var tpl_id;
Step(
function checkPerms(){
serverOptions.authorizedByAPIKey(req, this);
},
function updateTemplate(err, authenticated) {
if ( err ) throw err;
if (authenticated !== 1) {
err = new Error("Only authenticated user can list templated maps");
err.http_status = 401;
throw err;
}
if ( ! req.headers['content-type'] || req.headers['content-type'].split(';')[0] != 'application/json' )
throw new Error('template PUT data must be of type application/json');
template = req.body;
tpl_id = req.params.template_id.split('@');
if ( tpl_id.length > 1 ) {
if ( tpl_id[0] != cdbuser ) {
err = new Error("Invalid template id '"
+ req.params.template_id + "' for user '" + cdbuser + "'");
err.http_status = 404;
throw err;
}
tpl_id = tpl_id[1];
}
templateMaps.updTemplate(cdbuser, tpl_id, template, this);
},
function prepareResponse(err){
if ( err ) throw err;
return { template_id: cdbuser + '@' + tpl_id };
},
function finish(err, response){
if (err){
var statusCode = 400;
response = { error: ''+err };
if ( ! _.isUndefined(err.http_status) ) {
statusCode = err.http_status;
}
ws.sendError(res, response, statusCode, 'PUT TEMPLATE', err.message);
} else {
res.send(response, 200);
}
}
);
});
// Get a specific template
ws.get(template_baseurl + '/:template_id', function(req, res) {
ws.doCORS(res);
var that = this;
var response = {};
var cdbuser = ws.userByReq(req);
var template;
var tpl_id;
Step(
function checkPerms(){
serverOptions.authorizedByAPIKey(req, this);
},
function updateTemplate(err, authenticated) {
if ( err ) throw err;
if (authenticated !== 1) {
err = new Error("Only authenticated users can get template maps");
err.http_status = 401;
throw err;
}
tpl_id = req.params.template_id.split('@');
if ( tpl_id.length > 1 ) {
if ( tpl_id[0] != cdbuser ) {
var err = new Error("Cannot get template id '"
+ req.params.template_id + "' for user '" + cdbuser + "'");
err.http_status = 404;
throw err;
}
tpl_id = tpl_id[1];
}
templateMaps.getTemplate(cdbuser, tpl_id, this);
},
function prepareResponse(err, tpl_val){
if ( err ) throw err;
if ( ! tpl_val ) {
err = new Error("Cannot find template '" + tpl_id + "' of user '" + cdbuser + "'");
err.http_status = 404;
throw err;
}
// auth_id was added by ourselves,
// so we remove it before returning to the user
delete tpl_val.auth_id;
return { template: tpl_val };
},
function finish(err, response){
if (err){
var statusCode = 400;
response = { error: ''+err };
if ( ! _.isUndefined(err.http_status) ) {
statusCode = err.http_status;
}
ws.sendError(res, response, statusCode, 'GET TEMPLATE', err.message);
} else {
res.send(response, 200);
}
}
);
});
// Delete a specific template
ws.delete(template_baseurl + '/:template_id', function(req, res) {
ws.doCORS(res);
var that = this;
var response = {};
var cdbuser = ws.userByReq(req);
var template;
var tpl_id;
Step(
function checkPerms(){
serverOptions.authorizedByAPIKey(req, this);
},
function updateTemplate(err, authenticated) {
if ( err ) throw err;
if (authenticated !== 1) {
err = new Error("Only authenticated users can delete template maps");
err.http_status = 401;
throw err;
}
tpl_id = req.params.template_id.split('@');
if ( tpl_id.length > 1 ) {
if ( tpl_id[0] != cdbuser ) {
var err = new Error("Cannot find template id '"
+ req.params.template_id + "' for user '" + cdbuser + "'");
err.http_status = 404;
throw err;
}
tpl_id = tpl_id[1];
}
templateMaps.delTemplate(cdbuser, tpl_id, this);
},
function prepareResponse(err, tpl_val){
if ( err ) throw err;
return { status: 'ok' };
},
function finish(err, response){
if (err){
var statusCode = 400;
response = { error: ''+err };
if ( ! _.isUndefined(err.http_status) ) {
statusCode = err.http_status;
}
ws.sendError(res, response, statusCode, 'DELETE TEMPLATE', err.message);
} else {
res.send('', 204);
}
}
);
});
// Get a list of owned templates
ws.get(template_baseurl, function(req, res) {
ws.doCORS(res);
var that = this;
var response = {};
var cdbuser = ws.userByReq(req);
Step(
function checkPerms(){
serverOptions.authorizedByAPIKey(req, this);
},
function listTemplates(err, authenticated) {
if ( err ) throw err;
if (authenticated !== 1) {
err = new Error("Only authenticated user can list templated maps");
err.http_status = 401;
throw err;
}
templateMaps.listTemplates(cdbuser, this);
},
function prepareResponse(err, tpl_ids){
if ( err ) throw err;
// NOTE: might omit "cbduser" if == dbowner ...
var ids = _.map(tpl_ids, function(id) { return cdbuser + '@' + id; })
return { template_ids: ids };
},
function finish(err, response){
var statusCode = 200;
if (err){
response = { error: ''+err };
if ( ! _.isUndefined(err.http_status) ) {
statusCode = err.http_status;
}
ws.sendError(res, response, statusCode, 'GET TEMPLATE LIST', err.message);
} else {
res.send(response, statusCode);
}
}
);
});
ws.setDBParams = function(cdbuser, params, callback) {
Step(
function setAuth() {
serverOptions.setDBAuth(cdbuser, params, this);
},
function setConn(err) {
if ( err ) throw err;
serverOptions.setDBConn(cdbuser, params, this);
},
function finish(err) {
callback(err);
}
);
};
ws.options(template_baseurl + '/:template_id', function(req, res) {
ws.doCORS(res, "Content-Type");
return next();
});
// Instantiate a template
ws.post(template_baseurl + '/:template_id', function(req, res) {
ws.doCORS(res);
var that = this;
var response = {};
var template;
var signedMaps = serverOptions.signedMaps;
var layergroup;
var layergroupid;
var fakereq; // used for call to createLayergroup
var cdbuser = ws.userByReq(req);
// Format of template_id: [<template_owner>]@<template_id>
var tpl_id = req.params.template_id.split('@');
if ( tpl_id.length > 1 ) {
if ( tpl_id[0] ) cdbuser = tpl_id[0];
tpl_id = tpl_id[1];
}
var auth_token = req.query.auth_token;
Step(
function getTemplate(){
templateMaps.getTemplate(cdbuser, tpl_id, this);
},
function checkAuthorized(err, data) {
if ( err ) throw err;
if ( ! data ) {
err = new Error("Template '" + tpl_id + "' of user '" + cdbuser + "' not found");
err.http_status = 404;
throw err;
}
template = data;
var cert = templateMaps.getTemplateCertificate(template);
var authorized = false;
try {
// authorizedByCert will throw if unauthorized
authorized = signedMaps.authorizedByCert(cert, auth_token);
} catch (err) {
// we catch to add http_status
err.http_status = 401;
throw err;
}
if ( ! authorized ) {
err = new Error('Unauthorized template instanciation');
err.http_status = 401;
throw err;
}
if ( ! req.headers['content-type'] || req.headers['content-type'].split(';')[0] != 'application/json' )
throw new Error('template POST data must be of type application/json, it is instead ');
var template_params = req.body;
return templateMaps.instance(template, template_params);
},
function prepareParams(err, instance){
if ( err ) throw err;
layergroup = instance;
fakereq = { query: {}, params: {}, headers: _.clone(req.headers) };
ws.setDBParams(cdbuser, fakereq.params, this);
},
function createLayergroup(err) {
if ( err ) throw err;
ws.createLayergroup(layergroup, fakereq, this);
},
function signLayergroup(err, resp) {
if ( err ) throw err;
response = resp;
var signer = cdbuser;
var map_id = response.layergroupid.split(':')[0]; // dropping last_updated
var crt_id = template.auth_id; // check ?
if ( ! crt_id ) {
var errmsg = "Template '" + tpl_id + "' of user '" + cdbuser + "' has no signature";
// Is this really illegal ?
// Maybe we could just return an unsigned layergroupid
// in this case...
err = new Error(errmsg);
err.http_status = 403; // Forbidden, we refuse to respond to this
throw err;
}
signedMaps.signMap(signer, map_id, crt_id, this);
},
function prepareResponse(err) {
if ( err ) throw err;
//console.log("Response from createLayergroup: "); console.dir(response);
// Add the signature part to the token!
response.layergroupid = cdbuser + '@' + response.layergroupid;
return response;
},
function finish(err, response){
if (err){
var statusCode = 400;
response = { error: ''+err };
if ( ! _.isUndefined(err.http_status) ) {
statusCode = err.http_status;
}
ws.sendError(res, response, statusCode, 'POST INSTANCE TEMPLATE', err.message);
} else {
res.send(response, 200);
}
}
);
});
// ---- Template maps interface ends @}
return ws;
}

View File

@@ -78,17 +78,9 @@ module.exports = function(){
if (_.isString(api_key) && api_key != '') { qs.api_key = api_key; }
// call sql api
//
// NOTE: using POST to avoid size limits:
// Seehttp://github.com/CartoDB/Windshaft-cartodb/issues/111
//
// TODO: use "host" header to allow IP based specification
// of sqlapi address (and avoid a DNS lookup)
//
request.post({url:sqlapi, body:qs, json:true},
function(err, res, body){
request.get({url:sqlapi, qs:qs, json:true}, function(err, res, body){
if (err){
console.log('ERROR connecting to SQL API on ' + sqlapi + ': ' + err);
console.log('ERROR running connecting to SQL API on ' + sqlapi + ': ' + err);
callback(err);
return;
}
@@ -196,7 +188,7 @@ module.exports = function(){
}
var dbName = req.params.dbname;
var username = this.userByReq(req);
var username = req.headers.host.split('.')[0];
// strip out windshaft/mapnik inserted sql if present
var sql = req.params.sql.match(/^\((.*)\)\sas\scdbq$/);
@@ -258,7 +250,7 @@ module.exports = function(){
me.afterLayergroupCreate = function(req, mapconfig, response, callback) {
var token = response.layergroupid;
var username = this.userByReq(req);
var username = cartoData.userFromHostname(req.headers.host);
var tasksleft = 2; // redis key and affectedTables
var errors = [];
@@ -289,7 +281,7 @@ module.exports = function(){
sql = sql.join(';');
var dbName = req.params.dbname;
var usr = this.userByReq(req);
var usr = req.headers.host.split('.')[0];
var key = req.params.map_key;
var cacheKey = dbName + ':' + token;
@@ -313,224 +305,69 @@ module.exports = function(){
/* X-Cache-Channel generation } */
me.re_userFromHost = new RegExp(
global.environment.user_from_host ||
'^([^\\.]+)\\.' // would extract "strk" from "strk.cartodb.com"
);
me.userByReq = function(req) {
var host = req.headers.host;
var mat = host.match(this.re_userFromHost);
if ( ! mat ) {
console.error("ERROR: user pattern '" + this.re_userFromHost
+ "' does not match hostname '" + host + "'");
return;
}
// console.log("Matches: "); console.dir(mat);
if ( ! mat.length === 2 ) {
console.error("ERROR: pattern '" + this.re_userFromHost
+ "' gave unexpected matches against '" + host + "': " + mat);
return;
}
return mat[1];
}
// Set db authentication parameters to those of the given username
//
// @param username the cartodb username, mapped to a database username
// via CartodbRedis metadata records
//
// @param params the parameters to set auth options into
// added params are: "dbuser" and "dbpassword"
//
// @param callback function(err)
//
me.setDBAuth = function(username, params, callback) {
var user_params = {};
var auth_user = global.environment.postgres_auth_user;
var auth_pass = global.environment.postgres_auth_pass;
Step(
function getId() {
cartoData.getUserId(username, this);
},
function(err, user_id) {
if (err) throw err;
user_params['user_id'] = user_id;
var dbuser = _.template(auth_user, user_params);
_.extend(params, {dbuser:dbuser});
// skip looking up user_password if postgres_auth_pass
// doesn't contain the "user_password" label
if (!auth_pass || ! auth_pass.match(/\buser_password\b/) ) return null;
cartoData.getUserDBPass(username, this);
},
function(err, user_password) {
if (err) throw err;
user_params['user_password'] = user_password;
if ( auth_pass ) {
var dbpass = _.template(auth_pass, user_params);
_.extend(params, {dbpassword:dbpass});
}
return true;
},
function finish(err) {
callback(err);
}
);
};
// Set db connection parameters to those for the given username
//
// @param dbowner cartodb username of database owner,
// mapped to a database username
// via CartodbRedis metadata records
//
// @param params the parameters to set connection options into
// added params are: "dbname", "dbhost"
//
// @param callback function(err)
//
me.setDBConn = function(dbowner, params, callback) {
Step(
function getDatabaseHost(){
cartoData.getUserDBHost(dbowner, this);
},
function getDatabase(err, data){
if(err) throw err;
if ( data ) _.extend(params, {dbhost:data});
cartoData.getUserDBName(dbowner, this);
},
function getGeometryType(err, data){
if (err) throw err;
if ( data ) _.extend(params, {dbname:data});
return null;
},
function finish(err) {
callback(err);
}
);
};
// Check if a request is authorized by a signer
//
// Any existing signature for the given request will verified
// for authorization to this specific request (may require auth_token)
// See https://github.com/CartoDB/Windshaft-cartodb/wiki/Signed-maps
//
// @param req express request object
// @param callback function(err, signed_by) signed_by will be
// null if the request is not signed by anyone
// or will be a string cartodb username otherwise.
//
me.authorizedBySigner = function(req, callback)
{
if ( ! req.params.token || ! req.params.signer ) {
//console.log("No signature provided"); // debugging
callback(null, null); // no signer requested
return;
}
var signer = req.params.signer;
var layergroup_id = req.params.token;
var auth_token = req.params.auth_token;
console.log("Checking authorization from signer " + signer + " for resource " + layergroup_id + " with auth_token " + auth_token);
me.signedMaps.isAuthorized(signer, layergroup_id, auth_token,
function(err, authorized) {
callback(err, authorized ? signer : null);
});
};
// Check if a request is authorized by api_key
//
// @param req express request object
// @param callback function(err, authorized)
//
me.authorizedByAPIKey = function(req, callback)
{
var user = me.userByReq(req);
Step(
function (){
cartoData.getUserMapKey(user, this);
},
function checkApiKey(err, val){
if (err) throw err;
var valid = 0;
if ( val ) {
if ( val == req.query.map_key ) valid = 1;
else if ( val == req.query.api_key ) valid = 1;
// check also in request body
else if ( req.body && req.body.map_key && val == req.body.map_key ) valid = 1;
else if ( req.body && req.body.api_key && val == req.body.api_key ) valid = 1;
}
return valid;
},
function finish(err, authorized) {
callback(err, authorized);
}
);
};
/**
* Check access authorization
* Get privacy for cartodb table
*
* @param req - standard req object. Importantly contains table and host information
* @param callback function(err, allowed) is access allowed not?
* @param callback - is the table private or not?
*/
me.authorize = function(req, callback) {
me.authorize= function(req, callback) {
var that = this;
var user = me.userByReq(req);
Step(
function (){
that.authorizedByAPIKey(req, this);
function(){
cartoData.checkMapKey(req, this);
},
function checkApiKey(err, authorized){
function checkIfInternal(err, check_result){
if (err) throw err;
// if not authorized by api_key, continue
if (authorized !== 1) {
// not authorized by api_key,
// check if authorized by signer
that.authorizedBySigner(req, this);
return;
}
// if unauthorized continue to check table privacy
if (check_result !== 1) return true;
// authorized by api key, login as the given username and stop
that.setDBAuth(user, req.params, function(err) {
callback(err, true); // authorized (or error)
});
},
function checkSignAuthorized(err, signed_by){
if (err) throw err;
// authorized by key, login as db owner
var user_params = {};
var auth_user = global.environment.postgres_auth_user;
var auth_pass = global.environment.postgres_auth_pass;
Step(
function getId() {
cartoData.getId(req, this);
},
function(err, user_id) {
if (err) throw err;
user_params['user_id'] = user_id;
var dbuser = _.template(auth_user, user_params);
_.extend(req.params, {dbuser:dbuser});
if ( ! signed_by ) {
// request not authorized by signer, continue
// to check map privacy
return null;
}
// skip looking up user_password if postgres_auth_pass
// doesn't contain the "user_password" label
if (!auth_pass || ! auth_pass.match(/\buser_password\b/) ) return null;
// Authorized by "signed_by" !
that.setDBAuth(signed_by, req.params, function(err) {
callback(err, true); // authorized (or error)
});
},
function getDatabase(err){
cartoData.getDatabasePassword(req, this);
},
function(err, user_password) {
if (err) throw err;
user_params['user_password'] = user_password;
if ( auth_pass ) {
var dbpass = _.template(auth_pass, user_params);
_.extend(req.params, {dbpassword:dbpass});
}
return true;
},
function finish(err) {
callback(err, true); // authorized (or error)
}
);
}
,function getDatabase(err, data){
if (err) throw err;
// NOTE: only used to get to table privacy
cartoData.getUserDBName(user, this);
cartoData.getDatabase(req, this);
},
function getPrivacy(err, dbname){
function getPrivacy(err, data){
if (err) throw err;
cartoData.getTablePrivacy(dbname, req.params.table, this);
cartoData.getTablePrivacy(data, req.params.table, this);
},
function(err, privacy){
callback(err, privacy);
function(err, data){
callback(err, data);
}
);
};
@@ -572,7 +409,7 @@ console.log("Checking authorization from signer " + signer + " for resource " +
}
// Whitelist query parameters and attach format
var good_query = ['sql', 'geom_type', 'cache_buster', 'cache_policy', 'callback', 'interactivity', 'map_key', 'api_key', 'auth_token', 'style', 'style_version', 'style_convert', 'config' ];
var good_query = ['sql', 'geom_type', 'cache_buster', 'cache_policy', 'callback', 'interactivity', 'map_key', 'api_key', 'style', 'style_version', 'style_convert', 'config' ];
var bad_query = _.difference(_.keys(req.query), good_query);
_.each(bad_query, function(key){ delete req.query[key]; });
@@ -583,13 +420,6 @@ console.log("Checking authorization from signer " + signer + " for resource " +
var tksplit = req.params.token.split(':');
req.params.token = tksplit[0];
if ( tksplit.length > 1 ) req.params.cache_buster= tksplit[1];
tksplit = req.params.token.split('@');
if ( tksplit.length > 1 ) {
req.params.signer = this.userByReq(req);
if ( tksplit[0] ) req.params.signer = tksplit[0];
req.params.token = tksplit[1];
//console.log("Request for token " + req.params.token + " with signature from " + req.params.signer);
}
}
// bring all query values onto req.params object
@@ -622,26 +452,34 @@ console.log("Checking authorization from signer " + signer + " for resource " +
if (req.profiler) req.profiler.done('req2params.setup');
var user = me.userByReq(req);
Step(
function getPrivacy(){
me.authorize(req, this);
},
function gatekeep(err, data){
if (req.profiler) req.profiler.done('authorize');
if (req.profiler) req.profiler.done('cartoData.authorize');
if(err) throw err;
if(data === "0") throw new Error("Sorry, you are unauthorized (permission denied)");
return data;
},
function getDatabase(err, data){
function getDatabaseHost(err, data){
if(err) throw err;
that.setDBConn(user, req.params, this);
cartoData.getDatabaseHost(req, this);
},
function getGeometryType(err){
function getDatabase(err, data){
if (req.profiler) req.profiler.done('cartoData.getDatabaseHost');
if(err) throw err;
if ( data ) _.extend(req.params, {dbhost:data});
cartoData.getDatabase(req, this);
},
function getGeometryType(err, data){
if (req.profiler) req.profiler.done('cartoData.getDatabase');
if (err) throw err;
cartoData.getTableGeometryType(req.params.dbname, req.params.table, this);
_.extend(req.params, {dbname:data});
cartoData.getGeometryType(req, this);
},
function finishSetup(err, data){
if (req.profiler) req.profiler.done('cartoData.getGeometryType');
@@ -665,23 +503,14 @@ console.log("Checking authorization from signer " + signer + " for resource " +
*/
me.getInfowindow = function(req, callback){
var that = this;
var user = me.userByReq(req);
Step(
function(){
// TODO: if this step really needed ?
that.req2params(req, this);
},
function getDatabase(err){
if (err) throw err;
cartoData.getUserDBName(user, this);
},
function getInfowindow(err, dbname){
if (err) throw err;
cartoData.getTableInfowindow(dbname, req.params.table, this);
},
function(err, data){
callback(err, data);
if (err) callback(err, null);
else cartoData.getInfowindow(data, callback);
}
);
};
@@ -693,23 +522,14 @@ console.log("Checking authorization from signer " + signer + " for resource " +
*/
me.getMapMetadata = function(req, callback){
var that = this;
var user = me.userByReq(req);
Step(
function(){
// TODO: if this step really needed ?
that.req2params(req, this);
},
function getDatabase(err){
if (err) throw err;
cartoData.getUserDBName(user, this);
},
function getMapMetadata(err, dbname){
if (err) throw err;
cartoData.getTableMapMetadata(dbname, req.params.table, this);
},
function(err, data){
callback(err, data);
if (err) callback(err, null);
else cartoData.getMapMetadata(data, callback);
}
);
};

View File

@@ -1,347 +0,0 @@
var crypto = require('crypto');
var Step = require('step');
var _ = require('underscore');
// Class handling map signatures and user certificates
//
// See https://github.com/CartoDB/Windshaft-cartodb/wiki/Signed-maps
//
// @param redis_pool an instance of a "redis-mpool"
// See https://github.com/CartoDB/node-redis-mpool
// Needs version 0.x.x of the API.
//
function SignedMaps(redis_pool) {
this.redis_pool = redis_pool;
// Database containing signatures
// TODO: allow configuring ?
// NOTE: currently it is the same as
// the one containing layergroups
this.db_signatures = 0;
//
// Map signatures in redis are reference to signature certificates
// We have the following datastores:
//
// 1. User certificates: set of per-user authorization certificates
// 2. Map signatures: set of per-map certificate references
// 3. Certificate applications: set of per-certificate signed maps
// User certificates (HASH:crt_id->crt_val)
this.key_map_crt = "map_crt|<%= signer %>";
// Map signatures (SET:crt_id)
this.key_map_sig = "map_sig|<%= signer %>|<%= map_id %>";
// Certificates applications (SET:map_id)
//
// Everytime a map is signed, the map identifier (layergroup_id)
// is added to this set. The purpose of this set is to drop
// all map signatures when a certificate is removed
//
this.key_crt_sig = "crt_sig|<%= signer %>|<%= crt_id %>";
};
var o = SignedMaps.prototype;
//--------------- PRIVATE METHODS --------------------------------
o._acquireRedis = function(callback) {
this.redis_pool.acquire(this.db_signatures, callback);
};
o._releaseRedis = function(client) {
this.redis_pool.release(this.db_signatures, client);
};
/**
* Internal function to communicate with redis
*
* @param redisFunc - the redis function to execute
* @param redisArgs - the arguments for the redis function in an array
* @param callback - function to pass results too.
*/
o._redisCmd = function(redisFunc, redisArgs, callback) {
var redisClient;
var that = this;
var db = that.db_signatures;
Step(
function getRedisClient() {
that.redis_pool.acquire(db, this);
},
function executeQuery(err, data) {
if ( err ) throw err;
redisClient = data;
redisArgs.push(this);
redisClient[redisFunc.toUpperCase()].apply(redisClient, redisArgs);
},
function releaseRedisClient(err, data) {
if ( ! _.isUndefined(redisClient) ) that.redis_pool.release(db, redisClient);
callback(err, data);
}
);
};
//--------------- PUBLIC API -------------------------------------
// Check if the given certificate authorizes waiver of "auth"
o.authorizedByCert = function(cert, auth) {
//console.log("Checking cert: "); console.dir(cert);
if ( cert.version !== "0.0.1" ) {
throw new Error("Unsupported certificate version " + cert.version);
}
if ( ! cert.auth ) {
throw new Error("No certificate authorization");
}
if ( ! cert.auth.method ) {
throw new Error("No certificate authorization method");
}
// Open authentication certificates are always authorized
if ( cert.auth.method === 'open' ) return true;
// Token based authentication requires valid token
if ( cert.auth.method === 'token' ) {
var found = cert.auth.valid_tokens.indexOf(auth);
//if ( found !== -1 ) {
//console.log("Token " + auth + " is found at position " + found + " in valid tokens " + cert.auth.valid_tokens);
// return true;
//} else return false;
return cert.auth.valid_tokens.indexOf(auth) !== -1;
}
throw new Error("Unsupported authentication method: " + cert.auth.method);
};
// Check if shown credential are authorized to access a map
// by the given signer.
//
// @param signer a signer name (cartodb username)
// @param map_id a layergroup_id
// @param auth an authentication token, or undefined if none
// (can still be authorized by signature)
//
// @param callback function(Error, Boolean)
//
o.isAuthorized = function(signer, map_id, auth, callback) {
var that = this;
var authorized = false;
var certificate_id_list;
var missing_certificates = [];
console.log("Check auth from signer '" + signer + "' on map '" + map_id + "' with auth '" + auth + "'");
Step(
function getMapSignatures() {
var map_sig_key = _.template(that.key_map_sig, {signer:signer, map_id:map_id});
that._redisCmd('SMEMBERS', [ map_sig_key ], this);
},
function getCertificates(err, crt_lst) {
if ( err ) throw err;
console.log("Map '" + map_id + "' is signed by " + crt_lst.length + " certificates of user '" + signer + "': " + crt_lst);
certificate_id_list = crt_lst;
if ( ! crt_lst.length ) {
// No certs, avoid calling redis with short args list.
// Next step expects a list of certificate values so
// we directly send the empty list.
return crt_lst;
}
var map_crt_key = _.template(that.key_map_crt, {signer:signer});
that._redisCmd('HMGET', [ map_crt_key ].concat(crt_lst), this);
},
function checkCertificates(err, certs) {
if ( err ) throw err;
for (var i=0; i<certs.length; ++i) {
var crt_id = certificate_id_list[i];
if ( _.isNull(certs[i]) ) {
missing_certificates.push(crt_id);
continue;
}
var cert;
try {
//console.log("cert " + crt_id + ": " + certs[i]);
cert = JSON.parse(certs[i]);
authorized = that.authorizedByCert(cert, auth);
} catch (err) {
console.log("Certificate " + certificate_id_list[i] + " by user '" + signer + "' is malformed: " + err);
continue;
}
if ( authorized ) {
console.log("Access to map '" + map_id + "' authorized by cert '"
+ certificate_id_list[i] + "' of user '" + signer + "'");
//console.dir(cert);
break; // no need to further check certs
}
}
return null;
},
function finish(err) {
if ( missing_certificates.length ) {
console.log("WARNING: map '" + map_id + "' is signed by '" + signer
+ "' with " + missing_certificates.length
+ " missing certificates: "
+ missing_certificates + " (TODO: give cleanup instructions)");
}
callback(err, authorized);
}
);
};
// Add an authorization certificate from a user.
//
// @param signer a signer name (cartodb username)
// @param cert certificate object, see
// http://github.com/CartoDB/Windshaft-cartodb/wiki/Signed-maps
//
// @param callback function(err, crt_id) return certificate id
//
// TODO: allow for requesting error when certificate already exists ?
//
o.addCertificate = function(signer, cert, callback) {
var crt_val = JSON.stringify(cert);
var crt_id = crypto.createHash('md5').update(crt_val).digest('hex');
var usr_crt_key = _.template(this.key_map_crt, {signer:signer});
this._redisCmd('HSET', [ usr_crt_key, crt_id, crt_val ], function(err, created) {
// NOTE: created would be 0 if the field already existed, 1 otherwise
callback(err, crt_id);
});
};
// Remove an authorization certificate of a user, also removing
// any signature made with the certificate.
//
// @param signer a signer name (cartodb username)
// @param crt_id certificate identifier, as returned by addCertificate
// @param callback function(err)
//
o.delCertificate = function(signer, crt_id, callback) {
var db = this.db_signatures;
var crt_sig_key = _.template(this.key_crt_sig, {signer:signer, crt_id:crt_id});
var signed_map_list;
var redis_client;
var that = this;
Step (
function getRedisClient() {
that._acquireRedis(this);
},
function removeCertificate(err, data) {
if ( err ) throw err;
redis_client = data;
// Remove the certificate (would be enough to stop authorizing uses)
var usr_crt_key = _.template(that.key_map_crt, {signer:signer});
redis_client.HDEL(usr_crt_key, crt_id, this);
},
function getMapSignatures(err, deleted) {
if ( err ) throw err;
if ( ! deleted ) {
// debugging (how can this be possible?)
console.log("WARNING: authorization certificate '" + crt_id
+ "' by user '" + signer + "' did not exist on delete request");
}
// Get all signatures by this certificate
redis_client.SMEMBERS(crt_sig_key, this);
},
function delMapSignaturesReference(err, map_id_list) {
if ( err ) throw err;
signed_map_list = map_id_list;
console.log("Certificate '" + crt_id + "' from user '" + signer
+ "' was used to sign " + signed_map_list.length + " maps");
redis_client.DEL(crt_sig_key, this);
},
function delMapSignatures(err) {
if ( err ) throw err;
var crt_sig_key = _.template(that.key_crt_sig, {signer:signer, crt_id:crt_id});
var tx = redis_client.MULTI();
for (var i=0; i<signed_map_list.length; ++i) {
var map_id = signed_map_list[i];
var map_sig_key = _.template(that.key_map_sig, {signer:signer, map_id:map_id});
//console.log("Queuing removal of '" + crt_id + "' from '" + map_sig_key + "'");
tx.SREM( map_sig_key, crt_id )
}
tx.EXEC(this);
},
function reportTransaction(err, rets) {
if ( err ) throw err;
for (var i=0; i<signed_map_list.length; ++i) {
var ret = rets[i];
if ( ! ret ) {
console.log("No signature with certificate '" + crt_id
+ "' of user '" + signer + "' found in map '"
+ signed_map_list[i] + "'");
} else {
console.log("Signature with certificate '" + crt_id
+ "' of user '" + signer + "' removed from map '"
+ signed_map_list[i] + "'");
}
}
return null;
},
function finish(err) {
if ( ! _.isUndefined(redis_client) ) {
that._releaseRedis(redis_client);
}
callback(err);
}
);
};
// Sign a map with a certificate reference
//
// @param signer a signer name (cartodb username)
// @param map_id a layergroup_id
// @param crt_id signature certificate identifier
//
// @param callback function(Error)
//
o.signMap = function(signer, map_id, crt_id, callback) {
var that = this;
Step(
function addMapSignature() {
var map_sig_key = _.template(that.key_map_sig, {signer:signer, map_id:map_id});
console.log("Adding " + crt_id + " to " + map_sig_key);
that._redisCmd('SADD', [ map_sig_key, crt_id ], this);
},
function addCertificateUsage(err) {
// Add the map to the set of maps signed by the given cert
if ( err ) throw err;
var crt_sig_key = _.template(that.key_crt_sig, {signer:signer, crt_id:crt_id});
that._redisCmd('SADD', [ crt_sig_key, map_id ], this);
},
function finish(err) {
callback(err);
}
);
};
// Sign a map with a full certificate
//
// @param signer a signer name (cartodb username)
// @param map_id a layergroup_id
// @param cert_id signature certificate identifier
//
// @param callback function(Error, String) return certificate id
//
o.addSignature = function(signer, map_id, cert, callback) {
var that = this;
var certificate_id;
Step(
function addCertificate() {
that.addCertificate(signer, cert, this);
},
function signMap(err, cert_id) {
if ( err ) throw err;
if ( ! cert_id ) throw new Error("addCertificate returned no certificate id");
certificate_id = cert_id;
that.signMap(signer, map_id, cert_id, this);
},
function finish(err) {
callback(err, certificate_id);
}
);
};
module.exports = SignedMaps;

View File

@@ -1,586 +0,0 @@
var crypto = require('crypto');
var Step = require('step');
var _ = require('underscore');
// Templates in this hash (keyed as <username>@<template_name>)
// are being worked on.
var user_template_locks = {};
// Class handling map templates
//
// See http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps
//
// @param redis_pool an instance of a "redis-mpool"
// See https://github.com/CartoDB/node-redis-mpool
// Needs version 0.x.x of the API.
//
// @param signed_maps an instance of a "signed_maps" class,
// See signed_maps.js
//
function TemplateMaps(redis_pool, signed_maps) {
this.redis_pool = redis_pool;
this.signed_maps = signed_maps;
// Database containing templates
// TODO: allow configuring ?
// NOTE: currently it is the same as
// the one containing layergroups
this.db_signatures = 0;
//
// Map templates are owned by a user that specifies access permissions
// for their instances.
//
// We have the following datastores:
//
// 1. User teplates: set of per-user map templates
// NOTE: each template would have an associated auth
// reference, see signed_maps.js
// User templates (HASH:tpl_id->tpl_val)
this.key_usr_tpl = "map_tpl|<%= owner %>";
// User template locks (HASH:tpl_id->ctime)
this.key_usr_tpl_lck = "map_tpl|<%= owner %>|locks";
};
var o = TemplateMaps.prototype;
//--------------- PRIVATE METHODS --------------------------------
o._acquireRedis = function(callback) {
this.redis_pool.acquire(this.db_signatures, callback);
};
o._releaseRedis = function(client) {
this.redis_pool.release(this.db_signatures, client);
};
/**
* Internal function to communicate with redis
*
* @param redisFunc - the redis function to execute
* @param redisArgs - the arguments for the redis function in an array
* @param callback - function to pass results too.
*/
o._redisCmd = function(redisFunc, redisArgs, callback) {
var redisClient;
var that = this;
var db = that.db_signatures;
Step(
function getRedisClient() {
that.redis_pool.acquire(db, this);
},
function executeQuery(err, data) {
if ( err ) throw err;
redisClient = data;
redisArgs.push(this);
redisClient[redisFunc.toUpperCase()].apply(redisClient, redisArgs);
},
function releaseRedisClient(err, data) {
if ( ! _.isUndefined(redisClient) ) that.redis_pool.release(db, redisClient);
callback(err, data);
}
);
};
// @param callback function(err, obtained)
o._obtainTemplateLock = function(owner, tpl_id, callback) {
var usr_tpl_lck_key = _.template(this.key_usr_tpl_lck, {owner:owner});
var that = this;
var gotLock = false;
Step (
function obtainLock() {
var ctime = Date.now();
that._redisCmd('HSETNX', [usr_tpl_lck_key, tpl_id, ctime], this);
},
function checkLock(err, locked) {
if ( err ) throw err;
if ( ! locked ) {
// Already locked
// TODO: unlock if expired ?
throw new Error("Template '" + tpl_id + "' of user '" + owner + "' is locked");
}
return gotLock = true;
},
function finish(err) {
callback(err, gotLock);
}
);
};
// @param callback function(err, deleted)
o._releaseTemplateLock = function(owner, tpl_id, callback) {
var usr_tpl_lck_key = _.template(this.key_usr_tpl_lck, {owner:owner});
this._redisCmd('HDEL', [usr_tpl_lck_key, tpl_id], callback);
};
o._reValidIdentifier = /^[a-zA-Z][0-9a-zA-Z_]*$/;
o._checkInvalidTemplate = function(template) {
if ( template.version != '0.0.1' ) {
return new Error("Unsupported template version " + template.version);
}
var tplname = template.name;
if ( ! tplname ) {
return new Error("Missing template name");
}
if ( ! tplname.match(this._reValidIdentifier) ) {
return new Error("Invalid characters in template name '" + tplname + "'");
}
var phold = template.placeholders;
for (var k in phold) {
if ( ! k.match(this._reValidIdentifier) ) {
return new Error("Invalid characters in placeholder name '" + k + "'");
}
if ( ! phold[k].hasOwnProperty('default') ) {
return new Error("Missing default for placeholder '" + k + "'");
}
if ( ! phold[k].hasOwnProperty('type') ) {
return new Error("Missing type for placeholder '" + k + "'");
}
};
// TODO: run more checks over template format ?
};
//--------------- PUBLIC API -------------------------------------
// Extract a signature certificate from a template
//
// The certificate will be ready to be passed to
// SignedMaps.addCertificate or SignedMaps.authorizedByCert
//
o.getTemplateCertificate = function(template) {
var cert = {
version: '0.0.1',
template_id: template.name,
auth: template.auth
};
return cert;
};
// Add a template
//
// NOTE: locks user+template_name or fails
//
// @param owner cartodb username of the template owner
//
// @param template layergroup template, see
// http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps#template-format
//
// @param callback function(err, tpl_id)
// Return template identifier (only valid for given user)
//
o.addTemplate = function(owner, template, callback) {
var invalidError = this._checkInvalidTemplate(template);
if ( invalidError ) {
callback(invalidError);
return;
}
var tplname = template.name;
// Procedure:
//
// 0. Obtain a lock for user+template_name, fail if impossible
// 1. Check no other template exists with the same name
// 2. Install certificate extracted from template, extending
// it to contain a name to properly salt things out.
// 3. Modify the template object to reference certificate by id
// 4. Install template
// 5. Release lock
//
//
var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
var gotLock = false;
var that = this;
Step(
// try to obtain a lock
function obtainLock() {
that._obtainTemplateLock(owner, tplname, this);
},
function getExistingTemplate(err, locked) {
if ( err ) throw err;
if ( ! locked ) {
// Already locked
throw new Error("Template '" + tplname + "' of user '" + owner + "' is locked");
}
gotLock = true;
that._redisCmd('HEXISTS', [ usr_tpl_key, tplname ], this);
},
function installCertificate(err, exists) {
if ( err ) throw err;
if ( exists ) {
throw new Error("Template '" + tplname + "' of user '" + owner + "' already exists");
}
var cert = that.getTemplateCertificate(template);
that.signed_maps.addCertificate(owner, cert, this);
},
function installTemplate(err, crt_id) {
if ( err ) throw err;
template.auth_id = crt_id;
var tpl_val = JSON.stringify(template);
that._redisCmd('HSET', [ usr_tpl_key, tplname, tpl_val ], this);
},
function releaseLock(err, newfield) {
if ( ! err && ! newfield ) {
console.log("ERROR: addTemplate overridden existing template '"
+ tplname + "' of '" + owner
+ "' -- HSET returned " + overridden + ": someone added it without locking ?");
// TODO: how to recover this ?!
}
if ( ! gotLock ) {
if ( err ) throw err;
return null;
}
// release the lock
var next = this;
that._releaseTemplateLock(owner, tplname, function(e, d) {
if ( e ) {
console.log("Error removing lock on template '" + tplname
+ "' of user '" + owner + "': " + e);
} else if ( ! d ) {
console.log("ERROR: lock on template '" + tplname
+ "' of user '" + owner + "' externally removed during insert!");
}
next(err);
});
},
function finish(err) {
callback(err, tplname);
}
);
};
// Delete a template
//
// NOTE: locks user+template_name or fails
//
// Also deletes associated authentication certificate, which
// in turn deletes all instance signatures
//
// @param owner cartodb username of the template owner
//
// @param tpl_id template identifier as returned
// by addTemplate or listTemplates
//
// @param callback function(err)
//
o.delTemplate = function(owner, tpl_id, callback) {
var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
var gotLock = false;
var that = this;
Step(
// try to obtain a lock
function obtainLock() {
that._obtainTemplateLock(owner, tpl_id, this);
},
function getExistingTemplate(err, locked) {
if ( err ) throw err;
if ( ! locked ) {
// Already locked
throw new Error("Template '" + tpl_id + "' of user '" + owner + "' is locked");
}
gotLock = true;
that._redisCmd('HGET', [ usr_tpl_key, tpl_id ], this);
},
function delCertificate(err, tplval) {
if ( err ) throw err;
if ( ! tplval ) {
throw new Error("Template '" + tpl_id + "' of user '" + owner + "' does not exist");
}
var tpl = JSON.parse(tplval);
if ( ! tpl.auth_id ) {
// not sure this is an error, in case we'll ever
// allow unsigned templates...
console.log("ERROR: installed template '" + tpl_id
+ "' of user '" + owner + "' has no auth_id reference: "); console.dir(tpl);
return null;
}
var next = this;
that.signed_maps.delCertificate(owner, tpl.auth_id, function(err) {
if ( err ) {
var msg = "ERROR: could not delete certificate '"
+ tpl.auth_id + "' associated with template '"
+ tpl_id + "' of user '" + owner + "': " + err;
// I'm actually not sure we want this event to be fatal
// (avoiding a deletion of the template itself)
next(new Error(msg));
} else {
next();
}
});
},
function delTemplate(err) {
if ( err ) throw err;
that._redisCmd('HDEL', [ usr_tpl_key, tpl_id ], this);
},
function releaseLock(err, deleted) {
if ( ! err && ! deleted ) {
console.log("ERROR: template '" + tpl_id
+ "' of user '" + owner + "' externally removed during delete!");
}
if ( ! gotLock ) {
if ( err ) throw err;
return null;
}
// release the lock
var next = this;
that._releaseTemplateLock(owner, tpl_id, function(e, d) {
if ( e ) {
console.log("Error removing lock on template '" + tpl_id
+ "' of user '" + owner + "': " + e);
} else if ( ! d ) {
console.log("ERROR: lock on template '" + tpl_id
+ "' of user '" + owner + "' externally removed during delete!");
}
next(err);
});
},
function finish(err) {
callback(err);
}
);
};
// Update a template
//
// NOTE: locks user+template_name or fails
//
// Also deletes and re-creates associated authentication certificate,
// which in turn deletes all instance signatures
//
// @param owner cartodb username of the template owner
//
// @param tpl_id template identifier as returned by addTemplate
//
// @param template layergroup template, see
// http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps#template-format
//
// @param callback function(err)
//
o.updTemplate = function(owner, tpl_id, template, callback) {
var invalidError = this._checkInvalidTemplate(template);
if ( invalidError ) {
callback(invalidError);
return;
}
var tplname = template.name;
if ( tpl_id != tplname ) {
callback(new Error("Cannot update name of a map template ('" + tpl_id + "' != '" + tplname + "')"));
return;
}
var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
var gotLock = false;
var that = this;
Step(
// try to obtain a lock
function obtainLock() {
that._obtainTemplateLock(owner, tpl_id, this);
},
function getExistingTemplate(err, locked) {
if ( err ) throw err;
if ( ! locked ) {
// Already locked
throw new Error("Template '" + tpl_id + "' of user '" + owner + "' is locked");
}
gotLock = true;
that._redisCmd('HGET', [ usr_tpl_key, tpl_id ], this);
},
function delOldCertificate(err, tplval) {
if ( err ) throw err;
if ( ! tplval ) {
throw new Error("Template '" + tpl_id + "' of user '"
+ owner +"' does not exist");
}
var tpl = JSON.parse(tplval);
if ( ! tpl.auth_id ) {
// not sure this is an error, in case we'll ever
// allow unsigned templates...
console.log("ERROR: installed template '" + tpl_id
+ "' of user '" + owner + "' has no auth_id reference: "); console.dir(tpl);
return null;
}
var next = this;
that.signed_maps.delCertificate(owner, tpl.auth_id, function(err) {
if ( err ) {
var msg = "ERROR: could not delete certificate '"
+ tpl.auth_id + "' associated with template '"
+ tpl_id + "' of user '" + owner + "': " + err;
// I'm actually not sure we want this event to be fatal
// (avoiding a deletion of the template itself)
next(new Error(msg));
} else {
next();
}
});
},
function installNewCertificate(err) {
if ( err ) throw err;
var cert = that.getTemplateCertificate(template);
that.signed_maps.addCertificate(owner, cert, this);
},
function updTemplate(err, crt_id) {
if ( err ) throw err;
template.auth_id = crt_id;
var tpl_val = JSON.stringify(template);
that._redisCmd('HSET', [ usr_tpl_key, tplname, tpl_val ], this);
},
function releaseLock(err, newfield) {
if ( ! err && newfield ) {
console.log("ERROR: template '" + tpl_id
+ "' of user '" + owner + "' externally removed during update!");
}
if ( ! gotLock ) {
if ( err ) throw err;
return null;
}
// release the lock
var next = this;
that._releaseTemplateLock(owner, tpl_id, function(e, d) {
if ( e ) {
console.log("Error removing lock on template '" + tpl_id
+ "' of user '" + owner + "': " + e);
} else if ( ! d ) {
console.log("ERROR: lock on template '" + tpl_id
+ "' of user '" + owner + "' externally removed during update!");
}
next(err);
});
},
function finish(err) {
callback(err);
}
);
};
// List user templates
//
// @param owner cartodb username of the templates owner
//
// @param callback function(err, tpl_id_list)
// Returns a list of template identifiers
//
o.listTemplates = function(owner, callback) {
var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
this._redisCmd('HKEYS', [ usr_tpl_key ], callback);
};
// Get a templates
//
// @param owner cartodb username of the template owner
//
// @param tpl_id template identifier as returned
// by addTemplate or listTemplates
//
// @param callback function(err, template)
// Return full template definition
//
o.getTemplate = function(owner, tpl_id, callback) {
var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
var that = this;
Step(
function getTemplate() {
that._redisCmd('HGET', [ usr_tpl_key, tpl_id ], this);
},
function parseTemplate(err, tpl_val) {
if ( err ) throw err;
var tpl = JSON.parse(tpl_val);
// Should we strip auth_id ?
return tpl;
},
function finish(err, tpl) {
callback(err, tpl);
}
);
};
// Perform placeholder substitutions on a template
//
// @param template a template object (will not be modified)
//
// @param params an object containing named subsitution parameters
// Only the ones found in the template's placeholders object
// will be used, with missing ones taking default values.
//
// @returns a layergroup configuration
//
// @throws Error on malformed template or parameter
//
o._reNumber = /^([-+]?[\d\.]?\d+([eE][+-]?\d+)?)$/;
o._reCSSColorName = /^[a-zA-Z]+$/;
o._reCSSColorVal = /^#[0-9a-fA-F]{3,6}$/;
o._replaceVars = function(str, params) {
//return _.template(str, params); // lazy way, possibly dangerous
// Construct regular expressions for each param
if ( ! params._re ) {
params._re = {};
for (var k in params) {
params._re[k] = RegExp("<%= " + k + " %>", "g");
}
}
for (var k in params) str = str.replace(params._re[k], params[k]);
return str;
};
o.instance = function(template, params) {
var all_params = {};
var phold = template.placeholders;
for (var k in phold) {
var val = params.hasOwnProperty(k) ? params[k] : phold[k].default;
var type = phold[k].type;
// properly escape
if ( type === 'sql_literal' ) {
// duplicate any single-quote
val = val.replace(/'/g, "''");
}
else if ( type === 'sql_ident' ) {
// duplicate any double-quote
val = val.replace(/"/g, '""');
}
else if ( type === 'number' ) {
// check it's a number
if ( ! val.match(this._reNumber) ) {
throw new Error("Invalid number value for template parameter '"
+ k + "': " + val);
}
}
else if ( type === 'css_color' ) {
// check it only contains letters or
// starts with # and only contains hexdigits
if ( ! val.match(this._reCSSColorName) && ! val.match(this._reCSSColorVal) ) {
throw new Error("Invalid css_color value for template parameter '"
+ k + "': " + val);
}
}
else {
// NOTE: should be checked at template create/update time
throw new Error("Invalid placeholder type '" + type + "'");
}
all_params[k] = val;
}
// NOTE: we're deep-cloning the layergroup here
var layergroup = JSON.parse(JSON.stringify(template.layergroup));
for (var i=0; i<layergroup.layers.length; ++i) {
var lyropt = layergroup.layers[i].options;
lyropt.cartocss = this._replaceVars(lyropt.cartocss, all_params);
lyropt.sql = this._replaceVars(lyropt.sql, all_params);
// Anything else ?
}
return layergroup;
};
module.exports = TemplateMaps;

169
npm-shrinkwrap.json generated
View File

@@ -1,6 +1,6 @@
{
"name": "windshaft-cartodb",
"version": "1.6.2",
"version": "1.5.2",
"dependencies": {
"node-varnish": {
"version": "0.1.1"
@@ -9,14 +9,14 @@
"version": "1.3.3"
},
"windshaft": {
"version": "0.15.0",
"version": "0.14.5",
"dependencies": {
"grainstore": {
"version": "0.16.0",
"version": "0.15.2",
"dependencies": {
"carto": {
"version": "0.9.5-cdb2",
"from": "http://github.com/CartoDB/carto/tarball/0.9.5-cdb2",
"from": "git://github.com/CartoDB/carto.git#0.9.5-cdb2",
"dependencies": {
"underscore": {
"version": "1.4.4"
@@ -25,7 +25,7 @@
"version": "0.2.8",
"dependencies": {
"sax": {
"version": "0.5.8"
"version": "0.5.5"
}
}
},
@@ -131,136 +131,46 @@
"version": "0.3.8"
},
"zipfile": {
"version": "0.4.3"
"version": "0.4.2"
},
"sqlite3": {
"version": "2.2.0",
"version": "2.1.19",
"dependencies": {
"node-pre-gyp": {
"version": "0.2.5",
"tar.gz": {
"version": "0.1.1",
"dependencies": {
"nopt": {
"version": "2.1.2",
"fstream": {
"version": "0.1.25",
"dependencies": {
"abbrev": {
"version": "1.0.4"
"rimraf": {
"version": "2.2.4"
},
"graceful-fs": {
"version": "2.0.1"
},
"inherits": {
"version": "2.0.1"
}
}
},
"npmlog": {
"version": "0.0.6",
"dependencies": {
"ansi": {
"version": "0.2.1"
}
}
},
"semver": {
"version": "2.1.0"
},
"tar": {
"version": "0.1.19",
"version": "0.1.18",
"dependencies": {
"inherits": {
"version": "2.0.1"
},
"block-stream": {
"version": "0.0.7"
},
"fstream": {
"version": "0.1.25",
"dependencies": {
"graceful-fs": {
"version": "2.0.1"
}
}
}
}
},
"tar-pack": {
"version": "2.0.0",
"commander": {
"version": "1.1.1",
"dependencies": {
"uid-number": {
"version": "0.0.3"
},
"once": {
"version": "1.1.1"
},
"debug": {
"version": "0.7.4"
},
"fstream": {
"version": "0.1.25",
"dependencies": {
"graceful-fs": {
"version": "2.0.1"
},
"inherits": {
"version": "2.0.1"
}
}
},
"fstream-ignore": {
"version": "0.0.7",
"dependencies": {
"minimatch": {
"version": "0.2.14",
"dependencies": {
"sigmund": {
"version": "1.0.0"
}
}
},
"inherits": {
"version": "2.0.1"
}
}
},
"readable-stream": {
"version": "1.0.24"
},
"graceful-fs": {
"version": "1.2.3"
"keypress": {
"version": "0.1.0"
}
}
},
"aws-sdk": {
"version": "2.0.0-rc8",
"dependencies": {
"xml2js": {
"version": "0.2.4",
"dependencies": {
"sax": {
"version": "0.6.0"
}
}
},
"xmlbuilder": {
"version": "0.4.2"
}
}
},
"rc": {
"version": "0.3.2",
"dependencies": {
"optimist": {
"version": "0.3.7",
"dependencies": {
"wordwrap": {
"version": "0.0.2"
}
}
},
"deep-extend": {
"version": "0.2.6"
},
"ini": {
"version": "1.1.0"
}
}
},
"rimraf": {
"version": "2.2.5"
}
}
}
@@ -329,7 +239,7 @@
}
},
"tilelive-mapnik": {
"version": "0.6.5",
"version": "0.6.4",
"dependencies": {
"eio": {
"version": "0.2.2"
@@ -354,21 +264,10 @@
"version": "2.9.202"
},
"cartodb-redis": {
"version": "0.3.0"
},
"redis-mpool": {
"version": "0.0.3",
"version": "0.1.0",
"dependencies": {
"generic-pool": {
"version": "2.0.4"
},
"hiredis": {
"version": "0.1.16",
"dependencies": {
"bindings": {
"version": "1.1.1"
}
}
}
}
},
@@ -378,15 +277,23 @@
"lzma": {
"version": "1.2.3"
},
"strftime": {
"version": "0.6.2"
},
"semver": {
"version": "1.1.4"
},
"strftime": {
"version": "0.6.2"
},
"redis": {
"version": "0.8.6"
},
"hiredis": {
"version": "0.1.15",
"dependencies": {
"bindings": {
"version": "1.1.0"
}
}
},
"mocha": {
"version": "1.14.0",
"dependencies": {
@@ -420,7 +327,7 @@
"version": "3.2.3",
"dependencies": {
"minimatch": {
"version": "0.2.14",
"version": "0.2.12",
"dependencies": {
"lru-cache": {
"version": "2.5.0"

View File

@@ -1,7 +1,7 @@
{
"private": true,
"name": "windshaft-cartodb",
"version": "1.6.2",
"version": "1.5.2",
"description": "A map tile server for CartoDB",
"keywords": [
"cartodb"
@@ -24,11 +24,10 @@
"dependencies": {
"node-varnish": "0.1.1",
"underscore" : "~1.3.3",
"windshaft" : "~0.15.0",
"windshaft" : "~0.14.5",
"step": "0.0.x",
"request": "2.9.202",
"cartodb-redis": "~0.3.0",
"redis-mpool": "~0.0.2",
"cartodb-redis": "~0.1.0",
"mapnik": "~0.7.22",
"lzma": "~1.2.3"
},

View File

@@ -19,14 +19,6 @@ var serverOptions = require(__dirname + '/../../lib/cartodb/server_options');
var server = new CartodbWindshaft(serverOptions);
server.setMaxListeners(0);
// Check that the response headers do not request caching
// Throws on failure
function checkNoCache(res) {
assert.ok(!res.headers.hasOwnProperty('x-cache-channel'));
assert.ok(!res.headers.hasOwnProperty('cache-control')); // is this correct ?
assert.ok(!res.headers.hasOwnProperty('last-modified')); // is this correct ?
}
suite('multilayer', function() {
var redis_client = redis.createClient(global.environment.redis.port);
@@ -468,35 +460,6 @@ suite('multilayer', function() {
});
});
// Also tests that server doesn't crash:
// see http://github.com/CartoDB/Windshaft-cartodb/issues/109
test("layergroup creation fails if sql is bogus", function(done) {
var layergroup = {
stat_tag: 'random_tag',
version: '1.0.0',
layers: [
{ options: {
sql: 'select bogus(0,0) as the_geom_webmercator',
cartocss: '#layer { polygon-fill:red; }',
cartocss_version: '2.0.1'
} }
]
};
assert.response(server, {
url: '/tiles/layergroup',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(layergroup)
}, {}, function(res) {
assert.equal(res.statusCode, 400, res.body);
var parsed = JSON.parse(res.body);
var msg = parsed.errors[0];
assert.ok(msg.match(/bogus.*exist/), msg);
checkNoCache(res);
done();
});
});
test("layergroup with 2 private-table layers", function(done) {
var layergroup = {
@@ -900,71 +863,6 @@ suite('multilayer', function() {
);
});
// SQL strings can be of arbitrary length, when using POST
// See https://github.com/CartoDB/Windshaft-cartodb/issues/111
test("sql string can be very long", function(done){
var long_val = 'pretty';
for (var i=0; i<1024; ++i) long_val += ' long'
long_val += ' string';
var sql = "SELECT ";
for (var i=0; i<16; ++i)
sql += "'" + long_val + "'::text as pretty_long_field_name_" + i + ", ";
sql += "cartodb_id, the_geom_webmercator FROM gadm4 g";
var layergroup = {
version: '1.0.0',
layers: [
{ options: {
sql: sql,
cartocss: '#layer { marker-fill:red; }',
cartocss_version: '2.0.1'
} }
]
};
var errors = [];
var expected_token;
Step(
function do_post()
{
var data = JSON.stringify(layergroup);
assert.ok(data.length > 1024*64);
var next = this;
assert.response(server, {
url: '/tiles/layergroup?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: data
}, {}, function(res) { next(null, res); });
},
function check_result(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.statusCode + ': ' + res.body);
var parsedBody = JSON.parse(res.body);
var token_components = parsedBody.layergroupid.split(':');
expected_token = token_components[0];
return null;
},
function cleanup(err) {
if ( err ) errors.push(err.message);
if ( ! expected_token ) return null;
var next = this;
redis_client.keys("map_style|test_cartodb_user_1_db|~" + expected_token, function(err, matches) {
if ( err ) errors.push(err.message);
assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
redis_client.del(matches, function(err) {
if ( err ) errors.push(err.message);
next();
});
});
},
function finish(err) {
if ( err ) errors.push('' + err);
if ( errors.length ) done(new Error(errors.join(',')));
else done(null);
}
);
});
suiteTeardown(function(done) {
// This test will add map_style records, like

View File

@@ -107,7 +107,7 @@ suite('server', function() {
}, function(res) {
var parsed = JSON.parse(res.body);
assert.equal(parsed.style, _.template(default_style, {table: 'my_table'}));
assert.equal(parsed.style_version, mapnik_version);
assert.equal(parsed.style_version, mapnik.versions.mapnik);
done();
});
});
@@ -125,7 +125,6 @@ suite('server', function() {
assert.equal(res.statusCode, 400, res.body);
assert.deepEqual(JSON.parse(res.body),
{error: 'Sorry, you are unauthorized (permission denied)'});
assert.ok(!res.headers.hasOwnProperty('cache-control'));
done();
});
});
@@ -143,7 +142,6 @@ suite('server', function() {
assert.equal(res.statusCode, 400, res.statusCode + ': ' + res.body);
assert.deepEqual(JSON.parse(res.body),
{error:"missing unknown_user's database_name in redis (try CARTODB/script/restore_redis)"});
assert.ok(!res.headers.hasOwnProperty('cache-control'));
done();
});
});
@@ -160,7 +158,7 @@ suite('server', function() {
var parsed = JSON.parse(res.body);
var style = _.template(default_style, {table: 'test_table_private_1'});
assert.equal(parsed.style, style);
assert.equal(parsed.style_version, mapnik_version);
assert.equal(parsed.style_version, mapnik.versions.mapnik);
done();
});
});
@@ -214,12 +212,9 @@ suite('server', function() {
url: '/tiles/my_table/style',
method: 'POST'
},{
status: 400,
body: '{"error":"must send style information"}'
}, function(res) {
assert.equal(res.statusCode, 400);
assert.ok(!res.headers.hasOwnProperty('cache-control'));
done();
});
}, function() { done(); });
});
test("post'ing bad style returns 400 with error", function(done){
@@ -356,7 +351,7 @@ suite('server', function() {
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.equal(parsed.style, style);
assert.equal(parsed.style_version, mapnik_version);
assert.equal(parsed.style_version, mapnik.versions.mapnik);
done();
});
});
@@ -384,7 +379,7 @@ suite('server', function() {
var parsed = JSON.parse(res.body);
// NOTE: no transform expected for the specific style
assert.equal(parsed.style, style);
assert.equal(parsed.style_version, mapnik_version);
assert.equal(parsed.style_version, mapnik.versions.mapnik);
done();
});
});
@@ -771,8 +766,6 @@ suite('server', function() {
assert.equal(res.statusCode, 400, res.statusCode + ': ' + res.body);
assert.deepEqual(JSON.parse(res.body),
{error:"missing unknown_user's database_name in redis (try CARTODB/script/restore_redis)"});
assert.ok(!res.headers.hasOwnProperty('cache-control'),
"Unexpected Cache-Control: " + res.headers['cache-control']);
done();
});
});
@@ -793,9 +786,6 @@ suite('server', function() {
}, function(res) {
// 401 Unauthorized
assert.equal(res.statusCode, 401, res.statusCode + ': ' + res.body);
// Failed in 1.6.0 of https://github.com/CartoDB/Windshaft-cartodb/issues/107
assert.ok(!res.headers.hasOwnProperty('cache-control'),
"Unexpected Cache-Control: " + res.headers['cache-control']);
done();
});
});
@@ -1189,7 +1179,6 @@ suite('server', function() {
method: 'DELETE'
},{}, function(res) {
assert.equal(res.statusCode, 404, res.statusCode + ': ' + res.body);
assert.ok(!res.headers.hasOwnProperty('cache-control'));
done();
});
});
@@ -1221,7 +1210,6 @@ suite('server', function() {
},{}, function(res) {
// FIXME: should be 401 instead
assert.equal(res.statusCode, 500, res.statusCode + ': ' + res.body);
assert.ok(!res.headers.hasOwnProperty('cache-control'));
done();
});
});
@@ -1274,7 +1262,6 @@ suite('server', function() {
method: 'DELETE'
},{}, function(res) {
assert.equal(res.statusCode, 404, res.statusCode + ': ' + res.body);
assert.ok(!res.headers.hasOwnProperty('cache-control'));
done();
});
});

View File

@@ -1,936 +0,0 @@
var assert = require('../support/assert');
var tests = module.exports = {};
var _ = require('underscore');
var redis = require('redis');
var querystring = require('querystring');
var semver = require('semver');
var mapnik = require('mapnik');
var Step = require('step');
var strftime = require('strftime');
var SQLAPIEmu = require(__dirname + '/../support/SQLAPIEmu.js');
var redis_stats_db = 5;
require(__dirname + '/../support/test_helper');
var windshaft_fixtures = __dirname + '/../../node_modules/windshaft/test/fixtures';
var CartodbWindshaft = require(__dirname + '/../../lib/cartodb/cartodb_windshaft');
var serverOptions = require(__dirname + '/../../lib/cartodb/server_options');
var server = new CartodbWindshaft(serverOptions);
server.setMaxListeners(0);
suite('template_api', function() {
var redis_client = redis.createClient(global.environment.redis.port);
var sqlapi_server;
var expected_last_updated_epoch = 1234567890123; // this is hard-coded into SQLAPIEmu
var expected_last_updated = new Date(expected_last_updated_epoch).toISOString();
suiteSetup(function(done){
sqlapi_server = new SQLAPIEmu(global.environment.sqlapi.port, done);
// TODO: check redis is clean ?
});
var template_acceptance1 = {
version: '0.0.1',
name: 'acceptance1',
auth: { method: 'open' },
layergroup: {
version: '1.0.0',
layers: [
{ options: {
sql: 'select cartodb_id, ST_Translate(the_geom_webmercator, -5e6, 0) as the_geom_webmercator from test_table limit 2 offset 2',
cartocss: '#layer { marker-fill:blue; marker-allow-overlap:true; }',
cartocss_version: '2.0.2',
interactivity: 'cartodb_id'
} }
]
}
};
test("can add template, returning id", function(done) {
var errors = [];
var expected_failure = false;
var expected_tpl_id = "localhost@acceptance1";
var post_request_1 = {
url: '/tiles/template',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
Step(
function postUnauthenticated()
{
var next = this;
assert.response(server, post_request_1, {},
function(res) { next(null, res); });
},
function postTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 401);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('error'), res.body);
err = parsed.error;
assert.ok(err.match(/only.*authenticated.*user/i),
'Unexpected error response: ' + err);
post_request_1.url += '?api_key=1234';
var next = this;
assert.response(server, post_request_1, {},
function(res) { next(null, res); });
},
function rePostTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsedBody = JSON.parse(res.body);
var expectedBody = { template_id: expected_tpl_id };
assert.deepEqual(parsedBody, expectedBody);
var next = this;
assert.response(server, post_request_1, {},
function(res) { next(null, res); });
},
function checkFailure(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 400, res.body);
var parsedBody = JSON.parse(res.body);
assert.ok(parsedBody.hasOwnProperty('error'), res.body);
assert.ok(parsedBody.error.match(/already exists/i),
'Unexpected error for pre-existing template name: ' + parsedBody.error);
return null;
},
function finish(err) {
if ( err ) errors.push(err);
redis_client.keys("map_*|localhost", function(err, keys) {
if ( err ) errors.push(err.message);
var todrop = _.map(keys, function(m) {
if ( m.match(/^map_(tpl|crt)|/) )
return m;
});
if ( todrop.length != 2 ) {
errors.push(new Error("Unexpected keys in redis: " + todrop));
} else {
if ( todrop.indexOf('map_tpl|localhost') == -1 ) {
errors.push(new Error("Missing 'map_tpl|localhost' key in redis"));
}
if ( todrop.indexOf('map_crt|localhost') == -1 ) {
errors.push(new Error("Missing 'map_crt|localhost' key in redis"));
}
}
redis_client.del(todrop, function(err) {
if ( err ) errors.push(err.message);
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
});
});
}
);
});
test("instance endpoint should return CORS headers", function(done){
Step(function postTemplate1(err, res) {
var next = this;
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost.localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
};
assert.response(server, post_request, {}, function(res) { next(null, res); });
},
function testCORS() {
assert.response(server, {
url: '/tiles/template/acceptance1',
method: 'OPTIONS'
},{
status: 200,
headers: {
'Access-Control-Allow-Headers': 'X-Requested-With, X-Prototype-Version, X-CSRF-Token, Content-Type',
'Access-Control-Allow-Origin': '*'
}
}, function() { done(); });
});
});
test("can list templates", function(done) {
var errors = [];
var expected_failure = false;
var tplid1, tplid2;
Step(
function postTemplate1(err, res)
{
var next = this;
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function postTemplate2(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_id'),
"Missing 'template_id' from response body: " + res.body);
tplid1 = parsed.template_id;
var next = this;
var backup_name = template_acceptance1.name;
template_acceptance1.name += '_new';
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
template_acceptance1.name = backup_name;
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function litsTemplatesUnauthenticated(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_id'),
"Missing 'template_id' from response body: " + res.body);
tplid2 = parsed.template_id;
var next = this;
var get_request = {
url: '/tiles/template',
method: 'GET',
headers: {host: 'localhost'}
}
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function litsTemplates(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 401, res.statusCode + ': ' + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('error'),
'Missing error from response: ' + res.body);
err = parsed.error;
assert.ok(err.match(/authenticated user/), err);
var next = this;
var get_request = {
url: '/tiles/template?api_key=1234',
method: 'GET',
headers: {host: 'localhost'}
}
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function checkList(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_ids'),
"Missing 'template_ids' from response body: " + res.body);
var ids = parsed.template_ids;
assert.equal(ids.length, 2);
assert.ok(ids.indexOf(tplid1) != -1,
'Missing "' + tplid1 + "' from list response: " + ids.join(','));
assert.ok(ids.indexOf(tplid2) != -1,
'Missing "' + tplid2 + "' from list response: " + ids.join(','));
return null;
},
function finish(err) {
if ( err ) errors.push(err);
redis_client.keys("map_*|localhost", function(err, keys) {
if ( err ) errors.push(err.message);
var todrop = _.map(keys, function(m) {
if ( m.match(/^map_(tpl|crt)|/) )
return m;
});
if ( todrop.length != 2 ) {
errors.push(new Error("Unexpected keys in redis: " + todrop));
} else {
if ( todrop.indexOf('map_tpl|localhost') == -1 ) {
errors.push(new Error("Missing 'map_tpl|localhost' key in redis"));
}
if ( todrop.indexOf('map_crt|localhost') == -1 ) {
errors.push(new Error("Missing 'map_crt|localhost' key in redis"));
}
}
redis_client.del(todrop, function(err) {
if ( err ) errors.push(err.message);
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
});
});
}
);
});
test("can update template", function(done) {
var errors = [];
var expected_failure = false;
var tpl_id;
Step(
function postTemplate(err, res)
{
var next = this;
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function putMisnamedTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_id'),
"Missing 'template_id' from response body: " + res.body);
tpl_id = parsed.template_id;
var backup_name = template_acceptance1.name;
template_acceptance1.name = 'changed_name';
var put_request = {
url: '/tiles/template/' + tpl_id + '/?api_key=1234',
method: 'PUT',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
template_acceptance1.name = backup_name;
var next = this;
assert.response(server, put_request, {},
function(res) { next(null, res); });
},
function putUnexistentTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 400, res.statusCode + ": " + res.body);
var parsedBody = JSON.parse(res.body);
assert.ok(parsedBody.hasOwnProperty('error'), res.body);
assert.ok(parsedBody.error.match(/cannot update name/i),
'Unexpected error for invalid update: ' + parsedBody.error);
var put_request = {
url: '/tiles/template/unexistent/?api_key=1234',
method: 'PUT',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
var next = this;
assert.response(server, put_request, {},
function(res) { next(null, res); });
},
function putValidTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 400, res.statusCode + ": " + res.body);
var parsedBody = JSON.parse(res.body);
assert.ok(parsedBody.hasOwnProperty('error'), res.body);
assert.ok(parsedBody.error.match(/cannot update name/i),
'Unexpected error for invalid update: ' + parsedBody.error);
var put_request = {
url: '/tiles/template/' + tpl_id + '/?api_key=1234',
method: 'PUT',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
var next = this;
assert.response(server, put_request, {},
function(res) { next(null, res); });
},
function checkValidUpate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.statusCode + ": " + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_id'),
"Missing 'template_id' from response body: " + res.body);
assert.equal(tpl_id, parsed.template_id);
return null;
},
function finish(err) {
if ( err ) errors.push(err);
redis_client.keys("map_*|localhost", function(err, keys) {
if ( err ) errors.push(err.message);
var todrop = _.map(keys, function(m) {
if ( m.match(/^map_(tpl|crt)|/) )
return m;
});
if ( todrop.length != 2 ) {
errors.push(new Error("Unexpected keys in redis: " + todrop));
} else {
if ( todrop.indexOf('map_tpl|localhost') == -1 ) {
errors.push(new Error("Missing 'map_tpl|localhost' key in redis"));
}
if ( todrop.indexOf('map_crt|localhost') == -1 ) {
errors.push(new Error("Missing 'map_crt|localhost' key in redis"));
}
}
redis_client.del(todrop, function(err) {
if ( err ) errors.push(err.message);
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
});
});
}
);
});
test("can get a template by id", function(done) {
var errors = [];
var expected_failure = false;
var tpl_id;
Step(
function postTemplate(err, res)
{
var next = this;
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function getTemplateUnauthorized(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_id'),
"Missing 'template_id' from response body: " + res.body);
tpl_id = parsed.template_id;
var get_request = {
url: '/tiles/template/' + tpl_id,
method: 'GET',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function getTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 401, res.statusCode + ": " + res.body);
var parsedBody = JSON.parse(res.body);
assert.ok(parsedBody.hasOwnProperty('error'), res.body);
assert.ok(parsedBody.error.match(/only.*authenticated.*user/i),
'Unexpected error for unauthenticated template get: ' + parsedBody.error);
var get_request = {
url: '/tiles/template/' + tpl_id + '?api_key=1234',
method: 'GET',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function checkReturnTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.statusCode + ": " + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template'),
"Missing 'template' from response body: " + res.body);
assert.deepEqual(template_acceptance1, parsed.template);
return null;
},
function finish(err) {
if ( err ) errors.push(err);
redis_client.keys("map_*|localhost", function(err, keys) {
if ( err ) errors.push(err.message);
var todrop = _.map(keys, function(m) {
if ( m.match(/^map_(tpl|crt)|/) )
return m;
});
if ( todrop.length != 2 ) {
errors.push(new Error("Unexpected keys in redis: " + todrop));
} else {
if ( todrop.indexOf('map_tpl|localhost') == -1 ) {
errors.push(new Error("Missing 'map_tpl|localhost' key in redis"));
}
if ( todrop.indexOf('map_crt|localhost') == -1 ) {
errors.push(new Error("Missing 'map_crt|localhost' key in redis"));
}
}
redis_client.del(todrop, function(err) {
if ( err ) errors.push(err.message);
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
});
});
}
);
});
test("can delete a template by id", function(done) {
var errors = [];
var expected_failure = false;
var tpl_id;
Step(
function postTemplate(err, res)
{
var next = this;
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance1)
}
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function getTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_id'),
"Missing 'template_id' from response body: " + res.body);
tpl_id = parsed.template_id;
var get_request = {
url: '/tiles/template/' + tpl_id + '?api_key=1234',
method: 'GET',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function deleteTemplateUnauthorized(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.statusCode + ": " + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template'),
"Missing 'template' from response body: " + res.body);
assert.deepEqual(template_acceptance1, parsed.template);
var del_request = {
url: '/tiles/template/' + tpl_id,
method: 'DELETE',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, del_request, {},
function(res) { next(null, res); });
},
function deleteTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 401, res.statusCode + ": " + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('error'),
"Missing 'error' from response body: " + res.body);
assert.ok(parsed.error.match(/only.*authenticated.*user/i),
'Unexpected error for unauthenticated template get: ' + parsed.error);
var del_request = {
url: '/tiles/template/' + tpl_id + '?api_key=1234',
method: 'DELETE',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, del_request, {},
function(res) { next(null, res); });
},
function getMissingTemplate(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 204, res.statusCode + ': ' + res.body);
assert.ok(!res.body, 'Unexpected body in DELETE /template response');
var get_request = {
url: '/tiles/template/' + tpl_id + '?api_key=1234',
method: 'GET',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function checkGetFailure(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 404, res.statusCode + ': ' + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('error'),
"Missing 'error' from response body: " + res.body);
assert.ok(parsed.error.match(/cannot find/i),
'Unexpected error for missing template: ' + parsed.error);
return null;
},
function finish(err) {
if ( err ) errors.push(err);
redis_client.keys("map_*|localhost", function(err, keys) {
if ( err ) errors.push(err.message);
var todrop = _.map(keys, function(m) {
if ( m.match(/^map_(tpl|crt)|/) )
return m;
});
if ( todrop.length ) {
errors.push(new Error("Unexpected keys in redis: " + todrop));
redis_client.del(todrop, function(err) {
if ( err ) errors.push(err.message);
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
});
} else {
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
}
});
}
);
});
test("can instanciate a template by id", function(done) {
// This map fetches data from a private table
var template_acceptance2 = {
version: '0.0.1',
name: 'acceptance1',
auth: { method: 'token', valid_tokens: ['valid1','valid2'] },
layergroup: {
version: '1.0.0',
layers: [
{ options: {
sql: "select * from test_table_private_1 LIMIT 0",
cartocss: '#layer { marker-fill:blue; marker-allow-overlap:true; }',
cartocss_version: '2.0.2',
interactivity: 'cartodb_id'
} }
]
}
};
var template_params = {};
var errors = [];
var expected_failure = false;
var tpl_id;
var layergroupid;
Step(
function postTemplate(err, res)
{
var next = this;
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_acceptance2)
}
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function instanciateNoAuth(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('template_id'),
"Missing 'template_id' from response body: " + res.body);
tpl_id = parsed.template_id;
var post_request = {
url: '/tiles/template/' + tpl_id,
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_params)
}
var next = this;
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function instanciateAuth(err, res)
{
if ( err ) throw err;
assert.equal(res.statusCode, 401,
'Unexpected success instanciating template with no auth: '
+ res.statusCode + ': ' + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('error'),
"Missing 'error' from response body: " + res.body);
assert.ok(parsed.error.match(/unauthorized/i),
'Unexpected error for unauthorized instance : ' + parsed.error);
var post_request = {
url: '/tiles/template/' + tpl_id + '?auth_token=valid2',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template_params)
}
var next = this;
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function fetchTileNoAuth(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 200,
'Instantiating template: ' + res.statusCode + ': ' + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('layergroupid'),
"Missing 'layergroupid' from response body: " + res.body);
layergroupid = parsed.layergroupid;
assert.ok(layergroupid.match(/^localhost@/),
"Returned layergroupid does not start with signer name: "
+ layergroupid);
assert.ok(parsed.hasOwnProperty('last_updated'),
"Missing 'last_updated' from response body: " + res.body);
// TODO: check value of last_updated ?
var get_request = {
url: '/tiles/layergroup/' + layergroupid + ':cb0/0/0/0.png',
method: 'GET',
headers: {host: 'localhost' },
encoding: 'binary'
}
var next = this;
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function fetchTileAuth(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 401,
'Fetching tile with no auth: ' + res.statusCode + ': ' + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('error'),
"Missing 'error' from response body: " + res.body);
assert.ok(parsed.error.match(/permission denied/i),
'Unexpected error for unauthorized instance '
+ '(expected /permission denied): ' + parsed.error);
var get_request = {
url: '/tiles/layergroup/' + layergroupid + '/0/0/0.png?auth_token=valid1',
method: 'GET',
headers: {host: 'localhost' },
encoding: 'binary'
}
var next = this;
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function checkTile(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 200,
'Unexpected error for authorized instance: '
+ res.statusCode + ' -- ' + res.body);
assert.equal(res.headers['content-type'], "image/png");
return null;
},
function deleteTemplate(err)
{
if ( err ) throw err;
var del_request = {
url: '/tiles/template/' + tpl_id + '?api_key=1234',
method: 'DELETE',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, del_request, {},
function(res) { next(null, res); });
},
function fetchTileDeleted(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 204,
'Deleting template: ' + res.statusCode + ':' + res.body);
var get_request = {
url: '/tiles/layergroup/' + layergroupid + '/0/0/0.png?auth_token=valid1',
method: 'GET',
headers: {host: 'localhost' },
encoding: 'binary'
}
var next = this;
assert.response(server, get_request, {},
function(res) { next(null, res); });
},
function checkTileDeleted(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 401,
'Unexpected statusCode fetch tile after signature revokal: '
+ res.statusCode + ':' + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('error'),
"Missing 'error' from response body: " + res.body);
assert.ok(parsed.error.match(/permission denied/i),
'Unexpected error for unauthorized access : ' + parsed.error);
return null;
},
function finish(err) {
if ( err ) errors.push(err);
redis_client.keys("map_*|localhost", function(err, keys) {
if ( err ) errors.push(err.message);
var todrop = _.map(keys, function(m) {
if ( m.match(/^map_(tpl|crt)|/) )
return m;
});
if ( todrop.length ) {
errors.push(new Error("Unexpected keys in redis: " + todrop));
redis_client.del(todrop, function(err) {
if ( err ) errors.push(err.message);
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
});
} else {
if ( errors.length ) {
done(new Error(errors));
}
else done(null);
}
});
}
);
});
test("template instantiation raises mapviews counter", function(done) {
var layergroup = {
stat_tag: 'random_tag',
version: '1.0.0',
layers: [
{ options: {
sql: 'select 1 as cartodb_id, !pixel_height! as h, '
+ 'ST_Buffer(!bbox!, -32*greatest(!pixel_width!,!pixel_height!)) as the_geom_webmercator',
cartocss: '#layer { polygon-fill:red; }',
cartocss_version: '2.0.1'
} }
]
};
var template = {
version: '0.0.1',
name: 'stat_gathering',
auth: { method: 'open' },
layergroup: layergroup
};
var statskey = "user:localhost:mapviews";
var redis_stats_client = redis.createClient(global.environment.redis.port);
var template_id; // will be set on template post
var now = strftime("%Y%m%d", new Date());
var errors = [];
Step(
function clean_stats()
{
var next = this;
redis_stats_client.select(redis_stats_db, function(err) {
if ( err ) next(err);
else redis_stats_client.del(statskey+':global', next);
});
},
function do_post_tempate(err)
{
if ( err ) throw err;
var post_request = {
url: '/tiles/template?api_key=1234',
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify(template)
}
var next = this;
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function instantiateTemplate(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 200, res.body);
template_id = JSON.parse(res.body).template_id;
var post_request = {
url: '/tiles/template/' + template_id,
method: 'POST',
headers: {host: 'localhost', 'Content-Type': 'application/json' },
data: JSON.stringify({})
}
var next = this;
assert.response(server, post_request, {},
function(res) { next(null, res); });
},
function check_global_stats(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 200,
'Instantiating template: ' + res.statusCode + ': ' + res.body);
var parsed = JSON.parse(res.body);
assert.ok(parsed.hasOwnProperty('layergroupid'),
"Missing 'layergroupid' from response body: " + res.body);
layergroupid = parsed.layergroupid;
redis_stats_client.zscore(statskey + ":global", now, this);
},
function check_tag_stats(err, val) {
if ( err ) throw err;
assert.equal(val, 1, "Expected score of " + now + " in "
+ statskey + ":global to be 1, got " + val);
redis_stats_client.zscore(statskey+':stat_tag:random_tag', now, this);
},
function check_tag_stats_value(err, val) {
if ( err ) throw err;
assert.equal(val, 1, "Expected score of " + now + " in "
+ statskey + ":stat_tag:" + layergroup.stat_tag + " to be 1, got " + val);
return null;
},
function deleteTemplate(err)
{
if ( err ) throw err;
var del_request = {
url: '/tiles/template/' + template_id + '?api_key=1234',
method: 'DELETE',
headers: {host: 'localhost'}
}
var next = this;
assert.response(server, del_request, {},
function(res) { next(null, res); });
},
function cleanup_stats(err, res) {
if ( err ) throw err;
assert.equal(res.statusCode, 204, res.statusCode + ': ' + res.body);
if ( err ) errors.push('' + err);
redis_client.del([statskey+':global', statskey+':stat_tag:'+layergroup.stat_tag], this);
},
function finish(err) {
if ( err ) errors.push('' + err);
if ( errors.length ) done(new Error(errors.join(',')));
else done(null);
}
);
});
suiteTeardown(function(done) {
// This test will add map_style records, like
// 'map_style|null|publicuser|my_table',
redis_client.keys("map_*", function(err, keys) {
var todrop = _.map(keys, function(m) {
if ( m.match(/^map_(tpl|crt|sig)|/) ) return m;
});
redis_client.del(todrop, function(err) {
redis_client.select(5, function(err) {
redis_client.keys("user:localhost:mapviews*", function(err, keys) {
redis_client.del(keys, function(err) {
sqlapi_server.close(done);
});
});
});
});
});
});
});

View File

@@ -5,60 +5,31 @@ var o = function(port, cb) {
this.queries = [];
var that = this;
this.sqlapi_server = http.createServer(function(req,res) {
//console.log("server got request with method " + req.method);
var query;
if ( req.method == 'GET' ) {
query = url.parse(req.url, true).query;
that.handleQuery(query, res);
}
else if ( req.method == 'POST') {
var data = '';
req.on('data', function(chunk) {
//console.log("GOT Chunk " + chunk);
data += chunk;
});
req.on('end', function() {
//console.log("Data is: "); console.dir(data);
query = JSON.parse(data);
//console.log("Parsed is: "); console.dir(query);
//console.log("handleQuery is " + that.handleQuery);
that.handleQuery(query, res);
});
}
else {
that.handleQuery('SQLAPIEmu does not support method' + req.method, res);
}
var query = url.parse(req.url, true).query;
that.queries.push(query);
if ( query.q.match('SQLAPIERROR') ) {
res.statusCode = 400;
res.write(JSON.stringify({'error':'Some error occurred'}));
} else if ( query.q.match('EPOCH.* as max') ) {
// This is the structure of the known query sent by tiler
var row = {
'max': 1234567890.123
};
res.write(JSON.stringify({rows: [ row ]}));
} else {
var qs = JSON.stringify(query);
var row = {
// This is the structure of the known query sent by tiler
'cdb_querytables': '{' + qs + '}',
'max': qs
};
res.write(JSON.stringify({rows: [ row ]}));
}
res.end();
}).listen(port, cb);
};
o.prototype.handleQuery = function(query, res) {
this.queries.push(query);
if ( query.q.match('SQLAPIERROR') ) {
res.statusCode = 400;
res.write(JSON.stringify({'error':'Some error occurred'}));
} else if ( query.q.match('EPOCH.* as max') ) {
// This is the structure of the known query sent by tiler
var row = {
'max': 1234567890.123
};
res.write(JSON.stringify({rows: [ row ]}));
} else {
var qs = JSON.stringify(query);
var row = {
// This is the structure of the known query sent by tiler
'cdb_querytables': '{' + qs + '}',
'max': qs
};
var out_obj = {rows: [ row ]};
var out = JSON.stringify(out_obj);
res.write(out);
}
res.end();
};
o.prototype.close = function(cb) {
this.sqlapi_server.close(cb);
};

View File

@@ -83,27 +83,20 @@ fi
if test x"$PREPARE_REDIS" = xyes; then
echo "preparing redis..."
cat <<EOF | redis-cli -p ${REDIS_PORT} -n 5
HMSET rails:users:localhost id ${TESTUSERID} \
database_name '${TEST_DB}' \
map_key 1234
SADD rails:users:localhost:map_key 1235
EOF
echo "HSET rails:users:localhost id ${TESTUSERID}" | redis-cli -p ${REDIS_PORT} -n 5
echo 'HSET rails:users:localhost database_name "'"${TEST_DB}"'"' | redis-cli -p ${REDIS_PORT} -n 5
echo "HSET rails:users:localhost map_key 1234" | redis-cli -p ${REDIS_PORT} -n 5
echo "SADD rails:users:localhost:map_key 1235" | redis-cli -p ${REDIS_PORT} -n 5
# A user configured as with cartodb-2.5.0+
cat <<EOF | redis-cli -p ${REDIS_PORT} -n 5
HMSET rails:users:cartodb250user id ${TESTUSERID} \
database_name "${TEST_DB}" \
database_host "localhost" \
database_password "${TESTPASS}" \
map_key 4321
EOF
echo "HSET rails:users:cartodb250user id ${TESTUSERID}" | redis-cli -p ${REDIS_PORT} -n 5
echo 'HSET rails:users:cartodb250user database_name "'${TEST_DB}'"' | redis-cli -p ${REDIS_PORT} -n 5
echo 'HSET rails:users:cartodb250user database_host "localhost"' | redis-cli -p ${REDIS_PORT} -n 5
echo 'HSET rails:users:cartodb250user database_password "'${TESTPASS}'"' | redis-cli -p ${REDIS_PORT} -n 5
echo "HSET rails:users:cartodb250user map_key 4321" | redis-cli -p ${REDIS_PORT} -n 5
cat <<EOF | redis-cli -p ${REDIS_PORT} -n 0
HSET rails:${TEST_DB}:my_table infowindow "this, that, the other"
HSET rails:${TEST_DB}:test_table_private_1 privacy "0"
EOF
echo 'HSET rails:'"${TEST_DB}"':my_table infowindow "this, that, the other"' | redis-cli -p ${REDIS_PORT} -n 0
echo 'HSET rails:'"${TEST_DB}"':test_table_private_1 privacy "0"' | redis-cli -p ${REDIS_PORT} -n 0
fi

View File

@@ -15,7 +15,7 @@ suite('req2params', function() {
test('cleans up request', function(done){
opts.req2params({headers: { host:'localhost' }, query: {dbuser:'hacker',dbname:'secret'}}, function(err, req) {
if ( err ) { done(err); return; }
if ( err ) { console.log(err); throw new Error(err); }
assert.ok(_.isObject(req.query), 'request has query');
assert.ok(!req.query.hasOwnProperty('dbuser'), 'dbuser was removed from query');
assert.ok(req.hasOwnProperty('params'), 'request has params');
@@ -28,7 +28,7 @@ suite('req2params', function() {
test('sets dbname from redis metadata', function(done){
opts.req2params({headers: { host:'localhost' }, query: {} }, function(err, req) {
if ( err ) { done(err); return; }
if ( err ) { console.log(err); throw new Error(err); }
//console.dir(req);
assert.ok(_.isObject(req.query), 'request has query');
assert.ok(!req.query.hasOwnProperty('dbuser'), 'dbuser was removed from query');
@@ -44,7 +44,7 @@ suite('req2params', function() {
test('sets also dbuser for authenticated requests', function(done){
opts.req2params({headers: { host:'localhost' }, query: {map_key: '1234'} }, function(err, req) {
if ( err ) { done(err); return; }
if ( err ) { console.log(err); throw new Error(err); }
//console.dir(req);
assert.ok(_.isObject(req.query), 'request has query');
assert.ok(!req.query.hasOwnProperty('dbuser'), 'dbuser was removed from query');
@@ -70,8 +70,7 @@ suite('req2params', function() {
cache_buster: 5
};
test_helper.lzma_compress_to_base64(JSON.stringify(qo), 1, function(err, data) {
opts.req2params({ headers: { host:'localhost' }, query: { non_included: 'toberemoved', api_key: 'test', style: 'override', lzma: data }}, function(err, req) {
if ( err ) { done(err); return; }
opts.req2params({ query: { non_included: 'toberemoved', api_key: 'test', style: 'override', lzma: data }}, function(err, req) {
var query = req.params
assert.equal(qo.style, query.style)
assert.equal(qo.style_version, query.style_version)

View File

@@ -1,85 +0,0 @@
var assert = require('assert')
//, _ = require('underscore')
, RedisPool = require('redis-mpool')
, SignedMaps = require('../../../lib/cartodb/signed_maps.js')
, test_helper = require('../../support/test_helper')
, Step = require('step')
, tests = module.exports = {};
suite('signed_maps', function() {
// configure redis pool instance to use in tests
var redis_pool = RedisPool(global.environment.redis);
test('can sign map with open and token-based auth', function(done) {
var smap = new SignedMaps(redis_pool);
assert.ok(smap);
var sig = 'sig1';
var map = 'map1';
var tok = 'tok1';
var crt = {
version:'0.0.1',
layergroup_id:map,
auth: {}
};
var crt1_id; // by token
var crt2_id; // open
Step(
function() {
smap.isAuthorized(sig,map,tok,this);
},
function checkAuthFailure1(err, authorized) {
if ( err ) throw err;
assert.ok(!authorized, "unexpectedly authorized");
crt.auth.method = 'token';
crt.auth.valid_tokens = [tok];
smap.addSignature(sig, map, crt, this)
},
function getCert1(err, id) {
if ( err ) throw err;
assert.ok(id, "undefined signature id");
crt1_id = id; // keep note of it
//console.log("Certificate 1 is " + crt1_id);
smap.isAuthorized(sig,map,'',this);
},
function checkAuthFailure2(err, authorized) {
if ( err ) throw err;
assert.ok(!authorized, "unexpectedly authorized");
smap.isAuthorized(sig,map,tok,this);
},
function checkAuthSuccess1(err, authorized) {
if ( err ) throw err;
assert.ok(authorized, "unauthorized :(");
crt.auth.method = 'open';
delete crt.auth.valid_tokens;
smap.addSignature(sig, map, crt, this)
},
function getCert2(err, id) {
if ( err ) throw err;
assert.ok(id, "undefined signature id");
crt2_id = id; // keep note of it
//console.log("Certificate 2 is " + crt2_id);
smap.isAuthorized(sig,map,'arbitrary',this);
},
function checkAuthSuccess2_delCert2(err, authorized) {
if ( err ) throw err;
assert.ok(authorized, "unauthorized :(");
var next = this;
smap.delCertificate(sig, crt2_id, function(e) {
if (e) next(e);
else smap.isAuthorized(sig,map,'arbitrary',next);
});
},
function checkAuthFailure3_delCert2(err, authorized) {
if ( err ) throw err;
assert.ok(!authorized, "unexpectedly authorized");
smap.delCertificate(sig, crt1_id, this);
},
function finish(err) {
done(err);
}
);
});
});

View File

@@ -1,400 +0,0 @@
var assert = require('assert')
//, _ = require('underscore')
, RedisPool = require('redis-mpool')
, SignedMaps = require('../../../lib/cartodb/signed_maps.js')
, TemplateMaps = require('../../../lib/cartodb/template_maps.js')
, test_helper = require('../../support/test_helper')
, Step = require('step')
, tests = module.exports = {};
suite('template_maps', function() {
// configure redis pool instance to use in tests
var redis_pool = RedisPool(global.environment.redis);
var signed_maps = new SignedMaps(redis_pool);
test('does not accept template with unsupported version', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var tpl = { version:'6.6.6',
name:'k', auth: {}, layergroup: {} };
Step(
function() {
tmap.addTemplate('me', tpl, this);
},
function checkFailed(err) {
assert.ok(err);
assert.ok(err.message.match(/unsupported.*version/i), err);
return null;
},
function finish(err) {
done(err);
}
);
});
test('does not accept template with missing name', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var tpl = { version:'0.0.1',
auth: {}, layergroup: {} };
Step(
function() {
tmap.addTemplate('me', tpl, this);
},
function checkFailed(err) {
assert.ok(err);
assert.ok(err.message.match(/missing.*name/i), err);
return null;
},
function finish(err) {
done(err);
}
);
});
test('does not accept template with invalid name', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var tpl = { version:'0.0.1',
auth: {}, layergroup: {} };
var invalidnames = [ "ab|", "a b", "a@b", "1ab", "_x", "", " x", "x " ];
var testNext = function() {
if ( ! invalidnames.length ) { done(); return; }
var n = invalidnames.pop();
tpl.name = n;
tmap.addTemplate('me', tpl, function(err) {
if ( ! err ) {
done(new Error("Unexpected success with invalid name '" + n + "'"));
}
else if ( ! err.message.match(/template.*name/i) ) {
done(new Error("Unexpected error message with invalid name '" + n
+ "': " + err));
}
else {
testNext();
}
});
};
testNext();
});
test('does not accept template with invalid placeholder name', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var tpl = { version:'0.0.1',
name: "valid", placeholders: {},
auth: {}, layergroup: {} };
var invalidnames = [ "ab|", "a b", "a@b", "1ab", "_x", "", " x", "x " ];
var testNext = function() {
if ( ! invalidnames.length ) { done(); return; }
var n = invalidnames.pop();
tpl.placeholders = {};
tpl.placeholders[n] = { type:'number', default:1 };
tmap.addTemplate('me', tpl, function(err) {
if ( ! err ) {
done(new Error("Unexpected success with invalid name '" + n + "'"));
}
else if ( ! err.message.match(/invalid.*name/i) ) {
done(new Error("Unexpected error message with invalid name '" + n
+ "': " + err));
}
else {
testNext();
}
});
};
testNext();
});
test('does not accept template with missing placeholder default', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var tpl = { version:'0.0.1',
name: "valid", placeholders: { v: {} },
auth: {}, layergroup: {} };
tmap.addTemplate('me', tpl, function(err) {
if ( ! err ) {
done(new Error("Unexpected success with missing placeholder default"));
}
else if ( ! err.message.match(/missing default/i) ) {
done(new Error("Unexpected error message with missing placeholder default: "
+ err));
}
else {
done();
}
});
});
test('does not accept template with missing placeholder type', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var tpl = { version:'0.0.1',
name: "valid", placeholders: { v: { default:1 } },
auth: {}, layergroup: {} };
tmap.addTemplate('me', tpl, function(err) {
if ( ! err ) {
done(new Error("Unexpected success with missing placeholder type"));
}
else if ( ! err.message.match(/missing type/i) ) {
done(new Error("Unexpected error message with missing placeholder default: "
+ err));
}
else {
done();
}
});
});
test('add, get and delete a valid template', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var expected_failure = false;
var tpl_id;
var tpl = { version:'0.0.1',
name: 'first', auth: {}, layergroup: {} };
Step(
function() {
tmap.addTemplate('me', tpl, this);
},
function addOmonimousTemplate(err, id) {
if ( err ) throw err;
tpl_id = id;
assert.equal(tpl_id, 'first');
expected_failure = true;
// should fail, as it already exists
tmap.addTemplate('me', tpl, this);
},
function getTemplate(err) {
if ( ! expected_failure && err ) throw err;
assert.ok(err);
assert.ok(err.message.match(/already exists/i), err);
tmap.getTemplate('me', tpl_id, this);
},
function delTemplate(err, got_tpl) {
if ( err ) throw err;
assert.deepEqual(got_tpl, tpl);
tmap.delTemplate('me', tpl_id, this);
},
function finish(err) {
done(err);
}
);
});
test('add multiple templates, list them', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var expected_failure = false;
var tpl1 = { version:'0.0.1', name: 'first', auth: {}, layergroup: {} };
var tpl1_id;
var tpl2 = { version:'0.0.1', name: 'second', auth: {}, layergroup: {} };
var tpl2_id;
Step(
function addTemplate1() {
tmap.addTemplate('me', tpl1, this);
},
function addTemplate2(err, id) {
if ( err ) throw err;
tpl1_id = id;
tmap.addTemplate('me', tpl2, this);
},
function listTemplates(err, id) {
if ( err ) throw err;
tpl2_id = id;
tmap.listTemplates('me', this);
},
function checkTemplates(err, ids) {
if ( err ) throw err;
assert.equal(ids.length, 2);
assert.ok(ids.indexOf(tpl1_id) != -1, ids.join(','));
assert.ok(ids.indexOf(tpl2_id) != -1, ids.join(','));
return null;
},
function delTemplate1(err) {
if ( tpl1_id ) {
var next = this;
tmap.delTemplate('me', tpl1_id, function(e) {
if ( err || e ) next(new Error(err + '; ' + e));
else next();
});
} else {
if ( err ) throw err;
return null;
}
},
function delTemplate2(err) {
if ( tpl2_id ) {
var next = this;
tmap.delTemplate('me', tpl2_id, function(e) {
if ( err || e ) next(new Error(err + '; ' + e));
else next();
});
} else {
if ( err ) throw err;
return null;
}
},
function finish(err) {
done(err);
}
);
});
test('update templates', function(done) {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var expected_failure = false;
var owner = 'me';
var tpl = { version:'0.0.1',
name: 'first',
auth: { method: 'open' },
layergroup: {}
};
var tpl_id;
Step(
function addTemplate() {
tmap.addTemplate(owner, tpl, this);
},
// Updating template name should fail
function updateTemplateName(err, id) {
if ( err ) throw err;
tpl_id = id;
expected_failure = true;
tpl.name = 'second';
tmap.updTemplate(owner, tpl_id, tpl, this);
},
function updateTemplateAuth(err) {
if ( err && ! expected_failure) throw err;
expected_failure = false;
assert.ok(err);
tpl.name = 'first';
tpl.auth.method = 'token';
tmap.updTemplate(owner, tpl_id, tpl, this);
},
function updateTemplateWithInvalid(err) {
if ( err ) throw err;
tpl.version = '999.999.999';
expected_failure = true;
tmap.updTemplate(owner, tpl_id, tpl, this);
},
function updateUnexistentTemplate(err) {
if ( err && ! expected_failure) throw err;
expected_failure = false;
assert.ok(err);
assert.ok(err.message.match(/unsupported.*version/i), err);
tpl.version = '0.0.1';
expected_failure = true;
tmap.updTemplate(owner, 'unexistent', tpl, this);
},
function delTemplate(err) {
if ( err && ! expected_failure) throw err;
expected_failure = false;
assert.ok(err);
assert.ok(err.message.match(/cannot update name/i), err);
tmap.delTemplate(owner, tpl_id, this);
},
function finish(err) {
done(err);
}
);
});
test('instanciate templates', function() {
var tmap = new TemplateMaps(redis_pool, signed_maps);
assert.ok(tmap);
var tpl1 = {
version: '0.0.1',
name: 'acceptance1',
auth: { method: 'open' },
placeholders: {
fill: { type: "css_color", default: "red" },
color: { type: "css_color", default: "#a0fF9A" },
name: { type: "sql_literal", default: "test" },
zoom: { type: "number", default: "0" },
},
layergroup: {
version: '1.0.0',
global_cartocss_version: '2.0.2',
layers: [
{ options: {
sql: "select '<%= name %>' || id, g from t",
cartocss: '#layer { marker-fill:<%= fill %>; }'
} },
{ options: {
sql: "select fun('<%= name %>') g from x",
cartocss: '#layer { line-color:<%= color %>; marker-fill:<%= color %>; }'
} },
{ options: {
sql: "select g from x",
cartocss: '#layer[zoom=<%=zoom%>] { }'
} }
]
}
};
var inst = tmap.instance(tpl1, {});
var lyr = inst.layers[0].options;
assert.equal(lyr.sql, "select 'test' || id, g from t");
assert.equal(lyr.cartocss, '#layer { marker-fill:red; }');
lyr = inst.layers[1].options;
assert.equal(lyr.sql, "select fun('test') g from x");
assert.equal(lyr.cartocss, '#layer { line-color:#a0fF9A; marker-fill:#a0fF9A; }');
inst = tmap.instance(tpl1, {color:'yellow', name:"it's dangerous"});
lyr = inst.layers[0].options;
assert.equal(lyr.sql, "select 'it''s dangerous' || id, g from t");
assert.equal(lyr.cartocss, '#layer { marker-fill:red; }');
lyr = inst.layers[1].options;
assert.equal(lyr.sql, "select fun('it''s dangerous') g from x");
assert.equal(lyr.cartocss, '#layer { line-color:yellow; marker-fill:yellow; }');
// Invalid css_color
var err = null;
try { inst = tmap.instance(tpl1, {color:'##ff00ff'}); }
catch (e) { err = e; }
assert.ok(err);
assert.ok(err.message.match(/invalid css_color/i), err);
// Invalid css_color 2 (too few digits)
var err = null;
try { inst = tmap.instance(tpl1, {color:'#ff'}); }
catch (e) { err = e; }
assert.ok(err);
assert.ok(err.message.match(/invalid css_color/i), err);
// Invalid css_color 3 (too many digits)
var err = null;
try { inst = tmap.instance(tpl1, {color:'#1234567'}); }
catch (e) { err = e; }
assert.ok(err);
assert.ok(err.message.match(/invalid css_color/i), err);
// Invalid number
var err = null;
try { inst = tmap.instance(tpl1, {zoom:'#'}); }
catch (e) { err = e; }
assert.ok(err);
assert.ok(err.message.match(/invalid number/i), err);
// Invalid number 2
var err = null;
try { inst = tmap.instance(tpl1, {zoom:'23e'}); }
catch (e) { err = e; }
assert.ok(err);
assert.ok(err.message.match(/invalid number/i), err);
// Valid number
var err = null;
try { inst = tmap.instance(tpl1, {zoom:'-.23e10'}); }
catch (e) { err = e; }
assert.ok(!err);
});
});

View File

@@ -1,49 +0,0 @@
#!/bin/sh
verbose=no
tiler_url=http://dev.localhost.lan:8181/tiles/template
apikey=${CDB_APIKEY}
while test -n "$1"; do
if test "$1" = "-v"; then
verbose=yes
elif test "$1" = "-k"; then
shift
apikey="$1"
elif test "$1" = "-u"; then
shift
tiler_url="$1"
elif test -z "$cfg"; then
cfg="$1"
else
echo "Unused parameter $1" >&2
fi
shift
done
if test -z "$cfg"; then
echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_config>" >&2
echo "Default <tiler_url> is ${tiler_url}" >&2
echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
exit 1
fi
cmd="curl -skH Content-Type:application/json --data-binary @- ${tiler_url}?api_key=${apikey}"
if test x${verbose} = xyes; then
cmd="${cmd} -v"
fi
res=`cat ${cfg} | tr '\n' ' ' | ${cmd}`
if test $? -gt 0; then
echo "curl command failed: ${cmd}"
fi
if test x${verbose} = xyes; then
echo "${res}"
fi
# Successful response contains no space
echo "$res" | grep " " && { echo $res && exit 1; }
tok=`echo "$res" | sed 's/.*"template_id":"\([^"]*\)".*/\1/'`
echo $tok

View File

@@ -1,45 +0,0 @@
#!/bin/sh
verbose=no
tiler_url=http://dev.localhost.lan:8181/tiles/template
apikey=${CDB_APIKEY}
while test -n "$1"; do
if test "$1" = "-v"; then
verbose=yes
elif test "$1" = "-k"; then
shift
apikey="$1"
elif test "$1" = "-u"; then
shift
tiler_url="$1"
elif test -z "$tpl"; then
tpl="$1"
else
echo "Unused parameter $1" >&2
fi
shift
done
if test -z "$tpl"; then
echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_id>" >&2
echo "Default <tiler_url> is ${tiler_url}" >&2
echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
exit 1
fi
cmd="curl -X DELETE -skH Content-Type:application/json ${tiler_url}/${tpl}?api_key=${apikey}"
if test x${verbose} = xyes; then
cmd="${cmd} -v"
fi
res=`${cmd}`
if test $? -gt 0; then
echo "curl command failed: ${cmd}"
fi
if test x${verbose} = xyes; then
echo "${res}"
fi
tok=`echo "$res" | sed 's/.*"template_id":"\([^"]*\)".*/\1/'`
echo $tok

View File

@@ -1,53 +0,0 @@
#!/bin/sh
verbose=no
tiler_url=http://dev.localhost.lan:8181/tiles/template
apikey=${CDB_APIKEY}
while test -n "$1"; do
if test "$1" = "-v"; then
verbose=yes
elif test "$1" = "-k"; then
shift
apikey="$1"
elif test "$1" = "-u"; then
shift
tiler_url="$1"
elif test -z "$tpl"; then
tpl="$1"
elif test -z "$cfg"; then
cfg="$1"
else
echo "Unused parameter $1" >&2
fi
shift
done
if test -z "$tpl"; then
echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_id> [<template_params>]" >&2
echo "Default <tiler_url> is ${tiler_url}" >&2
echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
exit 1
fi
if test -z "$cfg"; then
cfg="/dev/null"
fi
tiler_url="${tiler_url}/${tpl}"
cmd="curl -X POST -skH Content-Type:application/json --data-binary @- ${tiler_url}?api_key=${apikey}"
if test x${verbose} = xyes; then
cmd="${cmd} -v"
fi
res=`cat ${cfg} | tr '\n' ' ' | ${cmd}`
if test $? -gt 0; then
echo "curl command failed: ${cmd}"
fi
if test x${verbose} = xyes; then
echo "${res}"
fi
tok=`echo "$res" | sed 's/.*"template_id":"\([^"]*\)".*/\1/'`
echo $tok

View File

@@ -1,54 +0,0 @@
#!/bin/sh
verbose=no
tiler_url=http://dev.localhost.lan:8181/tiles/template
apikey=${CDB_APIKEY}
while test -n "$1"; do
if test "$1" = "-k"; then
shift
apikey="$1"
elif test "$1" = "-u"; then
shift
tiler_url="$1"
elif test -z "$tpl"; then
tpl="$1"
else
echo "Unused parameter $1" >&2
fi
shift
done
if test -z "$tpl"; then
echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_config>" >&2
echo "Default <tiler_url> is ${tiler_url}" >&2
echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
exit 1
fi
basedir=$(cd $(dirname $0); cd ..; pwd)
export CDB_APIKEY=${apikey}
max=3000000
i=0
while test "$i" -le "$max"; do
tpln=`cat ${tpl} | sed "s/\"name\":\"\(.*\)\"/\"name\":\"\1${i}\"/"`
tpl_id=`echo ${tpln} | ${basedir}/create_template -u ${tiler_url} /dev/stdin`
if test $? -ne 0; then
echo $tpl_id >&2
break
fi
tpl_id=`echo ${tpln} | ${basedir}/update_template -u ${tiler_url} ${tpl_id} /dev/stdin`
if test $? -ne 0; then
echo $tpl_id >&2
break
fi
out=`${basedir}/delete_template -u ${tiler_url} ${tpl_id}`
if test $? -ne 0; then
echo $out >&2
break
fi
i=$((i+1))
if test `expr $i % 100` -eq 0; then
echo -n "."
fi
done

View File

@@ -2,84 +2,28 @@
# TODO: port to node, if you really need it
ENV='development';
BASEDIR=`cd $(dirname $0)/../; pwd`
REDIS_PORT=6379 # default port
if test -z "$1"; then
echo "Usage: $0 [--env <environment>] <username> [<tablename>|~<token>]" >&2
echo " environment defaults to 'development'"
exit 1
fi
username=""
token=""
while test -n "$1"; do
if test "$1" = "--env"; then
shift; ENV="$1"; shift
elif test -z "$username"; then
username="$1"; shift
elif test -z "$token"; then
token="$1"; shift
else
echo "Unused option $1" >&2
shift
fi
done
echo "Using environment '${ENV}'"
CONFIG="${BASEDIR}/config/environments/${ENV}.js"
REDIS_PORT=`node -e "console.log(require('${CONFIG}').redis.port)"`
if test $? -ne 0; then
echo "Usage: $0 <username> [<tablename>|~<token>]" >&2
exit 1
fi
username="$1"
token="$2"
dbname=`redis-cli -p ${REDIS_PORT} -n 5 hget "rails:users:${username}" "database_name"`
if test $? -ne 0; then
exit 1
fi
if test -z "${dbname}"; then
echo "Username ${username} unknown by redis on port ${REDIS_PORT} (try CARTODB/script/restore_redis?)" >&2
echo "Username ${username} unknown by redis (try CARTODB/script/restore_redis?)" >&2
exit 1
fi
echo "Database name for user ${username}: ${dbname}" # only if verbose?
if test -n "$token"; then
rec=`redis-cli get "map_style|${dbname}|${token}"`
if test -z "${rec}"; then
echo "${token}: no such map style known by redis on port ${REDIS_PORT}" >&2
exit 1
fi
#echo "${rec}"
escrec=`echo "${rec}" | sed -e 's/\\\\/\\\\\\\\/g'`
#echo "${escrec}"
node <<EOF
var x=JSON.parse('${escrec}');
console.log('style: ' + x.style);
console.log('version: ' + x.version);
global.environment = require('${CONFIG}');
var serverOptions = require('${BASEDIR}/lib/cartodb/server_options'); // _after_ setting global.environment
var grainstore = require('${BASEDIR}/node_modules/windshaft/node_modules/grainstore/lib/grainstore');
var mml_store = new grainstore.MMLStore(serverOptions.redis, serverOptions.grainstore);
var builderconfig = {dbname:'${dbname}'};
if ( '${token}'.match(/^~/) ) {
builderconfig.token = '${token}'.substring(1);
} else {
builderconfig.table = '${token}';
}
var mml_builder = mml_store.mml_builder(builderconfig,
function(err, payload) {
if ( err ) throw err;
mml_builder.toXML(function(err, xml) {
if ( err ) throw err;
console.log('- XML - ');
console.log(xml);
});
});
EOF
#echo "${rec}" | sed -e 's/\\n/\n/g' -e 's/\\//g'
redis-cli get "map_style|${dbname}|${token}" | sed -e 's/\\n/\n/g' -e 's/\\//g'
else
redis-cli keys "map_style|${dbname}|*"
fi

View File

@@ -1,47 +0,0 @@
#!/bin/sh
verbose=no
tiler_url=http://dev.localhost.lan:8181/tiles/template
apikey=${CDB_APIKEY}
while test -n "$1"; do
if test "$1" = "-v"; then
verbose=yes
elif test "$1" = "-k"; then
shift
apikey="$1"
elif test "$1" = "-u"; then
shift
tiler_url="$1"
elif test -z "$tpl"; then
tpl="$1"
elif test -z "$cfg"; then
cfg="$1"
else
echo "Unused parameter $1" >&2
fi
shift
done
if test -z "$cfg"; then
echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_id> <template_config>" >&2
echo "Default <tiler_url> is ${tiler_url}" >&2
echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
exit 1
fi
cmd="curl -X PUT -skH Content-Type:application/json --data-binary @- ${tiler_url}/${tpl}?api_key=${apikey}"
if test x${verbose} = xyes; then
cmd="${cmd} -v"
fi
res=`cat ${cfg} | tr '\n' ' ' | ${cmd}`
if test $? -gt 0; then
echo "curl command failed: ${cmd}"
fi
if test x${verbose} = xyes; then
echo "${res}"
fi
tok=`echo "$res" | sed 's/.*"template_id":"\([^"]*\)".*/\1/'`
echo $tok