Release 1.5.2

It sets it to the current grainstore default, so nothing changes.

.travis.yml

@@ -11,7 +11,6 @@ env:
 language: node_js
 node_js:
   - "0.8"
-  - "0.10"
 
 notifications:
   irc:

HOWTO_RELEASE

@@ -4,10 +4,8 @@
 4. Run npm install
 5. Test (make check or npm test), fix if broken before proceeding
 6. Run npm shrinkwrap
-7. Set "from" in npm-shrinkwrap.json for known packages
-   (windshaft, node-varnish, grainstore...)
-8. Commit package.json, npm-shrinwrap.json, NEWS
-9. git tag -a Major.Minor.Patch # use NEWS section as content
-10. Announce
-11. Stub NEWS/package for next version
+7. Commit package.json, npm-shrinwrap.json, NEWS
+8. Tag Major.Minor.Patch
+9. Announce
+10. Stub NEWS/package for next version
 

Makefile

@@ -17,8 +17,10 @@ config/environments/test.js: config.status--test
 
 check-local: config/environments/test.js
 	./run_tests.sh ${RUNTESTFLAGS} \
-    test/unit/cartodb/*.js \
-    test/acceptance/*.js 
+    test/unit/cartodb/req2params.test.js \
+    test/acceptance/cache_validator.js \
+    test/acceptance/server.js \
+    test/acceptance/multilayer.js
 
 check-submodules:
 	PATH="$$PATH:$(srcdir)/node_modules/.bin/"; \

NEWS.md

@@ -1,126 +1,3 @@
-1.8.2 -- 2014-02-25
--------------------
-
-Enhancements:
-
- * Allow using ":host" as part of statsd.prefix (#153)
- * Expand "addCacheChannel" stats
- * Allow using GET with sql-api for queries shorter than configured len (#155)
-   [ new sqlapi.max_get_sql_length directive, defaults to 2048 ]
- * Do not log an error for a legit request requiring no X-Cache-Channel
-
-Bug fixes:
-
- * Fix munin plugin after log format changes (#154)
-
-1.8.1 -- 2014-02-19
--------------------
-
-Enhancements:
-
- * Use log4js logger (#138)
-
-Bug fixes:
-
- * Always generate X-Cache-Channel for token-based tile responses (#152)
-
-1.8.0 -- 2014-02-18
--------------------
-
-Enhancements:
-
- * Add script to flush caches (#140)
- * Add script to list templates
- * Add statsd support (#139)
- * Add support for specifying a varnish password
- * Avoid sending multiple varnish invalidation at once (#135)
- * Tested with node-0.10 (#141)
- * Use single redis pooler for torque and grainstore
- * Reduce cost of garbage collection for localized resources
- * Allow limiting number of templates for each user (#136)
- * Allow configuring TTL of mapConfigs via "mapConfigTTL"
-
-1.7.1 -- 2014-02-11
--------------------
-
-Enhancements:
-
- * Disable debug logging unless "debug" config param evaluates to true (#137)
- * Require windshaft 0.17.2 for further reducing log noise (#137)
-
-1.7.0 -- 2014-02-11
--------------------
-
-New features:
-
- * Add support for torque tiles (#112)
- * Add attributes service (#118)
- * Implement Unified Map API (#126)
- * Make endpoints configurable (#127)
-
-Enhancements:
-
- * Allow specifying fixed sqlapi host address (#117)
- * Include template hash in template instance response, to keep caches
-   of different instances separated (#105)
-
-Bug fixes:
-
- * Allow space padding in template variables usage (#129)
- * Allow passing numbers as values for numeric template variables (#130)
-
-
-1.6.3 -- 2014-01-30
--------------------
-
-Bug fixes: 
-
-* layergroup accept both map_key and api_key (#91)
-* Fix public instanciation of signed template accessing private data (#114)
-* Fix show_style in presence of complex styles
-* Fix use of maxzoom in layergroup config (via windshaft-0.15.1)
-
-Enhancements:
-
-* Add support for instanciating a template map with JSONP (#116)
-* Stop processing XML on renderer creation, not needed anymore since 1.6.1
-  introduced on-demand XML generation.
-
-1.6.2 -- 2014-01-23
--------------------
-
-Bug fixes:
-
-* Fix support for long (>64k chars) queries in layergroup creation (#111)
-
-Enhancements:
-
-* Enhance tools/show_style to accept an environment parameter and
-  print XML style now it is not in redis anymore (#110)
-* Support CORS in template instanciation endpoint (#113)
-
-1.6.1 -- 2014-01-15
--------------------
-
-Bug fixes:
-
-* Drop cache headers from error responses (#107)
-* Localize external CartoCSS resources at renderer creation time (#108)
-
-1.6.0 -- 2014-01-10
--------------------
-
-New features:
-
- * Add 'user_from_host' directive to generalize username extraction (#100)
- * Implement signed template maps (#98)
-
-
-Other changes:
-
- * Update cartodb-redis dependency to "~0.3.0"
- * Update redis-server dependency to "2.4.0+"
-
 1.5.2 -- 2013-12-05
 -------------------
 

README.md

@@ -1,8 +1,7 @@
 Windshaft-CartoDB
 ==================
 
-[![Build Status](http://travis-ci.org/CartoDB/Windshaft-cartodb.png)]
-(http://travis-ci.org/CartoDB/Windshaft-cartodb)
+[![Build Status](https://travis-ci.org/CartoDB/Windshaft-cartodb.png)](http://travis-ci.org/CartoDB/Windshaft-cartodb)
 
 This is the CartoDB map tiler. It extends Windshaft with some extra
 functionality and custom filters for authentication
@@ -12,10 +11,8 @@ functionality and custom filters for authentication
 * gets the default geometry type from the cartodb redis store
 * allows tiles to be styled individually
 * provides a link to varnish high speed cache
-* provides a ``infowindow`` endpoint for windshaft (DEPRECATED)
-* provides a ``map_metadata`` endpoint for windshaft (DEPRECATED)
-* provides signed template maps API
-  (http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps)
+* provides a infowindow endpoint for windshaft
+* provides a ``map_metadata`` endpoint for windshaft
 
 Requirements
 ------------
@@ -24,16 +21,13 @@ Requirements
  - node-0.8.x+
  - PostgreSQL-8.3+
  - PostGIS-1.5.0+
- - Redis 2.4.0+ (http://www.redis.io)
+ - Redis 2.2.0+ (http://www.redis.io)
  - Mapnik 2.0 or 2.1 
 
  [for cache control]
  - CartoDB-SQL-API 1.0.0+
  - CartoDB 0.9.5+ (for ``CDB_QueryTables``)
- - Varnish (http://www.varnish-cache.org)
-
- [for running the testsuite]
- - Imagemagick (http://www.imagemagick.org)
+ - Varnish (https://www.varnish-cache.org)
 
 Configure
 ---------

app.js

@@ -18,25 +18,15 @@ if (ENV != 'development' && ENV != 'production' && ENV != 'staging' ){
 
 var _ = require('underscore')
     , Step       = require('step')
-    ;
+	, CartodbWindshaft = require('./lib/cartodb/cartodb_windshaft');
+
 // set environment specific variables
 global.settings     = require(__dirname + '/config/settings');
 global.environment  = require(__dirname + '/config/environments/' + ENV);
 _.extend(global.settings, global.environment);
 
-global.log4js = require('log4js')
-log4js.configure({
-  appenders: [
-    { type: "console", layout: { type:'basic' } }
-  ],
-  replaceConsole:true
-});
-
-
-// Include cartodb_windshaft only _after_ the "global" variable is set
-// See https://github.com/Vizzuality/Windshaft-cartodb/issues/28
-var CartodbWindshaft = require('./lib/cartodb/cartodb_windshaft');
-var serverOptions = require('./lib/cartodb/server_options')();
+var Windshaft = require('windshaft');
+var serverOptions = require('./lib/cartodb/server_options');
 
 ws = CartodbWindshaft(serverOptions);
 
@@ -61,4 +51,3 @@ process.on('SIGUSR1', function() {
 process.on('SIGUSR2', function() {
   ws.dumpCacheStats();
 });
-

config/environments/development.js.example

@@ -2,40 +2,14 @@ var config = {
      environment: 'development'
     ,port: 8181
     ,host: '127.0.0.1'
-    // Regular expression pattern to extract username
-    // from hostname. Must have a single grabbing block.
-    ,user_from_host: '^(.*)\\.localhost'
-
-    // Base URLs for the APIs
-    //
-    // See http://github.com/CartoDB/Windshaft-cartodb/wiki/Unified-Map-API
-    // 
-    // Base url for the Templated Maps API
-    // "/api/v1/map/named" is the new API,
-    // "/tiles/template" is for compatibility with versions up to 1.6.x
-    ,base_url_templated: '(?:/api/v1/map/named|/tiles/template)'
-    // Base url for the Detached Maps API
-    // "maps" is the the new API,
-    // "tiles/layergroup" is for compatibility with versions up to 1.6.x
-    ,base_url_detached: '(?:/api/v1/map|/tiles/layergroup)'
-    // Base url for the Inline Maps and Table Maps API
-    ,base_url_legacy: '/tiles/:table'
-
     // Maximum number of connections for one process
     // 128 is a good value with a limit of 1024 open file descriptors
     ,maxConnections:128
-    // Maximum number of templates per user. Unlimited by default.
-    ,maxUserTemplates:1024
-    // Seconds since "last creation" before a detached
-    // or template instance map expires. Or: how long do you want
-    // to be able to navigate the map without a reload ?
-    // Defaults to 7200 (2 hours)
-    ,mapConfigTTL: 7200
     // idle socket timeout, in miliseconds 
     ,socket_timeout: 600000
     ,enable_cors: true
     ,cache_enabled: false
-    ,log_format: ':req[X-Real-IP] :method :req[Host]:url :status :response-time ms -> :res[Content-Type] (:res[X-Tiler-Profiler])'
+    ,log_format: '[:date] :req[X-Real-IP] :method :req[Host]:url :status :response-time ms -> :res[Content-Type] (:res[X-Tiler-Profiler])'
     // Templated database username for authorized user
     // Supported labels: 'user_id' (read from redis)
     ,postgres_auth_user: 'development_cartodb_user_<%= user_id %>'
@@ -61,12 +35,6 @@ var config = {
         max_size: 500
     }
     ,mapnik_version: undefined
-    ,statsd: {
-        host: 'localhost',
-        port: 8125,
-        prefix: 'dev.'
-        // support all allowed node-statsd options
-    }
     ,renderer: {
       // Milliseconds since last access before renderer cache item expires
       cache_ttl: 60000,
@@ -93,25 +61,13 @@ var config = {
     }
     ,sqlapi: {
         protocol: 'http',
-        // If "host" is given, it will be used
-        // to connect to the SQL-API without a
-        // DNS lookup
-        host: '127.0.0.1',
-        port: 8080,
-        // The "domain" part will be appended to 
-        // the cartodb username and passed to
-        // SQL-API requests in the Host HTTP header
         domain: 'localhost.lan',
-        version: 'v1',
-        // Maximum lenght of SQL query for GET
-        // requests. Longer queries will be sent
-        // using POST. Defaults to 2048
-        max_get_sql_length: 2048
+        port: 8080,
+        version: 'v1'
     }
     ,varnish: {
         host: 'localhost',
         port: 6082,
-        secret: 'xxx',
         ttl: 86400
     }
     // If useProfiler is true every response will be served with an

config/environments/production.js.example

@@ -2,40 +2,14 @@ var config = {
     environment: 'production'
     ,port: 8181
     ,host: '127.0.0.1'
-    // Regular expression pattern to extract username
-    // from hostname. Must have a single grabbing block.
-    ,user_from_host: '^(.*)\\.cartodb\\.com$'
-
-    // Base URLs for the APIs
-    //
-    // See http://github.com/CartoDB/Windshaft-cartodb/wiki/Unified-Map-API
-    // 
-    // Base url for the Templated Maps API
-    // "/api/v1/map/named" is the new API,
-    // "/tiles/template" is for compatibility with versions up to 1.6.x
-    ,base_url_templated: '(?:/api/v1/map/named|/tiles/template)'
-    // Base url for the Detached Maps API
-    // "maps" is the the new API,
-    // "tiles/layergroup" is for compatibility with versions up to 1.6.x
-    ,base_url_detached: '(?:/api/v1/map|/tiles/layergroup)'
-    // Base url for the Inline Maps and Table Maps API
-    ,base_url_legacy: '/tiles/:table'
-
     // Maximum number of connections for one process
     // 128 is a good value with a limit of 1024 open file descriptors
     ,maxConnections:128
-    // Maximum number of templates per user. Unlimited by default.
-    ,maxUserTemplates:1024
-    // Seconds since "last creation" before a detached
-    // or template instance map expires. Or: how long do you want
-    // to be able to navigate the map without a reload ?
-    // Defaults to 7200 (2 hours)
-    ,mapConfigTTL: 7200
     // idle socket timeout, in miliseconds
     ,socket_timeout: 600000 
     ,enable_cors: true
     ,cache_enabled: true
-    ,log_format: ':req[X-Real-IP] :method :req[Host]:url :status :response-time ms -> :res[Content-Type] (:res[X-Tiler-Profiler])'
+    ,log_format: '[:date] :req[X-Real-IP] :method :req[Host]:url :status :response-time ms -> :res[Content-Type] (:res[X-Tiler-Profiler])'
     // Templated database username for authorized user
     // Supported labels: 'user_id' (read from redis)
     ,postgres_auth_user: 'cartodb_user_<%= user_id %>'
@@ -55,12 +29,6 @@ var config = {
         max_size: 500
     }
     ,mapnik_version: undefined
-    ,statsd: {
-        host: 'localhost',
-        port: 8125,
-        prefix: ':host.', // could be hostname, better not containing dots
-        // support all allowed node-statsd options
-    }
     ,renderer: {
       // Milliseconds since last access before renderer cache item expires
       cache_ttl: 60000,
@@ -87,25 +55,13 @@ var config = {
     }
     ,sqlapi: {
         protocol: 'https',
-        // If "host" is given, it will be used
-        // to connect to the SQL-API without a
-        // DNS lookup
-        //host: '127.0.0.1',
-        port: 8080,
-        // The "domain" part will be appended to 
-        // the cartodb username and passed to
-        // SQL-API requests in the Host HTTP header
         domain: 'cartodb.com',
-        version: 'v2',
-        // Maximum lenght of SQL query for GET
-        // requests. Longer queries will be sent
-        // using POST. Defaults to 2048
-        max_get_sql_length: 2048
+        port: 8080,
+        version: 'v2'
     }
     ,varnish: {
         host: 'localhost',
         port: 6082,
-        secret: 'xxx',
         ttl: 86400
     }
     // If useProfiler is true every response will be served with an

config/environments/staging.js.example

@@ -2,40 +2,14 @@ var config = {
     environment: 'production'
     ,port: 8181
     ,host: '127.0.0.1'
-    // Regular expression pattern to extract username
-    // from hostname. Must have a single grabbing block.
-    ,user_from_host: '^(.*)\\.cartodb\\.com$'
-
-    // Base URLs for the APIs
-    //
-    // See http://github.com/CartoDB/Windshaft-cartodb/wiki/Unified-Map-API
-    // 
-    // Base url for the Templated Maps API
-    // "/api/v1/maps/named" is the new API,
-    // "/tiles/template" is for compatibility with versions up to 1.6.x
-    ,base_url_templated: '(?:/api/v1/maps/named|/tiles/template)'
-    // Base url for the Detached Maps API
-    // "/api/v1/maps" is the the new API,
-    // "/tiles/layergroup" is for compatibility with versions up to 1.6.x
-    ,base_url_detached: '(?:/api/v1/maps|/tiles/layergroup)'
-    // Base url for the Inline Maps and Table Maps API
-    ,base_url_legacy: '/tiles/:table'
-
     // Maximum number of connections for one process
     // 128 is a good value with a limit of 1024 open file descriptors
     ,maxConnections:128
-    // Maximum number of templates per user. Unlimited by default.
-    ,maxUserTemplates:1024
-    // Seconds since "last creation" before a detached
-    // or template instance map expires. Or: how long do you want
-    // to be able to navigate the map without a reload ?
-    // Defaults to 7200 (2 hours)
-    ,mapConfigTTL: 7200
     // idle socket timeout, in miliseconds
     ,socket_timeout: 600000 
     ,enable_cors: true
     ,cache_enabled: true
-    ,log_format: ':req[X-Real-IP] :method :req[Host]:url :status :response-time ms (:res[X-Tiler-Profiler]) -> :res[Content-Type]'
+    ,log_format: '[:date] :req[X-Real-IP] :method :req[Host]:url :status :response-time ms (:res[X-Tiler-Profiler]) -> :res[Content-Type]'
     // Templated database username for authorized user
     // Supported labels: 'user_id' (read from redis)
     ,postgres_auth_user: 'cartodb_staging_user_<%= user_id %>'
@@ -55,12 +29,6 @@ var config = {
         max_size: 500
     }
     ,mapnik_version: undefined
-    ,statsd: {
-        host: 'localhost',
-        port: 8125,
-        prefix: 'stage.:host.'
-        // support all allowed node-statsd options
-    }
     ,renderer: {
       // Milliseconds since last access before renderer cache item expires
       cache_ttl: 60000,
@@ -87,25 +55,13 @@ var config = {
     }
     ,sqlapi: {
         protocol: 'https',
-        // If "host" is given, it will be used
-        // to connect to the SQL-API without a
-        // DNS lookup
-        //host: '127.0.0.1',
-        port: 8080,
-        // The "domain" part will be appended to 
-        // the cartodb username and passed to
-        // SQL-API requests in the Host HTTP header
         domain: 'cartodb.com',
-        version: 'v2',
-        // Maximum lenght of SQL query for GET
-        // requests. Longer queries will be sent
-        // using POST. Defaults to 2048
-        max_get_sql_length: 2048
+        port: 8080,
+        version: 'v2'
     }
     ,varnish: {
         host: 'localhost',
         port: 6082,
-        secret: 'xxx',
         ttl: 86400
     }
     // If useProfiler is true every response will be served with an

config/environments/test.js.example

@@ -2,35 +2,9 @@ var config = {
      environment: 'test'
     ,port: 8888
     ,host: '127.0.0.1'
-    // Regular expression pattern to extract username
-    // from hostname. Must have a single grabbing block.
-    ,user_from_host: '(.*)'
-
-    // Base URLs for the APIs
-    //
-    // See https://github.com/CartoDB/Windshaft-cartodb/wiki/Unified-Map-API
-    // 
-    // Base url for the Templated Maps API
-    // "/api/v1/map/named" is the new API,
-    // "/tiles/template" is for compatibility with versions up to 1.6.x
-    ,base_url_templated: '(?:/api/v1/map/named|/tiles/template)'
-    // Base url for the Detached Maps API
-    // "maps" is the the new API,
-    // "tiles/layergroup" is for compatibility with versions up to 1.6.x
-    ,base_url_detached: '(?:/api/v1/map|/tiles/layergroup)'
-    // Base url for the Inline Maps and Table Maps API
-    ,base_url_legacy: '/tiles/:table'
-
     // Maximum number of connections for one process
     // 128 is a good value with a limit of 1024 open file descriptors
     ,maxConnections:128
-    // Maximum number of templates per user. Unlimited by default.
-    ,maxUserTemplates:1024
-    // Seconds since "last creation" before a detached
-    // or template instance map expires. Or: how long do you want
-    // to be able to navigate the map without a reload ?
-    // Defaults to 7200 (2 hours)
-    ,mapConfigTTL: 7200
     // idle socket timeout, in miliseconds
     ,socket_timeout: 600000 
     ,enable_cors: true
@@ -38,10 +12,10 @@ var config = {
     ,log_format: '[:date] :req[X-Real-IP] :method :req[Host]:url :status :response-time ms -> :res[Content-Type] (:res[X-Tiler-Profiler])'
     // Templated database username for authorized user
     // Supported labels: 'user_id' (read from redis)
-    ,postgres_auth_user: 'test_windshaft_cartodb_user_<%= user_id %>'
+    ,postgres_auth_user: 'test_cartodb_user_<%= user_id %>'
     // Templated database password for authorized user
     // Supported labels: 'user_id', 'user_password' (both read from redis)
-    ,postgres_auth_pass: 'test_windshaft_cartodb_user_<%= user_id %>_pass'
+    ,postgres_auth_pass: 'test_cartodb_user_<%= user_id %>_pass'
     ,postgres: {
         // Parameters to pass to datasource plugin of mapnik
         // See http://github.com/mapnik/mapnik/wiki/PostGIS
@@ -54,13 +28,7 @@ var config = {
         simplify_geometries: true,
         max_size: 500
     }
-    ,mapnik_version: ''
-    ,statsd: {
-        host: 'localhost',
-        port: 8125,
-        prefix: 'test.:host.'
-        // support all allowed node-statsd options
-    }
+    ,mapnik_version: '2.0.2'
     ,renderer: {
       // Milliseconds since last access before renderer cache item expires
       cache_ttl: 60000,
@@ -87,33 +55,21 @@ var config = {
     }
     ,sqlapi: {
         protocol: 'http',
-        // If "host" is given, it will be used
-        // to connect to the SQL-API without a
-        // DNS lookup
-        host: '127.0.0.1',
-        port: 1080,
-        // The "domain" part will be appended to 
-        // the cartodb username and passed to
-        // SQL-API requests in the Host HTTP header
-        domain: 'donot_look_this_up',
+        domain: '',
         // This port will be used by "make check" for testing purposes
         // It must be available
-        version: 'v1',
-        // Maximum lenght of SQL query for GET
-        // requests. Longer queries will be sent
-        // using POST. Defaults to 2048
-        max_get_sql_length: 2048
+        port: 1080,
+        version: 'v1'
     }
     ,varnish: {
         host: '',
         port: null,
-        secret: 'xxx',
         ttl: 86400
     }
     // If useProfiler is true every response will be served with an
     // X-Tiler-Profile header containing elapsed timing for various 
     // steps taken for producing the response.
-    ,useProfiler:true
+    ,useProfiler:false
 };
 
 module.exports = config;

lib/cartodb/cache_validator.js

@@ -2,21 +2,16 @@ var _             = require('underscore'),
     Varnish       = require('node-varnish'),
     varnish_queue = null;
 
-function init(host, port, secret) {
-    varnish_queue = new Varnish.VarnishQueue(host, port, secret);
-    varnish_queue.on('error', function(e) {
-        console.log("[CACHE VALIDATOR ERROR] " + e);
-    });
+function init(host, port) {
+    varnish_queue = new Varnish.VarnishQueue(host, port);
 }
 
 function invalidate_db(dbname, table) {
-    var cmd = 'purge obj.http.X-Cache-Channel ~ "^' + dbname +
-              ':(.*'+ table +'.*)|(table)$"';
     try{
-        varnish_queue.run_cmd(cmd, false);
+        varnish_queue.run_cmd('purge obj.http.X-Cache-Channel ~ "^' + dbname + ':(.*'+ table +'.*)|(table)$"');
+        console.log('[SUCCESS FLUSHING CACHE]');
     } catch (e) {
-        console.log("[CACHE VALIDATOR ERROR] could not queue command " +
-                    cmd + " -- " + e);
+        console.log("[ERROR FLUSHING CACHE] Is enable_cache set to true? Failed for: " + 'purge obj.http.X-Cache-Channel ~ "^' + dbname + ':(.*'+ table +'.*)|(table)$"');
     }
 }
 

lib/cartodb/cartodb_windshaft.js

@@ -2,30 +2,13 @@
 var _ = require('underscore')
     , Step       = require('step')
     , Windshaft = require('windshaft')
-    , redisPool = new require('redis-mpool')(global.environment.redis)
-    // TODO: instanciate cartoData with redisPool
-    , cartoData  = require('cartodb-redis')(global.environment.redis)
-    , SignedMaps = require('./signed_maps.js')
-    , TemplateMaps = require('./template_maps.js')
-    , Cache = require('./cache_validator')
-    , os = require('os')
-;
+    , Cache = require('./cache_validator');
 
 var CartodbWindshaft = function(serverOptions) {
-   var debug = global.environment.debug;
-
-    // Perform keyword substitution in statsd
-    // See https://github.com/CartoDB/Windshaft-cartodb/issues/153
-    if ( global.environment.statsd ) {
-      if ( global.environment.statsd.prefix ) {
-        var host_token = os.hostname().split('.').reverse().join('.');
-        global.environment.statsd.prefix = global.environment.statsd.prefix.replace(/:host/, host_token);
-      }
-    }
 
     if(serverOptions.cache_enabled) {
         console.log("cache invalidation enabled, varnish on ", serverOptions.varnish_host, ' ', serverOptions.varnish_port);
-        Cache.init(serverOptions.varnish_host, serverOptions.varnish_port, serverOptions.varnish_secret);
+        Cache.init(serverOptions.varnish_host, serverOptions.varnish_port);
         serverOptions.afterStateChange = function(req, data, callback) {
             Cache.invalidate_db(req.params.dbname, req.params.table);
             callback(null, data);
@@ -34,18 +17,12 @@ var CartodbWindshaft = function(serverOptions) {
 
     serverOptions.beforeStateChange = function(req, callback) {
         var err = null;
-        if ( ! req.params.hasOwnProperty('_authorizedByApiKey') ) {
+        if ( ! req.params.hasOwnProperty('dbuser') ) {
           err = new Error("map state cannot be changed by unauthenticated request!");
         }
         callback(err, req);
     }
 
-    serverOptions.signedMaps = new SignedMaps(redisPool);
-    var templateMapsOpts = {
-      max_user_templates: global.environment.maxUserTemplates
-    }
-    var templateMaps = new TemplateMaps(redisPool, serverOptions.signedMaps, templateMapsOpts);
-
     // boot
     var ws = new Windshaft.Server(serverOptions);
 
@@ -57,42 +34,6 @@ var CartodbWindshaft = function(serverOptions) {
       return version;
     }
 
-    var ws_sendResponse = ws.sendResponse;
-    ws.sendResponse = function(res, args) {
-      var that = this;
-      var thatArgs = arguments;
-      var statusCode;
-      if ( args.length > 2 ) statusCode = args[2];
-      else {
-        statusCode = args[1] || 200;
-      }
-      var req = res.req;
-      Step (
-        function addCacheChannel() {
-          if ( ! req ) {
-            // having no associated request can happen when
-            // using fake response objects for testing layergroup
-            // creation
-            return false;
-          }
-          if ( ! req.params ) {
-            // service requests (/version, /) 
-            // have no need for an X-Cache-Channel
-            return false;
-          }
-          if ( statusCode != 200 ) {
-            // We do not want to cache
-            // unsuccessful responses
-            return false;
-          }
-          serverOptions.addCacheChannel(that, req, this);
-        },
-        function sendResponse(err, added) {
-          ws_sendResponse.apply(that, thatArgs);
-        }
-      );
-    };
-
     /**
      * Helper to allow access to the layer to be used in the maps infowindow popup.
      */
@@ -104,10 +45,10 @@ var CartodbWindshaft = function(serverOptions) {
             },
             function(err, data){
                 if (err){
-                    ws.sendError(res, {error: err.message}, 500, 'GET INFOWINDOW', err);
-                    //ws.sendResponse(res, [{error: err.message}, 500]);
+                    ws.sendError(res, {error: err.message}, 500, 'GET INFOWINDOW');
+                    //res.send({error: err.message}, 500);
                 } else {
-                    ws.sendResponse(res, [{infowindow: data}, 200]);
+                    res.send({infowindow: data}, 200);
                 }
             }
         );
@@ -125,10 +66,10 @@ var CartodbWindshaft = function(serverOptions) {
             },
             function(err, data){
                 if (err){
-                    ws.sendError(res, {error: err.message}, 500, 'GET MAP_METADATA', err);
-                    //ws.sendResponse(res, [err.message, 500]);
+                    ws.sendError(res, {error: err.message}, 500, 'GET MAP_METADATA');
+                    //res.send(err.message, 500);
                 } else {
-                    ws.sendResponse(res, [{map_metadata: data}, 200]);
+                    res.send({map_metadata: data}, 200);
                 }
             }
         );
@@ -146,476 +87,14 @@ var CartodbWindshaft = function(serverOptions) {
             },
             function sendResponse(err, data){
                 if (err){
-                    ws.sendError(res, {error: err.message}, 500, 'DELETE CACHE', err);
-                    //ws.sendResponse(res, [500]);
+                    ws.sendError(res, {error: err.message}, 500, 'DELETE CACHE');
+                    //res.send(500);
                 } else {
-                    ws.sendResponse(res, [{status: 'ok'}, 200]);
+                    res.send({status: 'ok'}, 200);
                 }
             }
         );
     });
-
-    // ---- Template maps interface starts @{
-
-    ws.userByReq = function(req) {
-        return serverOptions.userByReq(req);
-    }
-
-    // This is for Templated maps
-    //
-    // "named" is the official, "template" is for backward compatibility up to 1.6.x
-    //
-    var template_baseurl = global.environment.base_url_templated || '(?:/maps/named|/tiles/template)';
-
-    // Add a template
-    ws.post(template_baseurl, function(req, res) {
-      ws.doCORS(res);
-      var that = this;
-      var response = {};
-      var cdbuser = ws.userByReq(req);
-      Step(
-        function checkPerms(){
-            serverOptions.authorizedByAPIKey(req, this);
-        },
-        function addTemplate(err, authenticated) {
-          if ( err ) throw err;
-          if (authenticated !== 1) {
-            err = new Error("Only authenticated user can create templated maps");
-            err.http_status = 401;
-            throw err;
-          }
-          var next = this;
-          if ( ! req.headers['content-type'] || req.headers['content-type'].split(';')[0] != 'application/json' )
-            throw new Error('template POST data must be of type application/json');
-          var cfg = req.body;
-          templateMaps.addTemplate(cdbuser, cfg, this);
-        },
-        function prepareResponse(err, tpl_id){
-          if ( err ) throw err;
-          // NOTE: might omit "cdbuser" if == dbowner ...
-          return { template_id: cdbuser + '@' + tpl_id };
-        },
-        function finish(err, response){
-            if ( req.profiler ) {
-              var report = req.profiler.toString();
-              res.header('X-Tiler-Profiler', report);
-            }
-            if (err){
-                response = { error: ''+err };
-                var statusCode = 400;
-                if ( ! _.isUndefined(err.http_status) ) {
-                  statusCode = err.http_status;
-                }
-                ws.sendError(res, response, statusCode, 'POST TEMPLATE', err);
-            } else {
-              ws.sendResponse(res, [response, 200]);
-            }
-        }
-      );
-    });
-
-    // Update a template
-    ws.put(template_baseurl + '/:template_id', function(req, res) {
-      ws.doCORS(res);
-      var that = this;
-      var response = {};
-      var cdbuser = ws.userByReq(req);
-      var template;
-      var tpl_id;
-      Step(
-        function checkPerms(){
-            serverOptions.authorizedByAPIKey(req, this);
-        },
-        function updateTemplate(err, authenticated) {
-          if ( err ) throw err;
-          if (authenticated !== 1) {
-            err = new Error("Only authenticated user can list templated maps");
-            err.http_status = 401;
-            throw err;
-          }
-          if ( ! req.headers['content-type'] || req.headers['content-type'].split(';')[0] != 'application/json' )
-            throw new Error('template PUT data must be of type application/json');
-          template = req.body;
-          tpl_id = req.params.template_id.split('@');
-          if ( tpl_id.length > 1 ) {
-            if ( tpl_id[0] != cdbuser ) {
-              err = new Error("Invalid template id '"
-                + req.params.template_id + "' for user '" + cdbuser + "'");
-              err.http_status = 404;
-              throw err;
-            }
-            tpl_id = tpl_id[1];
-          }
-          templateMaps.updTemplate(cdbuser, tpl_id, template, this);
-        },
-        function prepareResponse(err){
-          if ( err ) throw err;
-          return { template_id: cdbuser + '@' + tpl_id };
-        },
-        function finish(err, response){
-            if ( req.profiler ) {
-              var report = req.profiler.toString();
-              res.header('X-Tiler-Profiler', report);
-            }
-            if (err){
-                var statusCode = 400;
-                response = { error: ''+err };
-                if ( ! _.isUndefined(err.http_status) ) {
-                  statusCode = err.http_status;
-                }
-                ws.sendError(res, response, statusCode, 'PUT TEMPLATE', err);
-            } else {
-              ws.sendResponse(res, [response, 200]);
-            }
-        }
-      );
-    });
-
-    // Get a specific template
-    ws.get(template_baseurl + '/:template_id', function(req, res) {
-      ws.doCORS(res);
-      var that = this;
-      var response = {};
-      var cdbuser = ws.userByReq(req);
-      var template;
-      var tpl_id;
-      Step(
-        function checkPerms(){
-            serverOptions.authorizedByAPIKey(req, this);
-        },
-        function updateTemplate(err, authenticated) {
-          if ( err ) throw err;
-          if (authenticated !== 1) {
-            err = new Error("Only authenticated users can get template maps");
-            err.http_status = 401;
-            throw err;
-          }
-          tpl_id = req.params.template_id.split('@');
-          if ( tpl_id.length > 1 ) {
-            if ( tpl_id[0] != cdbuser ) {
-              var err = new Error("Cannot get template id '"
-                + req.params.template_id + "' for user '" + cdbuser + "'");
-              err.http_status = 404;
-              throw err;
-            }
-            tpl_id = tpl_id[1];
-          }
-          templateMaps.getTemplate(cdbuser, tpl_id, this);
-        },
-        function prepareResponse(err, tpl_val){
-          if ( err ) throw err;
-          if ( ! tpl_val ) {
-            err = new Error("Cannot find template '" + tpl_id + "' of user '" + cdbuser + "'");
-            err.http_status = 404;
-            throw err;
-          }
-          // auth_id was added by ourselves,
-          // so we remove it before returning to the user
-          delete tpl_val.auth_id;
-          return { template: tpl_val };
-        },
-        function finish(err, response){
-            if (err){
-                var statusCode = 400;
-                response = { error: ''+err };
-                if ( ! _.isUndefined(err.http_status) ) {
-                  statusCode = err.http_status;
-                }
-                ws.sendError(res, response, statusCode, 'GET TEMPLATE', err);
-            } else {
-              ws.sendResponse(res, [response, 200]);
-            }
-        }
-      );
-    });
-
-    // Delete a specific template
-    ws.delete(template_baseurl + '/:template_id', function(req, res) {
-      ws.doCORS(res);
-      var that = this;
-      var response = {};
-      var cdbuser = ws.userByReq(req);
-      var template;
-      var tpl_id;
-      Step(
-        function checkPerms(){
-            serverOptions.authorizedByAPIKey(req, this);
-        },
-        function updateTemplate(err, authenticated) {
-          if ( err ) throw err;
-          if (authenticated !== 1) {
-            err = new Error("Only authenticated users can delete template maps");
-            err.http_status = 401;
-            throw err;
-          }
-          tpl_id = req.params.template_id.split('@');
-          if ( tpl_id.length > 1 ) {
-            if ( tpl_id[0] != cdbuser ) {
-              var err = new Error("Cannot find template id '"
-                + req.params.template_id + "' for user '" + cdbuser + "'");
-              err.http_status = 404;
-              throw err;
-            }
-            tpl_id = tpl_id[1];
-          }
-          templateMaps.delTemplate(cdbuser, tpl_id, this);
-        },
-        function prepareResponse(err, tpl_val){
-          if ( err ) throw err;
-          return { status: 'ok' };
-        },
-        function finish(err, response){
-            if (err){
-                var statusCode = 400;
-                response = { error: ''+err };
-                if ( ! _.isUndefined(err.http_status) ) {
-                  statusCode = err.http_status;
-                }
-                ws.sendError(res, response, statusCode, 'DELETE TEMPLATE', err);
-            } else {
-              ws.sendResponse(res, ['', 204]);
-            }
-        }
-      );
-    });
-
-    // Get a list of owned templates 
-    ws.get(template_baseurl, function(req, res) {
-      ws.doCORS(res);
-      var that = this;
-      var response = {};
-      var cdbuser = ws.userByReq(req);
-      Step(
-        function checkPerms(){
-            serverOptions.authorizedByAPIKey(req, this);
-        },
-        function listTemplates(err, authenticated) {
-          if ( err ) throw err;
-          if (authenticated !== 1) {
-            err = new Error("Only authenticated user can list templated maps");
-            err.http_status = 401;
-            throw err;
-          }
-          templateMaps.listTemplates(cdbuser, this);
-        },
-        function prepareResponse(err, tpl_ids){
-          if ( err ) throw err;
-          // NOTE: might omit "cbduser" if == dbowner ...
-          var ids = _.map(tpl_ids, function(id) { return cdbuser + '@' + id; })
-          return { template_ids: ids };
-        },
-        function finish(err, response){
-            var statusCode = 200;
-            if (err){
-                response = { error: ''+err };
-                if ( ! _.isUndefined(err.http_status) ) {
-                  statusCode = err.http_status;
-                }
-                ws.sendError(res, response, statusCode, 'GET TEMPLATE LIST', err);
-            } else {
-              ws.sendResponse(res, [response, statusCode]);
-            }
-        }
-      );
-    });
-
-    ws.setDBParams = function(cdbuser, params, callback) {
-      Step(
-        function setAuth() {
-          serverOptions.setDBAuth(cdbuser, params, this);
-        },
-        function setConn(err) {
-          if ( err ) throw err;
-          serverOptions.setDBConn(cdbuser, params, this);
-        },
-        function finish(err) {
-          callback(err);
-        }
-      );
-    };
-
-    ws.options(template_baseurl + '/:template_id', function(req, res) {
-      ws.doCORS(res, "Content-Type");
-      return next();
-    });
-
-    // Instantiate a template
-    function instanciateTemplate(req, res, template_params, callback) {
-      ws.doCORS(res);
-      if ( req.profiler ) req.profiler.done('cors');
-      var that = this;
-      var response = {};
-      var template;
-      var signedMaps = serverOptions.signedMaps;
-      var layergroup;
-      var layergroupid;
-      var fakereq; // used for call to createLayergroup
-      var cdbuser = ws.userByReq(req);
-      // Format of template_id: [<template_owner>]@<template_id>
-      var tpl_id = req.params.template_id.split('@');
-      if ( tpl_id.length > 1 ) {
-        if ( tpl_id[0] ) cdbuser = tpl_id[0];
-        tpl_id = tpl_id[1];
-      }
-      var auth_token = req.query.auth_token;
-      Step(
-        function getTemplate(){
-          templateMaps.getTemplate(cdbuser, tpl_id, this);
-        },
-        function checkAuthorized(err, data) {
-          if ( req.profiler ) req.profiler.done('getTemplate');
-          if ( err ) throw err;
-          if ( ! data ) {
-            err = new Error("Template '" + tpl_id + "' of user '" + cdbuser + "' not found");
-            err.http_status = 404;
-            throw err;
-          }
-          template = data;
-          var cert = templateMaps.getTemplateCertificate(template);
-          var authorized = false;
-          try {
-            // authorizedByCert will throw if unauthorized
-            authorized = signedMaps.authorizedByCert(cert, auth_token);
-          } catch (err) {
-            // we catch to add http_status
-            err.http_status = 401;
-            throw err;
-          }
-          if ( ! authorized ) {
-            err = new Error('Unauthorized template instanciation');
-            err.http_status = 401;
-            throw err;
-          }
-          /*if ( (! req.headers['content-type'] || req.headers['content-type'].split(';')[0] != 'application/json') && req.query.callback === undefined) {
-            throw new Error('template POST data must be of type application/json, it is instead ');
-          }*/
-          //var template_params = req.body;
-          if ( req.profiler ) req.profiler.done('authorizedByCert');
-          return templateMaps.instance(template, template_params);
-        },
-        function prepareParams(err, instance){
-          if ( req.profiler ) req.profiler.done('TemplateMaps_instance');
-          if ( err ) throw err;
-          layergroup = instance;
-          fakereq = { query: {}, params: {}, headers: _.clone(req.headers),
-            profiler: req.profiler
-          };
-          ws.setDBParams(cdbuser, fakereq.params, this);
-        },
-        function setApiKey(err){
-          if ( req.profiler ) req.profiler.done('setDBParams');
-          if ( err ) throw err;
-          cartoData.getUserMapKey(cdbuser, this);
-        },
-        function createLayergroup(err, val) {
-          if ( req.profiler ) req.profiler.done('getUserMapKey');
-          if ( err ) throw err;
-          fakereq.params.api_key = val;
-          ws.createLayergroup(layergroup, fakereq, this);
-        },
-        function signLayergroup(err, resp) {
-          // NOTE: createLayergroup uses profiler.start()/end() internally
-          //if ( req.profiler ) req.profiler.done('createLayergroup');
-          if ( err ) throw err;
-          response = resp;
-          var signer = cdbuser;
-          var map_id = response.layergroupid.split(':')[0]; // dropping last_updated 
-          var crt_id = template.auth_id; // check ?
-          if ( ! crt_id ) {
-            var errmsg = "Template '" + tpl_id + "' of user '" + cdbuser + "' has no signature";
-            // Is this really illegal ?
-            // Maybe we could just return an unsigned layergroupid
-            // in this case...
-            err = new Error(errmsg);
-            err.http_status = 403; // Forbidden, we refuse to respond to this
-            throw err;
-          }
-          signedMaps.signMap(signer, map_id, crt_id, this);
-        },
-        function prepareResponse(err) {
-          if ( req.profiler ) req.profiler.done('signMap');
-          if ( err ) throw err;
-          //console.log("Response from createLayergroup: "); console.dir(response);
-          // Add the signature part to the token!
-          var tplhash = templateMaps.fingerPrint(template).substring(0,8);
-          if ( req.profiler ) req.profiler.done('fingerPrint');
-          response.layergroupid = cdbuser + '@' + tplhash + '@' + response.layergroupid;
-          return response;
-        },
-        callback
-      );
-    }
-
-    function finish_instanciation(err, response, res, req) {
-        if ( req.profiler ) {
-          var report = req.profiler.toString();
-          res.header('X-Tiler-Profiler', report);
-        }
-        if (err) {
-            var statusCode = 400;
-            response = { error: ''+err };
-            if ( ! _.isUndefined(err.http_status) ) {
-              statusCode = err.http_status;
-            }
-            if(debug) {
-              response.stack = err.stack;
-            }
-            ws.sendError(res, response, statusCode, 'POST INSTANCE TEMPLATE', err);
-        } else {
-          ws.sendResponse(res, [response, 200]);
-        }
-        if ( req.profiler && req.profiler.statsd_client) {
-          req.profiler.sendStats();
-        }
-    }
-
-    ws.post(template_baseurl + '/:template_id', function(req, res) {
-      if ( req.profiler && req.profiler.statsd_client) {
-        req.profiler.start('windshaft-cartodb.instance_template_post');
-      }
-      Step(
-        function() {
-          if ( ! req.headers['content-type'] || req.headers['content-type'].split(';')[0] != 'application/json') {
-            throw new Error('template POST data must be of type application/json, it is instead ');
-          }
-          instanciateTemplate(req, res, req.body, this);
-        }, function(err, response) {
-          finish_instanciation(err, response, res, req);
-        }
-      );
-    });
-
-    /**
-     * jsonp endpoint, allows to instanciate a template with a json call.
-     * callback query argument is mandartoy
-     */
-    ws.get(template_baseurl + '/:template_id/jsonp', function(req, res) {
-      if ( req.profiler && req.profiler.statsd_client) {
-        req.profiler.start('windshaft-cartodb.instance_template_get');
-      }
-      Step(
-        function() {
-          if ( req.query.callback === undefined || req.query.callback.length === 0) {
-            throw new Error('callback parameter should be present and be a function name');
-          }
-          var config = {};
-          if(req.query.config) {
-            try {
-              config = JSON.parse(req.query.config);
-            } catch(e) {
-              throw new Error('badformed config parameter, should be a valid JSON');
-            }
-          }
-          instanciateTemplate(req, res, config, this);
-        }, function(err, response) {
-          finish_instanciation(err, response, res, req);
-        }
-      );
-    });
-
-
-    // ---- Template maps interface ends @}
-
     return ws;
 }
 

lib/cartodb/server_options.js

@@ -10,11 +10,7 @@ var   _          = require('underscore')
 
 // This is for backward compatibility with 1.3.3
 if ( _.isUndefined(global.environment.sqlapi.domain) ) {
-  // Only use "host" as "domain" if it contains alphanumeric characters
-  var host = global.environment.sqlapi.host;
-  if ( host && host.match(/[a-zA-Z]/) ) {
-    global.environment.sqlapi.domain = host;
-  }
+  global.environment.sqlapi.domain = global.environment.sqlapi.host;
 }
 
 module.exports = function(){
@@ -26,35 +22,19 @@ module.exports = function(){
     });
 
     var me = {
-        // This is for inline maps and table maps
-        base_url: global.environment.base_url_legacy || '/tiles/:table',
-
-        /// @deprecated with Windshaft-0.17.0
-        ///base_url_notable: '/tiles',
-
-        // This is for Detached maps
-        //
-        // "maps" is the official, while
-        // "tiles/layergroup" is for backward compatibility up to 1.6.x
-        //
-        base_url_mapconfig: global.environment.base_url_detached || '(?:/maps|/tiles/layergroup)',
-
+        base_url: '/tiles/:table',
+        base_url_notable: '/tiles',
         grainstore: {
-          map: {
-            // TODO: allow to specify in configuration
-            srid: 3857
-          },
           datasource: global.environment.postgres,
           cachedir: global.environment.millstone.cache_basedir,
           mapnik_version: global.environment.mapnik_version || mapnik.versions.mapnik,
-          default_layergroup_ttl: global.environment.mapConfigTTL || 7200, 
-          gc_prob: 0.01 // @deprecated since Windshaft-1.8.0
+          default_layergroup_ttl: 7200, // seconds (default is 300)
+          gc_prob: 0.01 // default is 0.01 TODO: make configurable via env config
         },
         mapnik: {
           metatile: rendererConfig.metatile,
           bufferSize: rendererConfig.bufferSize
         },
-        statsd: global.environment.statsd,
         renderCache: {
           ttl: rendererConfig.cache_ttl
         },
@@ -62,7 +42,6 @@ module.exports = function(){
         enable_cors: global.environment.enable_cors,
         varnish_host: global.environment.varnish.host,
         varnish_port: global.environment.varnish.port,
-        varnish_secret: global.environment.varnish.secret,
         cache_enabled: global.environment.cache_enabled,
         log_format: global.environment.log_format,
         useProfiler: global.environment.useProfiler
@@ -89,13 +68,9 @@ module.exports = function(){
       var api = global.environment.sqlapi;
 
       // build up api string
-      var sqlapihostname = username;
-      if ( api.domain ) sqlapihostname += '.' + api.domain;
-
-      var sqlapi = api.protocol + '://';
-      if ( api.host && api.host != api.domain ) sqlapi += api.host;
-      else sqlapi += sqlapihostname;
-      sqlapi += ':' + api.port + '/api/' + api.version + '/sql';
+      var sqlapi = api.protocol + '://' + username;
+      if ( api.domain ) sqlapi += '.' + api.domain;
+      sqlapi += ':' + api.port + '/api/' + api.version + '/sql'
 
       var qs  = { q: sql }
 
@@ -103,37 +78,9 @@ module.exports = function(){
       if (_.isString(api_key) && api_key != '') { qs.api_key = api_key; }
 
       // call sql api
-      //
-      // NOTE: using POST to avoid size limits:
-      // See http://github.com/CartoDB/Windshaft-cartodb/issues/111
-      //
-      // NOTE: uses "host" header to allow IP based specification
-      //       of sqlapi address (and avoid a DNS lookup)
-      //
-      // NOTE: allows for keeping up to "maxConnections" concurrent
-      //       sockets opened per SQL-API host.
-      // See http://nodejs.org/api/http.html#http_agent_maxsockets
-      //
-      var maxSockets = global.environment.maxConnections || 128;
-      var maxGetLen = api.max_get_sql_length || 2048;
-      var reqSpec = {
-            url:sqlapi,
-            json:true,
-            headers:{host: sqlapihostname}
-            // http://nodejs.org/api/http.html#http_agent_maxsockets
-            ,pool:{maxSockets:maxSockets}
-            //,timeout:100
-      }
-      if ( sql.length > maxGetLen ) {
-        reqSpec.method = 'POST';
-        reqSpec.body = qs;
-      } else {
-        reqSpec.method = 'GET';
-        reqSpec.qs = qs;
-      }
-      request(reqSpec, function(err, res, body) {
+      request.get({url:sqlapi, qs:qs, json:true}, function(err, res, body){
           if (err){
-            console.log('ERROR connecting to SQL API on ' + sqlapi + ': ' + err);
+            console.log('ERROR running connecting to SQL API on ' + sqlapi + ': ' + err);
             callback(err);
             return;
           }
@@ -206,101 +153,53 @@ module.exports = function(){
         return hash.digest('hex');
     }
 
-    me.generateCacheChannel = function(app, req, callback){
+    me.generateCacheChannel = function(req, callback){
+
+      // use key to call sql api with sql request if present, else
+      // just return dbname and table name base key
+      var dbName     = req.params.dbname;
 
-      // Build channelCache key
-      var dbName = req.params.dbname;
       var cacheKey = [ dbName ];
       if ( req.params.token ) cacheKey.push(req.params.token);
       else if ( req.params.sql ) cacheKey.push( me.generateMD5(req.params.sql) );
       cacheKey = cacheKey.join(':');
 
-      var that = this;
+      if ( me.channelCache.hasOwnProperty(cacheKey) ) {
+        callback(null, me.channelCache[cacheKey]);
+        return;
+      }
+      else if ( req.params.token ) {
+        // cached cache channel for token-based access should be constructed
+        // at cache creation time
+        callback(new Error('missing channel cache for token ' + req.params.token));
+        return;
+      }
 
-      Step (
-        function checkCached() {
-          if ( me.channelCache.hasOwnProperty(cacheKey) ) {
-            callback(null, me.channelCache[cacheKey]);
-            return;
-          }
-          return null;
-        },
-        function extractSQL(err) {
-          if ( err ) throw err;
+      if ( ! req.params.sql && ! req.params.token ) {
+        var cacheChannel = me.buildCacheChannel(dbName, [req.params.table]);
+        // not worth caching this
+        callback(null, cacheChannel);
+        return;
+      }
 
-          if ( req.params.token ) {
-            // TODO: cached cache channel for token-based access should
-            //       be constructed at renderer cache creation time
-            // See http://github.com/CartoDB/Windshaft-cartodb/issues/152
-            if ( ! app.mapStore ) {
-              throw new Error('missing channel cache for token ' + req.params.token);
-              return;
-            }
-            var next = this;
-            var mapStore = app.mapStore;
-            Step(
-              function loadFromStore() {
-                mapStore.load(req.params.token, this);
-              },
-              function getSQL(err, mapConfig) {
-                if (req.profiler) req.profiler.done('mapStore_load');
-                if ( err ) throw err;
-                var sql = [];
-                _.each(mapConfig.obj().layers, function(lyr) {
-                  sql.push(lyr.options.sql);
-                });
-                sql = sql.join(';');
-                return sql;
-              },
-              function finish(err, sql) {
-                next(err, sql);
-              }
-            );
-            return;
-          }
+      if ( ! req.params.sql ) {
+        callback(new Error("this request doesn't need an X-Cache-Channel generated"));
+        return;
+      }
 
-          if ( ! req.params.sql ) {
-            return null; // no sql
-          }
+      var dbName     = req.params.dbname;
+      var username   = req.headers.host.split('.')[0];
 
-          // We have sql, and no token...
+      // strip out windshaft/mapnik inserted sql if present
+      var sql = req.params.sql.match(/^\((.*)\)\sas\scdbq$/);
+      sql = (sql != null) ? sql[1] : req.params.sql;
 
-          // strip out windshaft/mapnik inserted sql if present
-          var sql = req.params.sql.match(/^\((.*)\)\sas\scdbq$/);
-          sql = (sql != null) ? sql[1] : req.params.sql;
-
-          return sql;
-        },
-        function findAffectedTables(err, sql) {
-          if ( err ) throw err;
-          if ( ! sql ) {
-            if ( ! req.params.table ) {
-              throw new Error("this request doesn't need an X-Cache-Channel generated");
-            }
-            return [req.params.table];
-          }
-          var username   = that.userByReq(req);
-          me.affectedTables(username, req.params.map_key, sql, this);
-        },
-        function buildCacheChannel(err, tableNames) {
-          if ( err ) throw err;
-          if (req.profiler && ! req.params.table ) {
-            req.profiler.done('affectedTables');
-          }
-
-          var dbName     = req.params.dbname;
+      me.affectedTables(username, req.params.map_key, sql, function(err, tableNames) {
+          if ( err ) { callback(err); return; }
           var cacheChannel = me.buildCacheChannel(dbName,tableNames);
-          // store for caching from me.generateCacheChannel
-          // (not worth when table was specified in params)
-          if ( ! req.params.table ) {
-            me.channelCache[cacheKey] = cacheChannel;
-          }
-          return cacheChannel;
-        },
-        function finish(err, cacheChannel) {
-          callback(err, cacheChannel);
-        }
-      );
+          me.channelCache[cacheKey] = cacheChannel; // store for caching
+          callback(null, cacheChannel);
+      });
     };
 
     // Set the cache chanel info to invalidate the cache on the frontend server
@@ -312,10 +211,9 @@ module.exports = function(){
     // @param cb function(err, channel) will be called when ready.
     //           the channel parameter will be null if nothing was added
     //
-    me.addCacheChannel = function(app, req, cb) {
+    me.addCacheChannel = function(req, cb) {
         // skip non-GET requests, or requests for which there's no response
         if ( req.method != 'GET' || ! req.res ) { cb(null, null); return; }
-        if (req.profiler) req.profiler.start('addCacheChannel');
         var res = req.res;
         var cache_policy = req.query.cache_policy;
         if ( req.params.token ) cache_policy = 'persist';
@@ -337,9 +235,7 @@ module.exports = function(){
         }
         res.header('Last-Modified', lastUpdated.toUTCString());
 
-        me.generateCacheChannel(app, req, function(err, channel){
-            if (req.profiler) req.profiler.done('generateCacheChannel');
-            if (req.profiler) req.profiler.end();
+        me.generateCacheChannel(req, function(err, channel){
             if ( ! err ) {
               res.header('X-Cache-Channel', channel);
               cb(null, channel);
@@ -354,7 +250,7 @@ module.exports = function(){
     me.afterLayergroupCreate = function(req, mapconfig, response, callback) {
         var token = response.layergroupid;
 
-        var username = this.userByReq(req);
+        var username = cartoData.userFromHostname(req.headers.host);
 
         var tasksleft = 2; // redis key and affectedTables
         var errors = [];
@@ -385,8 +281,8 @@ module.exports = function(){
         sql = sql.join(';');
 
         var dbName = req.params.dbname;
-        var usr    = this.userByReq(req);
-        var key    = req.params.map_key || req.params.api_key;
+        var usr    = req.headers.host.split('.')[0];
+        var key    = req.params.map_key;
 
         var cacheKey = dbName + ':' + token;
 
@@ -395,8 +291,7 @@ module.exports = function(){
 
             if ( err ) { done(err); return; }
             var cacheChannel = me.buildCacheChannel(dbName,tableNames);
-            // store for caching from me.afterLayergroupCreate
-            me.channelCache[cacheKey] = cacheChannel;
+            me.channelCache[cacheKey] = cacheChannel; // store for caching
             // find last updated
             me.findLastUpdated(usr, key, tableNames, function(err, lastUpdated) {
               if (req.profiler) req.profiler.done('findLastUpdated');
@@ -410,243 +305,69 @@ module.exports = function(){
 
 /* X-Cache-Channel generation } */
 
-    me.re_userFromHost = new RegExp(
-      global.environment.user_from_host ||
-      '^([^\\.]+)\\.' // would extract "strk" from "strk.cartodb.com"
-    );
-
-    me.userByReq = function(req) {
-      var host = req.headers.host;
-      var mat = host.match(this.re_userFromHost);
-      if ( ! mat ) {
-        console.error("ERROR: user pattern '" + this.re_userFromHost
-          + "' does not match hostname '" + host + "'");
-        return;
-      }
-      // console.log("Matches: "); console.dir(mat);
-      if ( ! mat.length === 2 ) {
-        console.error("ERROR: pattern '" + this.re_userFromHost
-          + "' gave unexpected matches against '" + host + "': " + mat);
-        return;
-      }
-      return mat[1];
-    }
-
-    // Set db authentication parameters to those of the given username
-    //
-    // @param username the cartodb username, mapped to a database username
-    //                 via CartodbRedis metadata records
-    //
-    // @param params the parameters to set auth options into
-    //               added params are: "dbuser" and "dbpassword"
-    //
-    // @param callback function(err) 
-    //
-    me.setDBAuth = function(username, params, callback) {
-
-      var user_params = {};
-      var auth_user = global.environment.postgres_auth_user;
-      var auth_pass = global.environment.postgres_auth_pass;
-      Step(
-        function getId() {
-          cartoData.getUserId(username, this);
-        },
-        function(err, user_id) {
-          if (err) throw err;
-          user_params['user_id'] = user_id;
-          var dbuser = _.template(auth_user, user_params);
-          _.extend(params, {dbuser:dbuser});
-
-          // skip looking up user_password if postgres_auth_pass
-          // doesn't contain the "user_password" label 
-          if (!auth_pass || ! auth_pass.match(/\buser_password\b/) ) return null;
-
-          cartoData.getUserDBPass(username, this);
-        },
-        function(err, user_password) {
-          if (err) throw err;
-          user_params['user_password'] = user_password;
-          if ( auth_pass ) {
-            var dbpass = _.template(auth_pass, user_params);
-            _.extend(params, {dbpassword:dbpass});
-          }
-          return true;
-        },
-        function finish(err) {
-          callback(err); 
-        }
-      );
-    };
-
-    // Set db connection parameters to those for the given username
-    //
-    // @param dbowner cartodb username of database owner,
-    //                mapped to a database username
-    //                via CartodbRedis metadata records
-    //
-    // @param params the parameters to set connection options into
-    //               added params are: "dbname", "dbhost"
-    //
-    // @param callback function(err) 
-    //
-    me.setDBConn = function(dbowner, params, callback) {
-      Step(
-        function getDatabaseHost(){
-            cartoData.getUserDBHost(dbowner, this);
-        },
-        function getDatabase(err, data){
-            if(err) throw err;
-            if ( data ) _.extend(params, {dbhost:data});
-            cartoData.getUserDBName(dbowner, this);
-        },
-        function extendParams(err, data){
-            if (err) throw err;
-            if ( data ) _.extend(params, {dbname:data});
-            return null;
-        },
-        function finish(err) {
-          callback(err); 
-        }
-      );
-    };
-
-
-    // Check if a request is authorized by a signer
-    //
-    // Any existing signature for the given request will verified
-    // for authorization to this specific request (may require auth_token)
-    // See https://github.com/CartoDB/Windshaft-cartodb/wiki/Signed-maps
-    //
-    // @param req express request object
-    // @param callback function(err, signed_by) signed_by will be
-    //                 null if the request is not signed by anyone
-    //                 or will be a string cartodb username otherwise.
-    //                 
-    me.authorizedBySigner = function(req, callback)
-    {
-      if ( ! req.params.token || ! req.params.signer ) {
-        //console.log("No signature provided"); // debugging
-        callback(null, null); // no signer requested
-        return;
-      }
-
-      var signer = req.params.signer;
-      var layergroup_id = req.params.token;
-      var auth_token = req.params.auth_token;
-
-      //console.log("Checking authorization from signer " + signer + " for resource " + layergroup_id + " with auth_token " + auth_token);
-
-      me.signedMaps.isAuthorized(signer, layergroup_id, auth_token,
-        function(err, authorized) {
-          callback(err, authorized ? signer : null);
-        });
-    };
-
-    // Check if a request is authorized by api_key
-    //
-    // @param req express request object
-    // @param callback function(err, authorized) 
-    //        NOTE: authorized is expected to be 0 or 1 (integer)
-    //                 
-    me.authorizedByAPIKey = function(req, callback)
-    {
-        var givenKey = req.query.api_key || req.query.map_key;
-        if ( ! givenKey && req.body ) {
-          // check also in request body
-          givenKey = req.body.api_key || req.body.map_key;
-        }
-        if ( ! givenKey ) {
-          callback(null, 0); // no api key, no authorization...
-          return;
-        }
-        //console.log("given ApiKey: " + givenKey);
-        var user = me.userByReq(req);
-        Step(
-          function (){
-              cartoData.getUserMapKey(user, this);
-          },
-          function checkApiKey(err, val){
-              if (err) throw err;
-              return ( val && givenKey == val ) ? 1 : 0;
-          },
-          function finish(err, authorized) {
-              callback(err, authorized);
-          }
-        );
-    };
-
     /**
-     * Check access authorization
+     * Get privacy for cartodb table
      *
      * @param req - standard req object. Importantly contains table and host information
-     * @param callback function(err, allowed) is access allowed not?
+     * @param callback - is the table private or not?
      */
-    me.authorize = function(req, callback) {
+    me.authorize= function(req, callback) {
         var that = this;
-        var user = me.userByReq(req);
 
         Step(
-            function (){
-                that.authorizedByAPIKey(req, this);
+            function(){
+                cartoData.checkMapKey(req, this);
             },
-            function checkApiKey(err, authorized){
-                if (req.profiler) req.profiler.done('authorizedByAPIKey');
+            function checkIfInternal(err, check_result){
                 if (err) throw err;
 
-                // if not authorized by api_key, continue 
-                if (authorized !== 1)  {
-                  // not authorized by api_key, 
-                  // check if authorized by signer
-                  that.authorizedBySigner(req, this);
-                  return;
-                }
+                // if unauthorized continue to check table privacy
+                if (check_result !== 1) return true;
 
-                _.extend(req.params, { _authorizedByApiKey: true });
+                // authorized by key, login as db owner 
+                var user_params = {};
+                var auth_user = global.environment.postgres_auth_user;
+                var auth_pass = global.environment.postgres_auth_pass;
+                Step(
+                  function getId() {
+                    cartoData.getId(req, this);
+                  },
+                  function(err, user_id) {
+                    if (err) throw err;
+                    user_params['user_id'] = user_id;
+                    var dbuser = _.template(auth_user, user_params);
+                    _.extend(req.params, {dbuser:dbuser});
 
-                // authorized by api key, login as the given username and stop
-                that.setDBAuth(user, req.params, function(err) {
-                  callback(err, true); // authorized (or error)
-                });
-            },
-            function checkSignAuthorized(err, signed_by){
-                if (err) throw err;
-                if (req.profiler) {
-                  if ( req.params._authorizedByApiKey ) {
-                    req.profiler.done('setDBAuth');
-                  } else {
-                    req.profiler.done('authorizedBySigner');
+                    // skip looking up user_password if postgres_auth_pass
+                    // doesn't contain the "user_password" label 
+                    if (!auth_pass || ! auth_pass.match(/\buser_password\b/) ) return null;
+
+                    cartoData.getDatabasePassword(req, this);
+                  },
+                  function(err, user_password) {
+                    if (err) throw err;
+                    user_params['user_password'] = user_password;
+                    if ( auth_pass ) {
+                      var dbpass = _.template(auth_pass, user_params);
+                      _.extend(req.params, {dbpassword:dbpass});
+                    }
+                    return true;
+                  },
+                  function finish(err) {
+                    callback(err, true); // authorized (or error)
                   }
-                }
-
-                if ( ! signed_by ) {
-                  // request not authorized by signer,
-                  // continue to check table privacy,
-                  // if table was given
-                  if ( req.params.table ) return null;
-                  // otherwise return no authorization
-                  callback(err, null);
-                  return;
-                }
-
-                // Authorized by "signed_by" !
-                that.setDBAuth(signed_by, req.params, function(err) {
-                  if (req.profiler) req.profiler.done('setDBAuth');
-                  callback(err, true); // authorized (or error)
-                });
-            },
-            function getDatabase(err){
+                );
+            }
+            ,function getDatabase(err, data){
                 if (err) throw err;
-                // NOTE: only used to get to table privacy
-                cartoData.getUserDBName(user, this);
+                cartoData.getDatabase(req, this);
             },
-            function getPrivacy(err, dbname){
+            function getPrivacy(err, data){
                 if (err) throw err;
-                if (req.profiler) req.profiler.done('tablePrivacy_getUserDBName');
-                cartoData.getTablePrivacy(dbname, req.params.table, this);
+                cartoData.getTablePrivacy(data, req.params.table, this);
             },
-            function(err, privacy){
-                if (req.profiler) req.profiler.done('getTablePrivacy');
-                callback(err, privacy);
+            function(err, data){
+                callback(err, data);
             }
         );
     };
@@ -688,7 +409,7 @@ module.exports = function(){
         }
 
         // Whitelist query parameters and attach format
-        var good_query = ['sql', 'geom_type', 'cache_buster', 'cache_policy', 'callback', 'interactivity', 'map_key', 'api_key', 'auth_token', 'style', 'style_version', 'style_convert', 'config' ];
+        var good_query = ['sql', 'geom_type', 'cache_buster', 'cache_policy', 'callback', 'interactivity', 'map_key', 'api_key', 'style', 'style_version', 'style_convert', 'config' ];
         var bad_query  = _.difference(_.keys(req.query), good_query);
 
         _.each(bad_query, function(key){ delete req.query[key]; });
@@ -699,16 +420,6 @@ module.exports = function(){
           var tksplit = req.params.token.split(':');
           req.params.token = tksplit[0];
           if ( tksplit.length > 1 ) req.params.cache_buster= tksplit[1];
-          tksplit = req.params.token.split('@');
-          if ( tksplit.length > 1 ) {
-            req.params.signer = tksplit.shift();
-            if ( ! req.params.signer ) req.params.signer = this.userByReq(req);
-            if ( tksplit.length > 1 ) {
-              var template_hash = tksplit.shift(); // unused
-            }
-            req.params.token = tksplit.shift(); 
-            //console.log("Request for token " + req.params.token + " with signature from " + req.params.signer);
-          }
         }
 
         // bring all query values onto req.params object
@@ -717,49 +428,70 @@ module.exports = function(){
         // for cartodb, ensure interactivity is cartodb_id or user specified
         req.params.interactivity = req.params.interactivity || 'cartodb_id';
 
+        req.params.processXML = function(req, xml, callback) {
+
+          // Replace dbuser
+          var dbuser = req.params.dbuser || global.environment.postgres.user;
+          if ( ! me.rx_dbuser ) me.rx_dbuser = /(<Parameter name="user"><!\[CDATA\[)[^\]]*(]]><\/Parameter>)/g;
+          xml = xml.replace(me.rx_dbuser, "$1" + dbuser + "$2");
+
+          // Replace dbpass
+          var dbpass = req.params.dbpassword || global.environment.postgres.password;
+          if ( ! me.rx_dbpass ) me.rx_dbpass = /(<Parameter name="password"><!\[CDATA\[)[^\]]*(]]><\/Parameter>)/g;
+          xml = xml.replace(me.rx_dbpass, "$1" + dbpass + "$2");
+
+          // Replace or set dbhost
+          var dbhost = req.params.dbhost || global.environment.postgres.host;
+          if ( ! me.rx_dbhost ) me.rx_dbhost = /(<Parameter name="host"><!\[CDATA\[)[^\]]*(]]><\/Parameter>)/g;
+          xml = xml.replace(me.rx_dbhost, "$1" + dbhost + "$2");
+
+          callback(null, xml);
+        }
+
         var that = this;
 
         if (req.profiler) req.profiler.done('req2params.setup');
 
-        var user = me.userByReq(req);
-
         Step(
             function getPrivacy(){
                 me.authorize(req, this);
             },
             function gatekeep(err, data){
-                if (req.profiler) req.profiler.done('authorize');
+                if (req.profiler) req.profiler.done('cartoData.authorize');
                 if(err) throw err;
                 if(data === "0") throw new Error("Sorry, you are unauthorized (permission denied)");
                 return data;
             },
-            function getDatabase(err, data){
+            function getDatabaseHost(err, data){
                 if(err) throw err;
-                that.setDBConn(user, req.params, this);
+
+                cartoData.getDatabaseHost(req, this);
             },
-            function getGeometryType(err){
-                if (req.profiler) req.profiler.done('setDBConn');
+            function getDatabase(err, data){
+                if (req.profiler) req.profiler.done('cartoData.getDatabaseHost');
+                if(err) throw err;
+                if ( data ) _.extend(req.params, {dbhost:data});
+
+                cartoData.getDatabase(req, this);
+            },
+            function getGeometryType(err, data){
+                if (req.profiler) req.profiler.done('cartoData.getDatabase');
                 if (err) throw err;
-                if ( ! req.params.table ) return null;
-                cartoData.getTableGeometryType(req.params.dbname, req.params.table, this);
+                _.extend(req.params, {dbname:data});
+
+                cartoData.getGeometryType(req, this);
             },
             function finishSetup(err, data){
-                if (req.profiler) req.profiler.done('cartoData.getTableGeometryType');
+                if (req.profiler) req.profiler.done('cartoData.getGeometryType');
                 if ( err ) { callback(err, req); return; }
 
                 if (!_.isNull(data))
                     _.extend(req.params, {geom_type: data});
 
-                // Add default database connection parameters
-                // if none given
-                _.defaults(req.params, {
-                  dbuser: global.environment.postgres.user,
-                  dbpassword: global.environment.postgres.password,
-                  dbhost: global.environment.postgres.host,
-                  dbport: global.environment.postgres.port
+                that.addCacheChannel(req, function(err) {
+                  if (req.profiler) req.profiler.done('addCacheChannel');
+                  callback(err, req);
                 });
-
-                callback(null, req);
             }
         );
     };
@@ -771,23 +503,14 @@ module.exports = function(){
      */
     me.getInfowindow = function(req, callback){
         var that = this;
-        var user = me.userByReq(req);
 
         Step(
             function(){
-                // TODO: if this step really needed ?
                 that.req2params(req, this);
             },
-            function getDatabase(err){
-                if (err) throw err;
-                cartoData.getUserDBName(user, this);
-            },
-            function getInfowindow(err, dbname){
-                if (err) throw err;
-                cartoData.getTableInfowindow(dbname, req.params.table, this);
-            },
             function(err, data){
-                callback(err, data);
+                if (err) callback(err, null);
+                else cartoData.getInfowindow(data, callback);
             }
         );
     };
@@ -799,23 +522,14 @@ module.exports = function(){
      */
     me.getMapMetadata = function(req, callback){
         var that = this;
-        var user = me.userByReq(req);
 
         Step(
             function(){
-                // TODO: if this step really needed ?
                 that.req2params(req, this);
             },
-            function getDatabase(err){
-                if (err) throw err;
-                cartoData.getUserDBName(user, this);
-            },
-            function getMapMetadata(err, dbname){
-                if (err) throw err;
-                cartoData.getTableMapMetadata(dbname, req.params.table, this);
-            },
             function(err, data){
-                callback(err, data);
+                if (err) callback(err, null);
+                else cartoData.getMapMetadata(data, callback);
             }
         );
     };
@@ -849,4 +563,4 @@ module.exports = function(){
     };
 
     return me;
-};
+}();

lib/cartodb/signed_maps.js

@@ -1,401 +0,0 @@
-var crypto    = require('crypto');
-var Step      = require('step');
-var _         = require('underscore');
-
-var debug = global.environment ? global.environment.debug : undefined;
-
-// Class handling map signatures and user certificates
-//
-// See https://github.com/CartoDB/Windshaft-cartodb/wiki/Signed-maps
-//
-// @param redis_pool an instance of a "redis-mpool"
-//        See https://github.com/CartoDB/node-redis-mpool
-//        Needs version 0.x.x of the API.
-//
-function SignedMaps(redis_pool) {
-  this.redis_pool = redis_pool;
-
-  // Database containing signatures
-  // TODO: allow configuring ?
-  // NOTE: currently it is the same as
-  //       the one containing layergroups
-  this.db_signatures = 0;
-
-  //
-  // Map signatures in redis are reference to signature certificates
-  // We have the following datastores:
-  //
-  //  1. User certificates: set of per-user authorization certificates
-  //  2. Map signatures: set of per-map certificate references 
-  //  3. Certificate applications: set of per-certificate signed maps
-
-  // User certificates (HASH:crt_id->crt_val)
-  this.key_map_crt = "map_crt|<%= signer %>";
-
-  // Map signatures (SET:crt_id)
-  this.key_map_sig = "map_sig|<%= signer %>|<%= map_id %>";
-  
-  // Certificates applications (SET:map_id)
-  //
-  // Everytime a map is signed, the map identifier (layergroup_id)
-  // is added to this set. The purpose of this set is to drop
-  // all map signatures when a certificate is removed
-  //
-  this.key_crt_sig = "crt_sig|<%= signer %>|<%= crt_id %>";
-
-};
-
-var o = SignedMaps.prototype;
-
-//--------------- PRIVATE METHODS --------------------------------
-
-o._acquireRedis = function(callback) {
-  this.redis_pool.acquire(this.db_signatures, callback);
-};
-
-o._releaseRedis = function(client) {
-  this.redis_pool.release(this.db_signatures, client);
-};
-
-/**
- * Internal function to communicate with redis
- *
- * @param redisFunc - the redis function to execute
- * @param redisArgs - the arguments for the redis function in an array
- * @param callback - function to pass results too.
- */
-o._redisCmd = function(redisFunc, redisArgs, callback) {
-  var redisClient;
-  var that = this;
-  var db = that.db_signatures;
-
-  Step(
-    function getRedisClient() {
-      that.redis_pool.acquire(db, this);
-    },
-    function executeQuery(err, data) {
-      if ( err ) throw err;
-      redisClient = data;
-      redisArgs.push(this);
-      redisClient[redisFunc.toUpperCase()].apply(redisClient, redisArgs);
-    },
-    function releaseRedisClient(err, data) {
-      if ( ! _.isUndefined(redisClient) ) that.redis_pool.release(db, redisClient);
-      callback(err, data);
-    }
-  );
-};
-
-o._getAuthMethod = function(auth) {
-  return auth.method || 'open';
-};
-
-//--------------- PUBLIC API -------------------------------------
-
-/// Check formal validity of a certificate
-//
-/// Return an Error instance if invalid, null otherwise
-///
-o.checkInvalidCertificate = function(cert) {
-  //console.log("Checking cert: "); console.dir(cert);
-  if ( cert.version !== "0.0.1" ) {
-    return new Error("Unsupported certificate version " + cert.version);
-  }
-
-  if ( ! cert.auth ) {
-console.log("Cert is : "); console.dir(cert);
-    return new Error("No certificate authorization");
-  }
-
-  var method = this._getAuthMethod(cert.auth);
-
-  switch ( method ) {
-    case 'open':
-      break;
-    case 'token':
-      if ( ! _.isArray(cert.auth.valid_tokens) )
-        return new Error("Invalid 'token' authentication: missing valid_tokens");
-      if ( ! cert.auth.valid_tokens.length )
-        return new Error("Invalid 'token' authentication: no valid_tokens");
-      break;
-    default:
-      return new Error("Unsupported authentication method: " + cert.auth.method);
-      break;
-  }
-
-  return null; // all valid
-}
-
-// Check if the given certificate authorizes waiver of "auth"
-o.authorizedByCert = function(cert, auth) {
-
-  var err = this.checkInvalidCertificate(cert);
-  if ( err ) throw err;
-
-  var method = this._getAuthMethod(cert.auth);
-
-  // Open authentication certificates are always authorized
-  if ( method === 'open' ) return true;
-
-  // Token based authentication requires valid token
-  if ( method === 'token' ) {
-    var found = cert.auth.valid_tokens.indexOf(auth);
-    //if ( found !== -1 ) {
-      //console.log("Token " + auth + " is found at position " + found + " in valid tokens " + cert.auth.valid_tokens);
-    //  return true;
-    //} else return false;
-    return cert.auth.valid_tokens.indexOf(auth) !== -1;
-  }
-
-  throw new Error("Unsupported authentication method: " + cert.auth.method);
-};
-
-// Check if shown credential are authorized to access a map
-// by the given signer.
-//
-// @param signer a signer name (cartodb username)
-// @param map_id a layergroup_id
-// @param auth an authentication token, or undefined if none
-//                    (can still be authorized by signature)
-//
-// @param callback function(Error, Boolean)
-//
-o.isAuthorized = function(signer, map_id, auth, callback) {
-  var that = this;
-  var redisClient;
-  var db = that.db_signatures;
-  var authorized = false;
-  var certificate_id_list;
-  var missing_certificates = [];
-  if ( debug ) {
-    console.log("Check auth from signer '" + signer + "' on map '" + map_id + "' with auth '" + auth + "'");
-  }
-  Step(
-    function getRedisClient() {
-      that.redis_pool.acquire(db, this);
-    },
-    function getMapSignatures(err, client) {
-      if ( err ) throw err;
-      redisClient = client;
-      var map_sig_key = _.template(that.key_map_sig, {signer:signer, map_id:map_id});
-      redisClient.SMEMBERS(map_sig_key, this);
-      //that._redisCmd('SMEMBERS', [ map_sig_key ], this);
-    },
-    function getCertificates(err, crt_lst) {
-      if ( err ) throw err;
-      if ( debug ) {
-        console.log("Map '" + map_id + "' is signed by " + crt_lst.length + " certificates of user '" + signer);
-      }
-      certificate_id_list = crt_lst;
-      if ( ! crt_lst.length ) {
-        // No certs, avoid calling redis with short args list.
-        // Next step expects a list of certificate values so
-        // we directly send the empty list.
-        return crt_lst;
-      }
-      var map_crt_key = _.template(that.key_map_crt, {signer:signer});
-      //that._redisCmd('HMGET', [ map_crt_key ].concat(crt_lst), this);
-      redisClient.HMGET(map_crt_key, crt_lst, this);
-    },
-    function checkCertificates(err, certs) {
-      if ( err ) throw err;
-      for (var i=0; i<certs.length; ++i) {
-        var crt_id = certificate_id_list[i];
-        if ( _.isNull(certs[i]) ) {
-          missing_certificates.push(crt_id);
-          continue;
-        }
-        var cert;
-        try {
-          //console.log("cert " + crt_id + ": " + certs[i]);
-          cert = JSON.parse(certs[i]);
-          authorized = that.authorizedByCert(cert, auth);
-        } catch (err) {
-          console.log("Certificate " + certificate_id_list[i] + " by user '" + signer + "' is malformed: " + err);
-          continue;
-        }
-        if ( authorized ) {
-          if ( debug ) {
-            console.log("Access to map '" + map_id + "' authorized by cert '"
-                        + certificate_id_list[i] + "' of user '" + signer + "'");
-          }
-          //console.dir(cert);
-          break; // no need to further check certs
-        }
-      }
-      return null;
-    },
-    function finish(err) {
-      if ( missing_certificates.length ) {
-        console.log("WARNING: map '" + map_id + "' is signed by '" + signer
-          + "' with " + missing_certificates.length
-          + " missing certificates: "
-          + missing_certificates + " (TODO: give cleanup instructions)");
-      }
-      if ( redisClient ) that.redis_pool.release(db, redisClient);
-      callback(err, authorized);
-    }
-  );
-};
-
-// Add an authorization certificate from a user.
-//
-// @param signer a signer name (cartodb username)
-// @param cert certificate object, see
-//             http://github.com/CartoDB/Windshaft-cartodb/wiki/Signed-maps
-//
-// @param callback function(err, crt_id) return certificate id
-//
-// TODO: allow for requesting error when certificate already exists ?
-//
-o.addCertificate = function(signer, cert, callback) {
-  var crt_val = JSON.stringify(cert);
-  var crt_id = crypto.createHash('md5').update(crt_val).digest('hex');
-
-  var usr_crt_key = _.template(this.key_map_crt, {signer:signer});
-  this._redisCmd('HSET', [ usr_crt_key, crt_id, crt_val ], function(err, created) {
-    // NOTE: created would be 0 if the field already existed, 1 otherwise
-    callback(err, crt_id);
-  });
-};
-
-// Remove an authorization certificate of a user, also removing
-// any signature made with the certificate.
-//
-// @param signer a signer name (cartodb username)
-// @param crt_id certificate identifier, as returned by addCertificate
-// @param callback function(err)
-//
-o.delCertificate = function(signer, crt_id, callback) {
-  var db = this.db_signatures;
-  var crt_sig_key = _.template(this.key_crt_sig, {signer:signer, crt_id:crt_id});
-  var signed_map_list;
-  var redis_client;
-  var that = this;
-  Step (
-    function getRedisClient() {
-      that._acquireRedis(this);
-    },
-    function removeCertificate(err, data) {
-      if ( err ) throw err;
-      redis_client = data;
-      // Remove the certificate (would be enough to stop authorizing uses)
-      var usr_crt_key = _.template(that.key_map_crt, {signer:signer});
-      redis_client.HDEL(usr_crt_key, crt_id, this);
-    },
-    function getMapSignatures(err, deleted) {
-      if ( err ) throw err;
-      if ( ! deleted ) {
-        // debugging (how can this be possible?)
-        console.log("WARNING: authorization certificate '" + crt_id
-          + "' by user '" + signer + "' did not exist on delete request");
-      }
-      // Get all signatures by this certificate
-      redis_client.SMEMBERS(crt_sig_key, this);
-    },
-    function delMapSignaturesReference(err, map_id_list) {
-      if ( err ) throw err;
-      signed_map_list = map_id_list;
-      if ( debug ) {
-        console.log("Certificate '" + crt_id + "' from user '" + signer
-          + "' was used to sign " + signed_map_list.length + " maps");
-      }
-      redis_client.DEL(crt_sig_key, this);
-    },
-    function delMapSignatures(err) {
-      if ( err ) throw err;
-      var crt_sig_key = _.template(that.key_crt_sig, {signer:signer, crt_id:crt_id});
-      var tx = redis_client.MULTI();
-      for (var i=0; i<signed_map_list.length; ++i) {
-        var map_id = signed_map_list[i];
-        var map_sig_key = _.template(that.key_map_sig, {signer:signer, map_id:map_id});
-        //console.log("Queuing removal of '" + crt_id + "' from '" + map_sig_key + "'");
-        tx.SREM( map_sig_key, crt_id )
-      }
-      tx.EXEC(this);
-    },
-    function reportTransaction(err, rets) {
-      if ( err ) throw err;
-      if ( debug ) {
-        for (var i=0; i<signed_map_list.length; ++i) {
-          var ret = rets[i];
-          if ( ! ret ) {
-            console.log("No signature with certificate '" + crt_id
-              + "' of user '" + signer + "' found in map '"
-              + signed_map_list[i] + "'");
-          } else {
-            console.log("Signature with certificate '" + crt_id
-              + "' of user '" + signer + "' removed from map '"
-              + signed_map_list[i] + "'");
-          }
-        }
-      }
-      return null;
-    },
-    function finish(err) {
-      if ( ! _.isUndefined(redis_client) ) {
-        that._releaseRedis(redis_client);
-      }
-      callback(err);
-    }
-  );
-};
-
-// Sign a map with a certificate reference
-//
-// @param signer a signer name (cartodb username)
-// @param map_id a layergroup_id
-// @param crt_id signature certificate identifier
-//
-// @param callback function(Error) 
-//
-o.signMap = function(signer, map_id, crt_id, callback) {
-  var that = this;
-  Step(
-    function addMapSignature() {
-      var map_sig_key = _.template(that.key_map_sig, {signer:signer, map_id:map_id});
-      if ( debug ) {
-        console.log("Adding " + crt_id + " to " + map_sig_key);
-      }
-      that._redisCmd('SADD', [ map_sig_key, crt_id ], this);
-    },
-    function addCertificateUsage(err) {
-      // Add the map to the set of maps signed by the given cert
-      if ( err ) throw err;
-      var crt_sig_key = _.template(that.key_crt_sig, {signer:signer, crt_id:crt_id});
-      that._redisCmd('SADD', [ crt_sig_key, map_id ], this);
-    },
-    function finish(err) {
-      callback(err);
-    }
-  );
-};
-
-// Sign a map with a full certificate
-//
-// @param signer a signer name (cartodb username)
-// @param map_id a layergroup_id
-// @param cert_id signature certificate identifier
-//
-// @param callback function(Error, String) return certificate id
-//
-o.addSignature = function(signer, map_id, cert, callback) {
-  var that = this;
-  var certificate_id;
-  Step(
-    function addCertificate() {
-      that.addCertificate(signer, cert, this);
-    },
-    function signMap(err, cert_id) {
-      if ( err ) throw err;
-      if ( ! cert_id ) throw new Error("addCertificate returned no certificate id");
-      certificate_id = cert_id;
-      that.signMap(signer, map_id, cert_id, this);
-    },
-    function finish(err) {
-      callback(err, certificate_id);
-    }
-  );
-};
-
-module.exports = SignedMaps;

lib/cartodb/template_maps.js

@@ -1,615 +0,0 @@
-var crypto    = require('crypto');
-var Step      = require('step');
-var _         = require('underscore');
-
-// Templates in this hash (keyed as <username>@<template_name>)
-// are being worked on.
-var user_template_locks = {};
-
-// Class handling map templates
-//
-// See http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps
-//
-// @param redis_pool an instance of a "redis-mpool"
-//        See https://github.com/CartoDB/node-redis-mpool
-//        Needs version 0.x.x of the API.
-//
-// @param signed_maps an instance of a "signed_maps" class,
-//        See signed_maps.js
-//
-// @param opts TemplateMap options. Supported elements:
-//    'max_user_templates' limit on the number of per-user
-//
-// 
-function TemplateMaps(redis_pool, signed_maps, opts) {
-  this.redis_pool = redis_pool;
-  this.signed_maps = signed_maps;
-  this.opts = opts || {};
-
-  // Database containing templates
-  // TODO: allow configuring ?
-  // NOTE: currently it is the same as
-  //       the one containing layergroups
-  this.db_signatures = 0;
-
-  //
-  // Map templates are owned by a user that specifies access permissions
-  // for their instances.
-  // 
-  // We have the following datastores:
-  //
-  //  1. User teplates: set of per-user map templates
-  //     NOTE: each template would have an associated auth
-  //           reference, see signed_maps.js
-
-  // User templates (HASH:tpl_id->tpl_val)
-  this.key_usr_tpl = "map_tpl|<%= owner %>";
-
-  // User template locks (HASH:tpl_id->ctime)
-  this.key_usr_tpl_lck = "map_tpl|<%= owner %>|locks";
-
-};
-
-var o = TemplateMaps.prototype;
-
-//--------------- PRIVATE METHODS --------------------------------
-
-o._userTemplateLimit = function() {
-  return this.opts['max_user_templates'] || 0;
-};
-
-o._acquireRedis = function(callback) {
-  this.redis_pool.acquire(this.db_signatures, callback);
-};
-
-o._releaseRedis = function(client) {
-  this.redis_pool.release(this.db_signatures, client);
-};
-
-/**
- * Internal function to communicate with redis
- *
- * @param redisFunc - the redis function to execute
- * @param redisArgs - the arguments for the redis function in an array
- * @param callback - function to pass results too.
- */
-o._redisCmd = function(redisFunc, redisArgs, callback) {
-  var redisClient;
-  var that = this;
-  var db = that.db_signatures;
-
-  Step(
-    function getRedisClient() {
-      that.redis_pool.acquire(db, this);
-    },
-    function executeQuery(err, data) {
-      if ( err ) throw err;
-      redisClient = data;
-      redisArgs.push(this);
-      redisClient[redisFunc.toUpperCase()].apply(redisClient, redisArgs);
-    },
-    function releaseRedisClient(err, data) {
-      if ( ! _.isUndefined(redisClient) ) that.redis_pool.release(db, redisClient);
-      callback(err, data);
-    }
-  );
-};
-
-// @param callback function(err, obtained)
-o._obtainTemplateLock = function(owner, tpl_id, callback) {
-  var usr_tpl_lck_key = _.template(this.key_usr_tpl_lck, {owner:owner});
-  var that = this;
-  var gotLock = false;
-  Step (
-    function obtainLock() {
-      var ctime = Date.now();
-      that._redisCmd('HSETNX', [usr_tpl_lck_key, tpl_id, ctime], this);
-    },
-    function checkLock(err, locked) {
-      if ( err ) throw err;
-      if ( ! locked ) {
-        // Already locked
-        // TODO: unlock if expired ?
-        throw new Error("Template '" + tpl_id + "' of user '" + owner + "' is locked");
-      }
-      return gotLock = true;
-    },
-    function finish(err) {
-      callback(err, gotLock);
-    }
-  );
-};
-
-// @param callback function(err, deleted)
-o._releaseTemplateLock = function(owner, tpl_id, callback) {
-  var usr_tpl_lck_key = _.template(this.key_usr_tpl_lck, {owner:owner});
-  this._redisCmd('HDEL', [usr_tpl_lck_key, tpl_id], callback);
-};
-
-o._reValidIdentifier = /^[a-zA-Z][0-9a-zA-Z_]*$/;
-o._checkInvalidTemplate = function(template) {
-  if ( template.version != '0.0.1' ) {
-    return new Error("Unsupported template version " + template.version);
-  }
-  var tplname = template.name;
-  if ( ! tplname ) {
-    return new Error("Missing template name");
-  }
-  if ( ! tplname.match(this._reValidIdentifier) ) {
-    return new Error("Invalid characters in template name '" + tplname + "'");
-  }
-
-  var phold = template.placeholders;
-  for (var k in phold) {
-    if ( ! k.match(this._reValidIdentifier) ) {
-      return new Error("Invalid characters in placeholder name '" + k + "'");
-    }
-    if ( ! phold[k].hasOwnProperty('default') ) {
-      return new Error("Missing default for placeholder '" + k + "'");
-    }
-    if ( ! phold[k].hasOwnProperty('type') ) {
-      return new Error("Missing type for placeholder '" + k + "'");
-    }
-  };
-
-  // Check certificate validity
-  var cert = this.getTemplateCertificate(template);
-  var err = this.signed_maps.checkInvalidCertificate(cert);
-  if ( err ) return err;
-
-  // TODO: run more checks over template format ?
-};
-
-//--------------- PUBLIC API -------------------------------------
-
-// Extract a signature certificate from a template
-//
-// The certificate will be ready to be passed to
-// SignedMaps.addCertificate or SignedMaps.authorizedByCert
-//
-o.getTemplateCertificate = function(template) {
-  var cert = {
-    version: '0.0.1',
-    template_id: template.name,
-    auth: template.auth
-  };
-  return cert;
-};
-
-// Add a template
-//
-// NOTE: locks user+template_name or fails
-//
-// @param owner cartodb username of the template owner
-//
-// @param template layergroup template, see
-//        http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps#template-format
-//
-// @param callback function(err, tpl_id) 
-//        Return template identifier (only valid for given user)
-//
-o.addTemplate = function(owner, template, callback) {
-  var invalidError = this._checkInvalidTemplate(template);
-  if ( invalidError ) {
-    callback(invalidError);
-    return;
-  }
-  var tplname = template.name;
-
-  // Procedure:
-  //
-  // - Check against limit
-  // 0. Obtain a lock for user+template_name, fail if impossible
-  // 1. Check no other template exists with the same name
-  // 2. Install certificate extracted from template, extending
-  //    it to contain a name to properly salt things out.
-  // 3. Modify the template object to reference certificate by id
-  // 4. Install template
-  // 5. Release lock
-  //
-  //
-
-  var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
-  var gotLock = false;
-  var that = this;
-  var limit = that._userTemplateLimit();
-  Step(
-    function checkLimit() {
-      if ( ! limit ) return 0;
-      that._redisCmd('HLEN', [ usr_tpl_key ], this);
-    },
-    // try to obtain a lock
-    function obtainLock(err, len) {
-      if ( err ) throw err;
-      if ( limit && len >= limit ) {
-        throw new Error("User '" + owner + "' reached limit on number of templates (" + len + "/" + limit + ")");
-      }
-      that._obtainTemplateLock(owner, tplname, this);
-    },
-    function getExistingTemplate(err, locked) {
-      if ( err ) throw err;
-      if ( ! locked ) {
-        // Already locked
-        throw new Error("Template '" + tplname + "' of user '" + owner + "' is locked");
-      }
-      gotLock = true;
-      that._redisCmd('HEXISTS', [ usr_tpl_key, tplname ], this);
-    },
-    function installCertificate(err, exists) {
-      if ( err ) throw err;
-      if ( exists ) {
-        throw new Error("Template '" + tplname + "' of user '" + owner + "' already exists"); 
-      }
-      var cert = that.getTemplateCertificate(template);
-      that.signed_maps.addCertificate(owner, cert, this);
-    },
-    function installTemplate(err, crt_id) {
-      if ( err ) throw err;
-      template.auth_id = crt_id;
-      var tpl_val = JSON.stringify(template);
-      that._redisCmd('HSET', [ usr_tpl_key, tplname, tpl_val ], this);
-    },
-    function releaseLock(err, newfield) {
-      if ( ! err && ! newfield ) {
-        console.log("ERROR: addTemplate overridden existing template '"
-          + tplname + "' of '" + owner
-          + "' -- HSET returned " + overridden + ": someone added it without locking ?");
-        // TODO: how to recover this ?!
-      }
-
-      if ( err && ! gotLock ) throw err;
-
-      // release the lock
-      var next = this;
-      that._releaseTemplateLock(owner, tplname, function(e, d) {
-        if ( e ) {
-          console.log("Error removing lock on template '" + tplname
-            + "' of user '" + owner + "': " + e);
-        } else if ( ! d ) {
-          console.log("ERROR: lock on template '" + tplname
-            + "' of user '" + owner + "' externally removed during insert!");
-        }
-        next(err);
-      });
-    },
-    function finish(err) {
-      callback(err, tplname);
-    }
-  );
-};
-
-// Delete a template
-//
-// NOTE: locks user+template_name or fails
-//
-// Also deletes associated authentication certificate, which
-// in turn deletes all instance signatures
-//
-// @param owner cartodb username of the template owner
-//
-// @param tpl_id template identifier as returned
-//        by addTemplate or listTemplates
-//
-// @param callback function(err)
-//
-o.delTemplate = function(owner, tpl_id, callback) {
-  var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
-  var gotLock = false;
-  var that = this;
-  Step(
-    // try to obtain a lock
-    function obtainLock() {
-      that._obtainTemplateLock(owner, tpl_id, this);
-    },
-    function getExistingTemplate(err, locked) {
-      if ( err ) throw err;
-      if ( ! locked ) {
-        // Already locked
-        throw new Error("Template '" + tpl_id + "' of user '" + owner + "' is locked");
-      }
-      gotLock = true;
-      that._redisCmd('HGET', [ usr_tpl_key, tpl_id ], this);
-    },
-    function delCertificate(err, tplval) {
-      if ( err ) throw err;
-      if ( ! tplval ) {
-        throw new Error("Template '" + tpl_id + "' of user '" + owner + "' does not exist"); 
-      }
-      var tpl = JSON.parse(tplval);
-      if ( ! tpl.auth_id ) {
-        // not sure this is an error, in case we'll ever
-        // allow unsigned templates...
-        console.log("ERROR: installed template '" + tpl_id
-            + "' of user '" + owner + "' has no auth_id reference: "); console.dir(tpl);
-        return null;
-      }
-      var next = this;
-      that.signed_maps.delCertificate(owner, tpl.auth_id, function(err) {
-        if ( err ) {
-          var msg = "ERROR: could not delete certificate '"
-                  + tpl.auth_id + "' associated with template '"
-                  + tpl_id + "' of user '" + owner + "': " + err;
-          // I'm actually not sure we want this event to be fatal
-          // (avoiding a deletion of the template itself) 
-          next(new Error(msg));
-        } else {
-          next();
-        }
-      });
-    },
-    function delTemplate(err) {
-      if ( err ) throw err;
-      that._redisCmd('HDEL', [ usr_tpl_key, tpl_id ], this);
-    },
-    function releaseLock(err, deleted) {
-      if ( ! err && ! deleted ) {
-          console.log("ERROR: template '" + tpl_id
-            + "' of user '" + owner + "' externally removed during delete!");
-      }
-
-      if ( ! gotLock ) {
-        if ( err ) throw err;
-        return null;
-      }
-
-      // release the lock
-      var next = this;
-      that._releaseTemplateLock(owner, tpl_id, function(e, d) {
-        if ( e ) {
-          console.log("Error removing lock on template '" + tpl_id
-            + "' of user '" + owner + "': " + e);
-        } else if ( ! d ) {
-          console.log("ERROR: lock on template '" + tpl_id
-            + "' of user '" + owner + "' externally removed during delete!");
-        }
-        next(err);
-      });
-    },
-    function finish(err) {
-      callback(err);
-    }
-  );
-};
-
-// Update a template
-//
-// NOTE: locks user+template_name or fails
-//
-// Also deletes and re-creates associated authentication certificate,
-// which in turn deletes all instance signatures
-//
-// @param owner cartodb username of the template owner
-//
-// @param tpl_id template identifier as returned by addTemplate
-//
-// @param template layergroup template, see
-//        http://github.com/CartoDB/Windshaft-cartodb/wiki/Template-maps#template-format
-//
-// @param callback function(err)
-//        
-o.updTemplate = function(owner, tpl_id, template, callback) {
-
-  var invalidError = this._checkInvalidTemplate(template);
-  if ( invalidError ) {
-    callback(invalidError);
-    return;
-  }
-
-  var tplname = template.name;
-
-  if ( tpl_id != tplname ) {
-    callback(new Error("Cannot update name of a map template ('" + tpl_id + "' != '" + tplname + "')"));
-    return;
-  }
-
-  var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
-  var gotLock = false;
-  var that = this;
-  Step(
-    // try to obtain a lock
-    function obtainLock() {
-      that._obtainTemplateLock(owner, tpl_id, this);
-    },
-    function getExistingTemplate(err, locked) {
-      if ( err ) throw err;
-      if ( ! locked ) {
-        // Already locked
-        throw new Error("Template '" + tpl_id + "' of user '" + owner + "' is locked");
-      }
-      gotLock = true;
-      that._redisCmd('HGET', [ usr_tpl_key, tpl_id ], this);
-    },
-    function delOldCertificate(err, tplval) {
-      if ( err ) throw err;
-      if ( ! tplval ) {
-        throw new Error("Template '" + tpl_id + "' of user '"
-                         + owner +"' does not exist");
-      }
-      var tpl = JSON.parse(tplval);
-      if ( ! tpl.auth_id ) {
-        // not sure this is an error, in case we'll ever
-        // allow unsigned templates...
-        console.log("ERROR: installed template '" + tpl_id
-            + "' of user '" + owner + "' has no auth_id reference: "); console.dir(tpl);
-        return null;
-      }
-      var next = this;
-      that.signed_maps.delCertificate(owner, tpl.auth_id, function(err) {
-        if ( err ) {
-          var msg = "ERROR: could not delete certificate '"
-                  + tpl.auth_id + "' associated with template '"
-                  + tpl_id + "' of user '" + owner + "': " + err;
-          // I'm actually not sure we want this event to be fatal
-          // (avoiding a deletion of the template itself) 
-          next(new Error(msg));
-        } else {
-          next();
-        }
-      });
-    },
-    function installNewCertificate(err) {
-      if ( err ) throw err;
-      var cert = that.getTemplateCertificate(template);
-      that.signed_maps.addCertificate(owner, cert, this);
-    },
-    function updTemplate(err, crt_id) {
-      if ( err ) throw err;
-      template.auth_id = crt_id;
-      var tpl_val = JSON.stringify(template);
-      that._redisCmd('HSET', [ usr_tpl_key, tplname, tpl_val ], this);
-    },
-    function releaseLock(err, newfield) {
-      if ( ! err && newfield ) {
-          console.log("ERROR: template '" + tpl_id
-            + "' of user '" + owner + "' externally removed during update!");
-      }
-
-      if ( ! gotLock ) {
-        if ( err ) throw err;
-        return null;
-      }
-
-      // release the lock
-      var next = this;
-      that._releaseTemplateLock(owner, tpl_id, function(e, d) {
-        if ( e ) {
-          console.log("Error removing lock on template '" + tpl_id
-            + "' of user '" + owner + "': " + e);
-        } else if ( ! d ) {
-          console.log("ERROR: lock on template '" + tpl_id
-            + "' of user '" + owner + "' externally removed during update!");
-        }
-        next(err);
-      });
-    },
-    function finish(err) {
-      callback(err);
-    }
-  );
-};
-
-// List user templates
-//
-// @param owner cartodb username of the templates owner
-//
-// @param callback function(err, tpl_id_list)
-//        Returns a list of template identifiers
-// 
-o.listTemplates = function(owner, callback) {
-  var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
-  this._redisCmd('HKEYS', [ usr_tpl_key ], callback);
-};
-
-// Get a templates
-//
-// @param owner cartodb username of the template owner
-//
-// @param tpl_id template identifier as returned
-//        by addTemplate or listTemplates
-//
-// @param callback function(err, template)
-//        Return full template definition
-//
-o.getTemplate = function(owner, tpl_id, callback) {
-  var usr_tpl_key = _.template(this.key_usr_tpl, {owner:owner});
-  var that = this;
-  Step(
-    function getTemplate() {
-      that._redisCmd('HGET', [ usr_tpl_key, tpl_id ], this);
-    },
-    function parseTemplate(err, tpl_val) {
-      if ( err ) throw err;
-      var tpl = JSON.parse(tpl_val);
-      // Should we strip auth_id ?
-      return tpl;
-    },
-    function finish(err, tpl) {
-      callback(err, tpl);
-    }
-  );
-};
-
-// Perform placeholder substitutions on a template
-//
-// @param template a template object (will not be modified)
-//
-// @param params an object containing named subsitution parameters
-//        Only the ones found in the template's placeholders object
-//        will be used, with missing ones taking default values.
-//
-// @returns a layergroup configuration 
-//
-// @throws Error on malformed template or parameter
-//
-o._reNumber = /^([-+]?[\d\.]?\d+([eE][+-]?\d+)?)$/;
-o._reCSSColorName = /^[a-zA-Z]+$/;
-o._reCSSColorVal = /^#[0-9a-fA-F]{3,6}$/;
-o._replaceVars = function(str, params) {
-  //return _.template(str, params); // lazy way, possibly dangerous 
-  // Construct regular expressions for each param
-  if ( ! params._re ) {
-    params._re = {};
-    for (var k in params) {
-      params._re[k] = RegExp("<%=\\s*" + k + "\\s*%>", "g");
-    }
-  }
-  for (var k in params) str = str.replace(params._re[k], params[k]);
-  return str;
-};
-o.instance = function(template, params) {
-  var all_params = {};
-  var phold = template.placeholders;
-  for (var k in phold) {
-    var val = params.hasOwnProperty(k) ? params[k] : phold[k].default;
-    var type = phold[k].type;
-    // properly escape
-    if ( type === 'sql_literal' ) {
-      // duplicate any single-quote
-      val = val.replace(/'/g, "''");
-    }
-    else if ( type === 'sql_ident' ) {
-      // duplicate any double-quote
-      val = val.replace(/"/g, '""');
-    }
-    else if ( type === 'number' ) {
-      // check it's a number
-      if ( typeof(val) !== 'number' && ! val.match(this._reNumber) ) {
-        throw new Error("Invalid number value for template parameter '"
-          + k + "': " + val);
-      }
-    }
-    else if ( type === 'css_color' ) {
-      // check it only contains letters or
-      // starts with # and only contains hexdigits
-      if ( ! val.match(this._reCSSColorName) && ! val.match(this._reCSSColorVal) ) {
-        throw new Error("Invalid css_color value for template parameter '"
-          + k + "': " + val);
-      }
-    }
-    else {
-      // NOTE: should be checked at template create/update time
-      throw new Error("Invalid placeholder type '" + type + "'");
-    }
-    all_params[k] = val;
-  }
-
-  // NOTE: we're deep-cloning the layergroup here
-  var layergroup = JSON.parse(JSON.stringify(template.layergroup));
-  for (var i=0; i<layergroup.layers.length; ++i) {
-    var lyropt = layergroup.layers[i].options;
-    if ( lyropt.cartocss ) lyropt.cartocss = this._replaceVars(lyropt.cartocss, all_params);
-    if ( lyropt.sql) lyropt.sql = this._replaceVars(lyropt.sql, all_params);
-    // Anything else ?
-  }
-  return layergroup;
-};
-
-// Return a fingerPrint of the object
-o.fingerPrint = function(template) {
-  return crypto.createHash('md5')
-    .update(JSON.stringify(template))
-    .digest('hex')
-  ;
-};
-
-module.exports = TemplateMaps;

npm-shrinkwrap.json

@@ -1,26 +1,52 @@
 {
   "name": "windshaft-cartodb",
-  "version": "1.8.2",
+  "version": "1.5.2",
   "dependencies": {
     "node-varnish": {
-      "version": "0.2.0",
-      "from": "http://github.com/Vizzuality/node-varnish/tarball/v0.2.0"
+      "version": "0.1.1"
     },
     "underscore": {
       "version": "1.3.3"
     },
     "windshaft": {
-      "version": "0.19.0",
-      "from": "http://github.com/CartoDB/Windshaft/tarball/0.19.0-rc1",
+      "version": "0.14.5",
       "dependencies": {
         "grainstore": {
-          "version": "0.18.0",
+          "version": "0.15.2",
           "dependencies": {
+            "carto": {
+              "version": "0.9.5-cdb2",
+              "from": "git://github.com/CartoDB/carto.git#0.9.5-cdb2",
+              "dependencies": {
+                "underscore": {
+                  "version": "1.4.4"
+                },
+                "xml2js": {
+                  "version": "0.2.8",
+                  "dependencies": {
+                    "sax": {
+                      "version": "0.5.5"
+                    }
+                  }
+                },
+                "optimist": {
+                  "version": "0.6.0",
+                  "dependencies": {
+                    "wordwrap": {
+                      "version": "0.0.2"
+                    },
+                    "minimist": {
+                      "version": "0.0.5"
+                    }
+                  }
+                }
+              }
+            },
             "mapnik-reference": {
               "version": "5.0.7"
             },
             "millstone": {
-              "version": "0.6.11",
+              "version": "0.6.8",
               "dependencies": {
                 "underscore": {
                   "version": "1.5.2"
@@ -35,7 +61,7 @@
                       "version": "5.0.0"
                     },
                     "forever-agent": {
-                      "version": "0.5.2"
+                      "version": "0.5.0"
                     },
                     "tunnel-agent": {
                       "version": "0.3.0"
@@ -95,151 +121,56 @@
                           }
                         },
                         "async": {
-                          "version": "0.2.10"
+                          "version": "0.2.9"
                         }
                       }
                     }
                   }
                 },
                 "srs": {
-                  "version": "0.3.10"
+                  "version": "0.3.8"
                 },
                 "zipfile": {
-                  "version": "0.4.3"
+                  "version": "0.4.2"
                 },
                 "sqlite3": {
-                  "version": "2.2.0",
+                  "version": "2.1.19",
                   "dependencies": {
-                    "node-pre-gyp": {
-                      "version": "0.2.6",
+                    "tar.gz": {
+                      "version": "0.1.1",
                       "dependencies": {
-                        "nopt": {
-                          "version": "2.1.2",
+                        "fstream": {
+                          "version": "0.1.25",
                           "dependencies": {
-                            "abbrev": {
-                              "version": "1.0.4"
+                            "rimraf": {
+                              "version": "2.2.4"
+                            },
+                            "graceful-fs": {
+                              "version": "2.0.1"
+                            },
+                            "inherits": {
+                              "version": "2.0.1"
                             }
                           }
                         },
-                        "npmlog": {
-                          "version": "0.0.6",
-                          "dependencies": {
-                            "ansi": {
-                              "version": "0.2.1"
-                            }
-                          }
-                        },
-                        "semver": {
-                          "version": "2.1.0"
-                        },
                         "tar": {
-                          "version": "0.1.19",
+                          "version": "0.1.18",
                           "dependencies": {
                             "inherits": {
                               "version": "2.0.1"
                             },
                             "block-stream": {
                               "version": "0.0.7"
-                            },
-                            "fstream": {
-                              "version": "0.1.25",
-                              "dependencies": {
-                                "graceful-fs": {
-                                  "version": "2.0.2"
-                                }
-                              }
                             }
                           }
                         },
-                        "tar-pack": {
-                          "version": "2.0.0",
+                        "commander": {
+                          "version": "1.1.1",
                           "dependencies": {
-                            "uid-number": {
-                              "version": "0.0.3"
-                            },
-                            "once": {
-                              "version": "1.1.1"
-                            },
-                            "debug": {
-                              "version": "0.7.4"
-                            },
-                            "fstream": {
-                              "version": "0.1.25",
-                              "dependencies": {
-                                "graceful-fs": {
-                                  "version": "2.0.2"
-                                },
-                                "inherits": {
-                                  "version": "2.0.1"
-                                }
-                              }
-                            },
-                            "fstream-ignore": {
-                              "version": "0.0.7",
-                              "dependencies": {
-                                "minimatch": {
-                                  "version": "0.2.14",
-                                  "dependencies": {
-                                    "sigmund": {
-                                      "version": "1.0.0"
-                                    }
-                                  }
-                                },
-                                "inherits": {
-                                  "version": "2.0.1"
-                                }
-                              }
-                            },
-                            "readable-stream": {
-                              "version": "1.0.25-1",
-                              "dependencies": {
-                                "string_decoder": {
-                                  "version": "0.10.25-1"
-                                }
-                              }
-                            },
-                            "graceful-fs": {
-                              "version": "1.2.3"
+                            "keypress": {
+                              "version": "0.1.0"
                             }
                           }
-                        },
-                        "aws-sdk": {
-                          "version": "2.0.0-rc9",
-                          "dependencies": {
-                            "xml2js": {
-                              "version": "0.2.4",
-                              "dependencies": {
-                                "sax": {
-                                  "version": "0.6.0"
-                                }
-                              }
-                            },
-                            "xmlbuilder": {
-                              "version": "0.4.2"
-                            }
-                          }
-                        },
-                        "rc": {
-                          "version": "0.3.3",
-                          "dependencies": {
-                            "optimist": {
-                              "version": "0.3.7",
-                              "dependencies": {
-                                "wordwrap": {
-                                  "version": "0.0.2"
-                                }
-                              }
-                            },
-                            "deep-extend": {
-                              "version": "0.2.8"
-                            },
-                            "ini": {
-                              "version": "1.1.0"
-                            }
-                          }
-                        },
-                        "rimraf": {
-                          "version": "2.2.6"
                         }
                       }
                     }
@@ -252,13 +183,13 @@
                   "version": "0.3.5"
                 },
                 "optimist": {
-                  "version": "0.6.1",
+                  "version": "0.6.0",
                   "dependencies": {
                     "wordwrap": {
                       "version": "0.0.2"
                     },
                     "minimist": {
-                      "version": "0.0.7"
+                      "version": "0.0.5"
                     }
                   }
                 }
@@ -308,7 +239,7 @@
           }
         },
         "tilelive-mapnik": {
-          "version": "0.6.5",
+          "version": "0.6.4",
           "dependencies": {
             "eio": {
               "version": "0.2.2"
@@ -323,62 +254,6 @@
         },
         "lru-cache": {
           "version": "2.3.1"
-        },
-        "carto": {
-          "version": "0.9.5-cdb2",
-          "from": "http://github.com/CartoDB/carto/tarball/0.9.5-cdb2",
-          "dependencies": {
-            "underscore": {
-              "version": "1.4.4"
-            },
-            "mapnik-reference": {
-              "version": "5.0.7"
-            },
-            "xml2js": {
-              "version": "0.2.8",
-              "dependencies": {
-                "sax": {
-                  "version": "0.5.8"
-                }
-              }
-            },
-            "optimist": {
-              "version": "0.6.1",
-              "dependencies": {
-                "wordwrap": {
-                  "version": "0.0.2"
-                },
-                "minimist": {
-                  "version": "0.0.7"
-                }
-              }
-            }
-          }
-        },
-        "underscore.string": {
-          "version": "1.1.6",
-          "dependencies": {
-            "underscore": {
-              "version": "1.1.7"
-            }
-          }
-        },
-        "pg": {
-          "version": "2.6.2",
-          "dependencies": {
-            "generic-pool": {
-              "version": "2.0.3"
-            },
-            "buffer-writer": {
-              "version": "1.0.0"
-            }
-          }
-        },
-        "torque.js": {
-          "version": "2.2.00"
-        },
-        "node-statsd": {
-          "version": "0.0.7"
         }
       }
     },
@@ -389,54 +264,35 @@
       "version": "2.9.202"
     },
     "cartodb-redis": {
-      "version": "0.3.0"
-    },
-    "redis-mpool": {
-      "version": "0.0.3",
+      "version": "0.1.0",
       "dependencies": {
         "generic-pool": {
           "version": "2.0.4"
-        },
-        "hiredis": {
-          "version": "0.1.16",
-          "dependencies": {
-            "bindings": {
-              "version": "1.1.1"
-            }
-          }
         }
       }
     },
     "mapnik": {
-      "version": "0.7.26"
+      "version": "0.7.25"
     },
     "lzma": {
       "version": "1.2.3"
     },
-    "log4js": {
-      "version": "0.6.10",
-      "dependencies": {
-        "async": {
-          "version": "0.1.15"
-        },
-        "readable-stream": {
-          "version": "1.0.25-1",
-          "dependencies": {
-            "string_decoder": {
-              "version": "0.10.25-1"
-            }
-          }
-        }
-      }
-    },
-    "redis": {
-      "version": "0.8.6"
+    "semver": {
+      "version": "1.1.4"
     },
     "strftime": {
       "version": "0.6.2"
     },
-    "semver": {
-      "version": "1.1.4"
+    "redis": {
+      "version": "0.8.6"
+    },
+    "hiredis": {
+      "version": "0.1.15",
+      "dependencies": {
+        "bindings": {
+          "version": "1.1.0"
+        }
+      }
     },
     "mocha": {
       "version": "1.14.0",
@@ -471,7 +327,7 @@
           "version": "3.2.3",
           "dependencies": {
             "minimatch": {
-              "version": "0.2.14",
+              "version": "0.2.12",
               "dependencies": {
                 "lru-cache": {
                   "version": "2.5.0"

package.json

@@ -1,7 +1,7 @@
 {
     "private": true,
     "name": "windshaft-cartodb",
-    "version": "1.8.2",
+    "version": "1.5.2",
     "description": "A map tile server for CartoDB",
     "keywords": [
       "cartodb"
@@ -22,16 +22,14 @@
       "Sandro Santilli <strk@vizzuality.com>"
     ],
     "dependencies": {
-        "node-varnish": "http://github.com/Vizzuality/node-varnish/tarball/v0.2.0",
+        "node-varnish": "0.1.1",
         "underscore"  : "~1.3.3",
-        "windshaft"  : "http://github.com/CartoDB/Windshaft/tarball/0.19.0",
+        "windshaft"  : "~0.14.5",
         "step": "0.0.x",
         "request": "2.9.202",
-        "cartodb-redis": "~0.3.0",
-        "redis-mpool": "~0.0.2",
+        "cartodb-redis": "~0.1.0",
         "mapnik": "~0.7.22",
-        "lzma": "~1.2.3",
-        "log4js": "~0.6.10"
+        "lzma": "~1.2.3"
     },
     "devDependencies": {
         "mocha": "1.14.0",

test/acceptance/multilayer.js

@@ -15,19 +15,10 @@ require(__dirname + '/../support/test_helper');
 var windshaft_fixtures = __dirname + '/../../node_modules/windshaft/test/fixtures';
 
 var CartodbWindshaft = require(__dirname + '/../../lib/cartodb/cartodb_windshaft');
-var ServerOptions = require(__dirname + '/../../lib/cartodb/server_options');
-serverOptions = ServerOptions();
+var serverOptions = require(__dirname + '/../../lib/cartodb/server_options');
 var server = new CartodbWindshaft(serverOptions);
 server.setMaxListeners(0);
 
-// Check that the response headers do not request caching
-// Throws on failure
-function checkNoCache(res) {
-  assert.ok(!res.headers.hasOwnProperty('x-cache-channel'));
-  assert.ok(!res.headers.hasOwnProperty('cache-control')); // is this correct ?
-  assert.ok(!res.headers.hasOwnProperty('last-modified')); // is this correct ?
-}
-
 suite('multilayer', function() {
 
     var redis_client = redis.createClient(global.environment.redis.port);
@@ -35,10 +26,6 @@ suite('multilayer', function() {
     var expected_last_updated_epoch = 1234567890123; // this is hard-coded into SQLAPIEmu
     var expected_last_updated = new Date(expected_last_updated_epoch).toISOString();
 
-    var test_user = _.template(global.environment.postgres_auth_user, {user_id:1});
-    var test_pubuser = global.environment.postgres.user;
-    var test_database = test_user + '_db';
-
     suiteSetup(function(done){
       sqlapi_server = new SQLAPIEmu(global.environment.sqlapi.port, done);
     });
@@ -63,7 +50,7 @@ suite('multilayer', function() {
         ]
       };
 
-      var expected_token; // = "e34dd7e235138a062f8ba7ad051aa3a7";
+      var expected_token = "e34dd7e235138a062f8ba7ad051aa3a7";
       Step(
         function do_post()
         {
@@ -88,7 +75,7 @@ suite('multilayer', function() {
               if ( expected_token ) {
                 assert.equal(parsedBody.layergroupid, expected_token + ':' + expected_last_updated_epoch);
               }
-              else expected_token = parsedBody.layergroupid.split(':')[0];
+              else expected_token = parsedBody.layergroupid;
               next(null, res);
           });
         },
@@ -112,7 +99,7 @@ suite('multilayer', function() {
               // Check X-Cache-Channel
               cc = res.headers['x-cache-channel'];
               assert.ok(cc); 
-              var dbname = test_database;
+              var dbname = 'test_cartodb_user_1_db'
               assert.equal(cc.substring(0, dbname.length), dbname);
               var jsonquery = cc.substring(dbname.length+1);
               var sentquery = JSON.parse(jsonquery);
@@ -169,7 +156,7 @@ suite('multilayer', function() {
             errors.push(err.message);
             console.log("Error: " + err);
           }
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
+          redis_client.keys("map_style|test_cartodb_user_1_db|~" + expected_token, function(err, matches) {
               if ( err ) errors.push(err.message);
               assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
               redis_client.del(matches, function(err) {
@@ -198,7 +185,7 @@ suite('multilayer', function() {
         ]
       };
 
-      var expected_token; //  = "6d8e4ad5458e2d25cf0eef38e38717a6";
+      var expected_token  = "6d8e4ad5458e2d25cf0eef38e38717a6";
       Step(
         function do_post()
         {
@@ -222,7 +209,7 @@ suite('multilayer', function() {
               if ( expected_token ) {
                 assert.equal(parsedBody.layergroupid, expected_token + ':' + expected_last_updated_epoch);
               }
-              else expected_token = parsedBody.layergroupid.split(':')[0];
+              else expected_token = parsedBody.layergroupid;
               next(null, res);
           });
         },
@@ -242,7 +229,7 @@ suite('multilayer', function() {
               // Check X-Cache-Channel
               var cc = res.headers['x-cache-channel'];
               assert.ok(cc); 
-              var dbname = test_database;
+              var dbname = 'test_cartodb_user_1_db'
               assert.equal(cc.substring(0, dbname.length), dbname);
               var jsonquery = cc.substring(dbname.length+1);
               var sentquery = JSON.parse(jsonquery);
@@ -275,7 +262,7 @@ suite('multilayer', function() {
               // Check X-Cache-Channel
               var cc = res.headers['x-cache-channel'];
               assert.ok(cc); 
-              var dbname = test_database;
+              var dbname = 'test_cartodb_user_1_db'
               assert.equal(cc.substring(0, dbname.length), dbname);
               var jsonquery = cc.substring(dbname.length+1);
               var sentquery = JSON.parse(jsonquery);
@@ -334,7 +321,7 @@ suite('multilayer', function() {
             errors.push(err.message);
             console.log("Error: " + err);
           }
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
+          redis_client.keys("map_style|test_cartodb_user_1_db|~" + expected_token, function(err, matches) {
               if ( err ) errors.push(err.message);
               assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
               redis_client.del(matches, function(err) {
@@ -430,7 +417,7 @@ suite('multilayer', function() {
           var next = this;
           // trip epoch
           expected_token = expected_token.split(':')[0];
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
+          redis_client.keys("map_style|test_cartodb_user_1_db|~" + expected_token, function(err, matches) {
               redis_client.del(matches, next);
           });
         },
@@ -473,35 +460,6 @@ suite('multilayer', function() {
       });
     });
 
-    // Also tests that server doesn't crash:
-    // see http://github.com/CartoDB/Windshaft-cartodb/issues/109
-    test("layergroup creation fails if sql is bogus", function(done) {
-      var layergroup =  {
-        stat_tag: 'random_tag',
-        version: '1.0.0',
-        layers: [
-           { options: {
-               sql: 'select bogus(0,0) as the_geom_webmercator',
-               cartocss: '#layer { polygon-fill:red; }', 
-               cartocss_version: '2.0.1' 
-             } }
-        ]
-      };
-      assert.response(server, {
-          url: '/tiles/layergroup',
-          method: 'POST',
-          headers: {host: 'localhost', 'Content-Type': 'application/json' },
-          data: JSON.stringify(layergroup)
-      }, {}, function(res) {
-          assert.equal(res.statusCode, 400, res.body);
-          var parsed = JSON.parse(res.body);
-          var msg = parsed.errors[0];
-          assert.ok(msg.match(/bogus.*exist/), msg);
-          checkNoCache(res);
-          done();
-      });
-    });
-
     test("layergroup with 2 private-table layers", function(done) {
 
       var layergroup =  {
@@ -522,7 +480,7 @@ suite('multilayer', function() {
         ]
       };
 
-      var expected_token; // = "b4ed64d93a411a59f330ab3d798e4009";
+      var expected_token = "b4ed64d93a411a59f330ab3d798e4009";
       Step(
         function do_post()
         {
@@ -547,7 +505,7 @@ suite('multilayer', function() {
               if ( expected_token ) {
                 assert.equal(parsedBody.layergroupid, expected_token + ':' + expected_last_updated_epoch);
               }
-              else expected_token = parsedBody.layergroupid.split(':')[0];
+              else expected_token = parsedBody.layergroupid;
               next(null, res);
           });
         },
@@ -567,7 +525,7 @@ suite('multilayer', function() {
               // Check X-Cache-Channel
               var cc = res.headers['x-cache-channel'];
               assert.ok(cc); 
-              var dbname = test_database;
+              var dbname = 'test_cartodb_user_1_db'
               assert.equal(cc.substring(0, dbname.length), dbname);
               next(err);
           });
@@ -655,7 +613,7 @@ suite('multilayer', function() {
             errors.push(err.message);
             console.log("Error: " + err);
           }
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
+          redis_client.keys("map_style|test_cartodb_user_1_db|~" + expected_token, function(err, matches) {
               if ( err ) errors.push(err.message);
               assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
               redis_client.del(matches, function(err) {
@@ -668,123 +626,6 @@ suite('multilayer', function() {
       );
     });
 
-    // See https://github.com/CartoDB/Windshaft-cartodb/issues/152
-    test("x-cache-channel still works for GETs after tiler restart", function(done) {
-
-      var layergroup =  {
-        version: '1.0.0',
-        layers: [
-           { options: {
-               sql: 'select * from test_table where cartodb_id=1',
-               cartocss: '#layer { marker-fill:red; marker-width:32; marker-allow-overlap:true; }', 
-               cartocss_version: '2.1.0',
-               interactivity: 'cartodb_id'
-             } }
-        ]
-      };
-
-      var expected_token; // = "b4ed64d93a411a59f330ab3d798e4009";
-      Step(
-        function do_post()
-        {
-          var next = this;
-          assert.response(server, {
-              url: '/tiles/layergroup?map_key=1234',
-              method: 'POST',
-              headers: {host: 'localhost', 'Content-Type': 'application/json' },
-              data: JSON.stringify(layergroup)
-          }, {}, function(res, err) { next(err, res); });
-        },
-        function check_post(err, res) {
-          if ( err ) throw err;
-          assert.equal(res.statusCode, 200, res.body);
-          var parsedBody = JSON.parse(res.body);
-          var expectedBody = { layergroupid: expected_token };
-          // check last modified
-          var qTables = JSON.stringify({
-            'q': 'SELECT CDB_QueryTables($windshaft$'
-                + layergroup.layers[0].options.sql
-                + '$windshaft$)'
-          });
-          assert.equal(parsedBody.last_updated, expected_last_updated);
-          if ( expected_token ) {
-            assert.equal(parsedBody.layergroupid, expected_token + ':' + expected_last_updated_epoch);
-          }
-          else expected_token = parsedBody.layergroupid.split(':')[0];
-          return null;
-        },
-        function do_get0(err)
-        {
-          if ( err ) throw err;
-          var next = this;
-          assert.response(server, {
-              url: '/tiles/layergroup/' + expected_token + ':cb0/0/0/0.png?map_key=1234',
-              method: 'GET',
-              headers: {host: 'localhost' },
-              encoding: 'binary'
-          }, {}, function(res, err) { next(err, res); });
-        },
-        function do_check0(err, res) {
-          if ( err ) throw err;
-          assert.equal(res.statusCode, 200, res.body);
-          assert.equal(res.headers['content-type'], "image/png");
-
-          // Check X-Cache-Channel
-          var cc = res.headers['x-cache-channel'];
-          assert.ok(cc, "Missing X-Cache-Channel"); 
-          var dbname = test_database;
-          assert.equal(cc.substring(0, dbname.length), dbname);
-          return null;
-        },
-        function do_restart_server(err, res) {
-          if ( err ) throw err;
-          // hack simulating restart...
-          serverOptions = ServerOptions();
-          server = new CartodbWindshaft(serverOptions);
-          return null;
-        },
-        function do_get1(err)
-        {
-          if ( err ) throw err;
-          var next = this;
-          assert.response(server, {
-              url: '/tiles/layergroup/' + expected_token + ':cb0/0/0/0.png?map_key=1234',
-              method: 'GET',
-              headers: {host: 'localhost' },
-              encoding: 'binary'
-          }, {}, function(res, err) { next(err, res); });
-        },
-        function do_check1(err, res) {
-          if ( err ) throw err;
-          assert.equal(res.statusCode, 200, res.body);
-          assert.equal(res.headers['content-type'], "image/png");
-
-          // Check X-Cache-Channel
-          var cc = res.headers['x-cache-channel'];
-          assert.ok(cc, "Missing X-Cache-Channel on restart");
-          var dbname = test_database;
-          assert.equal(cc.substring(0, dbname.length), dbname);
-          return null;
-        },
-        function finish(err) {
-          var errors = [];
-          if ( err ) {
-            errors.push(err.message);
-            console.log("Error: " + err);
-          }
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
-              if ( err ) errors.push(err.message);
-              assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
-              redis_client.del(matches, function(err) {
-                if ( err ) errors.push(err.message);
-                if ( errors.length ) done(new Error(errors.join(',')));
-                else done(null);
-              });
-          });
-        }
-      );
-    });
-
     // https://github.com/cartodb/Windshaft-cartodb/issues/81
     test("invalid text-name in CartoCSS", function(done) {
 
@@ -927,7 +768,7 @@ suite('multilayer', function() {
             errors.push(err.message);
             console.log("Error: " + err);
           }
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
+          redis_client.keys("map_style|test_cartodb_user_1_db|~" + expected_token, function(err, matches) {
               if ( err ) errors.push(err.message);
               assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
               redis_client.del(matches, function(err) {
@@ -1002,7 +843,7 @@ suite('multilayer', function() {
           if ( err ) errors.push(err.message);
           if ( ! expected_token ) return null;
           var next = this;
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
+          redis_client.keys("map_style|test_cartodb_user_1_db|~" + expected_token, function(err, matches) {
               if ( err ) errors.push(err.message);
               assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
               redis_client.del(matches, function(err) {
@@ -1022,113 +863,6 @@ suite('multilayer', function() {
       );
     });
 
-    // SQL strings can be of arbitrary length, when using POST
-    // See https://github.com/CartoDB/Windshaft-cartodb/issues/111
-    test("sql string can be very long", function(done){
-      var long_val = 'pretty';
-      for (var i=0; i<1024; ++i) long_val += ' long'
-      long_val += ' string';
-      var sql = "SELECT ";
-      for (var i=0; i<16; ++i) 
-        sql += "'" + long_val + "'::text as pretty_long_field_name_" + i + ", ";
-      sql += "cartodb_id, the_geom_webmercator FROM gadm4 g";
-      var layergroup =  {
-        version: '1.0.0',
-        layers: [
-           { options: {
-               sql: sql,
-               cartocss: '#layer { marker-fill:red; }', 
-               cartocss_version: '2.0.1'
-             } }
-        ]
-      };
-      var errors = [];
-      var expected_token; 
-      Step(
-        function do_post()
-        {
-          var data = JSON.stringify(layergroup);
-          assert.ok(data.length > 1024*64);
-          var next = this;
-          assert.response(server, {
-              url: '/tiles/layergroup?api_key=1234',
-              method: 'POST',
-              headers: {host: 'localhost', 'Content-Type': 'application/json' },
-              data: data
-          }, {}, function(res) { next(null, res); });
-        },
-        function check_result(err, res) {
-          if ( err ) throw err;
-          assert.equal(res.statusCode, 200, res.statusCode + ': ' + res.body);
-          var parsedBody = JSON.parse(res.body);
-          var token_components = parsedBody.layergroupid.split(':');
-          expected_token = token_components[0];
-          var last_request = sqlapi_server.getLastRequest();
-          assert.equal(last_request.method, 'POST');
-          return null;
-        },
-        function cleanup(err) {
-          if ( err ) errors.push('' + err);
-          if ( ! expected_token ) return null;
-          var next = this;
-          redis_client.keys("map_cfg|" + expected_token, function(err, matches) {
-              if ( err ) errors.push(err.message);
-              assert.equal(matches.length, 1, "Missing expected token " + expected_token + " from redis: " + matches);
-              redis_client.del(matches, function(err) {
-                if ( err ) errors.push(err.message);
-                next();
-              });
-          });
-        },
-        function finish(err) {
-          if ( err ) errors.push('' + err);
-          if ( errors.length ) done(new Error(errors.join(',')));
-          else done(null);
-        }
-      );
-    });
-
-    // See https://github.com/CartoDB/Windshaft-cartodb/issues/133
-    test("MapConfig with mapnik layer and no cartocss", function(done) {
-
-      var layergroup =  {
-        version: '1.0.0',
-        layers: [
-           { options: {
-               sql: 'select cartodb_id, ST_Translate(the_geom_webmercator, 5e6, 0) as the_geom_webmercator from test_table limit 2',
-               interactivity: 'cartodb_id'
-             } }
-        ]
-      };
-
-      Step(
-        function do_post()
-        {
-          var next = this;
-          assert.response(server, {
-              url: '/tiles/layergroup',
-              method: 'POST',
-              headers: {host: 'localhost', 'Content-Type': 'application/json' },
-              data: JSON.stringify(layergroup)
-          }, {}, function(res, err) { next(err, res); });
-        },
-        function check_post(err, res) {
-          if ( err ) throw err;
-          assert.equal(res.statusCode, 400, res.statusCode + ': ' + res.body);
-          var parsed = JSON.parse(res.body);
-          assert.ok(parsed.errors, 'Missing "errors" in response: ' + JSON.stringify(parsed));
-          assert.equal(parsed.errors.length, 1);
-          var msg = parsed.errors[0];
-          assert.equal(msg, 'Missing cartocss for layer 0 options');
-          return null;
-        },
-        function finish(err) {
-          done(err);
-        }
-      );
-    });
-
-
     suiteTeardown(function(done) {
 
         // This test will add map_style records, like

test/acceptance/server.js

@@ -12,7 +12,7 @@ var SQLAPIEmu   = require(__dirname + '/../support/SQLAPIEmu.js');
 var helper = require(__dirname + '/../support/test_helper');
 
 var CartodbWindshaft = require(__dirname + '/../../lib/cartodb/cartodb_windshaft');
-var serverOptions = require(__dirname + '/../../lib/cartodb/server_options')();
+var serverOptions = require(__dirname + '/../../lib/cartodb/server_options');
 var server = new CartodbWindshaft(serverOptions);
 server.setMaxListeners(0);
 
@@ -22,7 +22,7 @@ suite('server', function() {
     var sqlapi_server;
 
     var mapnik_version = global.environment.mapnik_version || mapnik.versions.mapnik;
-    var test_database = _.template(global.environment.postgres_auth_user, {user_id:1}) + '_db';
+    var test_database = 'test_cartodb_user_1_db';
     var default_style;
     if ( semver.satisfies(mapnik_version, '<2.1.0') ) {
       // 2.0.0 default
@@ -53,25 +53,12 @@ suite('server', function() {
     
     // TODO: I guess this should be a 404 instead...
     test("get call to server returns 200", function(done){
-      Step(
-        function doGet() {
-          var next = this;
-          assert.response(server, {
-              url: '/',
-              method: 'GET'
-          },{}, function(res, err) { next(err,res); });
-        },
-        function doCheck(err, res) {
-          if ( err ) throw err;
-          assert.ok(res.statusCode, 200);
-          var cc = res.headers['x-cache-channel'];
-          assert.ok(!cc);
-          return null;
-        },
-        function finish(err) {
-          done(err);
-        }
-      );
+        assert.response(server, {
+            url: '/',
+            method: 'GET'
+        },{
+            status: 200
+        }, function() { done(); });
     });
 
     /////////////////////////////////////////////////////////////////////////////////
@@ -120,7 +107,7 @@ suite('server', function() {
         }, function(res) {
             var parsed = JSON.parse(res.body);
             assert.equal(parsed.style, _.template(default_style, {table: 'my_table'}));
-            assert.equal(parsed.style_version, mapnik_version);
+            assert.equal(parsed.style_version, mapnik.versions.mapnik);
             done();
         });
     });
@@ -138,7 +125,6 @@ suite('server', function() {
           assert.equal(res.statusCode, 400, res.body);
           assert.deepEqual(JSON.parse(res.body),
             {error: 'Sorry, you are unauthorized (permission denied)'});
-          assert.ok(!res.headers.hasOwnProperty('cache-control'));
           done();
         });
     });
@@ -156,7 +142,6 @@ suite('server', function() {
           assert.equal(res.statusCode, 400, res.statusCode + ': ' + res.body);
           assert.deepEqual(JSON.parse(res.body),
             {error:"missing unknown_user's database_name in redis (try CARTODB/script/restore_redis)"});
-          assert.ok(!res.headers.hasOwnProperty('cache-control'));
           done();
         });
     });
@@ -173,7 +158,7 @@ suite('server', function() {
           var parsed = JSON.parse(res.body);
           var style = _.template(default_style, {table: 'test_table_private_1'});
           assert.equal(parsed.style, style);
-          assert.equal(parsed.style_version, mapnik_version); 
+          assert.equal(parsed.style_version, mapnik.versions.mapnik); 
           done();
         });
     });
@@ -188,8 +173,7 @@ suite('server', function() {
         },
         function setupRedisBase(err, matches) {
           if ( err ) throw err;
-          assert.equal(matches.length, 0,
-            'Unexpected redis keys at test start: ' + matches.join("\n"));
+          assert.equal(matches.length, 0);
           redis_client.set(base_key, 
             JSON.stringify({ style: style }),
           this);
@@ -228,12 +212,9 @@ suite('server', function() {
             url: '/tiles/my_table/style',
             method: 'POST'
         },{
+            status: 400,
             body: '{"error":"must send style information"}'
-        }, function(res) {
-          assert.equal(res.statusCode, 400);
-          assert.ok(!res.headers.hasOwnProperty('cache-control'));
-          done();
-        });
+        }, function() { done(); });
     });
     
     test("post'ing bad style returns 400 with error", function(done){
@@ -370,7 +351,7 @@ suite('server', function() {
                 assert.equal(res.statusCode, 200, res.body);
                 var parsed = JSON.parse(res.body);
                 assert.equal(parsed.style, style);
-                assert.equal(parsed.style_version, mapnik_version);
+                assert.equal(parsed.style_version, mapnik.versions.mapnik);
                 done();
               });
             });
@@ -398,7 +379,7 @@ suite('server', function() {
               var parsed = JSON.parse(res.body);
               // NOTE: no transform expected for the specific style
               assert.equal(parsed.style, style);
-              assert.equal(parsed.style_version, mapnik_version);
+              assert.equal(parsed.style_version, mapnik.versions.mapnik);
               done();
             });
         });
@@ -785,8 +766,6 @@ suite('server', function() {
           assert.equal(res.statusCode, 400, res.statusCode + ': ' + res.body);
           assert.deepEqual(JSON.parse(res.body),
             {error:"missing unknown_user's database_name in redis (try CARTODB/script/restore_redis)"});
-          assert.ok(!res.headers.hasOwnProperty('cache-control'),
-            "Unexpected Cache-Control: " + res.headers['cache-control']);
           done();
         });
     });
@@ -807,9 +786,6 @@ suite('server', function() {
         }, function(res) {
           // 401 Unauthorized
           assert.equal(res.statusCode, 401, res.statusCode + ': ' + res.body);
-          // Failed in 1.6.0 of https://github.com/CartoDB/Windshaft-cartodb/issues/107
-          assert.ok(!res.headers.hasOwnProperty('cache-control'),
-            "Unexpected Cache-Control: " + res.headers['cache-control']);
           done();
         });
     });
@@ -1125,8 +1101,8 @@ suite('server', function() {
             var ct = res.headers['content-type'];
             assert.equal(ct, 'image/png');
             var cc = res.headers['x-cache-channel'];
-            assert(cc, 'Missing X-Cache-Channel');
-            var dbname = test_database;
+            assert(cc);
+            var dbname = 'test_cartodb_user_1_db'
             assert.equal(cc.substring(0, dbname.length), dbname);
             var jsonquery = cc.substring(dbname.length+1);
             var sentquery = JSON.parse(jsonquery);
@@ -1140,38 +1116,6 @@ suite('server', function() {
         );
     });
 
-    test("passes hostname header to sqlapi", function(done){
-        var qo = {
-          sql: "SELECT * from gadm4",
-          map_key: 1234
-        };
-        var sqlapi;
-        Step(
-          function sendRequest(err) {
-            var next = this;
-            assert.response(server, {
-                headers: {host: 'localhost'},
-                url: '/tiles/gadm4/6/31/24.png?' + querystring.stringify(qo),
-                method: 'GET'
-            },{}, function(res) { next(null, res); });
-          },
-          function checkResponse(err, res) {
-            if ( err ) throw err;
-            assert.equal(res.statusCode, 200, res.statusCode + ': ' + res.body);
-            var last_request = sqlapi_server.getLastRequest();
-            assert.ok(last_request);
-            var host = last_request.headers['host'];
-            assert.ok(host);
-            assert.equal(last_request.method, 'GET');
-            assert.equal(host, 'localhost.donot_look_this_up');
-            return null;
-          },
-          function finish(err) {
-            done(err);
-          }
-        );
-    });
-
     test("requests to skip cache on sqlapi error", function(done){
         var qo = {
           sql: "SELECT g.cartodb_id, g.codineprov, t.the_geom_webmercator "
@@ -1222,20 +1166,6 @@ suite('server', function() {
         });
     });
 
-    // See https://github.com/CartoDB/Windshaft-cartodb/issues/115
-    test.skip("get'ing tile with not-strictly-valid style", function(done) {
-        var style = querystring.stringify({style: '#test_table{line-color:black}}', style_version: '2.0.0'});
-        assert.response(server, {
-            headers: {host: 'localhost'},
-            url: '/tiles/test_table/0/0/0.png?' + style, // madrid
-            method: 'GET',
-            encoding: 'binary'
-        },{}, function(res){
-          assert.equal(res.statusCode, 200, res.statusCode + ': ' + res.body);
-          done();
-        });
-    });
-
     /////////////////////////////////////////////////////////////////////////////////
     //
     // DELETE CACHE 
@@ -1249,7 +1179,6 @@ suite('server', function() {
             method: 'DELETE'
         },{}, function(res) {
           assert.equal(res.statusCode, 404, res.statusCode + ': ' + res.body);
-          assert.ok(!res.headers.hasOwnProperty('cache-control'));
           done();
         });
     });
@@ -1281,7 +1210,6 @@ suite('server', function() {
         },{}, function(res) {
           // FIXME: should be 401 instead
           assert.equal(res.statusCode, 500, res.statusCode + ': ' + res.body);
-          assert.ok(!res.headers.hasOwnProperty('cache-control'));
           done();
         });
     });
@@ -1334,7 +1262,6 @@ suite('server', function() {
             method: 'DELETE'
         },{}, function(res) {
           assert.equal(res.statusCode, 404, res.statusCode + ': ' + res.body);
-          assert.ok(!res.headers.hasOwnProperty('cache-control'));
           done();
         });
     });

test/support/SQLAPIEmu.js

@@ -1,80 +1,38 @@
 var http        = require('http');
 var url         = require('url');
-var _ = require('underscore');
 
 var o = function(port, cb) {
 
   this.queries = [];
   var that = this;
-  this.requests = [];
-
   this.sqlapi_server = http.createServer(function(req,res) {
-      //console.log("server got request with method " + req.method);
-      var query;
-
-      that.requests.push(req);
-
-      if ( req.method == 'GET' ) {
-        query = url.parse(req.url, true).query;
-        that.handleQuery(query, res);
-      }
-      else if ( req.method == 'POST') {
-        var data = '';
-        req.on('data', function(chunk) {
-          //console.log("GOT Chunk  " + chunk);
-          data += chunk;
-        });
-        req.on('end', function() {
-          //console.log("Data is: "); console.dir(data);
-          query = JSON.parse(data);
-          //console.log("handleQuery is " + that.handleQuery);
-          that.handleQuery(query, res);
-        });
-      }
-      else {
-        that.handleQuery('SQLAPIEmu does not support method' + req.method, res);
-      }
+        var query = url.parse(req.url, true).query;
+        that.queries.push(query);
+        if ( query.q.match('SQLAPIERROR') ) {
+          res.statusCode = 400;
+          res.write(JSON.stringify({'error':'Some error occurred'}));
+        } else if ( query.q.match('EPOCH.* as max') ) {
+          // This is the structure of the known query sent by tiler
+          var row = {
+            'max': 1234567890.123
+          };
+          res.write(JSON.stringify({rows: [ row ]}));
+        } else {
+          var qs = JSON.stringify(query);
+          var row = {
+            // This is the structure of the known query sent by tiler
+            'cdb_querytables': '{' + qs + '}',
+            'max': qs
+          };
+          res.write(JSON.stringify({rows: [ row ]}));
+        }
+        res.end();
    }).listen(port, cb);
 };
 
-o.prototype.handleQuery = function(query, res) {
-    this.queries.push(query);
-    if ( query.q.match('SQLAPIERROR') ) {
-      res.statusCode = 400;
-      res.write(JSON.stringify({'error':'Some error occurred'}));
-    } else if ( query.q.match('EPOCH.* as max') ) {
-      // This is the structure of the known query sent by tiler
-      var row = {
-        'max': 1234567890.123
-      };
-      res.write(JSON.stringify({rows: [ row ]}));
-    } else {
-      if ( query.q.match('_private_') && query.api_key === undefined) {
-        res.statusCode = 403;
-        res.write(JSON.stringify({'error':'forbidden: ' + JSON.stringify(query)}));
-      } else {
-        var qs = JSON.stringify(query);
-        var row = {
-          // This is the structure of the known query sent by tiler
-          'cdb_querytables': '{' + qs + '}',
-          'max': qs
-        };
-        var out_obj = {rows: [ row ]};
-        var out = JSON.stringify(out_obj);
-        res.write(out);
-      }
-    }
-    res.end();
-  };
-
-
 o.prototype.close = function(cb) {
   this.sqlapi_server.close(cb);
 };
 
-o.prototype.getLastRequest = function() {
-  return this.requests.pop();
-};
-
 module.exports = o;
 

test/support/prepare_db.sh

@@ -83,27 +83,20 @@ fi
 if test x"$PREPARE_REDIS" = xyes; then
 
   echo "preparing redis..."
-
-  cat <<EOF | redis-cli -p ${REDIS_PORT} -n 5
-HMSET rails:users:localhost id ${TESTUSERID} \
-                            database_name '${TEST_DB}' \
-                            map_key 1234
-SADD rails:users:localhost:map_key 1235
-EOF
+  echo "HSET rails:users:localhost id ${TESTUSERID}" | redis-cli -p ${REDIS_PORT} -n 5
+  echo 'HSET rails:users:localhost database_name "'"${TEST_DB}"'"' | redis-cli -p ${REDIS_PORT} -n 5
+  echo "HSET rails:users:localhost map_key 1234" | redis-cli -p ${REDIS_PORT} -n 5
+  echo "SADD rails:users:localhost:map_key 1235" | redis-cli -p ${REDIS_PORT} -n 5
 
   # A user configured as with cartodb-2.5.0+ 
-  cat <<EOF | redis-cli -p ${REDIS_PORT} -n 5
-HMSET rails:users:cartodb250user id ${TESTUSERID} \
-                                 database_name "${TEST_DB}" \
-                                 database_host "localhost" \
-                                 database_password "${TESTPASS}" \
-                                 map_key 4321
-EOF
+  echo "HSET rails:users:cartodb250user id ${TESTUSERID}" | redis-cli -p ${REDIS_PORT} -n 5
+  echo 'HSET rails:users:cartodb250user database_name "'${TEST_DB}'"' | redis-cli -p ${REDIS_PORT} -n 5
+  echo 'HSET rails:users:cartodb250user database_host "localhost"' | redis-cli -p ${REDIS_PORT} -n 5
+  echo 'HSET rails:users:cartodb250user database_password "'${TESTPASS}'"' | redis-cli -p ${REDIS_PORT} -n 5
+  echo "HSET rails:users:cartodb250user map_key 4321" | redis-cli -p ${REDIS_PORT} -n 5
 
-  cat <<EOF | redis-cli -p ${REDIS_PORT} -n 0
-HSET rails:${TEST_DB}:my_table infowindow "this, that, the other"
-HSET rails:${TEST_DB}:test_table_private_1 privacy "0"
-EOF
+  echo 'HSET rails:'"${TEST_DB}"':my_table infowindow "this, that, the other"' | redis-cli -p ${REDIS_PORT} -n 0
+  echo 'HSET rails:'"${TEST_DB}"':test_table_private_1 privacy "0"' | redis-cli -p ${REDIS_PORT} -n 0
 
 fi
 

test/support/sql/windshaft.test.sql

@@ -174,6 +174,5 @@ CREATE TABLE test_table_private_1 (
     CONSTRAINT enforce_srid_the_geom CHECK ((st_srid(the_geom) = 4326)),
     CONSTRAINT enforce_srid_the_geom_webmercator CHECK ((st_srid(the_geom_webmercator) = 3857))
 );
-INSERT INTO test_table_private_1 SELECT * from test_table;
 
 GRANT ALL ON TABLE test_table_private_1 TO :TESTUSER;

test/unit/cartodb/req2params.test.js

@@ -7,12 +7,7 @@ var assert = require('assert')
 suite('req2params', function() {
 
     // configure redis pool instance to use in tests
-    var opts = require('../../../lib/cartodb/server_options')();
-
-    var test_user = _.template(global.environment.postgres_auth_user, {user_id:1});
-    var test_pubuser = global.environment.postgres.user;
-    var test_database = test_user + '_db';
-
+    var opts = require('../../../lib/cartodb/server_options');
     
     test('can be found in server_options', function(){
       assert.ok(_.isFunction(opts.req2params));
@@ -20,45 +15,49 @@ suite('req2params', function() {
 
     test('cleans up request', function(done){
       opts.req2params({headers: { host:'localhost' }, query: {dbuser:'hacker',dbname:'secret'}}, function(err, req) {
-          if ( err ) { done(err); return; }
+          if ( err ) { console.log(err); throw new Error(err); }
           assert.ok(_.isObject(req.query), 'request has query');
           assert.ok(!req.query.hasOwnProperty('dbuser'), 'dbuser was removed from query');
           assert.ok(req.hasOwnProperty('params'), 'request has params');
           assert.ok(req.params.hasOwnProperty('interactivity'), 'request params have interactivity');
-          assert.equal(req.params.dbname, test_database, 'could forge dbname: '+ req.params.dbname);
-          assert.ok(req.params.dbuser === test_pubuser, 'could inject dbuser ('+req.params.dbuser+')');
+          assert.equal(req.params.dbname, 'test_cartodb_user_1_db', 'could forge dbname: '+ req.params.dbname);
+          assert.ok(!req.params.hasOwnProperty('dbuser'), 'could inject dbuser ('+req.params.dbuser+')');
           done();
       });
     });
 
     test('sets dbname from redis metadata', function(done){
       opts.req2params({headers: { host:'localhost' }, query: {} }, function(err, req) {
-          if ( err ) { done(err); return; }
+          if ( err ) { console.log(err); throw new Error(err); }
           //console.dir(req);
           assert.ok(_.isObject(req.query), 'request has query');
           assert.ok(!req.query.hasOwnProperty('dbuser'), 'dbuser was removed from query');
           assert.ok(req.hasOwnProperty('params'), 'request has params');
           assert.ok(req.params.hasOwnProperty('interactivity'), 'request params have interactivity');
-          assert.equal(req.params.dbname, test_database);
-          assert.ok(req.params.dbuser === test_pubuser, 'could inject dbuser ('+req.params.dbuser+')');
+          // database_name for user "localhost" (see test/support/prepare_db.sh)
+          assert.equal(req.params.dbname, 'test_cartodb_user_1_db');
+          // unauthenticated request gets no dbuser
+          assert.ok(!req.params.hasOwnProperty('dbuser'), 'could inject dbuser ('+req.params.dbuser+')');
           done();
       });
     });
 
     test('sets also dbuser for authenticated requests', function(done){
       opts.req2params({headers: { host:'localhost' }, query: {map_key: '1234'} }, function(err, req) {
-          if ( err ) { done(err); return; }
+          if ( err ) { console.log(err); throw new Error(err); }
           //console.dir(req);
           assert.ok(_.isObject(req.query), 'request has query');
           assert.ok(!req.query.hasOwnProperty('dbuser'), 'dbuser was removed from query');
           assert.ok(req.hasOwnProperty('params'), 'request has params');
           assert.ok(req.params.hasOwnProperty('interactivity'), 'request params have interactivity');
-          assert.equal(req.params.dbname, test_database);
-          assert.equal(req.params.dbuser, test_user);
+          // database_name for user "localhost" (see test/support/prepare_db.sh)
+          assert.equal(req.params.dbname, 'test_cartodb_user_1_db');
+          // id for user "localhost" (see test/support/prepare_db.sh)
+          assert.equal(req.params.dbuser, 'test_cartodb_user_1');
  
           opts.req2params({headers: { host:'localhost' }, query: {map_key: '1235'} }, function(err, req) {
               // wrong key resets params to no user
-              assert.ok(req.params.dbuser === test_pubuser, 'could inject dbuser ('+req.params.dbuser+')');
+              assert.ok(!req.params.hasOwnProperty('dbuser'), 'could inject dbuser ('+req.params.dbuser+')');
               done();
           });
       });
@@ -71,8 +70,7 @@ suite('req2params', function() {
         cache_buster: 5
       };
       test_helper.lzma_compress_to_base64(JSON.stringify(qo), 1, function(err, data) {
-        opts.req2params({ headers: { host:'localhost' }, query: { non_included: 'toberemoved', api_key: 'test', style: 'override', lzma: data }}, function(err, req) {
-          if ( err ) { done(err); return; }
+        opts.req2params({ query: { non_included: 'toberemoved', api_key: 'test', style: 'override', lzma: data }}, function(err, req) {
           var query = req.params
           assert.equal(qo.style, query.style)
           assert.equal(qo.style_version, query.style_version)

test/unit/cartodb/signed_maps.test.js

@@ -1,109 +0,0 @@
-var assert = require('assert')
-  //, _ = require('underscore')
-  , RedisPool = require('redis-mpool')
-  , SignedMaps = require('../../../lib/cartodb/signed_maps.js')
-  , test_helper = require('../../support/test_helper')
-  , Step = require('step')
-  , tests = module.exports = {};
-
-suite('signed_maps', function() {
-
-    // configure redis pool instance to use in tests
-    var redis_pool = RedisPool(global.environment.redis);
-    
-    test('can sign map with open and token-based auth', function(done) {
-      var smap = new SignedMaps(redis_pool);
-      assert.ok(smap);
-      var sig = 'sig1';
-      var map = 'map1';
-      var tok = 'tok1';
-      var crt = {
-        version:'0.0.1',
-        layergroup_id:map,
-        auth: {}
-      };
-      var crt1_id; // by token
-      var crt2_id; // open
-      Step(
-        function() {
-          smap.isAuthorized(sig,map,tok,this);
-        },
-        function checkAuthFailure1(err, authorized) {
-          if ( err ) throw err;
-          assert.ok(!authorized, "unexpectedly authorized");
-          crt.auth.method = 'token';
-          crt.auth.valid_tokens = [tok];
-          smap.addSignature(sig, map, crt, this)
-        },
-        function getCert1(err, id) {
-          if ( err ) throw err;
-          assert.ok(id, "undefined signature id");
-          crt1_id = id; // keep note of it
-//console.log("Certificate 1 is " + crt1_id);
-          smap.isAuthorized(sig,map,'',this);
-        },
-        function checkAuthFailure2(err, authorized) {
-          if ( err ) throw err;
-          assert.ok(!authorized, "unexpectedly authorized");
-          smap.isAuthorized(sig,map,tok,this);
-        },
-        function checkAuthSuccess1(err, authorized) {
-          if ( err ) throw err;
-          assert.ok(authorized, "unauthorized :(");
-          crt.auth.method = 'open';
-          delete crt.auth.valid_tokens;
-          smap.addSignature(sig, map, crt, this)
-        },
-        function getCert2(err, id) {
-          if ( err ) throw err;
-          assert.ok(id, "undefined signature id");
-          crt2_id = id; // keep note of it
-//console.log("Certificate 2 is " + crt2_id);
-          smap.isAuthorized(sig,map,'arbitrary',this);
-        },
-        function checkAuthSuccess2_delCert2(err, authorized) {
-          if ( err ) throw err;
-          assert.ok(authorized, "unauthorized :(");
-          var next = this;
-          smap.delCertificate(sig, crt2_id, function(e) {
-            if (e) next(e);
-            else smap.isAuthorized(sig,map,'arbitrary',next);
-          });
-        },
-        function checkAuthFailure3_delCert2(err, authorized) {
-          if ( err ) throw err;
-          assert.ok(!authorized, "unexpectedly authorized");
-          smap.delCertificate(sig, crt1_id, this);
-        },
-        function finish(err) {
-          done(err);
-        }
-      );
-    });
-
-    test('can validate certificates', function(done) {
-      var smap = new SignedMaps(redis_pool);
-      assert.ok(smap);
-      Step(
-        function invalidVersion() {
-          var cert = { version: '-1' };
-          var err = smap.checkInvalidCertificate(cert);
-          assert.ok(err);
-          assert.equal(err.message, "Unsupported certificate version -1");
-          return null;
-        },
-        function invalidTokenAuth() {
-          var cert = { version: '0.0.1', auth: { method:'token', valid_token:[] } };
-          var err = smap.checkInvalidCertificate(cert);
-          assert.ok(err);
-          assert.equal(err.message, "Invalid 'token' authentication: missing valid_tokens");
-          return null;
-        },
-        function finish(err) {
-          done(err);
-        }
-      );
-    });
-
-    
-});

test/unit/cartodb/template_maps.test.js

@@ -1,506 +0,0 @@
-var assert = require('assert')
-  //, _ = require('underscore')
-  , RedisPool = require('redis-mpool')
-  , SignedMaps = require('../../../lib/cartodb/signed_maps.js')
-  , TemplateMaps = require('../../../lib/cartodb/template_maps.js')
-  , test_helper = require('../../support/test_helper')
-  , Step = require('step')
-  , tests = module.exports = {};
-
-suite('template_maps', function() {
-
-  // configure redis pool instance to use in tests
-  var redis_pool = RedisPool(global.environment.redis);
-  var signed_maps = new SignedMaps(redis_pool);
-    
-  test('does not accept template with unsupported version', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var tpl = { version:'6.6.6',
-      name:'k', auth: {}, layergroup: {} };
-    Step(
-      function() {
-        tmap.addTemplate('me', tpl, this);
-      },
-      function checkFailed(err) {
-        assert.ok(err);
-        assert.ok(err.message.match(/unsupported.*version/i), err);
-        return null;
-      },
-      function finish(err) {
-        done(err);
-      }
-    );
-  });
-
-  test('does not accept template with missing name', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var tpl = { version:'0.0.1',
-      auth: {}, layergroup: {} };
-    Step(
-      function() {
-        tmap.addTemplate('me', tpl, this);
-      },
-      function checkFailed(err) {
-        assert.ok(err);
-        assert.ok(err.message.match(/missing.*name/i), err);
-        return null;
-      },
-      function finish(err) {
-        done(err);
-      }
-    );
-  });
-
-  test('does not accept template with invalid name', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var tpl = { version:'0.0.1',
-      auth: {}, layergroup: {} };
-    var invalidnames = [ "ab|", "a b", "a@b", "1ab", "_x", "", " x", "x " ];
-    var testNext = function() {
-      if ( ! invalidnames.length ) { done(); return; }
-      var n = invalidnames.pop();
-      tpl.name = n;
-      tmap.addTemplate('me', tpl, function(err) {
-        if ( ! err ) {
-          done(new Error("Unexpected success with invalid name '" + n + "'"));
-        }
-        else if ( ! err.message.match(/template.*name/i) ) {
-          done(new Error("Unexpected error message with invalid name '" + n
-            + "': " + err));
-        }
-        else {
-          testNext();
-        }
-      });
-    };
-    testNext();
-  });
-
-  test('does not accept template with invalid placeholder name', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var tpl = { version:'0.0.1',
-      name: "valid", placeholders: {},
-      auth: {}, layergroup: {} };
-    var invalidnames = [ "ab|", "a b", "a@b", "1ab", "_x", "", " x", "x " ];
-    var testNext = function() {
-      if ( ! invalidnames.length ) { done(); return; }
-      var n = invalidnames.pop();
-      tpl.placeholders = {};
-      tpl.placeholders[n] = { type:'number', default:1 };
-      tmap.addTemplate('me', tpl, function(err) {
-        if ( ! err ) {
-          done(new Error("Unexpected success with invalid name '" + n + "'"));
-        }
-        else if ( ! err.message.match(/invalid.*name/i) ) {
-          done(new Error("Unexpected error message with invalid name '" + n
-            + "': " + err));
-        }
-        else {
-          testNext();
-        }
-      });
-    };
-    testNext();
-  });
-
-  test('does not accept template with missing placeholder default', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var tpl = { version:'0.0.1',
-      name: "valid", placeholders: { v: {} },
-      auth: {}, layergroup: {} };
-    tmap.addTemplate('me', tpl, function(err) {
-        if ( ! err ) {
-          done(new Error("Unexpected success with missing placeholder default"));
-        }
-        else if ( ! err.message.match(/missing default/i) ) {
-          done(new Error("Unexpected error message with missing placeholder default: "
-            + err));
-        }
-        else {
-          done();
-        }
-      });
-  });
-
-  test('does not accept template with missing placeholder type', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var tpl = { version:'0.0.1',
-      name: "valid", placeholders: { v: { default:1 } },
-      auth: {}, layergroup: {} };
-    tmap.addTemplate('me', tpl, function(err) {
-        if ( ! err ) {
-          done(new Error("Unexpected success with missing placeholder type"));
-        }
-        else if ( ! err.message.match(/missing type/i) ) {
-          done(new Error("Unexpected error message with missing placeholder default: "
-            + err));
-        }
-        else {
-          done();
-        }
-      });
-  });
-
-  // See http://github.com/CartoDB/Windshaft-cartodb/issues/128
-  test('does not accept template with invalid token auth (undefined tokens)',
-  function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var tpl = { version:'0.0.1',
-      name: "invalid_auth1", placeholders: { },
-      auth: { method: 'token' }, layergroup: {} };
-    tmap.addTemplate('me', tpl, function(err) {
-        if ( ! err ) {
-          done(new Error("Unexpected success with invalid token auth (undefined tokens)"));
-        }
-        else if ( ! err.message.match(/invalid 'token' authentication/i) ) {
-          done(new Error("Unexpected error message with invalid token auth (undefined tokens): "
-            + err));
-        }
-        else {
-          done();
-        }
-      });
-  });
-
-  test('add, get and delete a valid template', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var expected_failure = false;
-    var tpl_id;
-    var tpl = { version:'0.0.1',
-      name: 'first', auth: {}, layergroup: {} };
-    Step(
-      function() {
-        tmap.addTemplate('me', tpl, this);
-      },
-      function addOmonimousTemplate(err, id) {
-        if ( err ) throw err;
-        tpl_id = id;
-        assert.equal(tpl_id, 'first');
-        expected_failure = true;
-        // should fail, as it already exists
-        tmap.addTemplate('me', tpl, this);
-      },
-      function getTemplate(err) {
-        if ( ! expected_failure && err ) throw err;
-        assert.ok(err);
-        assert.ok(err.message.match(/already exists/i), err);
-        tmap.getTemplate('me', tpl_id, this);
-      },
-      function delTemplate(err, got_tpl) {
-        if ( err ) throw err;
-        assert.deepEqual(got_tpl, tpl);
-        tmap.delTemplate('me', tpl_id, this);
-      },
-      function finish(err) {
-        done(err);
-      }
-    );
-  });
-
-  test('add multiple templates, list them', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var expected_failure = false;
-    var tpl1 = { version:'0.0.1', name: 'first', auth: {}, layergroup: {} };
-    var tpl1_id;
-    var tpl2 = { version:'0.0.1', name: 'second', auth: {}, layergroup: {} };
-    var tpl2_id;
-    Step(
-      function addTemplate1() {
-        tmap.addTemplate('me', tpl1, this);
-      },
-      function addTemplate2(err, id) {
-        if ( err ) throw err;
-        tpl1_id = id;
-        tmap.addTemplate('me', tpl2, this);
-      },
-      function listTemplates(err, id) {
-        if ( err ) throw err;
-        tpl2_id = id;
-        tmap.listTemplates('me', this);
-      },
-      function checkTemplates(err, ids) {
-        if ( err ) throw err;
-        assert.equal(ids.length, 2);
-        assert.ok(ids.indexOf(tpl1_id) != -1, ids.join(','));
-        assert.ok(ids.indexOf(tpl2_id) != -1, ids.join(','));
-        return null;
-      },
-      function delTemplate1(err) {
-        if ( tpl1_id ) {
-          var next = this;
-          tmap.delTemplate('me', tpl1_id, function(e) {
-            if ( err || e ) next(new Error(err + '; ' + e));
-            else next();
-          });
-        } else {
-          if ( err ) throw err;
-          return null;
-        }
-      },
-      function delTemplate2(err) {
-        if ( tpl2_id ) {
-          var next = this;
-          tmap.delTemplate('me', tpl2_id, function(e) {
-            if ( err || e ) next(new Error(err + '; ' + e));
-            else next();
-          });
-        } else {
-          if ( err ) throw err;
-          return null;
-        }
-      },
-      function finish(err) {
-        done(err);
-      }
-    );
-  });
-
-  test('update templates', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-    var expected_failure = false;
-    var owner = 'me';
-    var tpl = { version:'0.0.1',
-      name: 'first',
-      auth: { method: 'open' },
-      layergroup: {}
-    };
-    var tpl_id;
-    Step(
-      function addTemplate() {
-        tmap.addTemplate(owner, tpl, this);
-      },
-      // Updating template name should fail
-      function updateTemplateName(err, id) {
-        if ( err ) throw err;
-        tpl_id = id;
-        expected_failure = true;
-        tpl.name = 'second';
-        tmap.updTemplate(owner, tpl_id, tpl, this);
-      },
-      function updateTemplateAuth(err) {
-        if ( err && ! expected_failure) throw err;
-        expected_failure = false;
-        assert.ok(err);
-        tpl.name = 'first';
-        tpl.auth.method = 'token';
-        tpl.auth.valid_tokens = [ 'tok1' ];
-        tmap.updTemplate(owner, tpl_id, tpl, this);
-      },
-      function updateTemplateWithInvalid(err) {
-        if ( err ) throw err;
-        tpl.version = '999.999.999';
-        expected_failure = true;
-        tmap.updTemplate(owner, tpl_id, tpl, this);
-      },
-      function updateUnexistentTemplate(err) {
-        if ( err && ! expected_failure) throw err;
-        expected_failure = false;
-        assert.ok(err);
-        assert.ok(err.message.match(/unsupported.*version/i), err);
-        tpl.version = '0.0.1';
-        expected_failure = true;
-        tmap.updTemplate(owner, 'unexistent', tpl, this);
-      },
-      function delTemplate(err) {
-        if ( err && ! expected_failure) throw err;
-        expected_failure = false;
-        assert.ok(err);
-        assert.ok(err.message.match(/cannot update name/i), err);
-        tmap.delTemplate(owner, tpl_id, this);
-      },
-      function finish(err) {
-        done(err);
-      }
-    );
-  });
-
-  test('instanciate templates', function() {
-    var tmap = new TemplateMaps(redis_pool, signed_maps);
-    assert.ok(tmap);
-
-    var tpl1 =  {
-        version: '0.0.1',
-        name: 'acceptance1',
-        auth: { method: 'open' },
-        placeholders: {
-          fill: { type: "css_color", default: "red" },
-          color: { type: "css_color", default: "#a0fF9A" },
-          name: { type: "sql_literal", default: "test" },
-          zoom: { type: "number", default: "0" },
-          test_number: { type: "number", default: 23 },
-        },
-        layergroup: {
-          version: '1.0.0',
-          global_cartocss_version: '2.0.2',
-          layers: [
-             { options: {
-                 sql: "select '<%=name %>' || id, g from t",
-                 cartocss: '#layer { marker-fill:<%= fill %>; marker-width: <%=test_number  %>; }'
-               } },
-             { options: {
-                 sql: "select fun('<%=     name%>') g from x",
-                 cartocss: '#layer { line-color:<%= color %>; marker-fill:<%= color %>; }'
-               } },
-             { options: {
-                 sql: "select g from x",
-                 cartocss: '#layer[zoom=<%=zoom%>] { }'
-               } }
-          ]
-        }
-    };
-
-    var inst = tmap.instance(tpl1, {});
-
-    var lyr = inst.layers[0].options;
-    assert.equal(lyr.sql, "select 'test' || id, g from t");
-    assert.equal(lyr.cartocss, '#layer { marker-fill:red; marker-width: 23; }');
-
-    lyr = inst.layers[1].options;
-    assert.equal(lyr.sql, "select fun('test') g from x");
-    assert.equal(lyr.cartocss, '#layer { line-color:#a0fF9A; marker-fill:#a0fF9A; }');
-
-    inst = tmap.instance(tpl1, {color:'yellow', name:"it's dangerous"});
-
-    lyr = inst.layers[0].options;
-    assert.equal(lyr.sql, "select 'it''s dangerous' || id, g from t");
-    assert.equal(lyr.cartocss, '#layer { marker-fill:red; marker-width: 23; }');
-
-    lyr = inst.layers[1].options;
-    assert.equal(lyr.sql, "select fun('it''s dangerous') g from x");
-    assert.equal(lyr.cartocss, '#layer { line-color:yellow; marker-fill:yellow; }');
-
-    // Invalid css_color
-    var err = null;
-    try { inst = tmap.instance(tpl1, {color:'##ff00ff'}); }
-    catch (e) { err = e; }
-    assert.ok(err);
-    assert.ok(err.message.match(/invalid css_color/i), err);
-
-    // Invalid css_color 2 (too few digits)
-    var err = null;
-    try { inst = tmap.instance(tpl1, {color:'#ff'}); }
-    catch (e) { err = e; }
-    assert.ok(err);
-    assert.ok(err.message.match(/invalid css_color/i), err);
-
-    // Invalid css_color 3 (too many digits)
-    var err = null;
-    try { inst = tmap.instance(tpl1, {color:'#1234567'}); }
-    catch (e) { err = e; }
-    assert.ok(err);
-    assert.ok(err.message.match(/invalid css_color/i), err);
-
-    // Invalid number
-    var err = null;
-    try { inst = tmap.instance(tpl1, {zoom:'#'}); }
-    catch (e) { err = e; }
-    assert.ok(err);
-    assert.ok(err.message.match(/invalid number/i), err);
-
-    // Invalid number 2
-    var err = null;
-    try { inst = tmap.instance(tpl1, {zoom:'23e'}); }
-    catch (e) { err = e; }
-    assert.ok(err);
-    assert.ok(err.message.match(/invalid number/i), err);
-
-    // Valid number
-    var err = null;
-    try { inst = tmap.instance(tpl1, {zoom:'-.23e10'}); }
-    catch (e) { err = e; }
-    assert.ok(!err);
-  });
-
-  // Can set a limit on the number of user templates
-  test('can limit number of user templates', function(done) {
-    var tmap = new TemplateMaps(redis_pool, signed_maps, {
-      max_user_templates: 2
-    });
-    assert.ok(tmap);
-    var tpl = { version:'0.0.1', auth: {}, layergroup: {} };
-    var expectErr = false;
-    var idMe = [];
-    var idYou = [];
-    Step(
-      function oneForMe() {
-        tpl.name = 'oneForMe';
-        tmap.addTemplate('me', tpl, this);
-      },
-      function twoForMe(err, id) {
-        if ( err ) throw err;
-        assert.ok(id);
-        idMe.push(id);
-        tpl.name = 'twoForMe';
-        tmap.addTemplate('me', tpl, this);
-      },
-      function threeForMe(err, id) {
-        if ( err ) throw err;
-        assert.ok(id);
-        idMe.push(id);
-        tpl.name = 'threeForMe';
-        expectErr = true;
-        tmap.addTemplate('me', tpl, this);
-      },
-      function errForMe(err, id) {
-        if ( err && ! expectErr ) throw err;
-        expectErr = false;
-        assert.ok(err);
-        assert.ok(err.message.match(/limit.*template/), err);
-        return null;
-      },
-      function delOneMe(err) {
-        if ( err ) throw err;
-        tmap.delTemplate('me', idMe.shift(), this);
-      },
-      function threeForMeRetry(err) {
-        if ( err ) throw err;
-        tpl.name = 'threeForMe';
-        tmap.addTemplate('me', tpl, this);
-      },
-      function oneForYou(err, id) {
-        if ( err ) throw err;
-        assert.ok(id);
-        idMe.push(id);
-        tpl.name = 'oneForYou';
-        tmap.addTemplate('you', tpl, this);
-      },
-      function twoForYou(err, id) {
-        if ( err ) throw err;
-        assert.ok(id);
-        idYou.push(id);
-        tpl.name = 'twoForYou';
-        tmap.addTemplate('you', tpl, this);
-      },
-      function threeForYou(err, id) {
-        if ( err ) throw err;
-        assert.ok(id);
-        idYou.push(id);
-        tpl.name = 'threeForYou';
-        expectErr = true;
-        tmap.addTemplate('you', tpl, this);
-      },
-      function errForYou(err, id) {
-        if ( err && ! expectErr ) throw err;
-        expectErr = false;
-        assert.ok(err);
-        assert.ok(err.message.match(/limit.*template/), err);
-        return null;
-      },
-      function finish(err) {
-        // TODO: delete all templates
-        done(err);
-      }
-    );
-  });
-    
-});

tools/create_template

@@ -1,49 +0,0 @@
-#!/bin/sh
-
-verbose=no
-tiler_url=http://dev.localhost.lan:8181/tiles/template
-apikey=${CDB_APIKEY}
-
-while test -n "$1"; do
-  if test "$1" = "-v"; then
-    verbose=yes
-  elif test "$1" = "-k"; then
-    shift
-    apikey="$1"
-  elif test "$1" = "-u"; then
-    shift
-    tiler_url="$1"
-  elif test -z "$cfg"; then
-    cfg="$1"
-  else
-    echo "Unused parameter $1" >&2
-  fi
-  shift
-done
-
-if test -z "$cfg"; then
-  echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_config>" >&2
-  echo "Default <tiler_url> is ${tiler_url}" >&2
-  echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
-  exit 1
-fi
-
-cmd="curl -skH Content-Type:application/json --data-binary @- ${tiler_url}?api_key=${apikey}"
-if test x${verbose} = xyes; then
-  cmd="${cmd} -v"
-fi
-res=`cat ${cfg} | tr '\n' ' ' | ${cmd}`
-if test $? -gt 0; then
-  echo "curl command failed: ${cmd}"
-fi
-
-if test x${verbose} = xyes; then
-  echo "${res}"
-fi
-
-# Successful response contains no space
-echo "$res" | grep " " && { echo $res && exit 1; }
-
-tok=`echo "$res" | sed 's/.*"template_id":"\([^"]*\)".*/\1/'`
-echo $tok
-

tools/delete_template

@@ -1,45 +0,0 @@
-#!/bin/sh
-
-verbose=no
-tiler_url=http://dev.localhost.lan:8181/tiles/template
-apikey=${CDB_APIKEY}
-
-while test -n "$1"; do
-  if test "$1" = "-v"; then
-    verbose=yes
-  elif test "$1" = "-k"; then
-    shift
-    apikey="$1"
-  elif test "$1" = "-u"; then
-    shift
-    tiler_url="$1"
-  elif test -z "$tpl"; then
-    tpl="$1"
-  else
-    echo "Unused parameter $1" >&2
-  fi
-  shift
-done
-
-if test -z "$tpl"; then
-  echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_id>" >&2
-  echo "Default <tiler_url> is ${tiler_url}" >&2
-  echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
-  exit 1
-fi
-
-cmd="curl -X DELETE -skH Content-Type:application/json ${tiler_url}/${tpl}?api_key=${apikey}"
-if test x${verbose} = xyes; then
-  cmd="${cmd} -v"
-fi
-res=`${cmd}`
-if test $? -gt 0; then
-  echo "curl command failed: ${cmd}"
-fi
-
-if test x${verbose} = xyes; then
-  echo "${res}"
-fi
-
-tok=`echo "$res" | sed 's/.*"template_id":"\([^"]*\)".*/\1/'`
-echo $tok

tools/examples/mapconfig_simple.js

@@ -1,11 +0,0 @@
-{"version":"1.0.1",
-	"layers":[{
-    "type":"cartodb",
-	  "options":{
-      "sql":"select 1 as id, ST_SetSRID(ST_MakePoint(0,0),3857) as the_geom_webmercator",
-	    "cartocss":"#style{ marker-width: 12;}",
-	    "cartocss_version":"2.1.1",
-      "Interactivity":"id"
-    }
-  }]
-}

tools/examples/template_simple.js

@@ -1,17 +0,0 @@
-{
- "version":"0.0.1",
- "name":"simple",
- "placeholders":{},
- "auth":{ "method":"open" },
- "layergroup":{
-   "version":"1.0.1",
-    "layers":[{
-      "type":"cartodb",
-      "options":{
-        "sql":"select ST_SetSRID(ST_MakePoint(0,0),3857) as the_geom_webmercator",
-        "cartocss":"#s{ marker-width: 12;}",
-        "cartocss_version":"2.1.1"
-      }
-    }]
-  }
-}

tools/flush_cache

@@ -1,59 +0,0 @@
-#!/usr/bin/env node
-
-var path = require('path');
-var request    = require('request');
-
-function usage(me, exitcode) {
-  console.log("Usage: " + me + " [--env <environment>] <username> <tablename>");
-  process.exit(exitcode);
-}
-
-var node_path = process.argv.shift(); 
-var script_path = process.argv.shift(); 
-var basedir = path.dirname(script_path);
-var me = path.basename(script_path);
-
-var ENV = 'development.js';
-var username, table;
-var arg;
-while ( arg = process.argv.shift() ) {
-  if ( arg == '--env' ) {
-    ENV = process.argv.shift();
-  }
-  else if ( ! username ) {
-    username = arg;
-  }
-  else if ( ! table ) {
-    table = arg;
-  }
-  else {
-    console.warn("Unused parameter " + arg);
-  }
-}
-
-if ( ! table ) {
-  usage(me, 1);
-}
-
-global.environment = require('../config/environments/' + ENV);
-
-// _after_ setting global.environment
-var serverOptions = require('../lib/cartodb/server_options');
-
-var host = global.environment.host;
-var port = global.environment.port;
-var re = ''+serverOptions.re_userFromHost;
-var hostname = re.replace(/^\/\^/, '')
-                 .replace(/\/$$/, '')
-                 .replace(/\\/g,'')
-                 .replace(/\([^)]*\)/,username)
-              ;
-//console.log("re: " + re);
-//console.log("hostname: " + hostname);
-
-var url = 'http://' + host + ':' + port + '/tiles/' + table + '/flush_cache';
-request.del({ url: url, headers: { host: hostname } },
-function(err, res, body) {
-  if ( err ) throw err;
-  console.log(res.body);
-});

tools/instanciate_template

@@ -1,53 +0,0 @@
-#!/bin/sh
-
-verbose=no
-tiler_url=http://dev.localhost.lan:8181/tiles/template
-apikey=${CDB_APIKEY}
-
-while test -n "$1"; do
-  if test "$1" = "-v"; then
-    verbose=yes
-  elif test "$1" = "-k"; then
-    shift
-    apikey="$1"
-  elif test "$1" = "-u"; then
-    shift
-    tiler_url="$1"
-  elif test -z "$tpl"; then
-    tpl="$1"
-  elif test -z "$cfg"; then
-    cfg="$1"
-  else
-    echo "Unused parameter $1" >&2
-  fi
-  shift
-done
-
-if test -z "$tpl"; then
-  echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_id> [<template_params>]" >&2
-  echo "Default <tiler_url> is ${tiler_url}" >&2
-  echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
-  exit 1
-fi
-
-if test -z "$cfg"; then
-  cfg="/dev/null"
-fi
-
-tiler_url="${tiler_url}/${tpl}"
-
-cmd="curl -X POST -skH Content-Type:application/json --data-binary @- ${tiler_url}?api_key=${apikey}"
-if test x${verbose} = xyes; then
-  cmd="${cmd} -v"
-fi
-res=`cat ${cfg} | tr '\n' ' ' | ${cmd}`
-if test $? -gt 0; then
-  echo "curl command failed: ${cmd}"
-fi
-
-if test x${verbose} = xyes; then
-  echo "${res}"
-fi
-
-tok=`echo "$res" | sed 's/.*"layergroupid":"\([^"]*\)".*/\1/'`
-echo $tok

tools/list_templates

@@ -1,45 +0,0 @@
-#!/bin/sh
-
-verbose=no
-tiler_url=http://dev.localhost.lan:8181/tiles/template
-apikey=${CDB_APIKEY}
-
-while test -n "$1"; do
-  if test "$1" = "-v"; then
-    verbose=yes
-  elif test "$1" = "-k"; then
-    shift
-    apikey="$1"
-  elif test "$1" = "-u"; then
-    shift
-    tiler_url="$1"
-  elif test "$1" = "-h" -o "$1" = "-?"; then
-    echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>]" >&2
-    echo "Default <tiler_url> is ${tiler_url}" >&2
-    echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
-    exit 0
-  else
-    echo "Unused parameter $1" >&2
-  fi
-  shift
-done
-
-
-cmd="curl -X GET -sk ${tiler_url}?api_key=${apikey}"
-if test x${verbose} = xyes; then
-  cmd="${cmd} -v"
-fi
-res=`${cmd}`
-if test $? -gt 0; then
-  echo "curl command failed: ${cmd}"
-fi
-
-if test x${verbose} = xyes; then
-  echo "${res}"
-fi
-
-node <<EOF
-var parsed = JSON.parse('$res');
-console.dir(parsed);
-EOF
-

tools/munin/windshaft

@@ -68,7 +68,7 @@ for pid in ${pids}; do
   log=$(grep "${pid}" "${tmpreport}" | grep -w 1w | awk '{print $9}')
   if test -e "${log}"; then
     kill -USR2 "${pid}"
-    cnt=$(tac ${log} | sed -n -e '/ItemKey/p;/ RenderCache /q' | wc -l)
+    cnt=$(tac ${log} | sed -n -e '/ItemKey/p;/^RenderCache/q' | wc -l)
     if test $cnt -gt $maxcache; then maxcache=$cnt; fi
   else
     # report the error...

tools/performance/stresstest_templates.sh

@@ -1,54 +0,0 @@
-#!/bin/sh
-
-verbose=no
-tiler_url=http://dev.localhost.lan:8181/tiles/template
-apikey=${CDB_APIKEY}
-
-while test -n "$1"; do
-  if test "$1" = "-k"; then
-    shift
-    apikey="$1"
-  elif test "$1" = "-u"; then
-    shift
-    tiler_url="$1"
-  elif test -z "$tpl"; then
-    tpl="$1"
-  else
-    echo "Unused parameter $1" >&2
-  fi
-  shift
-done
-
-if test -z "$tpl"; then
-  echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_config>" >&2
-  echo "Default <tiler_url> is ${tiler_url}" >&2
-  echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
-  exit 1
-fi
-
-basedir=$(cd $(dirname $0); cd ..; pwd)
-export CDB_APIKEY=${apikey}
-max=3000000
-i=0
-while test "$i" -le "$max"; do
-  tpln=`cat ${tpl} | sed "s/\"name\":\"\(.*\)\"/\"name\":\"\1${i}\"/"`
-  tpl_id=`echo ${tpln} | ${basedir}/create_template -u ${tiler_url} /dev/stdin`
-  if test $? -ne 0; then
-    echo $tpl_id >&2
-    break
-  fi
-  tpl_id=`echo ${tpln} | ${basedir}/update_template -u ${tiler_url} ${tpl_id} /dev/stdin`
-  if test $? -ne 0; then
-    echo $tpl_id >&2
-    break
-  fi
-  out=`${basedir}/delete_template -u ${tiler_url} ${tpl_id}`
-  if test $? -ne 0; then
-    echo $out >&2
-    break
-  fi
-  i=$((i+1))
-  if test `expr $i % 100` -eq 0; then
-    echo -n "."
-  fi
-done

tools/show_style

@@ -1,147 +1,29 @@
-#!/usr/bin/env node
+#!/bin/sh
 
-var path = require('path');
-var redis = require('redis');
-var Step = require('step');
+# TODO: port to node, if you really need it
+
+REDIS_PORT=6379 # default port
 
 
-function usage(me, exitcode) {
-  console.log("Usage: " + me + " [--env <environment>] <username> [<tablename>|~<token>]");
-  process.exit(exitcode);
-}
+if test -z "$1"; then
+  echo "Usage: $0 <username> [<tablename>|~<token>]" >&2
+  exit 1
+fi
 
-var node_path = process.argv.shift(); 
-var script_path = process.argv.shift(); 
-var basedir = path.dirname(script_path);
-var me = path.basename(script_path);
-
-var ENV = 'development.js';
-var username, token;
-var arg;
-while ( arg = process.argv.shift() ) {
-  if ( arg == '--env' ) {
-    ENV = process.argv.shift();
-  }
-  else if ( ! username ) {
-    username = arg;
-  }
-  else if ( ! token ) {
-    token = arg;
-  }
-  else {
-    console.warn("Unused parameter " + arg);
-  }
-}
-
-if ( ! username ) usage(me, 1);
-
-console.log("Using environment " + ENV);
-
-global.environment = require('../config/environments/' + ENV);
-var serverOptions = require('../lib/cartodb/server_options'); // _after_ setting global.environment
-
-var client;
-var dbname;
-Step(
-  function getClient() {
-    client = redis.createClient(serverOptions.redis.port, serverOptions.redis.host);
-    client.on('connect', this);
-  },
-  function getUserMeta(err) {
-    if ( err ) throw err;
-    client.select(5);
-    client.hgetall('rails:users:' + username, this);
-  },
-  function readDB(err, data) {
-    if ( err ) throw err;
-    if ( ! data )
-      throw new Error('Username ' + username + ' unknown by redis on port '
-        + serverOptions.redis.port + ' (try CARTODB/script/restore_redis?)');
-    //console.log("Data:"); console.dir(data);
-    dbname = data['database_name'];
-    console.log("Database name for user " + username + ": " + dbname);
-    client.select(0);
-    return null;
-  },
-  function showTokens(err) {
-    if ( err ) throw err;
-    if ( token ) return null;
-    var next = this;
-    Step(
-      function getTokens() {
-        client.keys('map_style|' + dbname + '|*', this);
-      },
-      function showTokens(err, data) {
-        if (err) throw err;
-        if ( data ) console.log(data.join('\n'));
-        return null;
-      },
-      function showTokensFinish(err) {
-        next(err);
-      }
-    );
-  },
-  function showStyle(err) {
-    if ( err ) throw err;
-    if ( ! token ) return null;
-    var next = this;
-    Step(
-      function getStyle() {
-        client.get('map_style|' + dbname + '|' + token, this);
-      },
-      function showStyle(err, data) {
-        if ( err ) throw err;
-        if ( ! data ) {
-          throw new Error(token + ': no such map style known by redis on port '
-            + serverOptions.redis.port);
-        }
-//console.log("data: " + data);
-        var x=JSON.parse(data);
-        printMapnikStyle(x, this);
-      },
-      function showStyleFinish(err) {
-        next(err);
-      }
-    );
-  },
-  function finish(err) {
-    if ( err ) {
-      console.error(err.message)
-      process.exit(1);
-    }
-    process.exit(0);
-  }
-);
-
-function printMapnikStyle(x, callback) {
-  console.log('style: ' + x.style);
-  console.log('version: ' + x.version);
-  var grainstore = require(basedir + '/../node_modules/windshaft/node_modules/grainstore/lib/grainstore');
-  var mml_store = new grainstore.MMLStore(serverOptions.redis, serverOptions.grainstore);
-  var builderconfig = {dbname:dbname};
-  if ( token.match(/^~/) ) {
-    builderconfig.token = token.substring(1);
-  } else {
-    builderconfig.table = token;
-  }
-  var mml_builder;
-  Step(
-    function getBuilder() {
-      mml_builder = mml_store.mml_builder(builderconfig, this);
-    },
-    function getXML(err, builder) {
-      if ( err ) throw err;
-      mml_builder.toXML(this);
-    },
-    function showXML(err, xml) {
-      if ( err ) throw err;
-      console.log('- XML - ');
-      console.log(xml);
-      return null;
-    },
-    function finish(err) {
-      callback(err);
-    }
-  );
-}
+username="$1"
+token="$2"
 
+dbname=`redis-cli -p ${REDIS_PORT} -n 5 hget "rails:users:${username}" "database_name"`
+if test $? -ne 0; then
+  exit 1
+fi
+if test -z "${dbname}"; then
+  echo "Username ${username} unknown by redis (try CARTODB/script/restore_redis?)" >&2
+  exit 1
+fi
+echo "Database name for user ${username}: ${dbname}" # only if verbose?
+if test -n "$token"; then
+  redis-cli get "map_style|${dbname}|${token}" | sed -e 's/\\n/\n/g' -e 's/\\//g'
+else
+  redis-cli keys "map_style|${dbname}|*"
+fi

tools/update_template

@@ -1,47 +0,0 @@
-#!/bin/sh
-
-verbose=no
-tiler_url=http://dev.localhost.lan:8181/tiles/template
-apikey=${CDB_APIKEY}
-
-while test -n "$1"; do
-  if test "$1" = "-v"; then
-    verbose=yes
-  elif test "$1" = "-k"; then
-    shift
-    apikey="$1"
-  elif test "$1" = "-u"; then
-    shift
-    tiler_url="$1"
-  elif test -z "$tpl"; then
-    tpl="$1"
-  elif test -z "$cfg"; then
-    cfg="$1"
-  else
-    echo "Unused parameter $1" >&2
-  fi
-  shift
-done
-
-if test -z "$cfg"; then
-  echo "Usage: $0 [-v] [-k <api_key>] [-u <tiler_url>] <template_id> <template_config>" >&2
-  echo "Default <tiler_url> is ${tiler_url}" >&2
-  echo "Default <api_key> is read from CDB_APIKEY env variable" >&2
-  exit 1
-fi
-
-cmd="curl -X PUT -skH Content-Type:application/json --data-binary @- ${tiler_url}/${tpl}?api_key=${apikey}"
-if test x${verbose} = xyes; then
-  cmd="${cmd} -v"
-fi
-res=`cat ${cfg} | tr '\n' ' ' | ${cmd}`
-if test $? -gt 0; then
-  echo "curl command failed: ${cmd}"
-fi
-
-if test x${verbose} = xyes; then
-  echo "${res}"
-fi
-
-tok=`echo "$res" | sed 's/.*"template_id":"\([^"]*\)".*/\1/'`
-echo $tok