Release 7.2.0

Merge pull request #1125 from CartoDB/update_querytables
Update cartodb-query-tables to 0.6.2
2019-09-30 17:24:36 +02:00 · 2019-09-19 15:08:00 +02:00 · 2019-09-19 13:40:31 +02:00 · 2019-09-18 18:21:38 +02:00 · 2019-09-18 11:26:51 +02:00 · 2019-09-17 13:20:08 +02:00
44 changed files with 1714 additions and 1244 deletions
--- a/NEWS.md
+++ b/NEWS.md
@@ -1,5 +1,24 @@
 # Changelog

+## 7.2.0
+Released 2019-09-30
+
+Announcements:
+
+- Stop caching map template errors in Named Map Provider Cache
+- Gather metrics from Named Maps Providers Cache
+- Improved efficiency of query samples while instatiating a map (#1120).
+- Cache control header fine tuning. Set a shorter value for "max-age" directive if there is no way to know when to trigger the invalidation.
+- Update deps:
+  - Update `cartodb-query-tables` to version [`0.6.3`](https://github.com/CartoDB/node-cartodb-query-tables/blob/0.6.3/NEWS.md#version-063).
+  - Update `cartodb-psql` to [`0.14.0`](https://github.com/CartoDB/node-cartodb-psql/blob/0.14.0/NEWS.md#version-0140-2019-09-10)
+  - Upgrade `windshaft` to [`5.6.3`](https://github.com/CartoDB/Windshaft/blob/master/NEWS.md#version-563):
+    - Upgrade grainstore to [`2.0.1`](https://github.com/CartoDB/grainstore/releases/tag/2.0.1)
+    - Update @carto/mapnik to [`3.6.2-carto.16`](https://github.com/CartoDB/node-mapnik/blob/v3.6.2-carto.16/CHANGELOG.carto.md#362-carto16).
+    - Update turbo-carto to [`0.21.2`](https://github.com/CartoDB/turbo-carto/releases/tag/0.21.2)
+    - Upgrade `@carto/cartonik` to version [`0.7.0`](https://github.com/CartoDB/cartonik/blob/v0.7.0/CHANGELOG.md#cartonik-changelog).
+  - Upgrade `camshaft` to [`0.64.2`](https://github.com/CartoDB/camshaft/blob/8b89fcff276da20a71269bed28b7ad6704392898/CHANGELOG.md#0642) to update dependencies.
+
 ## 7.1.0
 Released 2019-05-06

--- a/carto-package.json
+++ b/carto-package.json
@@ -4,7 +4,7 @@
        "requires": {
            "node": "^10.15.1",
            "npm": "^6.4.1",
-            "mapnik": "==3.0.15.9",
+            "mapnik": "==3.0.15.16",
            "crankshaft": "~0.8.1"
        },
        "works_with": {
--- a/config/environments/development.js.example
+++ b/config/environments/development.js.example
@@ -326,6 +326,7 @@ var config = {
        purge_enabled: false, // whether the purge/invalidation mechanism is enabled in varnish or not
        secret: 'xxx',
        ttl: 86400,
+        fallbackTtl: 300,
        layergroupTtl: 86400 // the max-age for cache-control header in layergroup responses
    }
    // this [OPTIONAL] configuration enables invalidating by surrogate key in fastly
@@ -360,7 +361,7 @@ var config = {
    // Use this as a feature flags enabling/disabling mechanism
    ,enabledFeatures: {
        // whether it should intercept tile render errors an act based on them, enabled by default.
-        onTileErrorStrategy: true,
+        onTileErrorStrategy: false,
        // whether the affected tables for a given SQL must query directly postgresql or use the SQL API
        cdbQueryTablesFromPostgres: true,
        // whether in mapconfig is available stats & metadata for each layer
--- a/config/environments/production.js.example
+++ b/config/environments/production.js.example
@@ -326,6 +326,7 @@ var config = {
        purge_enabled: false, // whether the purge/invalidation mechanism is enabled in varnish or not
        secret: 'xxx',
        ttl: 86400,
+        fallbackTtl: 300,
        layergroupTtl: 86400 // the max-age for cache-control header in layergroup responses
    }
    // this [OPTIONAL] configuration enables invalidating by surrogate key in fastly
@@ -360,7 +361,7 @@ var config = {
    // Use this as a feature flags enabling/disabling mechanism
    ,enabledFeatures: {
        // whether it should intercept tile render errors an act based on them, enabled by default.
-        onTileErrorStrategy: true,
+        onTileErrorStrategy: false,
        // whether the affected tables for a given SQL must query directly postgresql or use the SQL API
        cdbQueryTablesFromPostgres: true,
        // whether in mapconfig is available stats & metadata for each layer
--- a/config/environments/staging.js.example
+++ b/config/environments/staging.js.example
@@ -326,6 +326,7 @@ var config = {
        purge_enabled: false, // whether the purge/invalidation mechanism is enabled in varnish or not
        secret: 'xxx',
        ttl: 86400,
+        fallbackTtl: 300,
        layergroupTtl: 86400 // the max-age for cache-control header in layergroup responses
    }
    // this [OPTIONAL] configuration enables invalidating by surrogate key in fastly
@@ -360,7 +361,7 @@ var config = {
    // Use this as a feature flags enabling/disabling mechanism
    ,enabledFeatures: {
        // whether it should intercept tile render errors an act based on them, enabled by default.
-        onTileErrorStrategy: true,
+        onTileErrorStrategy: false,
        // whether the affected tables for a given SQL must query directly postgresql or use the SQL API
        cdbQueryTablesFromPostgres: true,
        // whether in mapconfig is available stats & metadata for each layer
--- a/config/environments/test.js.example
+++ b/config/environments/test.js.example
@@ -328,6 +328,7 @@ var config = {
        purge_enabled: false, // whether the purge/invalidation mechanism is enabled in varnish or not
        secret: 'xxx',
        ttl: 86400,
+        fallbackTtl: 300,
        layergroupTtl: 86400 // the max-age for cache-control header in layergroup responses
    }
    // this [OPTIONAL] configuration enables invalidating by surrogate key in fastly
@@ -362,7 +363,7 @@ var config = {
    // Use this as a feature flags enabling/disabling mechanism
    ,enabledFeatures: {
        // whether it should intercept tile render errors an act based on them, enabled by default.
-        onTileErrorStrategy: true,
+        onTileErrorStrategy: false,
        // whether the affected tables for a given SQL must query directly postgresql or use the SQL API
        cdbQueryTablesFromPostgres: true,
        // whether in mapconfig is available stats & metadata for each layer
--- a/docs/reference/swagger.yaml
+++ b/docs/reference/swagger.yaml
@@ -90,10 +90,26 @@ paths:
      x-code-samples:
        - lang: Curl
          source: |
-            curl -X POST -H "Content-Type: application/json" -d '{ \
-              "q": "SELECT count(*) FROM cities", \
-              "filename": "number_of_cities.json" \
-            }' "https://username.carto.com/api/v2/sql"
+            # body.json
+            {
+              "version": "1.3.0",
+              "layers": [
+                {
+                  "type": "mapnik",
+                  "options": {
+                    "cartocss_version": "2.1.1",
+                    "cartocss": "#layer { polygon-fill: #FFF; }",
+                    "sql": "select * from european_countries_e",
+                    "interactivity": [
+                      "cartodb_id",
+                      "iso3"
+                    ]
+                  }
+                }
+              ]
+            }
+
+            curl -X POST -H "Content-Type: application/json" -d @body.json "https://username.carto.com/api/v2/sql"
  '/map/{layergroupid}/{z}/{x}/{y}.png':
    get:
      parameters:
@@ -277,57 +293,60 @@ paths:
      x-code-samples:
        - lang: Curl
          source: |
-            curl -X POST -H "Content-Type: application/json" -d '{ \
-              "version": "0.0.1", \
-              "name": "template_name", \
-              "auth": { \
-                "method": "token", \
-                "valid_tokens": [ \
-                  "auth_token1", \
-                  "auth_token2" \
-                ] \
-              }, \
-              "placeholders": { \
-                "color": { \
-                  "type": "css_color", \
-                  "default": "red" \
-                }, \
-                "cartodb_id": { \
-                  "type": "number", \
-                  "default": 1 \
-                } \
-              }, \
-              "layergroup": { \
-                "version": "1.7.0", \
-                "layers": [ \
-                  { \
-                    "type": "cartodb", \
-                    "options": { \
-                      "cartocss_version": "2.3.0", \
-                      "cartocss": "#layer { polygon-fill: <%= color %>; }", \
-                      "sql": "select * from european_countries_e WHERE cartodb_id = <%= cartodb_id %>" \
-                    } \
-                  } \
-                ] \
-              }, \
-              "view": { \
-                "zoom": 4, \
-                "center": { \
-                  "lng": 0, \
-                  "lat": 0 \
-                }, \
-                "bounds": { \
-                  "west": -45, \
-                  "south": -45, \
-                  "east": 45, \
-                  "north": 45 \
-                }, \
-                "preview_layers": { \
-                  "0": true, \
-                  "layer1": false \
-                } \
-              } \
-            }' "https://{username}.carto.com/api/v1/map/named?api_key={api_key}"
+            # body.json
+            {
+              "version": "0.0.1",
+              "name": "template_name",
+              "auth": {
+                "method": "token",
+                "valid_tokens": [
+                  "auth_token1",
+                  "auth_token2"
+                ]
+              },
+              "placeholders": {
+                "color": {
+                  "type": "css_color",
+                  "default": "red"
+                },
+                "cartodb_id": {
+                  "type": "number",
+                  "default": 1
+                }
+              },
+              "layergroup": {
+                "version": "1.7.0",
+                "layers": [
+                  {
+                    "type": "cartodb",
+                    "options": {
+                      "cartocss_version": "2.3.0",
+                      "cartocss": "#layer { polygon-fill: <%= color %>; }",
+                      "sql": "select * from european_countries_e WHERE cartodb_id = <%= cartodb_id %>"
+                    }
+                  }
+                ]
+              },
+              "view": {
+                "zoom": 4,
+                "center": {
+                  "lng": 0,
+                  "lat": 0
+                },
+                "bounds": {
+                  "west": -45,
+                  "south": -45,
+                  "east": 45,
+                  "north": 45
+                },
+                "preview_layers": {
+                  "0": true,
+                  "layer1": false
+                }
+              }
+            }
+
+            curl -X POST -H "Content-Type: application/json" -d @body.json "https://{username}.carto.com/api/v1/map/named?api_key={api_key}"
    get:
      summary: List user's templates
      description: |
@@ -424,59 +443,62 @@ paths:
      x-code-samples:
        - lang: Curl
          source: |
+            # body.json
+            {
+              "version": "0.0.1",
+              "name": "template_name",
+              "auth": {
+                "method": "token",
+                "valid_tokens": [
+                  "auth_token1",
+                  "auth_token2"
+                ]
+              },
+              "placeholders": {
+                "color": {
+                  "type": "css_color",
+                  "default": "red"
+                },
+                "cartodb_id": {
+                  "type": "number",
+                  "default": 1
+                }
+              },
+              "layergroup": {
+                "version": "1.7.0",
+                "layers": [
+                  {
+                    "type": "cartodb",
+                    "options": {
+                      "cartocss_version": "2.3.0",
+                      "cartocss": "#layer { polygon-fill: <%= color %>; }",
+                      "sql": "select * from european_countries_e WHERE cartodb_id = <%= cartodb_id %>"
+                    }
+                  }
+                ]
+              },
+              "view": {
+                "zoom": 4,
+                "center": {
+                  "lng": 0,
+                  "lat": 0
+                },
+                "bounds": {
+                  "west": -45,
+                  "south": -45,
+                  "east": 45,
+                  "north": 45
+                },
+                "preview_layers": {
+                  "0": true,
+                  "layer1": false
+                }
+              }
+            }
+
            curl -X PUT \
              -H 'Content-Type: application/json' \
-              -d '{ \
-                "version": "0.0.1", \
-                "name": "template_name", \
-                "auth": { \
-                  "method": "token", \
-                  "valid_tokens": [ \
-                    "auth_token1", \
-                    "auth_token2" \
-                  ] \
-                }, \
-                "placeholders": { \
-                  "color": { \
-                    "type": "css_color", \
-                    "default": "red" \
-                  }, \
-                  "cartodb_id": { \
-                    "type": "number", \
-                    "default": 1 \
-                  } \
-                }, \
-                "layergroup": { \
-                  "version": "1.7.0", \
-                  "layers": [ \
-                    { \
-                      "type": "cartodb", \
-                      "options": { \
-                        "cartocss_version": "2.3.0", \
-                        "cartocss": "#layer { polygon-fill: <%= color %>; }", \
-                        "sql": "select * from european_countries_e WHERE cartodb_id = <%= cartodb_id %>" \
-                      } \
-                    } \
-                  ] \
-                }, \
-                "view": { \
-                  "zoom": 4, \
-                  "center": { \
-                    "lng": 0, \
-                    "lat": 0 \
-                  }, \
-                  "bounds": { \
-                    "west": -45, \
-                    "south": -45, \
-                    "east": 45, \
-                    "north": 45 \
-                  }, \
-                  "preview_layers": { \
-                    "0": true, \
-                    "layer1": false \
-                  } \
-                } \
-              }' \
+              -d @body.json
              'https://{username}.carto.com/api/v1/map/named/{template_name}?api_key={api_key}'
    delete:
      summary: Delete template
@@ -548,12 +570,15 @@ paths:
      x-code-samples:
        - lang: Curl
          source: |
+            # body.json
+            {
+              "color": "#ff0000",
+              "cartodb_id": 3
+            }
+
            curl -X POST \
              -H 'Content-Type: application/json' \
-              -d '{ \
-                "color": "#ff0000", \
-                "cartodb_id": 3 \
-              }' \
+              -d @body.json
              'https://{username}.carto.com/api/v1/map/named/{template_name}?auth_token={auth_token}'
  '/map/named/{template_name}/jsonp':
    get:
--- a/docs/support/03-rate-limiting.md
+++ b/docs/support/03-rate-limiting.md
@@ -79,7 +79,7 @@ Below, you can find the values of the rate limit by user account type and endpoi
 | GET  /api/v1/map/named/{template_id}/{layer}/{z}/{x}/{y}.{format}  |25  |1  |25  |


-#### Professional plans
+#### Individual plans

 |Endpoint   |Request   |Time period  |Burst  |
 | :---         |          ---: |          ---: |          ---: |
--- a/docs/support/04-timeout-limiting.md
+++ b/docs/support/04-timeout-limiting.md
@@ -25,8 +25,8 @@ You are able to avoid common issues that trigger timeout limits following these

 ### Timeout Limits Chart

-Below, you can find the values of the timeout limit by user account type. 
+Below, you can find the values of the timeout limit by user account type.

-|Enterprise plans  |Professional plans  |Free plans  |
+|Enterprise plans  |Individual plans  |Free plans  |
 | ---  | ---  | ---  |
 | 25 seconds  | 15 seconds  | 5 seconds  |
--- a/docs/support/05-quota-limiting.md
+++ b/docs/support/05-quota-limiting.md
@@ -6,9 +6,9 @@ Maps API is affected by this kind of limiting.

 ### Quota Limits Chart

-Below, you can find the values of the different quota limits by user account type. 
+Below, you can find the values of the different quota limits by user account type.

-|Limit   |Enterprise plans   |Professional plans  |Free plans  |
+|Limit   |Enterprise plans   |Individual plans  |Free plans  |
 | :---         |          ---: |          ---: |          ---: |
 | Maximum Static Map image size |4000 X 4000 pixels  |4000 X 4000 pixels  |4000 X 4000 pixels  |
 | Maximum number of Named Maps |4096  |4096  |4096 |
--- a/lib/cartodb/api/api-router.js
+++ b/lib/cartodb/api/api-router.js
@@ -29,6 +29,7 @@ const VarnishHttpCacheBackend = require('../cache/backend/varnish_http');
 const FastlyCacheBackend = require('../cache/backend/fastly');
 const NamedMapProviderCache = require('../cache/named_map_provider_cache');
 const NamedMapsCacheEntry = require('../cache/model/named_maps_entry');
+const NamedMapProviderReporter = require('../stats/reporter/named-map-provider');

 const SqlWrapMapConfigAdapter = require('../models/mapconfig/adapter/sql-wrap-mapconfig-adapter');
 const MapConfigNamedLayersAdapter = require('../models/mapconfig/adapter/mapconfig-named-layers-adapter');
@@ -100,7 +101,9 @@ module.exports = class ApiRouter {

        const tileBackend = new windshaft.backend.Tile(rendererCache);
        const attributesBackend = new windshaft.backend.Attributes();
-        const previewBackend = new windshaft.backend.Preview(rendererCache);
+        const concurrency = serverOptions.renderer.mapnik.poolSize +
+                            serverOptions.renderer.mapnik.poolMaxWaitingClients;
+        const previewBackend = new windshaft.backend.Preview(rendererCache, { concurrency });
        const mapValidatorBackend = new windshaft.backend.MapValidator(tileBackend, attributesBackend);
        const mapBackend = new windshaft.backend.Map(rendererCache, mapStore, mapValidatorBackend);

@@ -159,10 +162,13 @@ module.exports = class ApiRouter {
            layergroupAffectedTablesCache
        );

-        ['update', 'delete'].forEach(function(eventType) {
-            templateMaps.on(eventType, namedMapProviderCache.invalidate.bind(namedMapProviderCache));
+        const namedMapProviderReporter = new NamedMapProviderReporter({
+            namedMapProviderCache,
+            intervalInMilliseconds: rendererCacheOpts.statsInterval
        });

+        namedMapProviderReporter.start();
+
        const collaborators = {
            analysisStatusBackend,
            attributesBackend,
--- a/lib/cartodb/api/map/preview-layergroup-controller.js
+++ b/lib/cartodb/api/map/preview-layergroup-controller.js
@@ -96,9 +96,10 @@ function getPreviewImageByCenter (previewBackend) {
        };

        const format = req.params.format === 'jpg' ? 'jpeg' : 'png';
-        const { mapConfigProvider: provider } = res.locals;
+        const { mapConfigProvider } = res.locals;
+        const options = { mapConfigProvider, format, width, height, zoom, center };

-        previewBackend.getImage(provider, format, width, height, zoom, center, (err, image, headers, stats = {}) => {
+        previewBackend.getImage(options, (err, image, stats = {}) => {
            req.profiler.done(`render-${format}`);
            req.profiler.add(stats);

@@ -107,11 +108,7 @@ function getPreviewImageByCenter (previewBackend) {
                return next(err);
            }

-            if (headers) {
-                res.set(headers);
-            }
-
-            res.set('Content-Type', headers['Content-Type'] || `image/${format}`);
+            res.set('Content-Type', `image/${format}`);

            res.statusCode = 200;
            res.body = image;
@@ -125,16 +122,17 @@ function getPreviewImageByBoundingBox (previewBackend) {
    return function getPreviewImageByBoundingBoxMiddleware (req, res, next) {
        const width = +req.params.width;
        const height = +req.params.height;
-        const bounds = {
+        const bbox = {
            west: +req.params.west,
            north: +req.params.north,
            east: +req.params.east,
            south: +req.params.south
        };
        const format = req.params.format === 'jpg' ? 'jpeg' : 'png';
-        const { mapConfigProvider: provider } = res.locals;
+        const { mapConfigProvider } = res.locals;
+        const options = { mapConfigProvider, format, width, height, bbox };

-        previewBackend.getImage(provider, format, width, height, bounds, (err, image, headers, stats = {}) => {
+        previewBackend.getImage(options, (err, image, stats = {}) => {
            req.profiler.done(`render-${format}`);
            req.profiler.add(stats);

@@ -143,11 +141,7 @@ function getPreviewImageByBoundingBox (previewBackend) {
                return next(err);
            }

-            if (headers) {
-                res.set(headers);
-            }
-
-            res.set('Content-Type', headers['Content-Type'] || `image/${format}`);
+            res.set('Content-Type', `image/${format}`);

            res.statusCode = 200;
            res.body = image;
--- a/lib/cartodb/api/map/preview-template-controller.js
+++ b/lib/cartodb/api/map/preview-template-controller.js
@@ -215,7 +215,6 @@ function getImageOptionsFromCoordinates (zoom, lon, lat) {
    }
 }

-
 function getImageOptionsFromTemplate (template, zoom) {
    if (template.view) {
        var zoomCenter = templateZoomCenter(template.view);
@@ -239,7 +238,7 @@ function getImageOptionsFromBoundingBox (bbox = '') {

    if (_bbox.length === 4 && _bbox.every(Number.isFinite)) {
        return {
-            bounds: {
+            bbox: {
                west: _bbox[0],
                south: _bbox[1],
                east: _bbox[2],
@@ -252,7 +251,7 @@ function getImageOptionsFromBoundingBox (bbox = '') {
 function getImage({ previewBackend, label }) {
    return function getImageMiddleware (req, res, next) {
        const { imageOpts, mapConfigProvider } = res.locals;
-        const { zoom, center, bounds } = imageOpts;
+        const { zoom, center, bbox } = imageOpts;

        let { width, height } = req.params;

@@ -262,8 +261,9 @@ function getImage({ previewBackend, label }) {
        const format = req.params.format === 'jpg' ? 'jpeg' : 'png';

        if (zoom !== undefined && center) {
-            return previewBackend.getImage(mapConfigProvider, format, width, height, zoom, center,
-                (err, image, headers, stats) => {
+            const options = { mapConfigProvider, format, width, height, zoom, center };
+
+            return previewBackend.getImage(options, (err, image, stats) => {
                req.profiler.add(stats);

                if (err) {
@@ -271,10 +271,6 @@ function getImage({ previewBackend, label }) {
                    return next(err);
                }

-                if (headers) {
-                    res.set(headers);
-                }
-
                res.statusCode = 200;
                res.body = image;

@@ -282,7 +278,9 @@ function getImage({ previewBackend, label }) {
            });
        }

-        previewBackend.getImage(mapConfigProvider, format, width, height, bounds, (err, image, headers, stats) => {
+        const options = { mapConfigProvider, format, width, height, bbox };
+
+        previewBackend.getImage(options, (err, image, stats) => {
            req.profiler.add(stats);
            req.profiler.done('render-' + format);

@@ -291,10 +289,6 @@ function getImage({ previewBackend, label }) {
                return next(err);
            }

-            if (headers) {
-                res.set(headers);
-            }
-
            res.statusCode = 200;
            res.body = image;

@@ -305,7 +299,9 @@ function getImage({ previewBackend, label }) {

 function setContentTypeHeader () {
    return function setContentTypeHeaderMiddleware(req, res, next) {
-        res.set('Content-Type', res.get('content-type') || res.get('Content-Type') || 'image/png');
+        const format = req.params.format === 'jpg' ? 'jpeg' : 'png';
+
+        res.set('Content-Type', `image/${format}`);

        next();
    };
@@ -354,7 +350,7 @@ function templateBounds(view) {

        if (hasAllBounds) {
            return {
-                bounds: {
+                bbox: {
                    west: view.bounds.west,
                    south: view.bounds.south,
                    east: view.bounds.east,
--- a/lib/cartodb/api/middlewares/cache-control-header.js
+++ b/lib/cartodb/api/middlewares/cache-control-header.js
@@ -1,21 +1,51 @@
 'use strict';

 const ONE_YEAR_IN_SECONDS = 60 * 60 * 24 * 365;
+const FIVE_MINUTES_IN_SECONDS = 60 * 5;
+const FALLBACK_TTL = global.environment.varnish.fallbackTtl || FIVE_MINUTES_IN_SECONDS;

-module.exports = function setCacheControlHeader ({ ttl = ONE_YEAR_IN_SECONDS, revalidate = false } = {}) {
+module.exports = function setCacheControlHeader ({
+    ttl = ONE_YEAR_IN_SECONDS,
+    fallbackTtl = FALLBACK_TTL,
+    revalidate = false
+} = {}) {
    return function setCacheControlHeaderMiddleware (req, res, next) {
        if (req.method !== 'GET') {
            return next();
        }

-        const directives = [ 'public', `max-age=${ttl}` ];
+        const { mapConfigProvider = { getAffectedTables: callback => callback() } } = res.locals;

-        if (revalidate) {
-            directives.push('must-revalidate');
-        }
+        mapConfigProvider.getAffectedTables((err, affectedTables) => {
+            if (err) {
+                global.logger.warn('ERROR generating Cache Control Header:', err);
+                return next();
+            }

-        res.set('Cache-Control', directives.join(','));
+            const directives = [ 'public' ];

-        next();
+            if (everyAffectedTableCanBeInvalidated(affectedTables)) {
+                directives.push(`max-age=${ttl}`);
+            } else {
+                directives.push(`max-age=${fallbackTtl}`);
+            }
+
+            if (revalidate) {
+                directives.push('must-revalidate');
+            }
+
+            res.set('Cache-Control', directives.join(','));
+
+            next();
+        });
    };
 };
+
+function everyAffectedTableCanBeInvalidated (affectedTables) {
+    const skipNotUpdatedAtTables = false;
+    const skipAnalysisCachedTables = true;
+
+    return affectedTables &&
+        affectedTables.getTables(skipNotUpdatedAtTables, skipAnalysisCachedTables)
+            .every(table => table.updated_at !== null);
+}
--- a/lib/cartodb/backends/cluster.js
+++ b/lib/cartodb/backends/cluster.js
@@ -5,6 +5,9 @@ const dbParamsFromReqParams = require('../utils/database-params');
 const debug = require('debug')('backend:cluster');
 const AggregationMapConfig = require('../models/aggregation/aggregation-mapconfig');

+const WebMercatorHelper = require('cartodb-query-tables').utils.webMercatorHelper;
+const webmercator = new WebMercatorHelper();
+
 module.exports = class ClusterBackend {
    getClusterFeatures (mapConfigProvider, params, callback) {
        mapConfigProvider.getMapConfig((err, _mapConfig) => {
@@ -127,7 +130,7 @@ function getClusterFeatures (pg, zoom, clusterId, columns, query, resolution, ag
    } , true); // use read-only transaction
 }

-const schemaQuery = ctx => `SELECT * FROM (${ctx.query}) __cdb_schema LIMIT 0`;
+const schemaQuery = ctx => `SELECT * FROM (${ctx.query}) __cdb_cluster_schema LIMIT 0`;
 const clusterFeaturesQuery = ctx => `
    WITH
    _cdb_params AS (
@@ -156,9 +159,8 @@ const clusterFeaturesQuery = ctx => `
 `;

 const gridResolution = ctx => {
-    const minimumResolution = 2*Math.PI*6378137/Math.pow(2,38);
-    const pixelSize = `CDB_XYZ_Resolution(${ctx.zoom})`;
-    return `GREATEST(${256/ctx.res}*${pixelSize}, ${minimumResolution})::double precision`;
+    const zoomResolution = webmercator.getResolution({ z : Math.min(38, ctx.zoom) });
+    return `${256/ctx.res} * (${zoomResolution})::double precision`;
 };

 const aggregationQuery = ctx => `
--- a/lib/cartodb/backends/layer-stats/mapnik-layer-stats.js
+++ b/lib/cartodb/backends/layer-stats/mapnik-layer-stats.js
@@ -36,7 +36,7 @@ function _getSQL(ctx, query, type='pre', zoom=0) {
    else {
        sql = ctx.aggrQuery;
    }
-    sql = queryUtils.subsituteTokensForZoom(sql, zoom || 0);
+    sql = queryUtils.substituteTokensForZoom(sql, zoom || 0);
    return query(sql);
 }

@@ -125,22 +125,43 @@ function mergeColumns(results) {
    }
 }

-const SAMPLE_SEED = 0.5;
 const DEFAULT_SAMPLE_ROWS = 100;

-function _sample(ctx, numRows) {
-    if (ctx.metaOptions.sample) {
-        const sampleProb = Math.min(ctx.metaOptions.sample.num_rows / numRows, 1);
-        // We'll use a safety limit just in case numRows is a bad estimate
-        const requestedRows = ctx.metaOptions.sample.num_rows || DEFAULT_SAMPLE_ROWS;
-        const limit = Math.ceil(requestedRows * 1.5);
-        let columns = ctx.metaOptions.sample.include_columns;
-        return queryUtils.queryPromise(ctx.dbConnection, _getSQL(
-            ctx,
-            sql => queryUtils.getQuerySample(sql, sampleProb, limit, SAMPLE_SEED, columns)
-        )).then(res => ({ sample: res.rows }));
+function _sample(ctx) {
+    if (!ctx.metaOptions.sample) {
+        return Promise.resolve();
    }
-    return Promise.resolve();
+
+    const limit = ctx.metaOptions.sample.num_rows || DEFAULT_SAMPLE_ROWS;
+    const columns = ctx.metaOptions.sample.include_columns;
+
+    const sqlMaxMin = _getSQL(ctx, sql => queryUtils.getMaxMinSpanColumnQuery(sql));
+    return queryUtils.queryPromise(ctx.dbConnection, sqlMaxMin)
+        .then(maxMinRes => {
+            const { min_id: min, id_span: span } = maxMinRes.rows[0];
+
+            if (!min || !span) {
+                return { rows: {} };
+            }
+
+            const values = _getSampleValuesFromRange(min, span, limit);
+            const sqlSample = _getSQL(ctx, sql => queryUtils.getSampleFromIdsQuery(sql, values, columns));
+
+            return queryUtils.queryPromise(ctx.dbConnection, sqlSample);
+        })
+        .then(res => ({ sample: res.rows }));
+}
+
+function _getSampleValuesFromRange (min, span, limit) {
+    const sample = new Set();
+
+    limit = limit < span ? limit : span;
+
+    while (sample.size < limit) {
+        sample.add(Math.floor(min + Math.random() * span));
+    }
+
+    return Array.from(sample);
 }

 function _columnsMetadataRequired(options) {
@@ -294,8 +315,8 @@ function (layer, dbConnection, callback) {

    Promise.all([
        _estimatedFeatureCount(ctx).then(
-            ({ estimatedFeatureCount }) => _sample(ctx, estimatedFeatureCount)
-                .then(sampleResults => mergeResults([sampleResults, { estimatedFeatureCount }]))
+            ({ estimatedFeatureCount }) => _sample(ctx)
+                .then(sampleResults => mergeResults([ sampleResults, { estimatedFeatureCount }] ))
        ),
        _featureCount(ctx),
        _aggrFeatureCount(ctx),
--- a/lib/cartodb/backends/tables-extent.js
+++ b/lib/cartodb/backends/tables-extent.js
@@ -41,7 +41,7 @@ TablesExtentBackend.prototype.getBounds = function (username, tables, callback)
        var result = null;
        if (rows.length > 0) {
            result = {
-                bounds: rows[0]
+                bbox: rows[0]
            };
        }
        callback(null, result);
--- a/lib/cartodb/cache/named_map_provider_cache.js
+++ b/lib/cartodb/cache/named_map_provider_cache.js
@@ -1,39 +1,43 @@
 'use strict';

-var _ = require('underscore');
-var dot = require('dot');
-var NamedMapMapConfigProvider = require('../models/mapconfig/provider/named-map-provider');
-var templateName = require('../backends/template_maps').templateName;
-var queue = require('queue-async');
+const LruCache = require('lru-cache');

-var LruCache = require("lru-cache");
+const NamedMapMapConfigProvider = require('../models/mapconfig/provider/named-map-provider');
+const { templateName } = require('../backends/template_maps');

-function NamedMapProviderCache(
-    templateMaps,
-    pgConnection,
-    metadataBackend,
-    userLimitsBackend,
-    mapConfigAdapter,
-    affectedTablesCache
-) {
-    this.templateMaps = templateMaps;
-    this.pgConnection = pgConnection;
-    this.metadataBackend = metadataBackend;
-    this.userLimitsBackend = userLimitsBackend;
-    this.mapConfigAdapter = mapConfigAdapter;
-    this.affectedTablesCache = affectedTablesCache;
+const TEN_MINUTES_IN_MILLISECONDS = 1000 * 60 * 10;
+const ACTIONS = ['update', 'delete'];

-    this.providerCache = new LruCache({ max: 2000 });
-}
+module.exports = class NamedMapProviderCache {
+    constructor (
+        templateMaps,
+        pgConnection,
+        metadataBackend,
+        userLimitsBackend,
+        mapConfigAdapter,
+        affectedTablesCache
+    ) {
+        this.templateMaps = templateMaps;
+        this.pgConnection = pgConnection;
+        this.metadataBackend = metadataBackend;
+        this.userLimitsBackend = userLimitsBackend;
+        this.mapConfigAdapter = mapConfigAdapter;
+        this.affectedTablesCache = affectedTablesCache;

-module.exports = NamedMapProviderCache;
+        this.providerCache = new LruCache({ max: 2000, maxAge: TEN_MINUTES_IN_MILLISECONDS });

-NamedMapProviderCache.prototype.get = function(user, templateId, config, authToken, params, callback) {
-    var namedMapKey = createNamedMapKey(user, templateId);
-    var namedMapProviders = this.providerCache.get(namedMapKey) || {};
+        ACTIONS.forEach(action => templateMaps.on(action, (user, templateId) => this.invalidate(user, templateId)));
+    }
+
+    get (user, templateId, config, authToken, params, callback) {
+        const namedMapKey = createNamedMapKey(user, templateId);
+        const namedMapProviders = this.providerCache.get(namedMapKey) || {};
+        const providerKey = createProviderKey(config, authToken, params);
+
+        if (namedMapProviders.hasOwnProperty(providerKey)) {
+            return callback(null, namedMapProviders[providerKey]);
+        }

-    var providerKey = createProviderKey(config, authToken, params);
-    if (!namedMapProviders.hasOwnProperty(providerKey)) {
        namedMapProviders[providerKey] = new NamedMapMapConfigProvider(
            this.templateMaps,
            this.pgConnection,
@@ -47,51 +51,32 @@ NamedMapProviderCache.prototype.get = function(user, templateId, config, authTok
            authToken,
            params
        );
+
        this.providerCache.set(namedMapKey, namedMapProviders);

-        // early exit, if provider did not exist we just return it
        return callback(null, namedMapProviders[providerKey]);
    }

-    var namedMapProvider = namedMapProviders[providerKey];
-
-    var self = this;
-    queue(2)
-        .defer(namedMapProvider.getTemplate.bind(namedMapProvider))
-        .defer(this.templateMaps.getTemplate.bind(this.templateMaps), user, templateId)
-        .awaitAll(function templatesQueueDone(err, results) {
-            if (err) {
-                return callback(err);
-            }
-
-            // We want to reset provider its template has changed
-            // Ideally this should be done in a passive mode where this cache gets notified of template changes
-            var uniqueFingerprints = _.uniq(results.map(self.templateMaps.fingerPrint)).length;
-            if (uniqueFingerprints > 1) {
-                namedMapProvider.reset();
-            }
-            return callback(null, namedMapProvider);
-        });
+    invalidate (user, templateId) {
+        this.providerCache.del(createNamedMapKey(user, templateId));
+    }
 };

-NamedMapProviderCache.prototype.invalidate = function(user, templateId) {
-    this.providerCache.del(createNamedMapKey(user, templateId));
-};
-
-function createNamedMapKey(user, templateId) {
-    return user + ':' + templateName(templateId);
+function createNamedMapKey (user, templateId) {
+    return `${user}:${templateName(templateId)}`;
 }

-var providerKey = '{{=it.authToken}}:{{=it.configHash}}:{{=it.format}}:{{=it.layer}}:{{=it.scale_factor}}';
-var providerKeyTpl = dot.template(providerKey);
+const providerKeyTpl = ctx => `${ctx.authToken}:${ctx.configHash}:${ctx.format}:${ctx.layer}:${ctx.scale_factor}`;

-function createProviderKey(config, authToken, params) {
-    var tplValues = _.defaults({}, params, {
+function createProviderKey (config, authToken, params) {
+    const defaults = {
        authToken: authToken || '',
        configHash: NamedMapMapConfigProvider.configHash(config),
        layer: '',
        format: '',
        scale_factor: 1
-    });
-    return providerKeyTpl(tplValues);
+    };
+    const ctx = Object.assign({}, defaults, params);
+
+    return providerKeyTpl(ctx);
 }
--- a/lib/cartodb/models/aggregation/aggregation-mapconfig.js
+++ b/lib/cartodb/models/aggregation/aggregation-mapconfig.js
@@ -201,7 +201,7 @@ module.exports = class AggregationMapConfig extends MapConfig {

    getLayerColumns (index, skipGeoms, callback) {
        const geomColumns = ['the_geom', 'the_geom_webmercator'];
-        const limitedQuery = ctx => `SELECT * FROM (${ctx.query}) __cdb_schema LIMIT 0`;
+        const limitedQuery = ctx => `SELECT * FROM (${ctx.query}) __cdb_aggregation_schema LIMIT 0`;
        const layer = this.getLayer(index);

        this.pgConnection.getConnection(this.user, (err, connection) => {
--- a/lib/cartodb/models/aggregation/aggregation-query.js
+++ b/lib/cartodb/models/aggregation/aggregation-query.js
@@ -3,31 +3,8 @@
 const timeDimension = require('./time-dimension');

 const DEFAULT_PLACEMENT = 'point-sample';
-
-
-/**
- * Returns a template function (function that accepts template parameters and returns a string)
- * to generate an aggregation query.
- * Valid options to define the query template are:
- * - placement
- * - columns
- * - dimensions*
- * The query template parameters taken by the result template function are:
- * - sourceQuery
- * - res
- * - columns
- * - dimensions
- */
-const templateForOptions = (options) => {
-    let templateFn = defaultAggregationQueryTemplate;
-    if (!options.isDefaultAggregation) {
-        templateFn = aggregationQueryTemplates[options.placement || DEFAULT_PLACEMENT];
-        if (!templateFn) {
-            throw new Error("Invalid Aggregation placement: '" + options.placement + "'");
-        }
-    }
-    return templateFn;
-};
+const WebMercatorHelper = require('cartodb-query-tables').utils.webMercatorHelper;
+const webmercator = new WebMercatorHelper();

 function optionsToParams (options) {
    return {
@@ -35,7 +12,9 @@ function optionsToParams (options) {
        res: 256/options.resolution,
        columns: options.columns,
        dimensions: options.dimensions,
-        filters: options.filters
+        filters: options.filters,
+        placement: options.placement || DEFAULT_PLACEMENT,
+        isDefaultAggregation: options.isDefaultAggregation
    };
 }

@@ -53,7 +32,7 @@ function optionsToParams (options) {
 * When placement, columns or dimensions are specified, columns are aggregated as requested
 * (by default only _cdb_feature_count) and with the_geom_webmercator as defined by placement.
 */
-const queryForOptions = (options) => templateForOptions(options)(optionsToParams(options));
+const queryForOptions = (options) => aggregationQueryTemplate(optionsToParams(options));

 module.exports = queryForOptions;

@@ -87,7 +66,7 @@ const SUPPORTED_AGGREGATE_FUNCTIONS = {
        sql: (column_name, params) => `max(${params.aggregated_column || column_name})`
    },
    'mode': {
-        sql: (column_name, params) => `_cdb_mode(${params.aggregated_column || column_name})`
+        sql: (column_name, params) => `mode() WITHIN GROUP (ORDER BY ${params.aggregated_column || column_name})`
    }
 };

@@ -106,16 +85,6 @@ const aggregateColumns = ctx => {
    }, ctx.columns || {});
 };

-const aggregateColumnNames = (ctx, table) => {
-    let columns = aggregateColumns(ctx);
-    if (table) {
-        return sep(Object.keys(columns).map(
-            column_name => `${table}.${column_name}`
-        ));
-    }
-    return sep(Object.keys(columns));
-};
-
 const aggregateExpression = (column_name, column_parameters) => {
    const aggregate_function = column_parameters.aggregate_function || 'count';
    const aggregate_definition = SUPPORTED_AGGREGATE_FUNCTIONS[aggregate_function];
@@ -297,156 +266,158 @@ const havingClause = ctx => {
 // (i.e. each tile is divided into ctx.res*ctx.res cells).
 // We limit the the minimum resolution to avoid division by zero problems. The limit used is
 // the pixel size of zoom level 30 (i.e. 1/2*(30+8) of the full earth web-mercator extent), which is about 0.15 mm.
-// Computing this using !scale_denominator!, !pixel_width! or !pixel_height! produces
-// inaccurate results due to rounding present in those values.
+//
+// NOTE: We'd rather use !pixel_width!, but in Mapnik this value is extent / 256 for raster
+// and extent / tile_extent {4096 default} for MVT, so since aggregations are always based
+// on 256 we can't have the same query in both cases
+// As this scale change doesn't happen in !scale_denominator! we use that instead
+// NOTE 2: The 0.00028 is used in Mapnik (and replicated in pg-mvt) and comes from
+// OGC's Styled Layer Descriptor Implementation Specification
 const gridResolution = ctx => {
-    const minimumResolution = 2*Math.PI*6378137/Math.pow(2,38);
-    const pixelSize = 'CDB_XYZ_Resolution(CDB_ZoomFromScale(!scale_denominator!))';
-    return `GREATEST(${256/ctx.res}*${pixelSize}, ${minimumResolution})::double precision`;
+    const minimumResolution = webmercator.getResolution({ z : 38 });
+    return `${256/ctx.res} * GREATEST(!scale_denominator! * 0.00028, ${minimumResolution})::double precision`;
 };

-// Each aggregation cell is defined by the cell coordinates Floor(x/res), Floor(y/res),
-// i.e. they include the West and South borders but not the East and North ones.
-// So, to avoid picking points that don't belong to cells in the tile, given the tile
-// limits Xmin, Ymin, Xmax, Ymax (bbox), we should select points that satisfy
-// Xmin <= x < Xmax and Ymin <= y < Ymax (with x, y from the_geom_webmercator)
-// On the other hand we can efficiently filter spatially (relying on spatial indexing)
-// with `the_geom_webmercator && bbox` which is equivalent to
-//   Xmin <= x <= Xmax and Ymin <= y <= Ymax
-// So, in order to be both efficient and accurate we will need to use both
-// conditions for spatial filtering.
-const spatialFilter = `
-  (_cdb_query.the_geom_webmercator && _cdb_params.bbox) AND
-    ST_X(_cdb_query.the_geom_webmercator) >= _cdb_params.xmin AND
-    ST_X(_cdb_query.the_geom_webmercator) < _cdb_params.xmax AND
-    ST_Y(_cdb_query.the_geom_webmercator) >= _cdb_params.ymin AND
-    ST_Y(_cdb_query.the_geom_webmercator) < _cdb_params.ymax
-`;
-
-// Notes:
-// * We need to filter spatially using !bbox! to make the queries efficient because
-//   the filter added by Mapnik (wrapping the query)
-//   is only applied after the aggregation.
-// * This queries are used for rendering and the_geom is omitted in the results for better performance
-// * If the MVT extent or tile buffer was 0 or a multiple of the resolution we could use directly
-//   the bbox for them, but in general we need to find the nearest cell limits inside the bbox.
-// * bbox coordinates can have an error in the last digits; we apply a small correction before
-//   applying CEIL or FLOOR to compensate for this, so that coordinates closer than a small (`eps`)
-//   fraction of the cell size to a cell limit are moved to the exact limit.
-const sqlParams = (ctx) => `
-_cdb_res AS (
+// SQL query to extract the boundaries of the area to be aggregated and the grid resolution
+// cdb_{x-y}{min_max} return the limits of the tile. Aggregations do [min, max) in both axis
+// cdb_res: Aggregation resolution (as specified by gridResolution)
+// cdb_point_bbox: Tile bounding box [min, max]
+const gridInfoQuery = ctx => {
+    return `
    SELECT
-    ${gridResolution(ctx)} AS res,
-    !bbox! AS bbox,
-    (1E-6::double precision) AS eps
-),
-_cdb_params AS (
-    SELECT
-      res,
-      bbox,
-      CEIL((ST_XMIN(bbox) - eps*res)/res)*res AS xmin,
-      FLOOR((ST_XMAX(bbox) + eps*res)/res)*res AS xmax,
-      CEIL((ST_YMIN(bbox) - eps*res)/res)*res AS ymin,
-      FLOOR((ST_YMAX(bbox) + eps*res)/res)*res AS ymax
-      FROM _cdb_res
-)
-`;
-
-// The special default aggregation includes all the columns of a sample row per grid cell and
-// the count (_cdb_feature_count) of the aggregated rows.
-const defaultAggregationQueryTemplate = ctx => `
-    WITH ${sqlParams(ctx)},
-    _cdb_clusters AS (
-        SELECT
-            MIN(cartodb_id) AS cartodb_id
-            ${dimensionDefs(ctx)}
-            ${aggregateColumnDefs(ctx)}
-        FROM (${ctx.sourceQuery}) _cdb_query, _cdb_params
-        WHERE ${spatialFilter}
-        GROUP BY
-            Floor(ST_X(_cdb_query.the_geom_webmercator)/_cdb_params.res),
-            Floor(ST_Y(_cdb_query.the_geom_webmercator)/_cdb_params.res)
-            ${dimensionNames(ctx)}
-    ) SELECT
-        _cdb_query.*
-        ${aggregateColumnNames(ctx)}
+        cdb_xmin,
+        cdb_ymin,
+        cdb_xmax,
+        cdb_ymax,
+        cdb_res,
+        ST_MakeEnvelope(cdb_xmin, cdb_ymin, cdb_xmax, cdb_ymax, 3857) AS cdb_point_bbox
    FROM
-        _cdb_clusters INNER JOIN (${ctx.sourceQuery}) _cdb_query
-        ON (_cdb_clusters.cartodb_id = _cdb_query.cartodb_id)
-`;
-
-const aggregationQueryTemplates = {
-    'centroid': ctx => `
-        WITH ${sqlParams(ctx)}
+    (
        SELECT
-            MIN(_cdb_query.cartodb_id)  AS cartodb_id,
-            ST_SetSRID(
-                ST_MakePoint(
-                    AVG(ST_X(_cdb_query.the_geom_webmercator)),
-                    AVG(ST_Y(_cdb_query.the_geom_webmercator))
-                ), 3857
-            ) AS the_geom_webmercator
-            ${dimensionDefs(ctx)}
-            ${aggregateColumnDefs(ctx)}
-        FROM (${ctx.sourceQuery}) _cdb_query, _cdb_params
-        WHERE ${spatialFilter}
-        GROUP BY
-            Floor(ST_X(_cdb_query.the_geom_webmercator)/_cdb_params.res),
-            Floor(ST_Y(_cdb_query.the_geom_webmercator)/_cdb_params.res)
-            ${dimensionNames(ctx)}
-        ${havingClause(ctx)}
-    `,
-
-    'point-grid': ctx => `
-        WITH ${sqlParams(ctx)},
-        _cdb_clusters AS (
-            SELECT
-            MIN(_cdb_query.cartodb_id) AS cartodb_id,
-            Floor(ST_X(_cdb_query.the_geom_webmercator)/_cdb_params.res)::int AS _cdb_gx,
-            Floor(ST_Y(_cdb_query.the_geom_webmercator)/_cdb_params.res)::int AS _cdb_gy
-            ${dimensionDefs(ctx)}
-            ${aggregateColumnDefs(ctx)}
-            FROM (${ctx.sourceQuery}) _cdb_query, _cdb_params
-            WHERE ${spatialFilter}
-            GROUP BY _cdb_gx, _cdb_gy ${dimensionNames(ctx)}
-            ${havingClause(ctx)}
-        )
-        SELECT
-            _cdb_clusters.cartodb_id AS cartodb_id,
-            ST_SetSRID(ST_MakePoint((_cdb_gx+0.5)*res, (_cdb_gy+0.5)*res), 3857) AS the_geom_webmercator
-            ${dimensionNames(ctx)}
-            ${aggregateColumnNames(ctx)}
-            FROM _cdb_clusters, _cdb_params
-    `,
-
-    'point-sample': ctx => `
-        WITH ${sqlParams(ctx)},
-        _cdb_clusters AS (
-            SELECT
-                MIN(cartodb_id) AS cartodb_id
-                ${dimensionDefs(ctx)}
-                ${aggregateColumnDefs(ctx)}
-            FROM (${ctx.sourceQuery}) _cdb_query, _cdb_params
-            WHERE ${spatialFilter}
-            GROUP BY
-                Floor(ST_X(_cdb_query.the_geom_webmercator)/_cdb_params.res),
-                Floor(ST_Y(_cdb_query.the_geom_webmercator)/_cdb_params.res)
-                ${dimensionNames(ctx)}
-            ${havingClause(ctx)}
-        )
-        SELECT
-            _cdb_clusters.cartodb_id,
-            the_geom_webmercator
-            ${dimensionNames(ctx, '_cdb_clusters')}
-            ${aggregateColumnNames(ctx, '_cdb_clusters')}
+            cdb_res,
+            CEIL (ST_XMIN(cdb_full_bbox) / cdb_res) * cdb_res AS cdb_xmin,
+            FLOOR(ST_XMAX(cdb_full_bbox) / cdb_res) * cdb_res AS cdb_xmax,
+            CEIL (ST_YMIN(cdb_full_bbox) / cdb_res) * cdb_res AS cdb_ymin,
+            FLOOR(ST_YMAX(cdb_full_bbox) / cdb_res) * cdb_res AS cdb_ymax
        FROM
-            _cdb_clusters INNER JOIN (${ctx.sourceQuery}) _cdb_query
-            ON (_cdb_clusters.cartodb_id = _cdb_query.cartodb_id)
-    `
-
+        (
+            SELECT
+                ${gridResolution(ctx)} AS cdb_res,
+                !bbox! cdb_full_bbox
+        ) _cdb_input_resources
+    ) _cdb_grid_bbox_margins
+`;
 };

-module.exports.SUPPORTED_PLACEMENTS = Object.keys(aggregationQueryTemplates);
+
+// Function to generate the resulting point for a cell from the aggregated data
+const aggregatedPointWebMercator = (ctx) => {
+    switch (ctx.placement) {
+
+        // For centroid, we return the average of the cell
+        case 'centroid':
+            return ', ST_SetSRID(ST_MakePoint(AVG(cdb_x), AVG(cdb_y)), 3857) AS the_geom_webmercator';
+
+        // Middle point of the cell
+        case 'point-grid':
+            return `, ST_SetSRID(ST_MakePoint(cdb_pos_grid_x, cdb_pos_grid_y), 3857) AS the_geom_webmercator`;
+
+        // For point-sample we'll get a single point directly from the source
+        // If it's default aggregation we'll add the extra columns to keep backwards compatibility
+        case 'point-sample':
+            return '';
+
+        default:
+            throw new Error(`Invalid aggregation placement "${ctx.placement}"`);
+    }
+};
+
+// Function to generate the resulting point for a cell from the a join with the source
+const aggregatedPointJoin = (ctx) => {
+    switch (ctx.placement) {
+
+        case 'centroid':
+            return '';
+
+        case 'point-grid':
+            return '';
+
+        // For point-sample we'll get a single point directly from the source
+        // If it's default aggregation we'll add the extra columns to keep backwards compatibility
+        case 'point-sample':
+            return `
+NATURAL JOIN
+(
+    SELECT ${ctx.isDefaultAggregation ? `*` : `cartodb_id, the_geom_webmercator`}
+    FROM
+    (
+        ${ctx.sourceQuery}
+    ) __cdb_src_query
+) __cdb_query_columns
+`;
+
+        default:
+            throw new Error('Invalid aggregation placement "${ctx.placement}"');
+    }
+};
+
+// Function to generate the values common to all points in a cell
+// By default we use the cell number (which is fast), but for point-grid we
+// get the coordinates of the mid point so we don't need to calculate them later
+// which requires extra data in the group by clause
+const aggregatedPosCoordinate = (ctx, coordinate) => {
+    switch (ctx.placement) {
+        // For point-grid we return the coordinate of the middle point of the grid
+        case `point-grid`:
+            return `(FLOOR(cdb_${coordinate} / __cdb_grid_params.cdb_res) + 0.5) * __cdb_grid_params.cdb_res`;
+
+        // For other, we return the cell position (relative to the world)
+        default:
+            return `FLOOR(cdb_${coordinate} / __cdb_grid_params.cdb_res)`;
+    }
+};
+
+
+const aggregationQueryTemplate = ctx => `
+WITH __cdb_grid_params AS
+(
+    ${gridInfoQuery(ctx)}
+)
+SELECT * FROM
+(
+    SELECT
+        min(cartodb_id) as cartodb_id
+        ${aggregatedPointWebMercator(ctx)}
+        ${dimensionDefs(ctx)}
+        ${aggregateColumnDefs(ctx)}
+    FROM
+    (
+        SELECT
+            *,
+            ${aggregatedPosCoordinate(ctx, 'x')} as cdb_pos_grid_x,
+            ${aggregatedPosCoordinate(ctx, 'y')} as cdb_pos_grid_y
+        FROM
+        (
+            SELECT
+                __cdb_src_query.*,
+                ST_X(the_geom_webmercator) cdb_x,
+                ST_Y(the_geom_webmercator) cdb_y
+            FROM
+            (
+                ${ctx.sourceQuery}
+            ) __cdb_src_query, __cdb_grid_params
+            WHERE the_geom_webmercator && cdb_point_bbox
+            OFFSET 0
+        ) __cdb_src_get_x_y, __cdb_grid_params
+        WHERE cdb_x < __cdb_grid_params.cdb_xmax AND cdb_y < __cdb_grid_params.cdb_ymax
+    ) __cdb_src_gridded
+    GROUP BY cdb_pos_grid_x, cdb_pos_grid_y ${dimensionNames(ctx)}
+    ${havingClause(ctx)}
+) __cdb_aggregation_src
+${aggregatedPointJoin(ctx)}
+`;
+
+module.exports.SUPPORTED_PLACEMENTS = ['centroid', 'point-grid', 'point-sample'];
 module.exports.GEOMETRY_COLUMN = 'the_geom_webmercator';

 const clusterFeaturesQuery = ctx => `
--- a/lib/cartodb/models/mapconfig/adapter/mapconfig-overviews-adapter.js
+++ b/lib/cartodb/models/mapconfig/adapter/mapconfig-overviews-adapter.js
@@ -2,6 +2,7 @@

 var queue = require('queue-async');
 var _ = require('underscore');
+const AggregationMapConfig = require('../../aggregation/aggregation-mapconfig');

 function MapConfigOverviewsAdapter(overviewsMetadataBackend, filterStatsBackend) {
    this.overviewsMetadataBackend = overviewsMetadataBackend;
@@ -14,7 +15,9 @@ MapConfigOverviewsAdapter.prototype.getMapConfig = function (user, requestMapCon
    var layers = requestMapConfig.layers;
    var analysesResults = context.analysesResults;

-    if (!layers || layers.length === 0) {
+    const aggMapConfig = new AggregationMapConfig(null, requestMapConfig);
+    if (aggMapConfig.isVectorOnlyMapConfig() || aggMapConfig.isAggregationMapConfig() ||
+        !layers || layers.length === 0) {
        return callback(null, requestMapConfig);
    }

--- a/lib/cartodb/models/mapconfig/adapter/turbo-carto-adapter.js
+++ b/lib/cartodb/models/mapconfig/adapter/turbo-carto-adapter.js
@@ -6,7 +6,7 @@ var queue = require('queue-async');
 var PSQL = require('cartodb-psql');
 var turboCarto = require('turbo-carto');

-var SubstitutionTokens = require('../../../utils/substitution-tokens');
+const SubstitutionTokens = require('cartodb-query-tables').utils.substitutionTokens;
 var PostgresDatasource = require('../../../backends/turbo-carto-postgres-datasource');

 var MapConfig = require('windshaft').model.MapConfig;
--- a/lib/cartodb/models/mapconfig/provider/base-mapconfig-adapter.js
+++ b/lib/cartodb/models/mapconfig/provider/base-mapconfig-adapter.js
@@ -1,6 +1,6 @@
 'use strict';

-const QueryTables = require('cartodb-query-tables');
+const QueryTables = require('cartodb-query-tables').queryTables;

 module.exports = class BaseMapConfigProvider {
    createAffectedTables (callback) {
@@ -34,7 +34,7 @@ module.exports = class BaseMapConfigProvider {
                    return callback(err);
                }

-                QueryTables.getAffectedTablesFromQuery(connection, sql, (err, affectedTables) => {
+                QueryTables.getQueryMetadataModel(connection, sql, (err, affectedTables) => {
                    if (err) {
                        return callback(err);
                    }
--- a/lib/cartodb/models/mapconfig/provider/named-map-provider.js
+++ b/lib/cartodb/models/mapconfig/provider/named-map-provider.js
@@ -48,7 +48,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
        this.affectedTablesCache = affectedTablesCache;

        // providing
-        this.err = null;
        this.mapConfig = null;
        this.rendererParams = null;
        this.context = {};
@@ -56,13 +55,12 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
    }

    getMapConfig (callback) {
-        if (!!this.err || this.mapConfig !== null) {
-            return callback(this.err, this.mapConfig, this.rendererParams, this.context);
+        if (this.mapConfig !== null) {
+            return callback(null, this.mapConfig, this.rendererParams, this.context);
        }

        this.getContext((err, context) => {
            if (err) {
-                this.err = err;
                return callback(err);
            }

@@ -75,8 +73,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
                        this.config;
                } catch (e) {
                    const err = new Error('malformed config parameter, should be a valid JSON');
-                    this.err = err;
-
                    return callback(err);
                }
            }
@@ -85,7 +81,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {

            this.getTemplate((err, template) => {
                if (err) {
-                    this.err = err;
                    return callback(err);
                }

@@ -94,7 +89,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
                try {
                    requestMapConfig = this.templateMaps.instance(template, templateParams);
                } catch (err) {
-                    this.err = err;
                    return callback(err);
                }

@@ -103,7 +97,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
                this.mapConfigAdapter.getMapConfig(
                    user, requestMapConfig, rendererParams, context, (err, mapConfig, stats = {}) => {
                    if (err) {
-                        this.err = err;
                        return callback(err);
                    }

@@ -148,7 +141,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {

                this.userLimitsBackend.getRenderLimits(this.user, this.params.api_key, (err, renderLimits) => {
                    if (err) {
-                        this.err = err;
                        return callback(err);
                    }

@@ -163,13 +155,12 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
    }

    getTemplate (callback) {
-        if (!!this.err || this.template !== null) {
-            return callback(this.err, this.template);
+        if (this.template !== null) {
+            return callback(null, this.template);
        }

        this.templateMaps.getTemplate(this.user, this.templateName, (err, tpl) => {
            if (err) {
-                this.err = err;
                return callback(err);
            }

@@ -177,8 +168,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
                const error = new Error(`Template '${this.templateName}' of user '${this.user}' not found`);
                error.http_status = 404;

-                this.err = error;
-
                return callback(error);
            }

@@ -190,15 +179,12 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {
                const error = new Error('Failed to authorize template');
                error.http_status = 403;

-                this.err = error;
-
                return callback(error);
            }

            if (!authorized) {
                const error = new Error('Unauthorized template instantiation');
                error.http_status = 403;
-                this.err = error;

                return callback(error);
            }
@@ -222,7 +208,6 @@ module.exports = class NamedMapMapConfigProvider extends BaseMapConfigProvider {

        this.affectedTables = null;

-        this.err = null;
        this.mapConfig = null;

        this.cacheBuster = Date.now();
--- a/lib/cartodb/server.js
+++ b/lib/cartodb/server.js
@@ -79,22 +79,22 @@ function getAndValidateVersions(options) {
        windshaft_cartodb: packageDefinition.version
    };

-    var dependenciesToValidate = ['camshaft', 'turbo-carto', 'windshaft'];
-    dependenciesToValidate.forEach(function(depName) {
-        var declaredDependencyVersion = declaredDependencies[depName];
-        var installedDependencyVersion = installedDependenciesVersions[depName];
-        if (!semver.satisfies(installedDependencyVersion,declaredDependencyVersion)) {
-            warn(
-                'Dependency="%s" installed version="%s" does not match declared version="%s". Check your installation.',
-                depName, installedDependencyVersion, declaredDependencyVersion
-            );
-        }
-    });
+    if (process.env.NODE_ENV !== 'test') {
+        var dependenciesToValidate = ['camshaft', 'turbo-carto', 'windshaft'];
+        dependenciesToValidate.forEach(function(depName) {
+            var declaredDependencyVersion = declaredDependencies[depName];
+            var installedDependencyVersion = installedDependenciesVersions[depName];
+            if (!semver.satisfies(installedDependencyVersion,declaredDependencyVersion)) {
+                warn(`Dependency="${depName}" installed version="${installedDependencyVersion}" does ` +
+                     `not match declared version="${declaredDependencyVersion}". Check your installation.`);
+            }
+        });

-    // Be nice and warn if configured mapnik version is != installed mapnik version
-    if (windshaft.mapnik.versions.mapnik !== options.grainstore.mapnik_version) {
-        warn('WARNING: detected mapnik version (' + windshaft.mapnik.versions.mapnik + ')' +
-            ' != configured mapnik version (' + options.grainstore.mapnik_version + ')');
+        // Be nice and warn if configured mapnik version is != installed mapnik version
+        if (windshaft.mapnik.versions.mapnik !== options.grainstore.mapnik_version) {
+            warn('WARNING: detected mapnik version (' + windshaft.mapnik.versions.mapnik + ')' +
+                ' != configured mapnik version (' + options.grainstore.mapnik_version + ')');
+        }
    }

    return installedDependenciesVersions;
--- a/lib/cartodb/stats/reporter/named-map-provider.js
+++ b/lib/cartodb/stats/reporter/named-map-provider.js
@@ -0,0 +1,33 @@
+'use strict';
+
+const statKeyTemplate = ctx => `windshaft.named-map-provider-cache.${ctx.metric}`;
+
+module.exports = class NamedMapProviderReporter {
+    constructor ({ namedMapProviderCache, intervalInMilliseconds } = {}) {
+        this.namedMapProviderCache = namedMapProviderCache;
+        this.intervalInMilliseconds = intervalInMilliseconds;
+        this.intervalId = null;
+    }
+
+    start () {
+        const { providerCache: cache } = this.namedMapProviderCache;
+        const { statsClient: stats } = global;
+
+        this.intervalId = setInterval(() => {
+            stats.gauge(statKeyTemplate({ metric: 'named-map.count' }), cache.length);
+            const providers = cache.dump();
+
+            const namedMapInstantiations = providers.reduce((acc, { v: providers }) => {
+                acc += Object.keys(providers).length;
+                return acc;
+            }, 0);
+
+            stats.gauge(statKeyTemplate({ metric: 'named-map.instantiation.count' }), namedMapInstantiations);
+        }, this.intervalInMilliseconds);
+    }
+
+    stop () {
+        clearInterval(this.intervalId);
+        this.intervalId = null;
+    }
+};
--- a/lib/cartodb/utils/query-utils.js
+++ b/lib/cartodb/utils/query-utils.js
@@ -1,48 +1,34 @@
 'use strict';

-const SubstitutionTokens = require('./substitution-tokens');
-
-function prepareQuery(sql) {
-    var affectedTableRegexCache = {
-        bbox: /!bbox!/g,
-        scale_denominator: /!scale_denominator!/g,
-        pixel_width: /!pixel_width!/g,
-        pixel_height: /!pixel_height!/g
-    };
-
-    return sql
-        .replace(affectedTableRegexCache.bbox, 'ST_MakeEnvelope(0,0,0,0)')
-        .replace(affectedTableRegexCache.scale_denominator, '0')
-        .replace(affectedTableRegexCache.pixel_width, '1')
-        .replace(affectedTableRegexCache.pixel_height, '1');
-}
-
-module.exports.extractTableNames = function extractTableNames(query) {
-    return [
-        'SELECT * FROM CDB_QueryTablesText($windshaft$',
-        prepareQuery(query),
-        '$windshaft$) as tablenames'
-    ].join('');
-};
+const SubstitutionTokens = require('cartodb-query-tables').utils.substitutionTokens;
+const WebMercatorHelper = require('cartodb-query-tables').utils.webMercatorHelper;

 module.exports.getQueryActualRowCount = function (query) {
-    return `select COUNT(*) AS rows FROM (${query}) AS __cdb_query`;
+    return `select COUNT(*) AS rows FROM (${substituteDummyTokens(query)}) AS __cdb_query`;
 };

 function getQueryRowEstimation(query) {
-    return 'select CDB_EstimateRowCount($windshaft$' + query + '$windshaft$) as rows';
+    return 'select CDB_EstimateRowCount($windshaft$' + substituteDummyTokens(query) + '$windshaft$) as rows';
 }
-
 module.exports.getQueryRowEstimation = getQueryRowEstimation;

+function getQueryGeometryType(query, geometryColumn) {
+    return `
+        SELECT ST_GeometryType(${geometryColumn}) AS geom_type
+            FROM (${substituteDummyTokens(query)}) AS __cdb_query
+            WHERE ${geometryColumn} IS NOT NULL
+            LIMIT 1
+    `;
+}
+module.exports.getQueryGeometryType = getQueryGeometryType;
+
 module.exports.getAggregationMetadata = ctx => `
    WITH
    rowEstimation AS (
        ${getQueryRowEstimation(ctx.query)}
    ),
    geometryType AS (
-        SELECT ST_GeometryType(${ctx.geometryColumn}) as geom_type
-        FROM (${ctx.query}) AS __cdb_query WHERE ${ctx.geometryColumn} IS NOT NULL LIMIT 1
+        ${getQueryGeometryType(ctx.query, ctx.geometryColumn)}
    )
    SELECT
        rows AS count,
@@ -85,7 +71,7 @@ module.exports.getQueryTopCategories = function (query, column, topN, includeNul
    const where = includeNulls ? '' : `WHERE ${column} IS NOT NULL`;
    return `
        SELECT ${column} AS category, COUNT(*) AS frequency
-        FROM (${query}) AS __cdb_query
+        FROM (${substituteDummyTokens(query)}) AS __cdb_query
        ${where}
        GROUP BY ${column} ORDER BY 2 DESC
        LIMIT ${topN}
@@ -105,71 +91,29 @@ function columnSelector(columns) {
    throw new TypeError(`Bad argument type for columns: ${typeof columns}`);
 }

-module.exports.getQuerySample = function (query, sampleProb, limit = null, randomSeed = 0.5, columns = null) {
-    const singleTable = simpleQueryTable(query);
-    if (singleTable) {
-        return getTableSample(singleTable.table, columns || singleTable.columns, sampleProb, limit, randomSeed);
-    }
-    const limitClause = limit ? `LIMIT ${limit}` : '';
+module.exports.getMaxMinSpanColumnQuery = function (query, column = 'cartodb_id') {
    return `
-        WITH __cdb_rndseed AS (
-            SELECT setseed(${randomSeed})
-        )
-        SELECT ${columnSelector(columns)}
-            FROM (${query}) AS __cdb_query
-            WHERE random() < ${sampleProb}
-            ${limitClause}
+        SELECT
+            min(${column}) AS min_id,
+            max(${column}) AS max_id,
+            (max(${column}) - min(${column})) AS id_span
+        FROM (${substituteDummyTokens(query)}) _cdb_metadata_max_min_span;
    `;
 };

-function getTableSample(table, columns, sampleProb, limit = null, randomSeed = 0.5) {
-    const limitClause = limit ? `LIMIT ${limit}` : '';
-    sampleProb *= 100;
-    randomSeed *= Math.pow(2, 31) - 1;
+module.exports.getSampleFromIdsQuery = function (query, ids, columns, column = 'cartodb_id') {
    return `
-        SELECT ${columnSelector(columns)}
-        FROM ${table}
-        TABLESAMPLE BERNOULLI (${sampleProb}) REPEATABLE (${randomSeed}) ${limitClause}
-    `;
-}
-
-function simpleQueryTable(sql) {
-    const basicQuery =
-        /\s*SELECT\s+([\*a-z0-9_,\s]+?)\s+FROM\s+((\"[^"]+\"|[a-z0-9_]+)\.)?(\"[^"]+\"|[a-z0-9_]+)\s*;?\s*/i;
-    const unwrappedQuery = new RegExp('^' + basicQuery.source + '$', 'i');
-    // queries for named maps are wrapped like this:
-    var wrappedQuery = new RegExp(
-        '^\\s*SELECT\\s+\\*\\s+FROM\\s+\\(' +
-        basicQuery.source +
-        '\\)\\s+AS\\s+wrapped_query\\s+WHERE\\s+\\d+=1\\s*$',
-        'i'
-    );
-    let match = sql.match(unwrappedQuery);
-    if (!match) {
-        match = sql.match(wrappedQuery);
-    }
-    if (match) {
-        const columns = match[1];
-        const schema = match[3];
-        const table = match[4];
-        return { table: schema ? `${schema}.${table}` : table, columns };
-    }
-    return false;
-}
-
-module.exports.getQueryGeometryType = function (query, geometryColumn) {
-    return `
-        SELECT ST_GeometryType(${geometryColumn}) AS geom_type
-            FROM (${query}) AS __cdb_query
-            WHERE ${geometryColumn} IS NOT NULL
-            LIMIT 1
+        SELECT
+            ${columnSelector(columns)}
+        FROM (${substituteDummyTokens(query)}) _cdb_metadata_sample
+        WHERE ${column} IN (${ids.join(',')})
    `;
 };

 function getQueryLimited(query, limit = 0) {
    return `
        SELECT *
-            FROM (${query}) AS __cdb_query
+            FROM (${substituteDummyTokens(query)}) AS __cdb_query
            LIMIT ${limit}
    `;
 }
@@ -181,35 +125,16 @@ function queryPromise(dbConnection, query) {
 }

 function substituteDummyTokens(sql) {
-    return sql && SubstitutionTokens.replace(sql, {
-        bbox: 'ST_MakeEnvelope(0,0,0,0)',
-        scale_denominator: '0',
-        pixel_width: '1',
-        pixel_height: '1'
-    });
+    return SubstitutionTokens.replace(sql);
 }

-function subsituteTokensForZoom(sql, zoom, singleTile=false) {
-    const tileRes = 256;
-    const wmSize = 6378137.0*2*Math.PI;
-    const nTiles = Math.pow(2, zoom);
-    const tileSize = wmSize / nTiles;
-    const resolution = tileSize / tileRes;
-    const scaleDenominator = resolution / 0.00028;
-    const x0 = -wmSize/2, y0 = -wmSize/2;
-    let bbox = `ST_MakeEnvelope(${x0}, ${y0}, ${x0+wmSize}, ${y0+wmSize})`;
-    if (singleTile) {
-        bbox = `ST_MakeEnvelope(${x0}, ${y0}, ${x0 + tileSize}, ${y0 + tileSize})`;
-    }
-    return SubstitutionTokens.replace(sql, {
-        bbox: bbox,
-        scale_denominator: scaleDenominator,
-        pixel_width: resolution,
-        pixel_height: resolution
-    });
+function substituteTokensForZoom(sql, zoom) {
+    const extent = new WebMercatorHelper().getExtent({ x : 0, y : 0, z : 0 });
+    const bbox = `ST_MakeEnvelope(${extent.xmin}, ${extent.ymin}, ${extent.xmax}, ${extent.ymax}, 3857)`;
+    return SubstitutionTokens.replaceXYZ(sql, { z : zoom, bbox: bbox });
 }

 module.exports.queryPromise = queryPromise;
 module.exports.getQueryLimited = getQueryLimited;
 module.exports.substituteDummyTokens = substituteDummyTokens;
-module.exports.subsituteTokensForZoom = subsituteTokensForZoom;
+module.exports.substituteTokensForZoom = substituteTokensForZoom;
--- a/package-lock.json
+++ b/package-lock.json
--- a/package.json
+++ b/package.json
@@ -1,7 +1,7 @@
 {
  "private": true,
  "name": "windshaft-cartodb",
-  "version": "7.1.0",
+  "version": "7.2.0",
  "description": "A map tile server for CartoDB",
  "keywords": [
    "cartodb"
@@ -27,9 +27,9 @@
    "@carto/fqdn-sync": "0.2.2",
    "basic-auth": "2.0.0",
    "body-parser": "1.18.3",
-    "camshaft": "^0.64.0",
-    "cartodb-psql": "0.13.1",
-    "cartodb-query-tables": "0.4.0",
+    "camshaft": "^0.64.2",
+    "cartodb-psql": "0.14.0",
+    "cartodb-query-tables": "^0.6.3",
    "cartodb-redis": "2.1.0",
    "debug": "3.1.0",
    "dot": "1.1.2",
@@ -47,9 +47,9 @@
    "request": "2.87.0",
    "semver": "5.5.0",
    "step-profiler": "0.3.0",
-    "turbo-carto": "0.21.0",
+    "turbo-carto": "0.21.2",
    "underscore": "1.6.0",
-    "windshaft": "5.2.0",
+    "windshaft": "^5.6.3",
    "yargs": "11.1.0"
  },
  "devDependencies": {
--- a/run_tests.sh
+++ b/run_tests.sh
@@ -5,7 +5,6 @@ OPT_CREATE_PGSQL=yes # create the PostgreSQL test environment
 OPT_DROP_REDIS=yes   # drop the redis test environment
 OPT_DROP_PGSQL=yes   # drop the PostgreSQL test environment
 OPT_COVERAGE=no      # run tests with coverage
-OPT_DOWNLOAD_SQL=yes # download a fresh copy of sql files
 OPT_REDIS_CELL=yes   # download redis cell

 export PGAPPNAME=cartodb_tiler_tester
@@ -75,10 +74,6 @@ while [ -n "$1" ]; do
                OPT_CREATE_REDIS=no
                shift
                continue
-        elif test "$1" = "--no-sql-download"; then
-                OPT_DOWNLOAD_SQL=no
-                shift
-                continue
        elif test "$1" = "--with-coverage"; then
                OPT_COVERAGE=yes
                shift
@@ -129,9 +124,6 @@ fi
 if test x"$OPT_CREATE_REDIS" != xyes; then
  PREPARE_DB_OPTS="$PREPARE_DB_OPTS --skip-redis"
 fi
-if test x"$OPT_DOWNLOAD_SQL" != xyes; then
-  PREPARE_DB_OPTS="$PREPARE_DB_OPTS --no-sql-download"
-fi

 echo "Preparing the environment"
 cd ${BASEDIR}/test/support
--- a/run_tests_docker.sh
+++ b/run_tests_docker.sh
@@ -4,6 +4,10 @@

 source /src/nodejs-install.sh

+# Install cartodb-postgresql extension
+git clone https://github.com/CartoDB/cartodb-postgresql.git
+cd cartodb-postgresql && make && make install && cd ..
+
 echo "Node.js version: "
 node -v

--- a/test/acceptance/aggregation.js
+++ b/test/acceptance/aggregation.js
@@ -6,6 +6,9 @@ const assert = require('../support/assert');
 const TestClient = require('../support/test-client');
 const serverOptions = require('../../lib/cartodb/server_options');

+const WebMercatorHelper = require('cartodb-query-tables').utils.webMercatorHelper;
+const webmercator = new WebMercatorHelper();
+
 const suites = [
    {
        desc: 'mvt (mapnik)',
@@ -96,9 +99,9 @@ describe('aggregation', function () {
    const POLYGONS_SQL_1 = `
    select
        x + 4 as cartodb_id,
-        st_buffer(st_setsrid(st_makepoint(x*10, x*10), 4326)::geography, 100000)::geometry as the_geom,
+        st_buffer(st_setsrid(st_makepoint(x*10, x*10), 4326), 10) as the_geom,
        st_transform(
-            st_buffer(st_setsrid(st_makepoint(x*10, x*10), 4326)::geography, 100000)::geometry,
+            st_buffer(st_setsrid(st_makepoint(x*10, x*10), 4326), 10),
            3857
        ) as the_geom_webmercator,
        x as value
@@ -109,15 +112,15 @@ describe('aggregation', function () {
    WITH hgrid AS (
        SELECT
            CDB_RectangleGrid (
-                ST_Expand(!bbox!, CDB_XYZ_Resolution(1) * 12),
-                CDB_XYZ_Resolution(1) * 12,
-                CDB_XYZ_Resolution(1) * 12
+                ST_Expand(!bbox!, ${webmercator.getResolution({ z : 1 })} * 12),
+                ${webmercator.getResolution({ z : 1 })} * 12,
+                ${webmercator.getResolution({ z : 1 })} * 12
            ) as cell
    )
    SELECT
        hgrid.cell as the_geom_webmercator,
        count(1) as agg_value,
-        count(1) /power( 12 * CDB_XYZ_Resolution(1), 2 ) as agg_value_density,
+        count(1) /power( 12 * ${webmercator.getResolution({ z : 1 })}, 2 ) as agg_value_density,
        row_number() over () as cartodb_id
    FROM hgrid, (<%= sql %>) i
    WHERE ST_Intersects(i.the_geom_webmercator, hgrid.cell) GROUP BY hgrid.cell
@@ -202,9 +205,9 @@ describe('aggregation', function () {
    //       |   |   |   |   |   |   |
    //    Tile 0, 1 -+---+---+---+- Tile 1,1
    //
-    const POINTS_SQL_GRID = `
+    const POINTS_SQL_GRID = (z, resolution) => `
        WITH params AS (
-            SELECT CDB_XYZ_Resolution($Z)*$resolution AS l -- cell size for Z, resolution
+            SELECT ${webmercator.getResolution({ z : z })}*${resolution} AS l -- cell size for Z, resolution
        )
        SELECT
            row_number() OVER () AS cartodb_id,
@@ -224,8 +227,8 @@ describe('aggregation', function () {
    const POINTS_SQL_CELL = `
      SELECT
        1 AS cartodb_id,
-        ST_SetSRID(ST_MakePoint(18181005.8, -18181043.9), 3857) AS the_geom_webmercator,
-        ST_Transform(ST_SetSRID(ST_MakePoint(18181005.8, -18181043.9), 3857), 4326) AS the_geom
+        ST_SetSRID(ST_MakePoint(18181005.82, -18181043.9), 3857) AS the_geom_webmercator,
+        ST_Transform(ST_SetSRID(ST_MakePoint(18181005.82, -18181043.9), 3857), 4326) AS the_geom
      UNION ALL SELECT
        2 AS cartodb_id,
        ST_SetSRID(ST_MakePoint(18181005.9, -18181044.0), 3857) AS the_geom_webmercator,
@@ -236,8 +239,8 @@ describe('aggregation', function () {
        ST_Transform(ST_SetSRID(ST_MakePoint(18181005.87, -18181043.94), 3857), 4326) AS the_geom
      UNION ALL SELECT
        4 AS cartodb_id,
-        ST_SetSRID(ST_MakePoint(18181005.8, -18181043.9), 3857) AS the_geom_webmercator,
-        ST_Transform(ST_SetSRID(ST_MakePoint(18181005.8, -18181043.9), 3857), 4326) AS the_geom
+        ST_SetSRID(ST_MakePoint(18181005.82, -18181043.9), 3857) AS the_geom_webmercator,
+        ST_Transform(ST_SetSRID(ST_MakePoint(18181005.82, -18181043.9), 3857), 4326) AS the_geom
    `;

    // Points positioned inside one cell of Z=20, X=1000000, X=1000000 (inner cell not on border)
@@ -249,8 +252,8 @@ describe('aggregation', function () {
        ST_Transform(ST_SetSRID(ST_MakePoint(18181005.95, -18181043.8), 3857), 4326) AS the_geom
      UNION ALL SELECT
        2 AS cartodb_id,
-        ST_SetSRID(ST_MakePoint(18181006.09, -18181043.72), 3857) AS the_geom_webmercator,
-        ST_Transform(ST_SetSRID(ST_MakePoint(18181006.09, -18181043.72), 3857), 4326) AS the_geom
+        ST_SetSRID(ST_MakePoint(18181006.09, -18181043.74), 3857) AS the_geom_webmercator,
+        ST_Transform(ST_SetSRID(ST_MakePoint(18181006.09, -18181043.74), 3857), 4326) AS the_geom
      UNION ALL SELECT
        3 AS cartodb_id,
        ST_SetSRID(ST_MakePoint(18181006.02, -18181043.79), 3857) AS the_geom_webmercator,
@@ -521,7 +524,7 @@ describe('aggregation', function () {
            });

            ['centroid', 'point-sample', 'point-grid'].forEach(placement => {
-                it('should provide all the requested columns in non-default aggregation ',
+                it('should provide all the requested columns in non-default aggregation: ' + placement,
                function (done) {
                    const response = {
                        status: 200,
@@ -570,7 +573,7 @@ describe('aggregation', function () {
                    });
                });

-                it('should provide only the requested columns in non-default aggregation ',
+                it('should provide only the requested columns in non-default aggregation: ' + placement,
                function (done) {
                    this.mapConfig = createVectorMapConfig([
                        {
@@ -1407,6 +1410,73 @@ describe('aggregation', function () {
                });
            });

+            it('aggregation dimension hour iso format with timezone', function (done) {
+                this.mapConfig = createVectorMapConfig([
+                    {
+                        type: 'cartodb',
+                        options: {
+                            // take four points per hour over two days
+                            sql: pointsWithTimeSQL(96, '2018-01-01T00:00:00+02', '2018-01-01T23:59:59+02', 0),
+                            dates_as_numbers: true,
+                            aggregation: {
+                                threshold: 1,
+                                dimensions: {
+                                    hour: {
+                                        column: 'date',
+                                        group: {
+                                            units: 'hour',
+                                            timezone: '+7200'
+                                        },
+                                        format: 'iso',
+                                    }
+                                }
+
+                            }
+                        }
+                    }
+                ]);
+
+                this.testClient = new TestClient(this.mapConfig);
+                const options = {
+                    format: 'mvt'
+                };
+                this.testClient.getTile(0, 0, 0, options, (err, res, tile) => {
+                    if (err) {
+                        return done(err);
+                    }
+                    const tileJSON = tile.toJSON();
+                    const resultHours = tileJSON[0].features.map(f => f.properties.hour).sort();
+                    assert.deepEqual(resultHours, [
+                        "2018-01-01T00",
+                        "2018-01-01T01",
+                        "2018-01-01T02",
+                        "2018-01-01T03",
+                        "2018-01-01T04",
+                        "2018-01-01T05",
+                        "2018-01-01T06",
+                        "2018-01-01T07",
+                        "2018-01-01T08",
+                        "2018-01-01T09",
+                        "2018-01-01T10",
+                        "2018-01-01T11",
+                        "2018-01-01T12",
+                        "2018-01-01T13",
+                        "2018-01-01T14",
+                        "2018-01-01T15",
+                        "2018-01-01T16",
+                        "2018-01-01T17",
+                        "2018-01-01T18",
+                        "2018-01-01T19",
+                        "2018-01-01T20",
+                        "2018-01-01T21",
+                        "2018-01-01T22",
+                        "2018-01-01T23"
+                    ]);
+                    tileJSON[0].features.forEach(f => assert.equal(f.properties._cdb_feature_count, 4));
+                    done();
+                });
+            });
+
            ['centroid', 'point-sample', 'point-grid'].forEach(placement => {
                it(`dimensions should work for ${placement} placement`, function(done) {
                    this.mapConfig = createVectorMapConfig([
@@ -2284,7 +2354,7 @@ describe('aggregation', function () {
                                    threshold: 1,
                                    columns: {
                                        value: {
-                                            aggregate_function: 'sum',
+                                            aggregate_function: 'mode',
                                            aggregated_column: 'value'
                                        }
                                    },
@@ -2330,7 +2400,7 @@ describe('aggregation', function () {
                                    threshold: 1,
                                    columns: {
                                        value: {
-                                            aggregate_function: 'sum',
+                                            aggregate_function: 'mode',
                                            aggregated_column: 'value'
                                        }
                                    },
@@ -2903,7 +2973,7 @@ describe('aggregation', function () {
                it(`for ${placement} each aggr. cell is in a single tile`, function (done) {
                    const z = 1;
                    const resolution = 1;
-                    const query = POINTS_SQL_GRID.replace('$Z', z).replace('$resolution', resolution);
+                    const query = POINTS_SQL_GRID(z, resolution);
                    this.mapConfig = {
                        version: '1.6.0',
                        buffersize: { 'mvt': 0 },
@@ -2948,32 +3018,17 @@ describe('aggregation', function () {
                                    }
                                    const tile11 = JSON.parse(mvt.toGeoJSONSync(0));

-                                    const tile00Expected = [
-                                        { cartodb_id: 4, _cdb_feature_count: 2 },
-                                        { cartodb_id: 7, _cdb_feature_count: 1 }
-                                    ];
-                                    const tile10Expected = [
-                                        { cartodb_id: 5, _cdb_feature_count: 2 },
-                                        { cartodb_id: 6, _cdb_feature_count: 1 },
-                                        { cartodb_id: 8, _cdb_feature_count: 1 },
-                                        { cartodb_id: 9, _cdb_feature_count: 1 }
-                                    ];
-                                    const tile01Expected = [
-                                        { cartodb_id: 1, _cdb_feature_count: 2 }
-                                    ];
-                                    const tile11Expected = [
-                                        { cartodb_id: 2, _cdb_feature_count: 2 },
-                                        { cartodb_id: 3, _cdb_feature_count: 1 }
-                                    ];
-                                    const tile00Actual = tile00.features.map(f => f.properties);
-                                    const tile10Actual = tile10.features.map(f => f.properties);
-                                    const tile01Actual = tile01.features.map(f => f.properties);
-                                    const tile11Actual = tile11.features.map(f => f.properties);
-                                    const orderById = (a, b) => a.cartodb_id - b.cartodb_id;
-                                    assert.deepEqual(tile00Actual.sort(orderById), tile00Expected);
-                                    assert.deepEqual(tile10Actual.sort(orderById), tile10Expected);
-                                    assert.deepEqual(tile01Actual.sort(orderById), tile01Expected);
-                                    assert.deepEqual(tile11Actual.sort(orderById), tile11Expected);
+                                    // There needs to be 13 points
+                                    const count_features = ((tile) =>
+                                        tile.features.map(f => f.properties)
+                                                     .map(f => f._cdb_feature_count)
+                                                     .reduce((a,b) => a + b, 0));
+
+                                    const tile00Count = count_features(tile00);
+                                    const tile10Count = count_features(tile10);
+                                    const tile01Count = count_features(tile01);
+                                    const tile11Count = count_features(tile11);
+                                    assert.equal(13, tile00Count + tile10Count + tile01Count + tile11Count);

                                    done();
                                });
@@ -2989,7 +3044,7 @@ describe('aggregation', function () {
                    const z = 1;
                    const resolution = 2;
                    // space the test points by half the resolution:
-                    const query = POINTS_SQL_GRID.replace('$Z', z).replace('$resolution', resolution/2);
+                    const query = POINTS_SQL_GRID(z, resolution / 2);

                    this.mapConfig = {
                        version: '1.6.0',
@@ -3066,20 +3121,11 @@ describe('aggregation', function () {
                });

                it(`for ${placement} includes complete cells in buffer`, function (done) {
-                    if (!usePostGIS && placement !== 'point-grid') {
-                        // Mapnik seem to filter query results by its (inaccurate) bbox,
-                        // which makes some aggregated clusters get lost here.
-                        // The point-grid placement is resilient to this problem because the result
-                        // coordinates are moved to cluster cell centers, so they're well within
-                        // bbox limits.
-                        this.testClient = new TestClient({});
-                        return done();
-                    }

                    // use buffersize coincident with resolution, the buffer should include neighbour cells
                    const z = 2;
                    const resolution = 1;
-                    const query = POINTS_SQL_GRID.replace('$Z', z).replace('$resolution', resolution);
+                    const query = POINTS_SQL_GRID(z, resolution);

                    this.mapConfig = {
                        version: '1.6.0',
@@ -3125,49 +3171,24 @@ describe('aggregation', function () {
                                    }
                                    const tile11 = JSON.parse(mvt.toGeoJSONSync(0));

-                                    const tile00Expected = [
-                                        { _cdb_feature_count: 2, cartodb_id: 1 },
-                                        { _cdb_feature_count: 2, cartodb_id: 2 },
-                                        { _cdb_feature_count: 2, cartodb_id: 4 },
-                                        { _cdb_feature_count: 2, cartodb_id: 5 },
-                                        { _cdb_feature_count: 1, cartodb_id: 7 },
-                                        { _cdb_feature_count: 1, cartodb_id: 8 }
-                                    ];
-                                    const tile10Expected = [
-                                        { _cdb_feature_count: 2, cartodb_id: 1 },
-                                        { _cdb_feature_count: 2, cartodb_id: 2 },
-                                        { _cdb_feature_count: 1, cartodb_id: 3 },
-                                        { _cdb_feature_count: 2, cartodb_id: 4 },
-                                        { _cdb_feature_count: 2, cartodb_id: 5 },
-                                        { _cdb_feature_count: 1, cartodb_id: 6 },
-                                        { _cdb_feature_count: 1, cartodb_id: 7 },
-                                        { _cdb_feature_count: 1, cartodb_id: 8 },
-                                        { _cdb_feature_count: 1, cartodb_id: 9 }
-                                    ];
-                                    const tile01Expected = [
-                                        { _cdb_feature_count: 2, cartodb_id: 1 },
-                                        { _cdb_feature_count: 2, cartodb_id: 2 },
-                                        { _cdb_feature_count: 2, cartodb_id: 4 },
-                                        { _cdb_feature_count: 2, cartodb_id: 5 }
-                                    ];
-                                    const tile11Expected = [
-                                        { _cdb_feature_count: 2, cartodb_id: 1 },
-                                        { _cdb_feature_count: 2, cartodb_id: 2 },
-                                        { _cdb_feature_count: 1, cartodb_id: 3 },
-                                        { _cdb_feature_count: 2, cartodb_id: 4 },
-                                        { _cdb_feature_count: 2, cartodb_id: 5 },
-                                        { _cdb_feature_count: 1, cartodb_id: 6 }
-                                    ];
+                                    // We check that if an id/cell is present in multiple tiles,
+                                    // it always contains the same amount of features
                                    const tile00Actual = tile00.features.map(f => f.properties);
                                    const tile10Actual = tile10.features.map(f => f.properties);
                                    const tile01Actual = tile01.features.map(f => f.properties);
                                    const tile11Actual = tile11.features.map(f => f.properties);
-                                    const orderById = (a, b) => a.cartodb_id - b.cartodb_id;
-                                    assert.deepEqual(tile00Actual.sort(orderById), tile00Expected);
-                                    assert.deepEqual(tile10Actual.sort(orderById), tile10Expected);
-                                    assert.deepEqual(tile01Actual.sort(orderById), tile01Expected);
-                                    assert.deepEqual(tile11Actual.sort(orderById), tile11Expected);

+                                    const allFeatures = [... tile00Actual, ...tile10Actual,
+                                                         ...tile01Actual, ...tile11Actual];
+                                    for (let i = 0; i < allFeatures.length; i++) {
+                                        for (let j = i + 1; j < allFeatures.length; j++) {
+                                            const c1 = allFeatures[i];
+                                            const c2 = allFeatures[j];
+                                            if (c1.cartodb_id === c2.cartodb_id) {
+                                                assert.equal(c1._cdb_feature_count, c2._cdb_feature_count);
+                                            }
+                                        }
+                                    }
                                    done();
                                });
                            });
--- a/test/acceptance/cache/cache-control-header.js
+++ b/test/acceptance/cache/cache-control-header.js
@@ -0,0 +1,179 @@
+'use strict';
+
+require('../../support/test_helper');
+
+const assert = require('../../support/assert');
+const TestClient = require('../../support/test-client');
+
+const ONE_YEAR_IN_SECONDS = 60 * 60 * 24 * 365;
+const FIVE_MINUTES_IN_SECONDS = 60 * 5;
+
+const defaultLayers = [{
+    type: 'cartodb',
+    options: {
+        sql: TestClient.SQL.ONE_POINT,
+        cartocss: TestClient.CARTOCSS.POINTS,
+        cartocss_version: '2.3.0'
+    }
+}];
+const defaultDatavies = {};
+const defaultAnalyses = [];
+
+function createMapConfig ({
+    layers = defaultLayers,
+    dataviews = defaultDatavies,
+    analyses = defaultAnalyses
+} = {}) {
+    return {
+        version: '1.8.0',
+        layers: layers,
+        dataviews: dataviews || {},
+        analyses: analyses || []
+    };
+}
+
+describe('cache-control header', function () {
+    describe('max-age directive', function () {
+        it('tile from a table which is included in cdb_tablemetada', function (done) {
+            const ttl = ONE_YEAR_IN_SECONDS;
+            const mapConfig = createMapConfig({
+                layers: [{
+                    type: 'cartodb',
+                    options: {
+                        sql: 'select * from test_table',
+                        cartocss: TestClient.CARTOCSS.POINTS,
+                        cartocss_version: '2.3.0'
+                    }
+                }]
+            });
+
+            const testClient = new TestClient(mapConfig);
+
+            testClient.getTile(0, 0, 0, {}, function (err, res) {
+                if (err) {
+                    return done(err);
+                }
+
+                assert.equal(res.headers['cache-control'], `public,max-age=${ttl}`);
+                testClient.drain(done);
+            });
+        });
+
+        it('tile from a table which is NOT included in cdb_tablemetada', function (done) {
+            const ttl = global.environment.varnish.fallbackTtl || FIVE_MINUTES_IN_SECONDS;
+            const mapConfig = createMapConfig({
+                layers: [{
+                    type: 'cartodb',
+                    options: {
+                        sql: 'select * from test_table_2',
+                        cartocss: TestClient.CARTOCSS.POINTS,
+                        cartocss_version: '2.3.0'
+                    }
+                }]
+            });
+
+            const testClient = new TestClient(mapConfig);
+
+            testClient.getTile(0, 0, 0, {}, function (err, res) {
+                if (err) {
+                    return done(err);
+                }
+
+                assert.equal(res.headers['cache-control'], `public,max-age=${ttl}`);
+                testClient.drain(done);
+            });
+        });
+
+        it('tile from joined tables which one of them is NOT included in cdb_tablemetada', function (done) {
+            const ttl = global.environment.varnish.fallbackTtl || FIVE_MINUTES_IN_SECONDS;
+            const mapConfig = createMapConfig({
+                layers: [{
+                    type: 'cartodb',
+                    options: {
+                        sql: `
+                            select
+                                t.cartodb_id,
+                                t.the_geom,
+                                t.the_geom_webmercator
+                            from
+                                test_table t,
+                                test_table_2 t2
+                            where
+                                t.cartodb_id = t2.cartodb_id
+                        `,
+                        cartocss: TestClient.CARTOCSS.POINTS,
+                        cartocss_version: '2.3.0'
+                    }
+                }]
+            });
+
+            const testClient = new TestClient(mapConfig);
+
+            testClient.getTile(0, 0, 0, {}, function (err, res) {
+                if (err) {
+                    return done(err);
+                }
+
+                assert.equal(res.headers['cache-control'], `public,max-age=${ttl}`);
+                testClient.drain(done);
+            });
+        });
+
+        it('tile from a dynamic query which doesn\'t use a table' , function (done) {
+            const ttl = ONE_YEAR_IN_SECONDS;
+            const mapConfig = createMapConfig();
+
+            const testClient = new TestClient(mapConfig);
+
+            testClient.getTile(0, 0, 0, {}, function (err, res) {
+                if (err) {
+                    return done(err);
+                }
+
+                assert.equal(res.headers['cache-control'], `public,max-age=${ttl}`);
+                testClient.drain(done);
+            });
+        });
+
+        it('tile from a cached analysis table which is not included in cdb_tablemetada', function (done) {
+            const ttl = ONE_YEAR_IN_SECONDS;
+            const mapConfig = createMapConfig({
+                layers: [{
+                    type: 'cartodb',
+                    options: {
+                        source: {
+                            id: 'HEAD'
+                        },
+                        cartocss: TestClient.CARTOCSS.POINTS,
+                        cartocss_version: '2.3.0'
+                    }
+                }],
+                analyses: [{
+                    id: 'HEAD',
+                    type: 'buffer',
+                    params: {
+                        source: {
+                            id: 'source_1',
+                            type: 'source',
+                            params: {
+                                query: 'select * from test_table'
+                            }
+                        },
+                        radius: 60000
+                    }
+                }]
+            });
+
+            const testClient = new TestClient(mapConfig, 1234);
+
+            testClient.getTile(0, 0, 0, {}, function (err, res) {
+                if (err) {
+                    return done(err);
+                }
+
+                assert.equal(res.headers['cache-control'], `public,max-age=${ttl}`);
+                testClient.drain(done);
+            });
+        });
+    });
+});
--- a/test/acceptance/multilayer.js
+++ b/test/acceptance/multilayer.js
@@ -21,7 +21,7 @@ var IMAGE_EQUALS_HIGHER_TOLERANCE_PER_MIL = 25;
 var CartodbWindshaft = require('../../lib/cartodb/server');
 var serverOptions = require('../../lib/cartodb/server_options');

-var QueryTables = require('cartodb-query-tables');
+var QueryTables = require('cartodb-query-tables').queryTables;

 ['/api/v1/map', '/user/localhost/api/v1/map'].forEach(function(layergroup_url) {

@@ -282,7 +282,7 @@ describe(suiteName, function() {
          var parsedBody = JSON.parse(res.body);
          expected_token = parsedBody.layergroupid.split(':')[0];
          helper.checkCache(res);
-          helper.checkSurrogateKey(res, new QueryTables.DatabaseTablesEntry([
+          helper.checkSurrogateKey(res, new QueryTables.QueryMetadata([
            {dbname: "test_windshaft_cartodb_user_1_db", table_name: "test_table", schema_name: "public"},
            {dbname: "test_windshaft_cartodb_user_1_db", table_name: "test_table_2", schema_name: "public"},
          ]).key().join(' '));
--- a/test/acceptance/multilayer_server.js
+++ b/test/acceptance/multilayer_server.js
@@ -9,7 +9,7 @@ var _ = require('underscore');
 var LayergroupToken = require('../../lib/cartodb/models/layergroup-token');

 var PgQueryRunner = require('../../lib/cartodb/backends/pg_query_runner');
-var QueryTables = require('cartodb-query-tables');
+var QueryTables = require('cartodb-query-tables').queryTables;
 var CartodbWindshaft = require('../../lib/cartodb/server');
 var serverOptions = require('../../lib/cartodb/server_options');

@@ -367,11 +367,9 @@ describe('tests from old api translated to multilayer', function() {
                keysToDelete['map_cfg|' + LayergroupToken.parse(JSON.parse(res.body).layergroupid).token] = 0;
                keysToDelete['user:localhost:mapviews:global'] = 5;

-                var affectedFn = QueryTables.getAffectedTablesFromQuery;
-                QueryTables.getAffectedTablesFromQuery = function(sql, username, query, callback) {
-                    affectedFn({query: function(query, callback) {
-                        return callback(new Error('fake error message'), []);
-                    }}, username, query, callback);
+                var affectedFn = QueryTables.getQueryMetadataModel;
+                QueryTables.getQueryMetadataModel = function(pg, sql, callback) {
+                    return callback(new Error('fake error message'));
                };

                // reset internal cacheChannel cache
@@ -396,8 +394,8 @@ describe('tests from old api translated to multilayer', function() {
                        status: 200
                    },
                    function(res) {
+                        QueryTables.getQueryMetadataModel = affectedFn;
                        assert.ok(!res.headers.hasOwnProperty('x-cache-channel'));
-                        QueryTables.getAffectedTablesFromQuery = affectedFn;
                        done();
                    }
                );
--- a/test/acceptance/named-map-cache-regressions.js
+++ b/test/acceptance/named-map-cache-regressions.js
@@ -0,0 +1,236 @@
+'use strict';
+
+require('../support/test_helper');
+
+const request = require('request');
+const assert = require('assert');
+const Server = require('../../lib/cartodb/server');
+const serverOptions = require('../../lib/cartodb/server_options');
+const { mapnik } = require('windshaft');
+const helper = require('../support/test_helper');
+
+const namedTileUrlTemplate = (ctx) => {
+    return `http://${ctx.address}/api/v1/map/static/named/${ctx.templateId}/256/256.png?api_key=${ctx.apiKey}`;
+};
+
+describe('named map cache regressions', function () {
+    const server = new Server(serverOptions);
+
+    const apiKey = 1234;
+
+    const template = {
+        version: '0.0.1',
+        name: 'named-map-cache-regression-missing-template',
+        layergroup: {
+            version: '1.8.0',
+            layers: [
+                {
+                    type: 'cartodb',
+                    options: {
+                        source: {
+                            id: 'a1'
+                        },
+                        cartocss: '#layer{marker-placement: point;marker-width: 5;marker-fill: red;}',
+                        cartocss_version: '2.3.0'
+                    }
+                }
+            ],
+            analyses: [
+                {
+                    id: 'a1',
+                    type: 'source',
+                    params: {
+                        query: 'select * from populated_places_simple_reduced'
+                    }
+                }
+            ]
+        }
+    };
+
+    const port = 0; // let the OS to choose a free port
+    const host = '127.0.0.1';
+
+    let listener;
+    let address;
+
+    const keysToDelete = {};
+
+    before(function (done) {
+        listener = server.listen(port, host);
+
+        listener.on('error', done);
+        listener.on('listening', () => {
+            const { address: host, port } = listener.address();
+
+            address = `${host}:${port}`;
+
+            done();
+        });
+    });
+
+    after(function (done) {
+        helper.deleteRedisKeys(keysToDelete, () => listener.close(done));
+    });
+
+    it('should not fail when a template gets recreated', function (done) {
+        this.timeout(10000);
+
+        const createTemplateRequest = {
+            url: `http://${address}/api/v1/map/named?api_key=${apiKey}`,
+            method: 'POST',
+            headers: {
+                host: 'localhost',
+                'Content-Type': 'application/json'
+            },
+            body: template,
+            json: true
+        };
+
+        request(createTemplateRequest, (err, res, body) => {
+            if (err) {
+                return done(err);
+            }
+
+            assert.strictEqual(res.statusCode, 200);
+
+            const templateId = body.template_id;
+
+            keysToDelete['map_tpl|localhost'] = 0;
+
+            const previewRequest = {
+                url: `http://${address}/api/v1/map/static/named/${templateId}/256/256.png?api_key=${apiKey}`,
+                encoding: 'binary',
+                method: 'GET',
+                headers: {
+                    host: 'localhost'
+                }
+            };
+
+            request(previewRequest, (err, res) => {
+                if (err) {
+                    return done(err);
+                }
+
+                assert.strictEqual(res.statusCode, 200);
+
+                const preview = mapnik.Image.fromBytes(Buffer.from(res.body, 'binary'));
+
+                assert.strictEqual(preview.width(), 256);
+                assert.strictEqual(preview.height(), 256);
+
+                const templateUpdate = Object.assign({}, template);
+
+                const newQuery = 'select * from populated_places_simple_reduced limit 100';
+                templateUpdate.layergroup.analyses[0].params.query = newQuery;
+
+                const updateTemplateRequest = {
+                    url: `http://${address}/api/v1/map/named/${templateId}?api_key=${apiKey}`,
+                    method: 'PUT',
+                    headers: {
+                        host: 'localhost',
+                        'Content-Type': 'application/json'
+                    },
+                    body: templateUpdate,
+                    json: true
+                };
+
+                request(updateTemplateRequest, (err, res) => {
+                    if (err) {
+                        return done(err);
+                    }
+
+                    assert.strictEqual(res.statusCode, 200);
+
+                    request(previewRequest, (err, res) => {
+                        if (err) {
+                            return done(err);
+                        }
+
+                        const preview = mapnik.Image.fromBytes(Buffer.from(res.body, 'binary'));
+
+                        assert.strictEqual(preview.width(), 256);
+                        assert.strictEqual(preview.height(), 256);
+
+                        request(previewRequest, (err, res) => {
+                            if (err) {
+                                return done(err);
+                            }
+
+                            const preview = mapnik.Image.fromBytes(Buffer.from(res.body, 'binary'));
+
+                            assert.strictEqual(preview.width(), 256);
+                            assert.strictEqual(preview.height(), 256);
+
+                            const deleteTemplateRequest = {
+                                url: `http://${address}/api/v1/map/named/${templateId}?api_key=${apiKey}`,
+                                method: 'DELETE',
+                                headers: {
+                                    host: 'localhost',
+                                }
+                            };
+
+                            request(deleteTemplateRequest, (err) => {
+                                if (err) {
+                                    return done(err);
+                                }
+
+                                delete keysToDelete['map_tpl|localhost'];
+
+                                assert.strictEqual(res.statusCode, 200);
+
+                                request(createTemplateRequest, (err, res, body) => {
+                                    if (err) {
+                                        return done(err);
+                                    }
+
+                                    assert.strictEqual(res.statusCode, 200);
+
+                                    const templateId = body.template_id;
+
+                                    keysToDelete['map_tpl|localhost'] = 0;
+
+                                    const previewRequest = {
+                                        url: namedTileUrlTemplate({ address, templateId, apiKey }),
+                                        encoding: 'binary',
+                                        method: 'GET',
+                                        headers: {
+                                            host: 'localhost'
+                                        }
+                                    };
+
+                                    request(previewRequest, (err, res) => {
+                                        if (err) {
+                                            return done(err);
+                                        }
+
+                                        assert.strictEqual(res.statusCode, 200);
+
+                                        const preview = mapnik.Image.fromBytes(Buffer.from(res.body, 'binary'));
+
+                                        assert.strictEqual(preview.width(), 256);
+                                        assert.strictEqual(preview.height(), 256);
+
+                                        request(deleteTemplateRequest, (err) => {
+                                            if (err) {
+                                                return done(err);
+                                            }
+
+                                            delete keysToDelete['map_tpl|localhost'];
+
+                                            assert.strictEqual(res.statusCode, 200);
+
+                                            keysToDelete['user:localhost:mapviews:global'] = 0;
+                                            keysToDelete['user:localhost:mapviews:global'] = 5;
+
+                                            helper.deleteRedisKeys(keysToDelete, done);
+                                        });
+                                    });
+                                });
+                            });
+                        });
+                    });
+                });
+            });
+        });
+    });
+});
--- a/test/acceptance/named_maps_cache.js
+++ b/test/acceptance/named_maps_cache.js
@@ -1,13 +1,12 @@
 'use strict';

 require('../support/test_helper');
-var RedisPool = require('redis-mpool');

+const helper = require('../support/test_helper');
 var assert = require('../support/assert');
 var mapnik = require('windshaft').mapnik;
 var CartodbWindshaft = require('../../lib/cartodb/server');
 var serverOptions = require('../../lib/cartodb/server_options');
-var TemplateMaps = require('../../lib/cartodb/backends/template_maps.js');

 describe('named maps provider cache', function() {
    var server;
@@ -16,14 +15,8 @@ describe('named maps provider cache', function() {
        server = new CartodbWindshaft(serverOptions);
    });

-    // configure redis pool instance to use in tests
-    var redisPool = new RedisPool(global.environment.redis);
-
-    var templateMaps = new TemplateMaps(redisPool, {
-        max_user_templates: global.environment.maxUserTemplates
-    });
-
    var username = 'localhost';
+    const apikey = 1234;
    var templateName = 'template_with_color';

    var IMAGE_TOLERANCE = 20;
@@ -31,7 +24,7 @@ describe('named maps provider cache', function() {
    function createTemplate(color) {
        return {
            version: '0.0.1',
-            name: templateName,
+            name: `${templateName}_${color}`,
            auth: {
                method: 'open'
            },
@@ -56,17 +49,13 @@ describe('named maps provider cache', function() {
        };
    }

-    afterEach(function (done) {
-        templateMaps.delTemplate(username, templateName, done);
-    });
-
-    function getNamedTile(options, callback) {
+    function getNamedTile(templateId, options, callback) {
        if (!callback) {
            callback = options;
            options = {};
        }

-        var url = '/api/v1/map/named/' + templateName + '/all/' + [0,0,0].join('/') + '.png';
+        var url = '/api/v1/map/named/' + templateId + '/all/' + [0,0,0].join('/') + '.png';

        var requestOptions = {
            url: url,
@@ -88,60 +77,116 @@ describe('named maps provider cache', function() {

        assert.response(server, requestOptions, expectedResponse, function (res, err) {
            var img;
-            if (statusCode === 200) {
-                img = mapnik.Image.fromBytes(new Buffer(res.body, 'binary'));
+            if (res.statusCode === 200) {
+                img = mapnik.Image.fromBytes(new Buffer.from(res.body, 'binary'));
            }
            return callback(err, res, img);
        });
    }

+    function addTemplate (template, callback) {
+        const createTemplateRequest = {
+            url: `/api/v1/map/named?api_key=${apikey}`,
+            method: 'POST',
+            headers: {
+                host: username,
+                'Content-Type': 'application/json'
+            },
+            body: JSON.stringify(template)
+        };
+
+        const expectedResponse = {
+            status: 200,
+            headers: {
+                'Content-Type': 'application/json; charset=utf-8'
+            }
+        };
+
+        assert.response(server, createTemplateRequest, expectedResponse, (res, err) => {
+            let template;
+
+            if (res.statusCode === 200) {
+                template = JSON.parse(res.body);
+            }
+
+            return callback(err, res, template);
+        });
+    }
+
+
+    function deleteTemplate (templateId, callback) {
+        const deleteTemplateRequest = {
+            url: `/api/v1/map/named/${templateId}?api_key=${apikey}`,
+            method: 'DELETE',
+            headers: {
+                host: 'localhost',
+            }
+        };
+
+        const expectedResponse = {
+            status: 204,
+        };
+
+        assert.response(server, deleteTemplateRequest, expectedResponse, (res, err) => {
+            return callback(err, res);
+        });
+    }
+
    function previewFixture(color) {
        return './test/fixtures/provider/populated_places_simple_reduced-' + color + '.png';
    }

-    var colors = ['red', 'red', 'green', 'blue'];
+    var colors = ['black', 'red', 'green', 'blue'];
    colors.forEach(function(color) {
        it('should return an image estimating its bounds based on dataset', function (done) {
-            templateMaps.addTemplate(username, createTemplate(color), function (err) {
+            addTemplate(createTemplate(color), function (err, res, template) {
                if (err) {
                    return done(err);
                }
-                getNamedTile(function(err, res, img) {
+
+                getNamedTile(template.template_id, function(err, res, img) {
                    assert.ok(!err);
-                    assert.imageIsSimilarToFile(img, previewFixture(color), IMAGE_TOLERANCE, done);
-                });
-            });
-        });
-    });
+                    assert.imageIsSimilarToFile(img, previewFixture(color), IMAGE_TOLERANCE, (err) => {
+                        assert.ifError(err);

-    it('should fail to use template from named map provider after template deletion', function (done) {
-        var color = 'black';
-        templateMaps.addTemplate(username, createTemplate(color), function (err) {
-            if (err) {
-                return done(err);
-            }
-            getNamedTile(function(err, res, img) {
-                assert.ok(!err);
-                assert.imageIsSimilarToFile(img, previewFixture(color), IMAGE_TOLERANCE, function(err) {
-                    assert.ok(!err);
-
-                    templateMaps.delTemplate(username, templateName, function (err) {
-                        assert.ok(!err);
-
-                        getNamedTile({ statusCode: 404 }, function(err, res) {
-                            assert.ok(!err);
-                            assert.deepEqual(
-                                JSON.parse(res.body).errors,
-                                ["Template 'template_with_color' of user 'localhost' not found"]
-                            );
-
-                            // add template again so it's clean in afterEach
-                            templateMaps.addTemplate(username, createTemplate(color), done);
-                        });
+                        const keysToDelete = {};
+                        keysToDelete['map_tpl|localhost'] = 0;
+                        helper.deleteRedisKeys(keysToDelete, done);
                    });
                });
            });
        });
    });

+    it('should fail to use template from named map provider after template deletion', function (done) {
+        const color = 'black';
+        const templateId = `${templateName}_${color}`;
+
+        addTemplate(createTemplate(color), function (err) {
+            assert.ifError(err);
+
+            getNamedTile(templateId, function(err, res, img) {
+                assert.ifError(err);
+
+                assert.imageIsSimilarToFile(img, previewFixture(color), IMAGE_TOLERANCE, function (err) {
+                    assert.ifError(err);
+
+                    deleteTemplate(templateId, function (err) {
+                        assert.ifError(err);
+
+                        getNamedTile(templateId, { statusCode: 404 }, function(err, res) {
+                            assert.ifError(err);
+
+                            assert.deepEqual(
+                                JSON.parse(res.body).errors,
+                                ["Template 'template_with_color_black' of user 'localhost' not found"]
+                            );
+
+                            done();
+                        });
+                    });
+                });
+            });
+        });
+    });
 });
--- a/test/acceptance/ported/support/ported_server_options.js
+++ b/test/acceptance/ported/support/ported_server_options.js
@@ -50,6 +50,7 @@ module.exports = _.extend({}, serverOptions, {
    renderer: {
        mapnik: {
            poolSize: 4,//require('os').cpus().length,
+            poolMaxWaitingClients: 32,
            metatile: 1,
            bufferSize: 64,
            snapToGrid: false,
--- a/test/acceptance/stats/mapnik_stats_layergroup.js
+++ b/test/acceptance/stats/mapnik_stats_layergroup.js
@@ -583,6 +583,33 @@ describe(`[${desc}] Create mapnik layergroup`, function() {
        });
    });

+    it('should not provide a sample when the source table is empty', function (done) {
+        var testClient = new TestClient({
+            version: "1.4.0",
+            layers: [
+                {
+                    type: "mapnik",
+                    options: {
+                        sql: "SELECT * FROM test_table_100 limit 0",
+                        cartocss_version: "2.3.0",
+                        cartocss: "#layer { line-width:16; }",
+                        metadata: {
+                            sample: {
+                                num_rows: 30
+                            }
+                        }
+                    }
+                }
+            ]
+        });
+
+        testClient.getLayergroup(function(err, layergroup) {
+            assert.ifError(err);
+            assert.deepStrictEqual(layergroup.metadata.layers[0].meta.stats.sample, {});
+            testClient.drain(done);
+        });
+    });
+
    it('can specify sample columns', function(done) {
        var testClient = new TestClient({
            version: '1.4.0',
--- a/test/acceptance/templates.js
+++ b/test/acceptance/templates.js
@@ -5,7 +5,7 @@ var _           = require('underscore');
 var redis       = require('redis');
 var step        = require('step');
 var strftime    = require('strftime');
-var QueryTables = require('cartodb-query-tables');
+var QueryTables = require('cartodb-query-tables').queryTables;
 var NamedMapsCacheEntry = require('../../lib/cartodb/cache/model/named_maps_entry');
 var redis_stats_db = 5;

@@ -1416,7 +1416,7 @@ describe('template_api', function() {
          // See https://github.com/CartoDB/Windshaft-cartodb/issues/176
          helper.checkCache(res);
          var expectedSurrogateKey = [
-            new QueryTables.DatabaseTablesEntry([{dbname: 'test_windshaft_cartodb_user_1_db', schema_name: 'public',
+            new QueryTables.QueryMetadata([{dbname: 'test_windshaft_cartodb_user_1_db', schema_name: 'public',
                                   table_name: 'test_table_private_1'}]).key(),
              new NamedMapsCacheEntry('localhost', template_acceptance_open.name).key()
          ].join(' ');
@@ -1500,7 +1500,7 @@ describe('template_api', function() {
          // See https://github.com/CartoDB/Windshaft-cartodb/issues/176
          helper.checkCache(res);
          var expectedSurrogateKey = [
-            new QueryTables.DatabaseTablesEntry([{dbname: 'test_windshaft_cartodb_user_1_db', schema_name: 'public',
+            new QueryTables.QueryMetadata([{dbname: 'test_windshaft_cartodb_user_1_db', schema_name: 'public',
                                   table_name: 'test_table_private_1'}]).key(),
              new NamedMapsCacheEntry('localhost', template_acceptance_open.name).key()
          ].join(' ');
--- a/test/integration/query-tables.js
+++ b/test/integration/query-tables.js
@@ -9,7 +9,7 @@ var cartodbRedis = require('cartodb-redis');

 var PgConnection = require('../../lib/cartodb/backends/pg_connection');

-var QueryTables = require('cartodb-query-tables');
+var QueryTables = require('cartodb-query-tables').queryTables;


 describe('QueryTables', function() {
@@ -34,18 +34,15 @@ describe('QueryTables', function() {

    it('should return an object with affected tables array and last updated time', function(done) {
        var query = 'select * from test_table';
-        QueryTables.getAffectedTablesFromQuery(connection, query, function(err, result) {
+        QueryTables.getQueryMetadataModel(connection, query, function(err, result) {
            assert.ok(!err, err);

            assert.equal(result.getLastUpdatedAt(), 1234567890123);

            assert.equal(result.tables.length, 1);
-            assert.deepEqual(result.tables[0], {
-                dbname: 'test_windshaft_cartodb_user_1_db',
-                schema_name: 'public',
-                table_name: 'test_table',
-                updated_at: new Date(1234567890123)
-            });
+            assert.equal(result.tables[0].dbname, 'test_windshaft_cartodb_user_1_db');
+            assert.equal(result.tables[0].schema_name, 'public');
+            assert.equal(result.tables[0].table_name, 'test_table');

            done();
        });
@@ -53,18 +50,15 @@ describe('QueryTables', function() {

    it('should work with private tables', function(done) {
        var query = 'select * from test_table_private_1';
-        QueryTables.getAffectedTablesFromQuery(connection, query, function(err, result) {
+        QueryTables.getQueryMetadataModel(connection, query, function(err, result) {
            assert.ok(!err, err);

            assert.equal(result.getLastUpdatedAt(), 1234567890123);

            assert.equal(result.tables.length, 1);
-            assert.deepEqual(result.tables[0], {
-                dbname: 'test_windshaft_cartodb_user_1_db',
-                schema_name: 'public',
-                table_name: 'test_table_private_1',
-                updated_at: new Date(1234567890123)
-            });
+            assert.equal(result.tables[0].dbname, 'test_windshaft_cartodb_user_1_db');
+            assert.equal(result.tables[0].schema_name, 'public');
+            assert.equal(result.tables[0].table_name, 'test_table_private_1');

            done();
        });
--- a/test/support/prepare_db.sh
+++ b/test/support/prepare_db.sh
@@ -12,7 +12,6 @@

 PREPARE_REDIS=yes
 PREPARE_PGSQL=yes
-DOWNLOAD_SQL_FILES=yes

 while [ -n "$1" ]; do
  OPTION=$(echo "$1" | tr -d '[:space:]')
@@ -22,9 +21,6 @@ while [ -n "$1" ]; do
  elif [[ "$OPTION" == "--skip-redis" ]]; then
    PREPARE_REDIS=no
    shift; continue
-  elif [[ "$OPTION" == "--no-sql-download" ]]; then
-    DOWNLOAD_SQL_FILES=no
-    shift; continue
  else
    shift; continue;
  fi
@@ -77,30 +73,16 @@ if test x"$PREPARE_PGSQL" = xyes; then
  echo "preparing postgres..."
  dropdb "${TEST_DB}"
  createdb -Ttemplate_postgis -EUTF8 "${TEST_DB}" || die "Could not create test database"
+  psql -c "CREATE EXTENSION IF NOT EXISTS cartodb CASCADE;" ${TEST_DB}

  LOCAL_SQL_SCRIPTS='analysis_catalog windshaft.test gadm4 countries_null_values ported/populated_places_simple_reduced cdb_analysis_check cdb_invalidate_varnish'
-  REMOTE_SQL_SCRIPTS='CDB_QueryStatements CDB_QueryTables CDB_CartodbfyTable CDB_TableMetadata CDB_ForeignTable CDB_UserTables CDB_ColumnNames CDB_ZoomFromScale CDB_OverviewsSupport CDB_Overviews CDB_QuantileBins CDB_JenksBins CDB_HeadsTailsBins CDB_EqualIntervalBins CDB_Hexagon CDB_XYZ CDB_EstimateRowCount CDB_RectangleGrid'
-
-  if test x"$DOWNLOAD_SQL_FILES" = xyes; then
-    CURL_ARGS=""
-    for i in ${REMOTE_SQL_SCRIPTS}
-    do
-        CURL_ARGS="${CURL_ARGS}\"https://github.com/CartoDB/cartodb-postgresql/raw/master/scripts-available/$i.sql\" -o sql/$i.sql "
-    done
-    echo "Downloading and updating: ${REMOTE_SQL_SCRIPTS}"
-    echo ${CURL_ARGS} | xargs curl -L -s
-  fi
-
-  psql -c "CREATE EXTENSION IF NOT EXISTS plpythonu;" ${TEST_DB}
-  ALL_SQL_SCRIPTS="${REMOTE_SQL_SCRIPTS} ${LOCAL_SQL_SCRIPTS}"
-  for i in ${ALL_SQL_SCRIPTS}
+  for i in ${LOCAL_SQL_SCRIPTS}
  do
    cat sql/${i}.sql |
-      sed -e 's/cartodb\./public./g' -e "s/''cartodb''/''public''/g" |
-      sed "s/:PUBLICUSER/${PUBLICUSER}/" |
-      sed "s/:PUBLICPASS/${PUBLICPASS}/" |
-      sed "s/:TESTUSER/${TESTUSER}/" |
-      sed "s/:TESTPASS/${TESTPASS}/" |
+      sed -e "s/:PUBLICUSER/${PUBLICUSER}/g" |
+      sed -e "s/:PUBLICPASS/${PUBLICPASS}/g" |
+      sed -e "s/:TESTUSER/${TESTUSER}/g" |
+      sed -e "s/:TESTPASS/${TESTPASS}/g" |
      PGOPTIONS='--client-min-messages=WARNING' psql -q -v ON_ERROR_STOP=1 ${TEST_DB} > /dev/null || exit 1
  done
 fi
--- a/test/support/sql/windshaft.test.sql
+++ b/test/support/sql/windshaft.test.sql
@@ -11,17 +11,21 @@ SET standard_conforming_strings = off;
 SET check_function_bodies = false;
 SET client_min_messages = warning;
 SET escape_string_warning = off;
-SET search_path = public, pg_catalog;
+SET search_path = public, cartodb, pg_catalog;
 SET default_tablespace = '';
 SET default_with_oids = false;

 -- public user role
 DROP USER IF EXISTS :PUBLICUSER;
 CREATE USER :PUBLICUSER WITH PASSWORD ':PUBLICPASS';
+GRANT USAGE ON SCHEMA cartodb TO :PUBLICUSER;
+GRANT ALL ON CDB_TableMetadata TO :PUBLICUSER;

 -- db owner role
 DROP USER IF EXISTS :TESTUSER;
 CREATE USER :TESTUSER WITH PASSWORD ':TESTPASS';
+GRANT USAGE ON SCHEMA cartodb TO :TESTUSER;
+GRANT ALL ON CDB_TableMetadata TO :TESTUSER;

 -- regular user role 1
 DROP USER IF EXISTS test_windshaft_regular1;
@@ -184,12 +188,6 @@ INSERT INTO test_table_private_1 SELECT * from test_table;

 GRANT ALL ON TABLE test_table_private_1 TO :TESTUSER;

-CREATE TABLE IF NOT EXISTS
-  CDB_TableMetadata (
-    tabname regclass not null primary key,
-    updated_at timestamp with time zone not null default now()
-  );
-
 INSERT INTO CDB_TableMetadata (tabname, updated_at) VALUES ('test_table'::regclass, '2009-02-13T23:31:30.123Z');
 INSERT INTO CDB_TableMetadata (tabname, updated_at) VALUES ('test_table_private_1'::regclass, '2009-02-13T23:31:30.123Z');

--- a/test/unit/cartodb/stats/reporter/named-map-provider.js
+++ b/test/unit/cartodb/stats/reporter/named-map-provider.js
@@ -0,0 +1,61 @@
+'use strict';
+
+const assert = require('assert');
+const NamedMapProviderReporter = require('../../../../../lib/cartodb/stats/reporter/named-map-provider');
+
+describe('named-map-provider-reporter', function () {
+    it('should report metrics every 100 ms', function (done) {
+        const oldStatsClient = global.statsClient;
+
+        global.statsClient = {
+            gauge: function (metric, value) {
+                this[metric] = value;
+            }
+        };
+
+        const dummyCacheEntries = [
+            {
+                k: 'foo:template_1',
+                v: { 'instantiation_1': 1 }
+            },
+            {
+                k: 'bar:template_2',
+                v: { 'instantiation_1': 1, 'instantiation_2': 2 }
+            },
+            {
+                k: 'buz:template_3',
+                v: { 'instantiation_1': 1, 'instantiation_2': 2, 'instantiation_3': 3 }
+            }
+        ];
+
+        const reporter = new NamedMapProviderReporter({
+            namedMapProviderCache: {
+                providerCache: {
+                    dump: () => dummyCacheEntries,
+                    length: dummyCacheEntries.length
+                }
+            },
+            intervalInMilliseconds: 100
+        });
+
+        reporter.start();
+
+        setTimeout(() => {
+            reporter.stop();
+
+            assert.strictEqual(
+                global.statsClient['windshaft.named-map-provider-cache.named-map.count'],
+                3
+            );
+
+            assert.strictEqual(
+                global.statsClient['windshaft.named-map-provider-cache.named-map.instantiation.count'],
+                6
+            );
+
+            global.statsClient = oldStatsClient;
+
+            done();
+        }, 110);
+    });
+});
Author	SHA1	Message	Date
Daniel García Aubert	b881bec668	Release 7.2.0	2019-09-30 17:24:36 +02:00
Raúl Marín	e6f3c63675	Merge pull request #1125 from CartoDB/update_querytables Update cartodb-query-tables to 0.6.2	2019-09-19 15:08:00 +02:00
Raul Marin	f2afece658	Update cartodb-query-tables to 0.6.3	2019-09-19 13:40:31 +02:00
Raul Marin	b69ceeeee8	Update cartodb-query-tables to 0.6.2	2019-09-18 18:21:38 +02:00
Raúl Marín	b93caa7410	Merge pull request #1123 from Algunenano/query_tables_update Query tables update	2019-09-18 11:26:51 +02:00
Raul Marin	7476879bde	Update cartodb-query-tables to 0.6.1	2019-09-17 13:20:08 +02:00
Daniel G. Aubert	ce884732f3	Merge pull request #1122 from CartoDB/fix-missing-template Improve named map provider cache	2019-09-16 13:37:52 +02:00
Daniel García Aubert	455932b032	Update NEWS	2019-09-16 13:26:50 +02:00
Daniel García Aubert	68a9b4ccae	Typo	2019-09-16 13:09:23 +02:00
Daniel García Aubert	f6c205baf9	Typo	2019-09-16 13:08:10 +02:00
Daniel García Aubert	738d10409f	Style	2019-09-16 11:35:40 +02:00
Daniel García Aubert	89c5a3e0a9	Merge branch 'master' into fix-missing-template	2019-09-16 11:30:01 +02:00
Daniel García Aubert	5dac9d956c	Add test	2019-09-16 11:18:50 +02:00
Daniel García Aubert	c19c723795	Add named map providers reporter to gather some stats	2019-09-13 20:01:03 +02:00
Daniel García Aubert	788cd9d6fb	Be explicit while forwarding parametes	2019-09-13 18:15:11 +02:00
Daniel García Aubert	824d41ef0f	Consitent quotes	2019-09-13 18:11:47 +02:00
Daniel García Aubert	329b5d9b9e	Use ES6 class syntax	2019-09-13 18:07:50 +02:00
Daniel García Aubert	b55c2ec55c	Use template string	2019-09-13 18:03:39 +02:00
Daniel García Aubert	d99e5a44f5	Use const instead of var	2019-09-13 18:00:13 +02:00
Daniel García Aubert	e8d5e42300	Use Object.assign() instead of _.defaults()	2019-09-13 17:58:23 +02:00
Daniel García Aubert	c0afd42fa2	Use template strings instead of dot module	2019-09-13 17:53:10 +02:00
Daniel García Aubert	1bb6a2ac0d	Move invalidation closer to its definition	2019-09-13 17:42:56 +02:00
Daniel García Aubert	3d2f554be9	Use early return pattern	2019-09-13 16:53:53 +02:00
Raul Marin	a673e6d138	Update dependencies	2019-09-13 16:32:37 +02:00
Raul Marin	f9b6e92745	Stop throwing up warnings during testing Yeah, I test with different repos and branches. Big deal. Shut up.	2019-09-13 16:26:46 +02:00
Daniel García Aubert	6229455d25	Remove mechanism to reset named map's provider as, in the end, it's reading from storage (redis) always so cache isn't doing its job. There is already a mechanism to invalidate cache entry when a template is modified (see template-maps emits on "update" and "delete", and listeners attached at server startup)	2019-09-12 20:34:18 +02:00
Daniel García Aubert	9d6726227a	Add maxAge param to lru-cache to be able to refresh entries when staled	2019-09-12 18:02:13 +02:00
Daniel García Aubert	64b4efef17	Do not cache map template CRUD errors in Named Map provider	2019-09-12 17:23:19 +02:00
Pablo Alonso	eb71601cd6	Merge pull request #1121 from CartoDB/professional-plan-renaming Professional -> Individual	2019-09-10 16:31:01 +02:00
Daniel G. Aubert	0297e09c17	Merge pull request #1120 from CartoDB/improve-metadata-sample Improve efficiency of query samples (esp. for FDW's)	2019-09-04 12:37:41 +02:00
Daniel García Aubert	8f6447b67e	Update NEWS	2019-09-04 12:30:12 +02:00
Pablo Alonso Garcia	4ae4ce477f	Professional -> Individual	2019-09-02 17:32:06 +02:00
Daniel García Aubert	7bf5deb4c1	Styling	2019-09-02 16:40:43 +02:00
Daniel García Aubert	2fbd9893bd	Going green: do not fail when source is empty	2019-09-02 14:09:13 +02:00
Daniel García Aubert	5a01c1c5eb	Going red: test to avoid error when the source table is empty	2019-09-02 13:38:28 +02:00
Daniel García Aubert	9a85b661b0	Remove safe limit	2019-08-23 18:04:19 +02:00
Daniel García Aubert	d4d981909b	Remove unused code	2019-08-23 17:43:20 +02:00
Daniel García Aubert	16035131bc	Rename method	2019-08-23 17:25:37 +02:00
Daniel García Aubert	b2adb8f058	Use substituteDummyTokens	2019-08-23 17:21:01 +02:00
Daniel García Aubert	780cb80c8c	Move method	2019-08-23 17:16:35 +02:00
Daniel García Aubert	2a8a8f6e6a	Export methods to get max, min, values od a column and sample based on a range	2019-08-23 17:10:46 +02:00
Daniel García Aubert	850bda9669	Use modified sample method	2019-08-23 17:09:53 +02:00
Daniel García Aubert	25e3395580	Modify sample metadata	2019-08-23 17:09:24 +02:00
Román Jiménez	71dba04d83	Merge pull request #1119 from CartoDB/fix-curl-docs-windows Fix cURL docs for Windows users by using files	2019-08-23 11:08:15 +02:00
Román Jiménez	2a3312e779	Fix cURL docs for Windows users by using files	2019-08-22 18:12:27 +02:00
Daniel G. Aubert	c4484dcc54	Merge pull request #1115 from CartoDB/upgrade-windshaft-5.6.0 Update windshaft to version 5.6.0	2019-07-30 12:24:08 +02:00
Daniel García Aubert	61883b13ef	Update windshaft to version 5.6.0	2019-07-30 12:00:28 +02:00
Daniel García Aubert	a220af4fad	Please jshint	2019-07-29 19:20:38 +02:00
Daniel García Aubert	945c122dda	Update windshaft devel branch	2019-07-29 19:14:27 +02:00
Daniel García Aubert	61b66e88d5	Update NEWS	2019-07-23 18:08:48 +02:00
Daniel G. Aubert	9dfd5f3012	Merge pull request #1114 from CartoDB/upgrade-windshaft-5.5.1 Upgrade windshaft to version 5.5.1	2019-07-23 15:22:28 +02:00
Daniel García Aubert	ff634e32db	Upgrade windshaft to version 5.5.1	2019-07-23 13:47:56 +02:00
Daniel García Aubert	3b583ebd56	Update windshaft#master	2019-07-23 12:15:49 +02:00
Daniel García Aubert	1a9b410540	Update to master branch	2019-07-23 11:14:40 +02:00
Daniel García Aubert	d7a439477c	Use https	2019-07-23 11:03:55 +02:00
Daniel García Aubert	c7f3da237c	Upgrade windshaft to devel branch with fix	2019-07-23 09:13:09 +02:00
Daniel García Aubert	da144de57b	Update windshaft devel branch	2019-07-22 17:14:31 +02:00
Raúl Marín	58d38682eb	Merge pull request #1113 from Algunenano/audit_dependencies package-lock.json: Update js-yaml and lodash	2019-07-16 16:29:44 +02:00
Raul Marin	402579f7e2	package-lock.json: Update js-yaml and lodash	2019-07-16 16:11:03 +02:00
Raúl Marín	ebf373e680	Merge pull request #1111 from Algunenano/cartodbless Render MVTs and aggregations without cartodb-postgresql	2019-07-16 16:06:40 +02:00
Raul Marin	ffe347cfdc	package.json: Revert changes to match master	2019-07-16 13:44:13 +02:00
Raul Marin	b572b979a1	Style	2019-07-16 13:43:46 +02:00
Raul Marin	de49aa0bd4	Aggregation: Style improvements	2019-07-16 12:49:09 +02:00
Raul Marin	286daa9bec	Simplify aggregation templates	2019-07-16 12:49:09 +02:00
Raul Marin	65beb6e460	Tweak the tests to accept Mapnik precision as valid	2019-07-16 12:49:09 +02:00
Raul Marin	63b6af2ac7	Query utils: Use webmercator utils, reuse code and always substitute tokens	2019-07-16 12:49:09 +02:00
Raul Marin	bdbe132311	Update lodash	2019-07-16 12:49:09 +02:00
Raul Marin	492bcbfdaa	Aggregation tests: Enable mapnik and rework the complete cells test Before it used to test exact tiles, now it test that if a cell/value is included it will contain exactly the features that we expect	2019-07-16 12:48:58 +02:00
Raul Marin	46600bf4fc	Please jshint	2019-07-16 12:48:58 +02:00
Raul Marin	aed456bf32	Cluster: Use new webmercator utilities	2019-07-16 12:48:58 +02:00
Raul Marin	d3e807583a	Use proper mode	2019-07-16 12:48:58 +02:00
Raul Marin	262f957218	Aggregation: Improve speeeeeeed	2019-07-16 12:48:58 +02:00
Raul Marin	8454eef6e9	Aggregation: Extract the query to get the grid data	2019-07-16 12:48:58 +02:00
Raul Marin	892479d9b9	API changes	2019-07-16 12:48:58 +02:00
Raul Marin	5e24f650af	Rework how aggregations are calculated Pending fixing Mapnik tiles (pg-mvt work ok)	2019-07-16 12:48:12 +02:00
Raul Marin	de38f1f6fd	Tests: Avoid using st_buffer(geography) It's really slow with PROJ6 since it involves creating different projections for almost each point, and that's particularly slow in the new release	2019-07-16 12:48:12 +02:00
Raul Marin	a3e8f45552	WIP to change how aggregations are calculated	2019-07-16 12:48:12 +02:00
Raul Marin	cd8624ae2d	Improve subquery naming	2019-07-16 12:48:12 +02:00
Raul Marin	7b731a24d1	Overviews Adapter: Do not check overviews if using MVT pure style or aggregations	2019-07-16 12:48:12 +02:00
Daniel G. Aubert	f9bd3d39f0	Merge pull request #1112 from CartoDB/upgrade-windshaft-5.5.0 Upgrade windshaft to version 5.5.0	2019-07-16 12:38:52 +02:00
Daniel García Aubert	dbc926b0b8	Upgrade windshaft to version 5.5.0	2019-07-16 12:27:49 +02:00
Daniel G. Aubert	0f1a3dfb34	Merge pull request #1055 from CartoDB/1054-test Test for time dimension hour format	2019-07-11 16:04:06 +02:00
Daniel G. Aubert	cbb9285fb3	Merge pull request #863 from CartoDB/cartofante removing raster image on timeout error	2019-07-11 15:59:38 +02:00
Daniel García Aubert	d7412aab45	Update NEWS	2019-07-09 16:53:19 +02:00
Daniel G. Aubert	d8ca29509f	Merge pull request #1109 from CartoDB/update-query-tables [WIP] Update cartodb-query-tables	2019-07-09 16:50:16 +02:00
Daniel García Aubert	619cad9c35	Typo	2019-07-09 15:44:34 +02:00
Daniel García Aubert	bca723bcf3	PLease jshint	2019-07-09 14:59:16 +02:00
Daniel García Aubert	ef39f23d1f	Fix tests to use map-config builder properly	2019-07-09 14:47:40 +02:00
Daniel García Aubert	c066e2c3cf	Update cartodb-query-tables to version 0.5.0	2019-07-09 12:45:33 +02:00
Daniel García Aubert	45af291e6a	Remove test filter	2019-07-08 18:08:08 +02:00
Daniel García Aubert	a84184852e	Going green: skip analyses table while computing max-age directive	2019-07-08 18:05:30 +02:00
Daniel García Aubert	6cdb872bb5	Going red: add test to check the bad behavour of max-age directive when an analysis table is in affected tables of the map	2019-07-08 17:55:04 +02:00
Daniel G. Aubert	27b76aefd2	Merge pull request #1107 from CartoDB/max-age-directive Set directive 'max-age' to a fallback value	2019-07-05 15:38:33 +02:00
Daniel García Aubert	8820e34870	Update news	2019-07-05 15:31:17 +02:00
Daniel García Aubert	4def4b0341	Improve condition	2019-07-04 16:41:30 +02:00
Daniel García Aubert	ec0c0eb810	Improve readability	2019-07-04 16:24:17 +02:00
Daniel García Aubert	5ca498d0f3	Typo	2019-07-04 16:18:43 +02:00
Daniel García Aubert	a894194b6b	Add more specific tests	2019-07-04 15:58:39 +02:00
Daniel García Aubert	2e8a5d0d86	Add test	2019-07-04 15:34:37 +02:00
Daniel García Aubert	a374deaf30	Please linter	2019-07-03 17:16:09 +02:00
Daniel García Aubert	bc29587c55	Set directive 'max-age' to 5 min when there are affacted tables where we can't know when were updated for the last time, e.g: non cartodified tables or foreing tables without cartodb support	2019-07-03 17:15:14 +02:00
Simon Martín	9cb149fa32	Merge pull request #1103 from CartoDB/fix-torque-cartocss Upgrade windshaft to version 5.4.0	2019-06-25 11:17:14 +02:00
Simon Martín	412c4af7b0	Upgrade windshaft to version 5.4.0	2019-06-25 11:01:02 +02:00
Simon Martín	28ff0bdfc4	updating lock2	2019-06-24 17:33:32 +02:00
Simon Martín	f9250cda1a	updating lock	2019-06-24 17:24:37 +02:00
Raúl Marín	b821b9b038	Merge pull request #1105 from Algunenano/carto-package-mapnik16 carto-package.json: Update mapnik dependency to match what's installed	2019-06-24 17:13:22 +02:00
Raul Marin	6f35f9cbbb	carto-package.json: Update mapnik dependency to match what's installed	2019-06-24 17:05:06 +02:00
Simon Martín	23d9cd81c8	test torque fix	2019-06-21 15:26:22 +02:00
Daniel G. Aubert	dc63edf0c7	Merge pull request #1099 from CartoDB/abaculus-cartofy Upgrade windshaft: integrate abaculus into cartonik	2019-06-17 15:52:46 +02:00
Daniel García Aubert	ab6ee52b43	Merge branch 'master' into abaculus-cartofy	2019-06-17 15:42:12 +02:00
Daniel García Aubert	3121d907c1	Update NEWS	2019-06-17 15:40:31 +02:00
Daniel García Aubert	b09a35f272	Upgrade windshaft to version 5.3.0	2019-06-17 15:37:51 +02:00
Daniel García Aubert	f21eda2b40	Adapt endpoints to the new preview interface	2019-06-17 10:43:30 +02:00
Daniel García Aubert	ef4370c213	Update devel branch	2019-06-12 18:33:00 +02:00
Rafa de la Torre	9f03e978dd	Merge pull request #1102 from CartoDB/update-turbo-carto-0.21.2 Update turbo-carto to 0.21.2	2019-06-12 10:54:04 +02:00
Rafa de la Torre	54190a9cd2	Update turbo-carto to 0.21.2	2019-06-12 10:37:53 +02:00
Daniel García Aubert	f43ccd4c4e	Update devel branch	2019-06-10 19:04:22 +02:00
Daniel García Aubert	20fe04de38	Do not get headers from abaculus	2019-06-10 13:16:05 +02:00
Daniel García Aubert	dc16f4cebf	Update windshaft devel branch	2019-06-10 12:49:56 +02:00
Raúl Marín	bfaf764f13	Merge pull request #1098 from Algunenano/extension_changes Adapt to extension changes	2019-06-03 17:53:53 +02:00
Raul Marin	62d9fb1365	Docker: Install cartodb extension for testing purposes	2019-06-03 16:44:59 +02:00
Raul Marin	9f2b5330d5	Install the cartodb extension directly	2019-06-03 16:23:41 +02:00
Raul Marin	9a552a7cc4	Adapt to fully qualification in the extension	2019-06-03 13:41:22 +02:00
Raúl Marín	08998a3d17	Merge pull request #1096 from CartoDB/mapnik15 Mapnik 3.0.15	2019-05-21 12:34:12 +02:00
Raul Marin	b41f43f4bf	Mapnik 3.0.15	2019-05-21 12:06:07 +02:00
Javier Goizueta	b0d1d5a07a	Fix test	2018-10-31 12:39:06 +01:00
Javier Goizueta	c63380427e	Fix comment	2018-10-31 12:02:23 +01:00
Javier Goizueta	102e75ce95	Add test for hours time dimension It tests the problem solved in #1054	2018-10-31 12:00:40 +01:00
Simon Martín	47c8e828df	removing raster image on timeout error	2018-02-02 16:38:47 +01:00