Compare commits

..

59 Commits

Author SHA1 Message Date
Raul Ochoa
f2f342e14c Release 2.12.0 2015-08-27 17:46:29 +02:00
Raul Ochoa
11bd07ee6b Merge pull request #328 from CartoDB/upgrade-windshaft
Upgrades windshaft to 0.51.0
2015-08-27 17:39:14 +02:00
Raul Ochoa
fa85dcbc4c Upgrades windshaft to 0.51.0 2015-08-27 17:24:38 +02:00
Raul Ochoa
473ae13a03 Merge pull request #327 from CartoDB/http-agent-configuration
Make http and https globalAgent options configurable
2015-08-27 16:44:55 +02:00
Raul Ochoa
0561a28a4d Make http and https globalAgent options configurable 2015-08-27 16:28:16 +02:00
Raul Ochoa
b3467116fe Stubs next version 2015-08-26 09:55:15 +02:00
Raul Ochoa
e667d10eb8 Release 2.11.0 2015-08-26 09:52:53 +02:00
Raul Ochoa
a5bda00cdd Merge pull request #326 from CartoDB/upgrade-windshaft
Upgrades windshaft to 0.50.0
2015-08-25 19:32:28 +02:00
Raul Ochoa
627bc672bc Upgrades windshaft to 0.50.0 2015-08-25 19:27:00 +02:00
Raul Ochoa
9ef96080a6 Log PID on start 2015-08-24 12:45:21 +02:00
Raul Ochoa
bf4844e664 Stubs next version 2015-08-18 16:35:02 +02:00
Raul Ochoa
06f454abcf Release 2.10.0 2015-08-18 16:33:44 +02:00
Raul Ochoa
9d4e4a99bc Merge pull request #325 from CartoDB/upgrade-windshaft-tilelive-cache
Upgrades windshaft
2015-08-18 16:29:22 +02:00
Raul Ochoa
b77be76f51 Bump version and add news 2015-08-18 15:22:27 +02:00
Raul Ochoa
3121ed9a95 Config samples for metatileCache with default behaviour 2015-08-18 15:18:58 +02:00
Raul Ochoa
37dfd8fc12 Upgrade windshaft 2015-08-18 15:18:44 +02:00
Raul Ochoa
de2719b0c5 Stubs next version 2015-08-06 18:03:00 +02:00
Raul Ochoa
e3d5abc9a2 Release 2.9.0 2015-08-06 18:02:09 +02:00
Raul Ochoa
ea7a5da1c1 Merge pull request #324 from CartoDB/memory-stats
Send memory usage stats
2015-08-06 16:12:18 +02:00
Raul Ochoa
2e063cc2d2 Send memory usage stats 2015-08-06 16:06:42 +02:00
Raul Ochoa
decd9077e4 Stubs next version 2015-07-15 11:44:59 +02:00
Raul Ochoa
f6c47bf85e Release 2.8.0 2015-07-15 11:43:54 +02:00
Raul Ochoa
423191c13b Merge pull request #319 from CartoDB/upgrade-windshaft
Upgrades windshaft to 0.48.0
2015-07-15 11:43:09 +02:00
Raul Ochoa
7a5928d957 Upgrades windshaft to 0.48.0 2015-07-15 11:31:26 +02:00
Raul Ochoa
436c334f5a Stubs next version 2015-07-14 16:33:12 +02:00
Raul Ochoa
4f84138ade Release 2.7.2 2015-07-14 16:32:04 +02:00
Raul Ochoa
a0d86ac5dc Update news 2015-07-14 16:28:51 +02:00
Raul Ochoa
e2be4f1275 Merge pull request #314 from CartoDB/use-cdb-query-tables-text
Use CDB_QueryTablesText instead of CDB_QueryTables
2015-07-14 16:20:23 +02:00
Raul Ochoa
9a393fa793 Adds some notes about uv_threadpool_size and mapnik renderer pool size 2015-07-07 12:27:09 +02:00
Raul Ochoa
7614f72df6 Stubs next version 2015-07-06 11:58:08 +02:00
Raul Ochoa
ef2db78567 Release 2.7.1 2015-07-06 11:56:30 +02:00
Raul Ochoa
8708468444 fix 2.7.0 release date 2015-07-06 11:56:18 +02:00
Raul Ochoa
cd28a4fbcc redis-mpool noReadyCheck and unwatchOnRelease options from config
do not extend them as it disallows to pick from config
2015-07-06 11:52:34 +02:00
Raul Ochoa
19f488095b Stubs next version 2015-07-06 01:07:30 +02:00
Raul Ochoa
cd65c6dd0e Release 2.7.0 2015-07-06 01:05:28 +02:00
Raul Ochoa
0c670cfdfd Merge pull request #318 from CartoDB/upgrade-windshaft
Upgrades windshaft to 0.47.0
2015-07-05 21:51:34 +02:00
Raul Ochoa
27ff1ac4f6 Upgrades windshaft to 0.47.0 2015-07-05 21:16:12 +02:00
Raul Ochoa
3f06de93f7 Merge pull request #317 from CartoDB/improve-redis
Improve redis
2015-07-05 21:11:34 +02:00
Raul Ochoa
0da6495330 Fixes unwatchOnRelease redis config 2015-07-05 21:00:25 +02:00
Raul Ochoa
bf24347328 Exposes redis noReadyCheck config 2015-07-05 20:59:52 +02:00
Raul Ochoa
7a5d73f9df Upgrades redis-mpool 2015-07-05 20:58:39 +02:00
Raul Ochoa
0d9f34fd48 Stubs next version 2015-07-02 15:48:58 +02:00
Raul Ochoa
da55a3bdd2 Release 2.6.1 2015-07-02 15:47:40 +02:00
Raul Ochoa
333334e598 Upgrades windshaft 2015-07-02 15:41:48 +02:00
Raul Ochoa
7168e4410c Stubs next version 2015-07-02 14:12:41 +02:00
Raul Ochoa
3ff8571f4a Release 2.6.0 2015-07-02 14:11:39 +02:00
Raul Ochoa
75ddcbbd01 Updates windshaft to 0.46.0 and documents new config formatMetatile 2015-07-02 13:28:37 +02:00
Raul Ochoa
9b3e18f333 Merge pull request #315 from CartoDB/readme_update
Update instructions
2015-07-01 09:37:57 +02:00
Juan Ignacio Sánchez Lara
fcb0a4a7e6 Less specific upgrade help 2015-07-01 07:22:37 +02:00
Juan Ignacio Sánchez Lara
94e38cef9d Update instructions 2015-06-30 11:00:32 +02:00
Raul Ochoa
9e30f05e7d Reverts to use cdb branch as is already published 2015-06-29 16:46:07 +02:00
Raul Ochoa
0df725112b Update CDB_QueryTables function 2015-06-29 16:42:55 +02:00
Raul Ochoa
9ea2029f81 Deprecating scripts from tools directory 2015-06-25 18:07:48 +02:00
Raul Ochoa
2715f47a22 Points CDB_QueryTables script to the branch with CDB_QueryTablesText 2015-06-24 19:07:41 +02:00
Rafa de la Torre
90d0b23441 Use CDB_QueryTablesText instead of CDB_QueryTables
This avoids trouble with len(schema.table_name) > 63
See https://github.com/CartoDB/cartodb-postgresql/issues/86
2015-06-24 15:43:04 +02:00
Raul Ochoa
b59e0a00a0 Merge pull request #313 from CartoDB/cartodb-layer-docs
Update Map-API docs
2015-06-23 15:04:16 +02:00
Andrew Thompson
790571fd2c Update Map-API docs
It was not clear to me what the difference was between a mapnik layer and a cartodb layer until I read the comments in the Mapconfig spec! Let's save some users the extra step :)
2015-06-22 11:14:57 -04:00
Raul Ochoa
6ecebae110 Adds test to validate (once it is fixed) long table names do not fail 2015-06-18 16:29:59 +02:00
Raul Ochoa
849470a3c0 Stubs next version 2015-06-18 12:55:38 +02:00
15 changed files with 724 additions and 417 deletions

95
NEWS.md
View File

@@ -1,5 +1,100 @@
# Changelog
## 2.12.0
Released 2015-08-27
Announcements:
- Upgrades windshaft to [0.51.0](https://github.com/CartoDB/Windshaft/releases/tag/0.51.0)
New features:
- Make http and https globalAgent options configurable
* If config is not provided it configures them with default values
## 2.11.0
Released 2015-08-26
Announcements:
- Upgrades windshaft to [0.50.0](https://github.com/CartoDB/Windshaft/releases/tag/0.50.0)
## 2.10.0
Released 2015-08-18
New features:
- Exposes metatile cache configuration for tilelive-mapnik, see configuration sample files for more information.
Announcements:
- Upgrades windshaft to [0.49.0](https://github.com/CartoDB/Windshaft/releases/tag/0.49.0)
## 2.9.0
Released 2015-08-06
New features:
- Send memory usage stats
## 2.8.0
Released 2015-07-15
Announcements:
- Upgrades windshaft to [0.48.0](https://github.com/CartoDB/Windshaft/releases/tag/0.48.0)
## 2.7.2
Released 2015-07-14
Enhancements:
- Replaces `CDB_QueryTables` with `CDB_QueryTablesText` to avoid issues with long schema+table names
## 2.7.1
Released 2015-07-06
Bug fixes:
- redis-mpool `noReadyCheck` and `unwatchOnRelease` options from config and defaulted
## 2.7.0
Released 2015-07-06
Announcements:
- Upgrades windshaft to [0.47.0](https://github.com/CartoDB/Windshaft/releases/tag/0.47.0)
- Upgrades redis-mpool to [0.4.0](https://github.com/CartoDB/node-redis-mpool/releases/tag/0.4.0)
New features:
- Exposes redis `noReadyCheck` config
Bug fixes:
- Fixes `unwatchOnRelease` redis config
## 2.6.1
Released 2015-07-02
Announcements:
- Upgrades windshaft to [0.46.1](https://github.com/CartoDB/Windshaft/releases/tag/0.46.1)
## 2.6.0
Released 2015-07-02
Announcements:
- Upgrades windshaft to [0.46.0](https://github.com/CartoDB/Windshaft/releases/tag/0.46.0)
- New config to set metatile by format
## 2.5.0
Released 2015-06-18

View File

@@ -59,6 +59,14 @@ happen to have startup errors you may need to force rebuilding those
modules. At any time just wipe out the node_modules/ directory and run
```npm install``` again.
Upgrading
---------
Checkout your commit/branch. If you need to reinstall dependencies (you can check [NEWS](NEWS.md)) do the following:
```
rm -rf node_modules; npm install
```
Run
---

36
app.js
View File

@@ -9,6 +9,8 @@
var path = require('path');
var fs = require('fs');
var http = require('http');
var https = require('https');
var RedisPool = require('redis-mpool');
var _ = require('underscore');
@@ -65,8 +67,23 @@ if ( global.environment.log_filename ) {
global.log4js.configure(log4js_config, { cwd: __dirname });
global.logger = global.log4js.getLogger();
var redisOpts = _.extend(global.environment.redis, { name: 'windshaft' }),
redisPool = new RedisPool(redisOpts);
var redisOpts = _.defaults(global.environment.redis, {
name: 'windshaft',
unwatchOnRelease: false,
noReadyCheck: true
});
var redisPool = new RedisPool(redisOpts);
// set global HTTP and HTTPS agent default configurations
// ref https://nodejs.org/api/http.html#http_new_agent_options
var agentOptions = _.defaults(global.environment.httpAgent || {}, {
keepAlive: false,
keepAliveMsecs: 1000,
maxSockets: Infinity,
maxFreeSockets: 256
});
http.globalAgent = new http.Agent(agentOptions);
https.globalAgent = new https.Agent(agentOptions);
// Include cartodb_windshaft only _after_ the "global" variable is set
// See https://github.com/Vizzuality/Windshaft-cartodb/issues/28
@@ -82,6 +99,13 @@ if (global.statsClient) {
global.statsClient.gauge(keyPrefix + 'unused', status.unused);
global.statsClient.gauge(keyPrefix + 'waiting', status.waiting);
});
setInterval(function() {
var memoryUsage = process.memoryUsage();
Object.keys(memoryUsage).forEach(function(k) {
global.statsClient.gauge('windshaft.memory.' + k, memoryUsage[k]);
});
}, 5000);
}
// Maximum number of connections for one process
@@ -95,10 +119,10 @@ ws.listen(global.environment.port, global.environment.host);
var version = require("./package").version;
ws.on('listening', function() {
console.log(
"Windshaft tileserver %s started on %s:%s (%s)",
version, global.environment.host, global.environment.port, ENV
);
console.log(
"Windshaft tileserver %s started on %s:%s PID=%d (%s)",
version, global.environment.host, global.environment.port, process.pid, ENV
);
});
process.on('SIGHUP', function() {

View File

@@ -2,6 +2,9 @@ var config = {
environment: 'development'
,port: 8181
,host: '127.0.0.1'
// Size of the threadpool which can be used to run user code and get notified in the loop thread
// Its default size is 4, but it can be changed at startup time (the absolute maximum is 128).
// See http://docs.libuv.org/en/latest/threadpool.html
,uv_threadpool_size: undefined
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
@@ -86,8 +89,10 @@ var config = {
cache_ttl: 60000,
statsInterval: 5000, // milliseconds between each report to statsd about number of renderers and mapnik pool status
mapnik: {
// The size of the pool of internal mapnik renderers
// Check the configuration of uv_threadpool_size to use suitable value
// The size of the pool of internal mapnik backend
// This pool size is per mapnik renderer created in Windshaft's RendererFactory
// See https://github.com/CartoDB/Windshaft/blob/master/lib/windshaft/renderers/renderer_factory.js
// Important: check the configuration of uv_threadpool_size to use suitable value
poolSize: 8,
// Metatile is the number of tiles-per-side that are going
@@ -96,6 +101,23 @@ var config = {
// wasted time.
metatile: 2,
// tilelive-mapnik uses an internal cache to store tiles/grids
// generated when using metatile. This options allow to tune
// the behaviour for that internal cache.
metatileCache: {
// Time an object must stay in the cache until is removed
ttl: 0,
// Whether an object must be removed after the first hit
// Usually you want to use `true` here when ttl>0.
deleteOnHit: false
},
// Override metatile behaviour depending on the format
formatMetatile: {
png: 2,
'grid.json': 1
},
// Buffer size is the tickness in pixel of a buffer
// around the rendered (meta?)tile.
//
@@ -170,7 +192,16 @@ var config = {
},
emitter: {
statusInterval: 5000 // time, in ms, between each status report is emitted from the pool, status is sent to statsd
}
},
unwatchOnRelease: false, // Send unwatch on release, see http://github.com/CartoDB/Windshaft-cartodb/issues/161
noReadyCheck: true // Check `no_ready_check` at https://github.com/mranney/node_redis/tree/v0.12.1#overloading
}
// For more details about this options check https://nodejs.org/api/http.html#http_new_agent_options
,httpAgent: {
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 25,
maxFreeSockets: 256
}
,varnish: {
host: 'localhost',

View File

@@ -2,6 +2,9 @@ var config = {
environment: 'production'
,port: 8181
,host: '127.0.0.1'
// Size of the threadpool which can be used to run user code and get notified in the loop thread
// Its default size is 4, but it can be changed at startup time (the absolute maximum is 128).
// See http://docs.libuv.org/en/latest/threadpool.html
,uv_threadpool_size: undefined
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
@@ -80,8 +83,10 @@ var config = {
cache_ttl: 60000,
statsInterval: 5000, // milliseconds between each report to statsd about number of renderers and mapnik pool status
mapnik: {
// The size of the pool of internal mapnik renderers
// Check the configuration of uv_threadpool_size to use suitable value
// The size of the pool of internal mapnik backend
// This pool size is per mapnik renderer created in Windshaft's RendererFactory
// See https://github.com/CartoDB/Windshaft/blob/master/lib/windshaft/renderers/renderer_factory.js
// Important: check the configuration of uv_threadpool_size to use suitable value
poolSize: 8,
// Metatile is the number of tiles-per-side that are going
@@ -90,6 +95,23 @@ var config = {
// wasted time.
metatile: 2,
// tilelive-mapnik uses an internal cache to store tiles/grids
// generated when using metatile. This options allow to tune
// the behaviour for that internal cache.
metatileCache: {
// Time an object must stay in the cache until is removed
ttl: 0,
// Whether an object must be removed after the first hit
// Usually you want to use `true` here when ttl>0.
deleteOnHit: false
},
// Override metatile behaviour depending on the format
formatMetatile: {
png: 2,
'grid.json': 1
},
// Buffer size is the tickness in pixel of a buffer
// around the rendered (meta?)tile.
//
@@ -164,7 +186,16 @@ var config = {
},
emitter: {
statusInterval: 5000 // time, in ms, between each status report is emitted from the pool, status is sent to statsd
}
},
unwatchOnRelease: false, // Send unwatch on release, see http://github.com/CartoDB/Windshaft-cartodb/issues/161
noReadyCheck: true // Check `no_ready_check` at https://github.com/mranney/node_redis/tree/v0.12.1#overloading
}
// For more details about this options check https://nodejs.org/api/http.html#http_new_agent_options
,httpAgent: {
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 25,
maxFreeSockets: 256
}
,varnish: {
host: 'localhost',

View File

@@ -2,6 +2,9 @@ var config = {
environment: 'production'
,port: 8181
,host: '127.0.0.1'
// Size of the threadpool which can be used to run user code and get notified in the loop thread
// Its default size is 4, but it can be changed at startup time (the absolute maximum is 128).
// See http://docs.libuv.org/en/latest/threadpool.html
,uv_threadpool_size: undefined
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
@@ -80,8 +83,10 @@ var config = {
cache_ttl: 60000,
statsInterval: 5000, // milliseconds between each report to statsd about number of renderers and mapnik pool status
mapnik: {
// The size of the pool of internal mapnik renderers
// Check the configuration of uv_threadpool_size to use suitable value
// The size of the pool of internal mapnik backend
// This pool size is per mapnik renderer created in Windshaft's RendererFactory
// See https://github.com/CartoDB/Windshaft/blob/master/lib/windshaft/renderers/renderer_factory.js
// Important: check the configuration of uv_threadpool_size to use suitable value
poolSize: 8,
// Metatile is the number of tiles-per-side that are going
@@ -90,6 +95,23 @@ var config = {
// wasted time.
metatile: 2,
// tilelive-mapnik uses an internal cache to store tiles/grids
// generated when using metatile. This options allow to tune
// the behaviour for that internal cache.
metatileCache: {
// Time an object must stay in the cache until is removed
ttl: 0,
// Whether an object must be removed after the first hit
// Usually you want to use `true` here when ttl>0.
deleteOnHit: false
},
// Override metatile behaviour depending on the format
formatMetatile: {
png: 2,
'grid.json': 1
},
// Buffer size is the tickness in pixel of a buffer
// around the rendered (meta?)tile.
//
@@ -164,7 +186,16 @@ var config = {
},
emitter: {
statusInterval: 5000 // time, in ms, between each status report is emitted from the pool, status is sent to statsd
}
},
unwatchOnRelease: false, // Send unwatch on release, see http://github.com/CartoDB/Windshaft-cartodb/issues/161
noReadyCheck: true // Check `no_ready_check` at https://github.com/mranney/node_redis/tree/v0.12.1#overloading
}
// For more details about this options check https://nodejs.org/api/http.html#http_new_agent_options
,httpAgent: {
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 25,
maxFreeSockets: 256
}
,varnish: {
host: 'localhost',

View File

@@ -2,6 +2,9 @@ var config = {
environment: 'test'
,port: 8888
,host: '127.0.0.1'
// Size of the threadpool which can be used to run user code and get notified in the loop thread
// Its default size is 4, but it can be changed at startup time (the absolute maximum is 128).
// See http://docs.libuv.org/en/latest/threadpool.html
,uv_threadpool_size: undefined
// Regular expression pattern to extract username
// from hostname. Must have a single grabbing block.
@@ -80,8 +83,10 @@ var config = {
cache_ttl: 60000,
statsInterval: 5000, // milliseconds between each report to statsd about number of renderers and mapnik pool status
mapnik: {
// The size of the pool of internal mapnik renderers
// Check the configuration of uv_threadpool_size to use suitable value
// The size of the pool of internal mapnik backend
// This pool size is per mapnik renderer created in Windshaft's RendererFactory
// See https://github.com/CartoDB/Windshaft/blob/master/lib/windshaft/renderers/renderer_factory.js
// Important: check the configuration of uv_threadpool_size to use suitable value
poolSize: 8,
// Metatile is the number of tiles-per-side that are going
@@ -90,6 +95,23 @@ var config = {
// wasted time.
metatile: 2,
// tilelive-mapnik uses an internal cache to store tiles/grids
// generated when using metatile. This options allow to tune
// the behaviour for that internal cache.
metatileCache: {
// Time an object must stay in the cache until is removed
ttl: 0,
// Whether an object must be removed after the first hit
// Usually you want to use `true` here when ttl>0.
deleteOnHit: false
},
// Override metatile behaviour depending on the format
formatMetatile: {
png: 2,
'grid.json': 1
},
// Buffer size is the tickness in pixel of a buffer
// around the rendered (meta?)tile.
//
@@ -166,7 +188,16 @@ var config = {
},
emitter: {
statusInterval: 5000 // time, in ms, between each status report is emitted from the pool, status is sent to statsd
}
},
unwatchOnRelease: false, // Send unwatch on release, see http://github.com/CartoDB/Windshaft-cartodb/issues/161
noReadyCheck: true // Check `no_ready_check` at https://github.com/mranney/node_redis/tree/v0.12.1#overloading
}
// For more details about this options check https://nodejs.org/api/http.html#http_new_agent_options
,httpAgent: {
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 25,
maxFreeSockets: 256
}
,varnish: {
host: '',

View File

@@ -880,6 +880,8 @@ By manipulating the `"urlTemplate"` custom basemaps can be used in generating st
**CartoDB**
As described in the [Mapconfig documentation](https://github.com/CartoDB/Windshaft/blob/0.44.1/doc/MapConfig-1.3.0.md), a "cartodb" type layer is now just an alias to a "mapnik" type layer as above, intended for backwards compatibility.
```javascript
{
"type": "cartodb",

View File

@@ -14,7 +14,7 @@ module.exports = QueryTablesApi;
QueryTablesApi.prototype.getAffectedTablesInQuery = function (username, sql, callback) {
var query = 'SELECT CDB_QueryTables($windshaft$' + prepareSql(sql) + '$windshaft$)';
var query = 'SELECT CDB_QueryTablesText($windshaft$' + prepareSql(sql) + '$windshaft$)';
this.pgQueryRunner.run(username, query, handleAffectedTablesInQueryRows, callback);
};
@@ -25,9 +25,9 @@ function handleAffectedTablesInQueryRows(err, rows, callback) {
callback(new Error('could not fetch source tables: ' + msg));
return;
}
var qtables = rows[0].cdb_querytables;
var tableNames = qtables.split(/^\{(.*)\}$/)[1];
tableNames = tableNames ? tableNames.split(',') : [];
// This is an Array, so no need to split into parts
var tableNames = rows[0].cdb_querytablestext;
callback(null, tableNames);
}
@@ -35,7 +35,7 @@ QueryTablesApi.prototype.getAffectedTablesAndLastUpdatedTime = function (usernam
var query = [
'WITH querytables AS (',
'SELECT * FROM CDB_QueryTables($windshaft$' + prepareSql(sql) + '$windshaft$) as tablenames',
'SELECT * FROM CDB_QueryTablesText($windshaft$' + prepareSql(sql) + '$windshaft$) as tablenames',
')',
'SELECT (SELECT tablenames FROM querytables), EXTRACT(EPOCH FROM max(updated_at)) as max',
'FROM CDB_TableMetadata m',
@@ -54,8 +54,8 @@ function handleAffectedTablesAndLastUpdatedTimeRows(err, rows, callback) {
var result = rows[0];
var tableNames = result.tablenames.split(/^\{(.*)\}$/)[1];
tableNames = tableNames ? tableNames.split(',') : [];
// This is an Array, so no need to split into parts
var tableNames = result.tablenames;
var lastUpdatedTime = result.max || 0;

740
npm-shrinkwrap.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,7 @@
{
"private": true,
"name": "windshaft-cartodb",
"version": "2.5.0",
"version": "2.12.0",
"description": "A map tile server for CartoDB",
"keywords": [
"cartodb"
@@ -24,14 +24,14 @@
"dependencies": {
"underscore" : "~1.6.0",
"dot": "~1.0.2",
"windshaft": "0.45.0",
"windshaft": "0.51.0",
"step": "~0.0.5",
"queue-async": "~1.0.7",
"request": "~2.9.203",
"cartodb-redis": "~0.13.0",
"cartodb-psql": "~0.4.0",
"fastly-purge": "~1.0.0",
"redis-mpool": "~0.3.0",
"redis-mpool": "~0.4.0",
"lzma": "~1.3.7",
"log4js": "https://github.com/CartoDB/log4js-node/tarball/cdb"
},

View File

@@ -256,6 +256,41 @@ describe('tests from old api translated to multilayer', function() {
);
});
// https://github.com/CartoDB/cartodb-postgresql/issues/86
it.skip("should not fail with long table names because table name length limit", function(done) {
var tableName = 'long_table_name_with_enough_chars_to_break_querytables_function';
var expectedCacheChannel = _.template('<%= databaseName %>:public.<%= tableName %>', {
databaseName: _.template(global.environment.postgres_auth_user, {user_id:1}) + '_db',
tableName: tableName
});
var layergroup = singleLayergroupConfig('select * from ' + tableName, '#layer { marker-fill: red; }');
assert.response(server,
{
url: layergroupUrl + '?config=' + encodeURIComponent(JSON.stringify(layergroup)),
method: 'GET',
headers: {
host: 'localhost'
}
},
{
status: 200
},
function(res) {
var parsed = JSON.parse(res.body);
assert.ok(parsed.layergroupid);
assert.ok(res.headers.hasOwnProperty('x-cache-channel'));
assert.equal(res.headers['x-cache-channel'], expectedCacheChannel);
assert.equal(res.headers['x-layergroup-id'], parsed.layergroupid);
done();
}
);
});
it("creates layergroup fails when postgresql queries fail to figure affected tables in query", function(done) {
var runQueryFn = PgQueryRunner.prototype.run;

View File

@@ -2,12 +2,12 @@
--
-- Requires PostgreSQL 9.x+
--
CREATE OR REPLACE FUNCTION CDB_QueryTables(query text)
RETURNS name[]
CREATE OR REPLACE FUNCTION CDB_QueryTablesText(query text)
RETURNS text[]
AS $$
DECLARE
exp XML;
tables NAME[];
tables text[];
rec RECORD;
rec2 RECORD;
BEGIN
@@ -41,11 +41,11 @@ BEGIN
xpath('//x:Relation-Name/text()', exp, ARRAY[ARRAY['x', 'http://www.postgresql.org/2009/explain']]) as x,
xpath('//x:Relation-Name/../x:Schema/text()', exp, ARRAY[ARRAY['x', 'http://www.postgresql.org/2009/explain']]) as s
)
SELECT unnest(x)::name as p, unnest(s)::name as sc from inp
SELECT unnest(x) as p, unnest(s) as sc from inp
LOOP
-- RAISE DEBUG 'tab: %', rec2.p;
-- RAISE DEBUG 'sc: %', rec2.sc;
tables := array_append(tables, (rec2.sc || '.' || rec2.p)::name);
tables := array_append(tables, (rec2.sc || '.' || rec2.p));
END LOOP;
-- RAISE DEBUG 'Tables: %', tables;
@@ -65,3 +65,14 @@ BEGIN
return tables;
END
$$ LANGUAGE 'plpgsql' VOLATILE STRICT;
-- Keep CDB_QueryTables with same signature for backwards compatibility.
-- It should probably be removed in the future.
CREATE OR REPLACE FUNCTION CDB_QueryTables(query text)
RETURNS name[]
AS $$
BEGIN
RETURN CDB_QueryTablesText(query)::name[];
END
$$ LANGUAGE 'plpgsql' VOLATILE STRICT;

View File

@@ -189,3 +189,31 @@ INSERT INTO CDB_TableMetadata (tabname, updated_at) VALUES ('test_table_private_
-- GRANT SELECT ON CDB_TableMetadata TO :PUBLICUSER;
GRANT SELECT ON CDB_TableMetadata TO :TESTUSER;
-- long name table
CREATE TABLE
long_table_name_with_enough_chars_to_break_querytables_function
(
updated_at timestamp without time zone DEFAULT now(),
created_at timestamp without time zone DEFAULT now(),
cartodb_id integer NOT NULL,
name character varying,
address character varying,
the_geom geometry,
the_geom_webmercator geometry
);
INSERT INTO long_table_name_with_enough_chars_to_break_querytables_function SELECT * from test_table;
ALTER TABLE ONLY long_table_name_with_enough_chars_to_break_querytables_function
ADD CONSTRAINT long_table_name_with_enough_chars_to_break_querytables_func_pkey PRIMARY KEY (cartodb_id);
CREATE INDEX long_table_name_the_geom_idx
ON long_table_name_with_enough_chars_to_break_querytables_function USING gist (the_geom);
CREATE INDEX long_table_name_the_geom_webmercator_idx
ON long_table_name_with_enough_chars_to_break_querytables_function USING gist (the_geom_webmercator);
GRANT ALL ON TABLE long_table_name_with_enough_chars_to_break_querytables_function TO :TESTUSER;
GRANT SELECT ON TABLE long_table_name_with_enough_chars_to_break_querytables_function TO :PUBLICUSER;
INSERT INTO CDB_TableMetadata (tabname, updated_at) VALUES ('long_table_name_with_enough_chars_to_break_querytables_function'::regclass, '2009-02-13T23:31:30.123Z');

8
tools/README.md Normal file
View File

@@ -0,0 +1,8 @@
Deprecated tools
================
All tools and scripts found in this directory are deprecated and no longer maintained.
Use at your own peril.
In future releases they might get removed.