diff --git a/docs/examples/01-example.md b/docs/examples/01-example.md new file mode 100644 index 00000000..5bc9ae3e --- /dev/null +++ b/docs/examples/01-example.md @@ -0,0 +1 @@ +## Example 1 \ No newline at end of file diff --git a/docs/guides/01-quickstart.md b/docs/guides/01-quickstart.md new file mode 100644 index 00000000..03de7527 --- /dev/null +++ b/docs/guides/01-quickstart.md @@ -0,0 +1,100 @@ +## Quickstart + +### Anonymous Maps + +Here is an example of how to create an Anonymous Map with JavaScript: + +```javascript +var mapconfig = { + "version": "1.3.1", + "layers": [{ + "type": "cartodb", + "options": { + "cartocss_version": "2.1.1", + "cartocss": "#layer { polygon-fill: #FFF; }", + "sql": "select * from european_countries_e" + } + }] +} + +$.ajax({ + crossOrigin: true, + type: 'POST', + dataType: 'json', + contentType: 'application/json', + url: 'https://{username}.carto.com/api/v1/map', + data: JSON.stringify(mapconfig), + success: function(data) { + var templateUrl = 'https://{username}.carto.com/api/v1/map/' + data.layergroupid + '/{z}/{x}/{y}.png' + console.log(templateUrl); + } +}) +``` + +### Named Maps + +Let's create a Named Map using some private tables in a CARTO account. +The following map config sets up a map of European countries that have a white fill color: + +```javascript +{ + "version": "0.0.1", + "name": "test", + "auth": { + "method": "open" + }, + "layergroup": { + "layers": [{ + "type": "mapnik", + "options": { + "cartocss_version": "2.1.1", + "cartocss": "#layer { polygon-fill: #FFF; }", + "sql": "select * from european_countries_e" + } + }] + } +} +``` + +The MapConfig needs to be sent to CARTO's Map API using an authenticated call. Here we will use a command line tool called `curl`. For more info about this tool, see [this blog post](http://quickleft.com/blog/command-line-tutorials-curl), or type `man curl` in bash. Using `curl`, and storing the config from above in a file `MapConfig.json`, the call would look like: + +##### Call + +```bash +curl 'https://{username}.carto.com/api/v1/map/named?api_key={api_key}' -H 'Content-Type: application/json' -d @mapconfig.json +``` + +To get the `URL` to fetch the tiles you need to instantiate the map, where `template_id` is the template name from the previous response. + +##### Call + +```bash +curl -X POST 'https://{username}.carto.com/api/v1/map/named/{template_id}' -H 'Content-Type: application/json' +``` + +The response will return JSON with properties for the `layergroupid`, the timestamp (`last_updated`) of the last data modification and some key/value pairs with `metadata` for the `layers`. + +Note: all `layers` in `metadata` will always have a `type` string and a `meta` dictionary with the key/value pairs. + +##### Response + +```javascript +{ + "layergroupid": "c01a54877c62831bb51720263f91fb33:0", + "last_updated": "1970-01-01T00:00:00.000Z", + "metadata": { + "layers": [ + { + "type": "mapnik", + "meta": {} + } + ] + } +} +``` + +You can use the `layergroupid` to instantiate a URL template for accessing tiles on the client. Here we use the `layergroupid` from the example response above in this URL template: + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/{z}/{x}/{y}.png +``` diff --git a/docs/guides/02-general-concepts.md b/docs/guides/02-general-concepts.md new file mode 100644 index 00000000..4eeec096 --- /dev/null +++ b/docs/guides/02-general-concepts.md @@ -0,0 +1,27 @@ +## General Concepts + +The following concepts are the same for every endpoint in the API except when it's noted explicitly. + +### Auth + +By default, users do not have access to private tables in CARTO. In order to instantiate a map from private table data an API Key is required. Additionally, to include some endpoints, an API Key must be included (e.g. creating a Named Map). + +To execute an authorized request, `api_key=YOURAPIKEY` should be added to the request URL. The param can be also passed as POST param. Using HTTPS is mandatory when you are performing requests that include your `api_key`. + +### Errors + +Errors are reported using standard HTTP codes and extended information encoded in JSON with this format: + +```javascript +{ + "errors": [ + "access forbidden to table TABLE" + ] +} +``` + +If you use JSONP, the 200 HTTP code is always returned so the JavaScript client can receive errors from the JSON object. + +### CORS Support + +All the endpoints, which might be accessed using a web browser, add CORS headers and allow OPTIONS method. \ No newline at end of file diff --git a/docs/guides/03-anonymous-maps.md b/docs/guides/03-anonymous-maps.md new file mode 100644 index 00000000..56d61502 --- /dev/null +++ b/docs/guides/03-anonymous-maps.md @@ -0,0 +1,396 @@ +## Anonymous Maps + +Anonymous Maps allows you to instantiate a map given SQL and CartoCSS. It also allows you to add interaction capabilities using [UTF Grid.](https://github.com/mapbox/utfgrid-spec). +Alternatively, you can get the data for the map (geometry and attributes for each layer) using vector tiles (in which case CartoCSS is not required). + + +### Instantiate + +##### Definition + +```html +POST /api/v1/map +``` + +##### Params + +```javascript +{ + "version": "1.3.0", + "layers": [{ + "type": "mapnik", + "options": { + "cartocss_version": "2.1.1", + "cartocss": "#layer { polygon-fill: #FFF; }", + "sql": "select * from european_countries_e", + "interactivity": ["cartodb_id", "iso3"] + } + }] +} +``` + +See [MapConfig File Formats](http://docs.carto.com/carto-engine/maps-api/mapconfig/) for details. + +##### Response + +The response includes: + +Attributes | Description +--- | --- +layergroupid | The ID for that map, used to compose the URL for the tiles. The final URL is: `https://{username}.carto.com/api/v1/map/{layergroupid}/{z}/{x}/{y}.png` +updated_at | The ISO date of the last time the data involved in the query was updated. +metadata | Includes information about the layers. +cdn_url | URLs to fetch the data using the best CDN for your zone. + +**Improved response metadata** + +Originally, you needed to concantenate the `layergroupid` with the correct domain and the path for the tiles. +Now, for convenience, the layergroup includes the final URLs in two formats: +1. Leaflet's urlTemplate alike: useful when working with raster tiles or with libraries with an API similar to Leaflet's one. +1. [TileJSON spec](https://github.com/mapbox/tilejson-spec): useful when working with Mapbox GL or any other library that supports TileJSON. + +#### Example + +##### Call + +```bash +curl 'https://{username}.carto.com/api/v1/map' -H 'Content-Type: application/json' -d @mapconfig.json +``` + +##### Response + +```javascript +{ + "layergroupid": "c01a54877c62831bb51720263f91fb33:0", + "last_updated": "1970-01-01T00:00:00.000Z", + "metadata": { + "layers": [ + { + "type": "mapnik", + "meta": {} + } + ], + "tilejson": { + "raster": { + "tilejson": "2.2.0", + "tiles": [ + "http://a.cdb.com/c01a54877c62831bb51720263f91fb33/{z}/{x}/{y}.png", + "http://b.cdb.com/c01a54877c62831bb51720263f91fb33/{z}/{x}/{y}.png" + ] + } + }, + "url": { + "raster": { + "urlTemplate": "http://{s}.cdb.com/c01a54877c62831bb51720263f91fb33/{z}/{x}/{y}.png", + "subdomains": ["a", "b"] + } + } + }, + "cdn_url": { + "http": "http://cdb.com", + "https": "https://cdb.com", + "templates": { + "http": { "subdomains": ["a","b"], "url": "http://{s}.cdb.com" }, + "https": { "subdomains": ["a","b"], "url": "https://{s}.example.com" }, + } + } +} +``` + +### Map Tile Rendering + +Map tiles are used to create the graphic representation of your map in a web browser. Tiles can be requested either as pre-rendered *raster* tiles (images) or as *vector* map data to be rendered by the client (browser). + +- **Raster**: If a tile is requested as a raster image format, like PNG, the map will be rendered on the server, using the CartoCSS styles defined in the layers of the map. It is necessary that all the layers of a map define CartoCSS styles in order to obtain raster tiles. Raster tiles are made up of 256x256 pixels; to avoid graphic quality issues tiles should be used unscaled to represent the zoom level (Z) for which they are requested. In order to render tiles, data will be retrieved from the database (in vector format) on the server-side. + +- **Vector**: Tiles can also be requested as MVT (Mapbox Vector Tiles). In this case, only the geospatial vector data, without any styling, is returned. These tiles should be processed in the client-side to render the map. In this case layers do not need to define CartoCSS, as any rendering and styling will be performed on the client side. The vector data of a tile represents real-world geometries by defining the vertices of points, lines or polygons in a tile-specific coordinate system. + +### Retrieve resources from the layergroup + +When you have a layergroup, there are several resources for retrieving layergoup details such as, accessing Mapnik tiles, getting individual layers, accessing defined Attributes, and blending and layer selection. + +#### Raster tiles + +These raster tiles are PNG images that represent only the Mapnik layers of a map. See [individual layers](#individual-layers) for details about how to retrieve other layers. + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/{z}/{x}/{y}.png +``` + +#### Mapbox Vector Tiles (MVT) + +[Mapbox Vector Tiles (MVT)](https://www.mapbox.com/vector-tiles/specification/) are map tiles that transfer geographic vector data to the client-side. Browser performance is fast since you can pan and zoom without having to query the server. + +CARTO uses Web Graphics Library (WebGL) to process MVT files on the browser. This is useful since WebGL is compatible with most web browsers, include support for multiple client-side mapping engines, and do not require additional information from the server; which makes it more efficient for rendering map tiles. However, you can use any implementation tool for processing MVT files. + +The following examples describe how to fetch MVT tiles with a cURL request. + +##### MVT and Windshaft + +CARTO uses Windshaft as the map tiler library to render multilayer maps with the Maps API. You can use Windshaft to request MVT using the same layer type that is used for requesting raster tiles (Mapnik layer). Simply change the file format `.mvt` in the URL. + + +```bash +https://{username}.cartodb.com/api/v1/map/HASH/:layer/{z}/{x}/{y}.mvt +``` + +The following example instantiates an anonymous map with layer options: + +```bash +{ + user_name: 'mycartodbuser', + sublayers: [{ + sql: "SELECT * FROM table_name"; + cartocss: '#layer { marker-fill: #F0F0F0; }' + }], + maps_api_template: 'https://{user}.cartodb.com' // Optional +} +``` + +**Note**: If no layer type is specified, Mapnik tiles are used by default. To access MVT tiles, specify `https://{username}.cartodb.com/api/v1/map/HASH/{z}/{x}/{y}.mvt` as the `maps_api_template` variable. + +**Tip:** If you are using [Named Maps](https://carto.com/docs/carto-engine/maps-api/named-maps/) to instantiate a layer, indicate the MVT file format and layer in the response: + +```bash +https://{username}.cartodb.com/api/v1/map/named/:templateId/:layer/{z}/{x}/{y}.mvt +``` + +For all layers in a Named Map, you must indicate Mapnik as the layer filter: + +```bash +https://{username}.cartodb.com/api/v1/map/named/:templateId/mapnik/{z}/{x}/{y}.mvt +``` + +##### Layergroup Filter for MVT Tiles + +To filter layers using Windshaft, use the following request where layers are numbered: + +```bash +https://{username}.cartodb.com/api/v1/map/HASH/0,1,2/{z}/{x}/{y}.mvt +``` + +To request all layers, remove the layergroup filter parameter: + +```bash +https://{username}.cartodb.com/api/v1/map/HASH/{z}/{x}/{y}.mvt +``` + +To filter a specific layer: + +```bash +https://{username}.cartodb.com/api/v1/map/HASH/2/{z}/{x}/{y}.mvt +``` + +##### Example 1: MVT Tiles with Windshaft, CARTO.js, and MapboxGL + +1) Import the required libraries: + +```bash + + + +``` + +2) Configure Map Client: + +```bash +mapboxgl.accessToken = '{yourMapboxToken}'; +``` + +3) Create Map Object (Mapbox): + +```bash +var map = new mapboxgl.Map({ +container: 'map', +zoom: 1, +minZoom: 0, +maxZoom: 18, +center: [30, 0] +}); +``` + +4) Define Layer Options (CARTO): + +```bash +var layerOptions = { +user_name: "{username}", +sublayers: [{ +sql: "SELECT * FROM {table_name}", +cartocss: "...", + }] +}; +``` + +5) Request Tiles (from CARTO) and Set to Map Object (Mapbox): + +**Note:** By default, [CARTO core functions](https://carto.com/docs/carto-engine/carto-js/core-api/) retrieve URLs for fully rendered tiles. You must replace the default format (.png) with the MVT format (.mvt). + + +```bash +cartodb.Tiles.getTiles(layerOptions, function(result, err) { +var tiles = result.tiles.map(function(tileUrl) { +return tileUrl +.replace('{s}', 'a') +.replace(/\.png/, '.mvt'); +}); +map.setStyle(simpleStyle(tiles)); +}); +``` + +##### Example 2: MVT Libraries with Windshaft and MapboxGL + +When you are not including CARTO.js to implement MVT tiles, you must use the `map.setStyle` parameter to specify vector map rendering. + +1) Import the required libraries: + +```bash + + +``` + +2) Configure Map Client: + +```bash +mapboxgl.accessToken = '{yourMapboxToken}'; +``` + +3) Create Map Object (Mapbox): + +```bash +var map = new mapboxgl.Map({ +container: 'map', +zoom: 1, +minZoom: 0, +maxZoom: 18, +center: [30, 0] +}); +``` + +4) Set the Style + +```bash +map.setStyle({ + "version": 7, + "glyphs": "...", + "constants": {...}, + "sources": { + "cartodb": { + "type": "vector", + "tiles": [ "http://{username}.cartodb.com/api/v1/map/named/templateId/mapnik/{z}/{x}/{y}.mvt" + ], + "maxzoom": 18 + } + }, + "layers": [{...}] +}); +``` + +**Tip:** If you are using MapboxGL, see the following resource for additional information. + +- [MapboxGL API Reference](https://www.mapbox.com/mapbox-gl-js/api/) +- [MapboxGL Style Specifications](https://www.mapbox.com/mapbox-gl-js/style-spec/) +- [Example of MapboxGL Implementation](https://www.mapbox.com/mapbox-gl-js/examples/) + +#### Individual layers + +The MapConfig specification holds the layers definition in a 0-based index. Layers can be requested individually, in different formats, depending on the layer type. + +Individual layers can be accessed using that 0-based index. For UTF grid tiles: + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/{layer}/{z}/{x}/{y}.grid.json +``` + +In this case, `layer` as 0 returns the UTF grid tiles/attributes for layer 0, the only layer in the example MapConfig. + +If the MapConfig had a Torque layer at index 1 it could be possible to request it with: + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/1/{z}/{x}/{y}.torque.json +``` + +#### Attributes defined in `attributes` section + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/{layer}/attributes/{feature_id} +``` + +Which returns JSON with the attributes defined, such as: + +```javascript +{ "c": 1, "d": 2 } +``` + +#### Blending and layer selection + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/{layer_filter}/{z}/{x}/{y}.png +``` + +Note: currently format is limited to `png`. + +`layer_filter` can be used to select some layers to be rendered together. `layer_filter` supports two formats: + +- `all` alias + +Using `all` as `layer_filter` will blend all layers in the layergroup + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/all/{z}/{x}/{y}.png +``` + +- Filter by layer index + +A list of comma separated layer indexes can be used to just render a subset of layers. For example `0,3,4` will filter and blend layers with indexes 0, 3, and 4. + +```bash +https://{username}.carto.com/api/v1/map/{layergroupid}/0,3,4/{z}/{x}/{y}.png +``` + +Some notes about filtering: + + - Invalid index values or out of bounds indexes will end in `Invalid layer filtering` errors. + - Ordering is not considered. So right now filtering layers 0,3,4 is the very same thing as filtering 3,4,0. As this may change in the future, **it is recommended** to always select the layers in ascending order so that you will always get consistent behavior. + +### Create JSONP + +The JSONP endpoint is provided in order to allow web browsers access which don't support CORS. + +##### Definition + +```bash +GET /api/v1/map?callback=method +``` + +##### Params + +Param | Description +--- | --- +config | Encoded JSON with the params for creating Named Maps (the variables defined in the template). +lmza | This attribute contains the same as config but LZMA compressed. It cannot be used at the same time as `config`. +callback | JSON callback name. + +#### Example + +##### Call + +```bash +curl "https://{username}.carto.com/api/v1/map?callback=callback&config=%7B%22version%22%3A%221.0.1%22%2C%22layers%22%3A%5B%7B%22type%22%3A%22cartodb%22%2C%22options%22%3A%7B%22sql%22%3A%22select+%2A+from+european_countries_e%22%2C%22cartocss%22%3A%22%23european_countries_e%7B+polygon-fill%3A+%23FF6600%3B+%7D%22%2C%22cartocss_version%22%3A%222.3.0%22%2C%22interactivity%22%3A%5B%22cartodb_id%22%5D%7D%7D%5D%7D" +``` + +##### Response + +```javascript +callback({ + layergroupid: "d9034c133262dfb90285cea26c5c7ad7:0", + cdn_url: { + "http": "http://cdb.com", + "https": "https://cdb.com" + }, + last_updated: "1970-01-01T00:00:00.000Z" +}) +``` + +### Remove + +Anonymous Maps cannot be removed by an API call. They will expire after about five minutes, or sometimes longer. If an Anonymous Map expires and tiles are requested from it, an error will be raised. This could happen if a user leaves a map open and after time, returns to the map and attempts to interact with it in a way that requires new tiles (e.g. zoom). The client will need to go through the steps of creating the map again to fix the problem. diff --git a/docs/guides/04-named-maps.md b/docs/guides/04-named-maps.md new file mode 100644 index 00000000..fb91adb3 --- /dev/null +++ b/docs/guides/04-named-maps.md @@ -0,0 +1,568 @@ +## Named Maps + +Named Maps are essentially the same as Anonymous Maps except the MapConfig is stored on the server, and the map is given a unique name. You can create Named Maps from private data, and users without an API Key can view your Named Map (while keeping your data private). + +The Named Map workflow consists of uploading a MapConfig file to CARTO servers, to select data from your CARTO user database by using SQL, and specifying the CartoCSS for your map. + +The response back from the API provides the template_id of your Named Map as the `name` (the identifier of your Named Map), which is the name that you specified in the MapConfig. You can which you can then use to create your Named Map details, or [fetch XYZ tiles](#fetching-xyz-tiles-for-named-maps) directly for Named Maps. + +**Tip:** You can also use a Named Map that you created (which is defined by its `name`), to create a map using CARTO.js. This is achieved by adding the [`namedmap` type](http://docs.carto.com/carto-engine/carto-js/layer-source-object/#named-maps-layer-source-object-type-namedmap) layer source object to draw the Named Map. + +The main differences, compared to Anonymous Maps, is that Named Maps include: + +- **auth token** + This allows you to control who is able to see the map based on an auth token, and create a secure Named Map with password-protection. + +- **template map** + The template map is static and may contain placeholders, enabling you to modify your maps appearance by using variables. Templates maps are persistent with no preset expiration. They can only be created, or deleted, by a CARTO user with a valid API KEY (See [auth argument](#arguments)). + + Uploading a MapConfig creates a Named Map. MapConfigs are uploaded to the server by sending the server a "template".json file, which contain the [MapConfig specifications](http://docs.carto.com/carto-engine/maps-api/mapconfig/). + +**Note:** There is a limit of 4,096 Named Maps allowed per account. If you need to create more Named Maps, it is recommended to use a single Named Map and change the variables using [placeholders](#placeholder-format), instead of uploading multiple [Named Map MapConfigs](http://docs.carto.com/carto-engine/maps-api/mapconfig/#named-map-layer-options). + +### Create + +##### Definition + +```html +POST /api/v1/map/named +``` + +##### Params + +Params | Description +--- | --- +api_key | is required +MapConfig | a [Named Map MapConfig](http://docs.carto.com/carto-engine/maps-api/mapconfig/#named-map-layer-options) is required to create a Named Map + +##### template.json + +The `name` argument defines how to name this "template_name".json. Note that there are some requirements for how to name a Named Map template. See the [`name`](#arguments) argument description for details. + +```javascript +{ + "version": "0.0.1", + "name": "template_name", + "auth": { + "method": "token", + "valid_tokens": [ + "auth_token1", + "auth_token2" + ] + }, + "placeholders": { + "color": { + "type": "css_color", + "default": "red" + }, + "cartodb_id": { + "type": "number", + "default": 1 + } + }, + "layergroup": { + "version": "1.0.1", + "layers": [ + { + "type": "cartodb", + "options": { + "cartocss_version": "2.1.1", + "cartocss": "#layer { polygon-fill: <%= color %>; }", + "sql": "select * from european_countries_e WHERE cartodb_id = <%= cartodb_id %>" + } + } + ] + }, + "view": { + "zoom": 4, + "center": { + "lng": 0, + "lat": 0 + }, + "bounds": { + "west": -45, + "south": -45, + "east": 45, + "north": 45 + }, + "preview_layers": { + "0": true, + "layer1": false + } + } +} +``` + +##### Arguments + +Params | Description +--- | --- +name | There can only be _one_ template with the same name for any user. Valid names start with a letter or a number, and only contain letters, numbers, dashes (-), or underscores (_). _This is specific to the name of your Named Map that is specified in the `name` property of the template file_. + +auth | +--- | --- +|_ method | `"token"` or `"open"` (`"open"` is the default if no method is specified. Use `"token"` to password-protect your map) +|_ valid_tokens | when `"method"` is set to `"token"`, the values listed here allow you to instantiate the Named Map. See this [example](http://docs.carto.com/faqs/manipulating-your-data/#how-to-create-a-password-protected-named-map) for how to create a password-protected map. +placeholders | Placeholders are variables that can be placed in your template.json file's SQL or CartoCSS. +layergroup | the layergroup configurations, as specified in the template. See [MapConfig File Format](http://docs.carto.com/carto-engine/maps-api/mapconfig/) for more information. +view (optional) | extra keys to specify the view area for the map. It can be used to have a static preview of a Named Map without having to instantiate it. It is possible to specify it with `center` + `zoom` or with a bounding box `bbox`. Center+zoom takes precedence over bounding box. Also it is possible to choose which layers are visible or not with `preview_layers` indicating its visibility by layer index or id (visible by default). +--- | --- +|_ zoom | The zoom level to use + +|_ center | +--- | --- +|_ |_ lng | The longitude to use for the center +|_ |_ lat | The latitude to use for the center + +|_ bounds | +--- | --- +|_ |_ west | LowerCorner longitude for the bounding box, in decimal degrees (aka most western) +|_ |_ south | LowerCorner latitude for the bounding box, in decimal degrees (aka most southern) +|_ |_ east | UpperCorner longitude for the bounding box, in decimal degrees (aka most eastern) +|_ |_ north | UpperCorner latitude for the bounding box, in decimal degrees (aka most northern) + + +#### Placeholder Format + +Placeholders are variables that can be placed in your template.json file. Placeholders need to be defined with a `type` and a default value for MapConfigs. See details about defining a MapConfig `type` for [Layergroup configurations](http://docs.carto.com/carto-engine/maps-api/mapconfig/#layergroup-configurations). + +Valid placeholder names start with a letter and can only contain letters, numbers, or underscores. They have to be written between the `<%=` and `%>` strings in order to be replaced inside the Named Maps API. + +##### Example + +```javascript +<%= my_color %> +``` + +The set of supported placeholders for a template need to be explicitly defined with a specific type, and default value, for each placeholder. + +#### Placeholder Types + +The placeholder type will determine the kind of escaping for the associated value. Supported types are: + +Types | Description +--- | --- +sql_literal | internal single-quotes will be sql-escaped +sql_ident | internal double-quotes will be sql-escaped +number | can only contain numerical representation +css_color | can only contain color names or hex-values + +Placeholder default values will be used whenever new values are not provided as options, at the time of creation on the client. They can also be used to test the template by creating a default version with new options provided. + +When using templates, be very careful about your selections as they can give broad access to your data if they are defined loosely. + +##### Call + +This is the call for creating the Named Map. It is sending the template.json file to the service, and the server responds with the template id. + +```bash +curl -X POST \ + -H 'Content-Type: application/json' \ + -d @template.json \ + 'https://{username}.carto.com/api/v1/map/named?api_key={api_key}' +``` + +##### Response + +The response back from the API provides the name of your MapConfig as a template, enabling you to edit the Named Map details by inserting your variables into the template where placeholders are defined, and create custom queries using SQL. + +```javascript +{ + "template_id":"name" +} +``` + +### Instantiate + +Instantiating a Named Map allows you to fetch the map tiles. You can use the Maps API to instantiate, or use the CARTO.js `createLayer()` function. The result is an Anonymous Map. + +##### Definition + +```html +POST /api/v1/map/named/{template_name} +``` + +##### Param + +Param | Description +--- | --- +auth_token | `"token"` or `"open"` (`"open"` is the default if not specified. Use `"token"` to password-protect your map) + +```javascript +// params.json, this is required if the Named Map allows variables (if placeholders were defined in the template.json by the user) +{ + "color": "#ff0000", + "cartodb_id": 3 +} +``` + +The fields you pass as `params.json` depend on the variables allowed by the Named Map. If there are variables missing, it will raise an error (HTTP 400). + +**Note:** It is required that you include a `params.json` file to instantiate a Named Map that contains variables, even if you have no fields to pass and the JSON is empty. (This is specific to when a Named Map allows variables (if placeholders were defined in the template.json by the user). + +##### Example + +You can initialize a template map by passing all of the required parameters in a POST to `/api/v1/map/named/{template_name}`. + +Valid auth token will be needed, if required by the template. + + +##### Call + +```bash +curl -X POST \ + -H 'Content-Type: application/json' \ + -d @params.json \ + 'https://{username}.carto.com/api/v1/map/named/{template_name}?auth_token={auth_token}' +``` + +##### Response + +```javascript +{ + "layergroupid": "docs@fd2861af@c01a54877c62831bb51720263f91fb33:123456788", + "last_updated": "2013-11-14T11:20:15.000Z" +} +``` + +##### Error + +```javascript +{ + "errors" : ["Some error string here"] +} +``` + +You can then use the `layergroupid` for fetching tiles and grids as you would normally (see [Anonymous Maps](http://docs.carto.com/carto-engine/maps-api/anonymous-maps/)). + +### Update + +##### Definition + +```bash +PUT /api/v1/map/named/{template_name} +``` + +##### Params + +Param | Description +--- | --- +api_key | is required + +##### Response + +Same as updating a map. + +#### Other Information + +Updating a Named Map removes all the Named Map instances, so they need to be initialized again. + +#### Example + +##### Call + +```bash +curl -X PUT \ + -H 'Content-Type: application/json' \ + -d @template.json \ + 'https://{username}.carto.com/api/v1/map/named/{template_name}?api_key={api_key}' +``` + +##### Response + +```javascript +{ + "template_id": "@template_name" +} +``` + +If any template has the same name, it will be updated. + +If a template with the same name does NOT exist, a 400 HTTP response is generated with an error in this format: + +```javascript +{ + "errors" : ["error string here"] +} +``` + +### Delete + +Deletes the specified template map from the server, and disables any previously initialized versions of the map. + +##### Definition + +```bash +DELETE /api/v1/map/named/{template_name} +``` + +##### Params + +Param | Description +--- | --- +api_key | is required + +#### Example + +##### Call + +```bash +curl -X DELETE 'https://{username}.carto.com/api/v1/map/named/{template_name}?api_key={api_key}' +``` + +##### Response + +```javascript +{ + "errors" : ["Some error string here"] +} +``` + +On success, a 204 (No Content) response will be issued. Otherwise a 4xx response with an error will be returned. + +### Listing Available Templates + +This allows you to get a list of all available templates. + +##### Definition + +```bash +GET /api/v1/map/named/ +``` + +##### Params + +Param | Description +--- | --- +api_key | is required + +#### Example + +##### Call + +```bash +curl -X GET 'https://{username}.carto.com/api/v1/map/named?api_key={api_key}' +``` + +##### Response + +```javascript +{ + "template_ids": ["@template_name1","@template_name2"] +} +``` + +##### Error + +```javascript +{ + "errors" : ["Some error string here"] +} +``` + +### Get Template Definition + +This gets the definition of a requested template. + +##### Definition + +```bash +GET /api/v1/map/named/{template_name} +``` + +##### Params + +Param | Description +--- | --- +api_key | is required + +#### Example + +##### Call + +```bash +curl -X GET 'https://{username}.carto.com/api/v1/map/named/{template_name}?api_key={api_key}' +``` + +##### Response + +```javascript +{ + "template": {...} // see [template.json](#templatejson) +} +``` + +##### Error + +```javascript +{ + "errors" : ["Some error string here"] +} +``` + +### JSONP for Named Maps + +If using a [JSONP](https://en.wikipedia.org/wiki/JSONP) (for old browsers) request, there is a special endpoint used to initialize and create a Named Map. + +##### Definition + +```bash +GET /api/v1/map/named/{template_name}/jsonp +``` + +##### Params + +Params | Description +--- | --- +auth_token | `"token"` or `"open"` (`"open"` is the default if no method is specified. Use `"token"` to password-protect your map) +params | Encoded JSON with the params (variables) needed for the Named Map +lmza | You can use an LZMA compressed file instead of a params JSON file +callback | JSON callback name + +##### Call + +```bash +curl 'https://{username}.carto.com/api/v1/map/named/{template_name}/jsonp?auth_token={auth_token}&callback=callback&config=template_params_json' +``` + +##### Response + +```javascript +callback({ + "layergroupid":"c01a54877c62831bb51720263f91fb33:0", + "last_updated":"1970-01-01T00:00:00.000Z" + "cdn_url": { + "http": "http://cdb.com", + "https": "https://cdb.com" + } +}) +``` + +This takes the `callback` function (required), `auth_token` if the template needs auth, and `config` which is the variable for the template (in cases where it has variables). + +```javascript +url += "config=" + encodeURIComponent( +JSON.stringify({ color: 'red' }); +``` + +The response is: + +```javascript +callback({ + layergroupid: "dev@744bd0ed9b047f953fae673d56a47b4d:1390844463021.1401", + last_updated: "2014-01-27T17:41:03.021Z" +}) +``` + +### CARTO.js for Named Maps + +You can use a Named Map that you created (which is defined by its `name`), to create a map using CARTO.js. This is achieved by adding the [`namedmap` type](http://docs.carto.com/carto-engine/carto-js/layer-source-object/#named-maps-layer-source-object-type-namedmap) layer source object to draw the Named Map. + +```javascript +{ + user_name: '{username}', // Required + type: 'namedmap', // Required + named_map: { + name: '{name_of_map}', // Required, the 'name' of the Named Map that you have created + // Optional + layers: [{ + layer_name: "sublayer0", // Optional + interactivity: "column1, column2, ..." // Optional + }, + { + layer_name: "sublayer1", + interactivity: "column1, column2, ..." + }, + ... + ], + // Optional + params: { + color: "hex_value", + num: 2 + } + } +} +``` + +**Note:** Instantiating a Named Map over a `createLayer` does not require an API Key and by default, does not include auth tokens. _If_ you defined auth tokens for the Named Map configuration, then you will have to include them. + +[CARTO.js](http://docs.carto.com/carto-engine/carto-js/) has methods for accessing your Named Maps. + +1. [layer.setParams()](http://docs.carto.com/carto-engine/carto-js/api-methods/#layersetparamskey-value) allows you to change the template variables (in the placeholders object) via JavaScript + + **Note:** The CARTO.js `layer.setParams()` function is not supported when using Named Maps for Torque. Alternatively, you can create a [Torque layer in a Named Map](http://bl.ocks.org/iriberri/de37be6406f9cc7cfe5a) + +2. [layer.setAuthToken()](http://docs.carto.com/carto-engine/carto-js/api-methods/#layersetauthtokenauthtoken) allows you to set the auth tokens to create the layer + +#### Torque Layer in a Named Map + +If you are creating a Torque layer in a Named Map without using the Torque.js library, you can apply the Torque layer by applying the following code with CARTO.js: + +```javascript + // add cartodb layer with one sublayer + cartodb.createLayer(map, { + user_name: '{username}', + type: 'torque', + order: 1, + options: { + query: "", + table_name: "named_map_tutorial_table", + user_name: "{username}", + tile_style: 'Map { -torque-frame-count:512; -torque-animation-duration:10; -torque-time-attribute:"cartodb_id"; -torque-aggregation-function:"count(cartodb_id)"; -torque-resolution:2; -torque-data-aggregation:linear; } #named_map_tutorial_table_copy{ comp-op: lighter; marker-fill-opacity: 0.9; marker-line-color: #FFF; marker-line-width: 1.5; marker-line-opacity: 1; marker-type: ellipse; marker-width: 6; marker-fill: #FF9900; } #named_map_tutorial_table_copy[frame-offset=1] { marker-width:8; marker-fill-opacity:0.45; } #named_map_tutorial_table_copy[frame-offset=2] { marker-width:10; marker-fill-opacity:0.225; }' + + }, + named_map: { + name: "{namedmap_example}", + layers: [{ + layer_name: "t" + }] + } + }) + .addTo(map) + .done(function(layer) { + + }); +} +``` + +##### Examples of Named Maps created with CARTO.js + +- [Named Map selectors with interaction](http://bl.ocks.org/andy-esch/515a8af1f99d5e690484) + +- [Named Map with interactivity](http://bl.ocks.org/andy-esch/d1a45b8ff5e7bd90cd68) + +- [Toggling sublayers in a Named Map](http://bl.ocks.org/andy-esch/c1a0f4913610eec53cd3) + +### Fetching XYZ Tiles for Named Maps + +Optionally, authenticated users can fetch projected tiles (XYZ tiles or Mapnik Retina tiles) for your Named Map. + +#### Fetch XYZ Tiles Directly with a URL + +Authenticated users, with an auth token, can use XYZ-based URLs to fetch tiles directly, and instantiate the Named Map as part of the request to your application. You do not have to do any other steps to initialize your map. + +To call a template_id in a URL: + +`/{template_id}/{layer}/{z}/{x}/{y}.{format}` + +For example, a complete URL might appear as: + +"https://{username}.carto.com/api/v1/map/named/{template_id}/{layer}/{z}/{x}/{y}.png" + +The placeholders indicate the following: + +- [`template_id`](http://docs.carto.com/carto-engine/maps-api/named-maps/#response) is the response of your Named Map. +- layers can be a number (referring to the ## layer of your map), all layers of your map, or a list of layers. + - To show just the basemap layer, enter the number value `0` in the layer placeholder "https://{username}.carto.com/api/v1/map/named/{template_id}/0/{z}/{x}/{y}.png" + - To show the first layer, enter the number value `1` in the layer placeholder "https://{username}.carto.com/api/v1/map/named/{template_id}/1/{z}/{x}/{y}.png" + - To show all layers, enter the value `all` for the layer placeholder "https://{username}.carto.com/api/v1/map/named/{template_id}/all/{z}/{x}/{y}.png" + - To show a [list of layers](http://docs.carto.com/carto-engine/maps-api/anonymous-maps/#blending-and-layer-selection), enter the comma separated layer value as 0,1,2 in the layer placeholder. For example, to show the basemap and the first layer, "https://{username}.carto.com/api/v1/map/named/{template_id}/0,1/{z}/{x}/{y}.png" + + +#### Get Mapnik Retina Tiles + +Mapnik Retina tiles are not directly supported for Named Maps, so you cannot use the Named Map template_id. To fetch Mapnik Retina tiles, get the [layergroupid](http://docs.carto.com/carto-engine/maps-api/named-maps/#response-1) to initialize the map. + +Instantiate the map by using your `layergroupid` in the token placeholder: + + `{token}/{z}/{x}/{y}@{scale_factor}?{x}.{format}` diff --git a/docs/guides/05-static-maps-API.md b/docs/guides/05-static-maps-API.md new file mode 100644 index 00000000..d36d83b0 --- /dev/null +++ b/docs/guides/05-static-maps-API.md @@ -0,0 +1,226 @@ +## Static Maps API + +The Static Maps API can be initiated using both Named and Anonymous Maps using the 'layergroupid' token. The API can be used to create static images of parts of maps and thumbnails for use in web design, graphic design, print, field work, and many other applications that require standard image formats. + +### Maps API endpoints + +Begin by instantiating either a Named or Anonymous Map using the `layergroupid token` as demonstrated in the Maps API documentation above. The `layergroupid` token calls to the map and allows for parameters in the definition to generate static images. + +#### Zoom + center + +##### Definition + +```bash +GET /api/v1/map/static/center/{token}/{z}/{lat}/{lng}/{width}/{height}.{format} +``` + +##### Params + +Param | Description +--- | --- +token | the layergroupid token from the map instantiation +z | the zoom level of the map +lat | the latitude for the center of the map + +format | the format for the image, supported types: `png`, `jpg` +--- | --- +|_ jpg | will have a default quality of 85. + +#### Bounding Box + +##### Definition + +```bash +GET /api/v1/map/static/bbox/{token}/{bbox}/{width}/{height}.{format}` +``` + +##### Params + +Param | Description +--- | --- +token | the layergroupid token from the map instantiation + +bbox | the bounding box in WGS 84 (EPSG:4326), comma separated values for: +--- | --- + | LowerCorner longitude, in decimal degrees (aka most western) + | LowerCorner latitude, in decimal degrees (aka most southern) + | UpperCorner longitude, in decimal degrees (aka most eastern) + | UpperCorner latitude, in decimal degrees (aka most northern) +width | the width in pixels for the output image +height | the height in pixels for the output image +format | the format for the image, supported types: `png`, `jpg` +--- | --- +|_ jpg | will have a default quality of 85. + +Note: you can see this endpoint as + +```bash +GET /api/v1/map/static/bbox/{token}/{west},{south},{east},{north}/{width}/{height}.{format}` +``` + +#### Named Map + +##### Definition + +```bash +GET /api/v1/map/static/named/{name}/{width}/{height}.{format} +``` + +##### Params + +Param | Description +--- | --- +name | the name of the Named Map +width | the width in pixels for the output image +height | the height in pixels for the output image + +format | the format for the image, supported types: `png`, `jpg` +--- | --- +|_ jpg | will have a default quality of 85. + +A Named Maps static image will get its constraints from the [`view` argument of the Create Named Map function](http://docs.carto.com/carto-engine/maps-api/named-maps/#arguments). If `view` is not defined, it will estimate the extent based on the involved tables, otherwise it fallbacks to `"zoom": 1`, `"lng": 0` and `"lat": 0`. + +##### Layers + +The Static Maps API allows for multiple layers of incorporation into the `MapConfig` to allow for maximum versatility in creating a static map. The examples below were used to generate the static image example in the next section, and appear in the specific order designated. + +**Basemaps** + +```javascript +{ + "type": "http", + "options": { + "urlTemplate": "http://{s}.basemaps.cartocdn.com/dark_nolabels/{z}/{x}/{y}.png", + "subdomains": [ + "a", + "b", + "c" + ] + } +} +``` + +By manipulating the `"urlTemplate"` custom basemaps can be used in generating static images. Supported map types for the Static Maps API are: + +```javascript +'http://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png', +'http://{s}.basemaps.cartocdn.com/dark_nolabels/{z}/{x}/{y}.png', +'http://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}.png', +'http://{s}.basemaps.cartocdn.com/light_nolabels/{z}/{x}/{y}.png', +``` + +**Mapnik** + +```javascript +{ + "type": "mapnik", + "options": { + "sql": "select null::geometry the_geom_webmercator", + "cartocss": "#layer {\n\tpolygon-fill: #FF3300;\n\tpolygon-opacity: 0;\n\tline-color: #333;\n\tline-width: 0;\n\tline-opacity: 0;\n}", + "cartocss_version": "2.2.0" + } +}, +``` + +**CARTO** + +As described in the [MapConfig File Format](http://docs.carto.com/carto-engine/maps-api/mapconfig/), a "cartodb" type layer is now just an alias to a "mapnik" type layer as above, intended for backwards compatibility. + +```javascript +{ + "type": "cartodb", + "options": { + "sql": "select * from park", + "cartocss": "/** simple visualization */\n\n#park{\n polygon-fill: #229A00;\n polygon-opacity: 0.7;\n line-color: #FFF;\n line-width: 0;\n line-opacity: 1;\n}", + "cartocss_version": "2.1.1" + } +} +``` + +Additionally, static images from Torque maps and other map layers can be used together to generate highly customizable and versatile static maps. + + +#### Caching + +It is important to note that generated images are cached from the live data referenced with the `layergroupid token` on the specified CARTO account. This means that if the data changes, the cached image will also change. When linking dynamically, it is important to take into consideration the state of the data and longevity of the static image to avoid broken images or changes in how the image is displayed. To obtain a static snapshot of the map as it is today and preserve the image long-term regardless of changes in data, the image must be saved and stored locally. + +#### Limits + +* While images can encompass an entirety of a map, the limit for pixel range is 8192 x 8192. +* Image resolution is set to 72 DPI +* JPEG quality is 85% +* Timeout limits for generating static maps are the same across CARTO Builder and CARTO Engine. It is important to ensure timely processing of queries. +* If you are publishing your map as a static image with the API, you must manually add [attributions](https://carto.com/attribution) for your static map image. For example, add the following attribution code: + +{% highlight javascript %} +attribution: '© OpenStreetMap contributors, © CARTO +{% endhighlight %} + +### Examples + +After instantiating a map from a CARTO account: + +##### Call + +```bash + GET /api/v1/map/static/center/{layergroupid}/{z}/{x}/{y}/{width}/{height}.png +``` + +##### Response + +

static-api

+ +#### MapConfig + +For this map, the multiple layers, order, and stylings are defined by the MapConfig. + +```javascript +{ + "version": "1.3.0", + "layers": [ + { + "type": "http", + "options": { + "urlTemplate": "http://{s}.basemaps.cartocdn.com/dark_nolabels/{z}/{x}/{y}.png", + "subdomains": [ + "a", + "b", + "c" + ] + } + }, + { + "type": "mapnik", + "options": { + "sql": "select null::geometry the_geom_webmercator", + "cartocss": "#layer {\n\tpolygon-fill: #FF3300;\n\tpolygon-opacity: 0;\n\tline-color: #333;\n\tline-width: 0;\n\tline-opacity: 0;\n}", + "cartocss_version": "2.2.0" + } + }, + { + "type": "cartodb", + "options": { + "sql": "select * from park", + "cartocss": "/** simple visualization */\n\n#park{\n polygon-fill: #229A00;\n polygon-opacity: 0.7;\n line-color: #FFF;\n line-width: 0;\n line-opacity: 1;\n}", + "cartocss_version": "2.1.1" + } + }, + { + "type": "cartodb", + "options": { + "sql": "select * from residential_zoning_2009", + "cartocss": "/** simple visualization */\n\n#residential_zoning_2009{\n polygon-fill: #c7eae5;\n polygon-opacity: 1;\n line-color: #FFF;\n line-width: 0.2;\n line-opacity: 0.5;\n}", + "cartocss_version": "2.1.1" + } + }, + { + "type": "cartodb", + "options": { + "sql": "select * from nycha_developments_july2011", + "cartocss": "/** simple visualization */\n\n#nycha_developments_july2011{\n polygon-fill: #ef3b2c;\n polygon-opacity: 0.7;\n line-color: #FFF;\n line-width: 0;\n line-opacity: 1;\n}", + "cartocss_version": "2.1.1" + } + } + ] +} +``` diff --git a/docs/guides/06-tile-aggregation.md b/docs/guides/06-tile-aggregation.md new file mode 100644 index 00000000..ef615b2e --- /dev/null +++ b/docs/guides/06-tile-aggregation.md @@ -0,0 +1,187 @@ +## Tile Aggregation + +To be able to represent a large amount of data (say, hundred of thousands to millions of points) in a tile. This can be useful both for raster tiles (where the aggregation reduces the number of features to be rendered) and vector tiles (the tile contais less features). + +Aggregation is available only for point geometries. During aggregation the points are grouped using a grid; all the points laying in the same cell of the grid are summarized in a single aggregated result point. + - The position of the aggregated point is controlled by the `placement` parameter. + - The aggregated rows always contain at least a column, named `_cdb_feature_count`, which contains the number of the original points that the aggregated point represents. + +#### Special default aggregation + +When no placement or columns are specified a special default aggregation is performed. + +This special mode performs only spatial aggregation (using a grid defined by the requested tile and the resolution, parameter, as all the other cases), and returns a _random_ record from each group (grid cell) with all its columns and an additional `_cdb_features_count` with the number of features in the group. + +Regarding the randomness of the sample: currently we use the row with the minimum `cartodb_id` value in each group. + +The rationale behind having this special aggregation with all the original columns is to provide a mostly transparent way to handle large datasets without having to provide special map configurations for those cases (i.e. preserving the logic used to produce the maps with smaller datasets). [Overviews have been used so far with this intent](https://carto.com/docs/tips-and-tricks/back-end-data-performance/), but they are inflexible. + +#### User defined aggregations + +When either a explicit placement or columns are requested we no longer use the special, query; we use one determined by the placement (which will default to "centroid"), and it will have as columns only the aggregated columns specified, in addition to `_cdb_features_count`, which is always present. + +We might decide in the future to allow sampling column values for any of the different placement modes. + +#### Behaviour for raster and vector tiles + +The vector tiles from a vector-only map will be aggregated by default. +However, Raster tiles (or vector tiles from a map which defines CartoCSS styles) will be aggregated only upon request. + +Aggregation that would otherwise occur can be disabled by passing an `aggregation=false` parameter to the map instantiation HTTP call. + +To control how aggregation is performed, an aggregation option can be added to the layer: + +```json +{ + "layers": [ + { + "options": { + "sql": "SELECT * FROM data", + "aggregation": { + "placement": "centroid", + "columns": { + "value": { + "aggregate_function": "sum", + "aggregated_column": "value" + } + } + } + } + } + ] +} +``` + +Even if aggregation is explicitly requested it may not be activated, e.g., if the geometries are not points +or the whole dataset is too small. The map instantiation response contains metadata that informs if any particular +layer will be aggregated when tiles are requested, both for vector (mvt) and raster (png) tiles. + +```json +{ + "layergroupid": "7b97b6e76590fef889b63edd2efb1c79:1513608333045", + "metadata": { + "layers": [ + { + "type": "mapnik", + "id": "layer0", + "meta": { + "stats": { + "estimatedFeatureCount": 6232136 + }, + "aggregation": { + "png": true, + "mvt": true + } + } + } + ] + } +} +``` + +### Aggregation parameters + +The aggregation parameters for a layer are defined inside an `aggregation` option of the layer: + +```json +{ + "layers": [ + { + "options": { + "sql": "SELECT * FROM data", + "aggregation": {"...": "..."} + } + } + ] +} +``` + +#### `placement` + +Determines the kind of aggregated geometry generated: + +##### `point-sample` + +This is the default placement. It will place the aggregated point at a random sample of the grouped points, +like the default aggregation does. No other attribute is sampled, though, the point will contain the aggregated attributes determined by the `columns` parameter. + +##### `point-grid` + +Generates points at the center of the aggregation grid cells (squares). + +##### `centroid` + +Generates points with the averaged coordinated of the grouped points (i.e. the points inside each grid cell). + +#### `columns` + +The aggregated attributes defined by `columns` are computed by a applying an _aggregate function_ to all the points in each group. +Valid aggregate functions are `sum`, `avg` (average), `min` (minimum), `max` (maximum) and `mode` (the most frequent value in the group). +The values to be aggregated are defined by the _aggregated column_ of the source data. The column keys define the name of the resulting column in the aggregated dataset. + +For example here we define three aggregate attributes named `total`, `max_price` and `price` which are all computed with the same column, `price`, +of the original dataset applying three different aggregate functions. + +```json +{ + "columns": { + "total": { "aggregate_function": "sum", "aggregated_column": "price" }, + "max_price": { "aggregate_function": "max", "aggregated_column": "price" }, + "price": { "aggregate_function": "avg", "aggregated_column": "price" } + } +} +``` + +> Note that you can use the original column names as names of the result, but all the result column names must be unique. In particular, the names `cartodb_id`, `the_geom`, `the_geom_webmercator` and `_cdb_feature_count` cannot be used for aggregated columns, as they correspond to columns always present in the result. + +#### `resolution` + +Defines the cell-size of the spatial aggregation grid. This is equivalent to the [CartoCSS `-torque-resolution`](https://carto.com/docs/carto-engine/cartocss/properties-for-torque/#-torque-resolution-float) property of Torque maps. + +The aggregation cells are `resolution`×`resolution` pixels in size, where pixels here are defined to be 1/256 of the (linear) size of a tile. +The default value is 1, so that aggregation coincides with raster pixels. A value of 2 would make each cell to be 4 (2×2) pixels, and a value of +0.5 would yield 4 cells per pixel. In teneral values less than 1 produce sub-pixel precision. + +> Note that is independent of the number of pixels for raster tile or the coordinate resolution (mvt_extent) of vector tiles. + + +#### `threshold` + +This is the minimum number of (estimated) rows in the dataset (query results) for aggregation to be applied. If the number of rows estimate is less than the threshold aggregation will be disabled for the layer; the instantiation response will reflect that and tiles will be generated without aggregation. + +#### Example + +```json +{ + "version": "1.7.0", + "extent": [-20037508.5, -20037508.5, 20037508.5, 20037508.5], + "srid": 3857, + "maxzoom": 18, + "minzoom": 3, + "layers": [ + { + "type": "mapnik", + "options": { + "sql": "select * from table", + "cartocss": "#table { marker-width: [total]; marker-fill: ramp(value, (red, green, blue), jenks); }", + "cartocss_version": "2.3.0", + "aggregation": { + "placement": "centroid", + "columns": { + "value": { + "aggregate_function": "avg", + "aggregated_column": "value" + }, + "total": { + "aggregate_function": "sum", + "aggregated_column": "value" + } + }, + "resolution": 2, + "threshold": 500000 + } + } + } + ] +} +``` diff --git a/docs/guides/07-MapConfig-file-format.md b/docs/guides/07-MapConfig-file-format.md new file mode 100644 index 00000000..2f8068a9 --- /dev/null +++ b/docs/guides/07-MapConfig-file-format.md @@ -0,0 +1,270 @@ +{% comment %} +The original resource for this was: +https://github.com/CartoDB/Windshaft/blob/master/doc/MapConfig-1.4.0.md. However this is internal documenation only. This file (07-mapconfig.md) contains select content from the Windshaft internal doc. *I instructed @rochoa to add new Doc issues if/when they make a change to this content - so that the public docs can also be updated. +{% endcomment %} + +## MapConfig File Format + +CARTO uses Windshaft as the map tiler library to render multilayer maps with the [Maps API]({{ site.baseurl }}/carto-engine/maps-api/). The MapConfig file is where these Windshaft layers are stored and applied. You can configure tiles and use the MapConfig document to request different resources for your map. + +This section describes the MapConfig specifications, and required formats, when using the Maps API. + +### Layergroup Configurations + +The following MapConfig Layergroup configurations are applied using the [RFC 4627](http://www.ietf.org/rfc/rfc4627.txt) JSON format. + +Layergroup Configuration | Description | Optional or Required? +--- | --- +`version` | Spec version to use for validation.

**Note:** The default value is `"1.0.0"`. | Optional +`extent` | The default map extent for the map projection.

**Note:** Currently, only webmercator is supported. | Optional +`srid` | The spatial reference identifier for the map. The default is `3857`. | Optional +`maxzoom` | The maximum zoom level for your map. A request beyond the defined maxzoom returns a 404 error.

**Note:** The default value is undefined (infinite). | Optional +`minzoom` | The minimum zoom level for your map. A request beyond the defined minzoom returns a 404 error.

**Note:** The default value is `0`. | Optional +`layers` | Defines the layer type, and the layers, in rendering order.

**Note:** The following layers options are available: | +--- | --- + type | A string value that defines the layer type. You can define up to four values:

`mapnik`, rasterized tiles

`cartodb`, an alias for mapnik (for backward compatibility)

`torque`, render vector tiles in torque format

`http`, load tiles over HTTP

`plain`, color or background image url

`named`, use a Named Map as a layer | Required + options | An object value that sets different options for each layer type.

**Note:** Options that are not defined in different layers will be discarded. | Required + +#### Example of MapConfig + +{% highlight json %} +{ + "version": "1.7.0", + "extent": [-20037508.5, -20037508.5, 20037508.5, 20037508.5], + "srid": 3857, + "maxzoom": 18, + "minzoom": 3, + "layers": [ + { + "type": "mapnik", + "options": { + "sql": "select * from table", + "cartocss": "#table { marker-placement: point; }", + "cartocss_version": "2.3.0" + } + } + ] +} +{% endhighlight %} + +--- + +### Mapnik Layer Options + +If you are using Mapnik as a layer resource, the following configurations are required in your MapConfig file. + +Mapnik Layer Option | Description | Optional or Required? +--- | --- +`sql` | A string value, the SQL request to the user database that will fetch the rendered data.

**Tip:** The SQL request should include the following Mapnik layer configurations: `geom_column`, `interactivity`, and `attributes`, as described in this section.

**Note:** The SQL request may contain substitutions tokens, such as `!bbox!`, `!pixel_width!`, and `!pixel_height!`. It is suggested to define the layergroup `minzoom` and `extent` variables to prevent errors. | Required +`cartocss` | A string value, specifying the CartoCSS style to render the tiles. If this is not present, only vector tiles can be requested for this layer. For a map to be valid either all the layers or none of them must have CartoCSS style.

**Note:** The CartoCSS specification is dependent on the layer type. For details, see [mapnik-reference.json](https://github.com/mapnik/mapnik-reference). | Optional +`cartocss_version` | A string value, specifying the CartoCSS style version of the CartoCSS attribute.

**Note:** The CartoCSS version is specific to the layer type. | Optional +`geom_column` | The name of the column containing the geometry. The default is `the_geom_webmercator`.

*You must specify this value as part of the Mapnik layer `SQL`configuration. | *Optional +`geom_type` | Defines the type of column as either `geometry` (the default) or `raster`.

**Note:** `geom_type` is not compatible with the Mapnik layer `interactivity` option. | Optional +`raster_band` | Defines the raster band (this option is only applicable when the `geom_type=raster`. The default value is `0`.

**Note:** If the default, or no value is specified, raster bands are interpreted as either: grayscale (for single bands), RGB (for 3 bands), or RGBA (for 4 bands). | Optional +`srid` | The spatial reference identifier for the geometry column. The default is `3857`. | Optional +`affected_tables` | A string of values containing the tables that the Mapnik layer `SQL` configuration is using. This value is used if there is a problem guessing what the affected tables are from the SQL configuration (i.e. when using PL/SQL functions). | Optional +`interactivity` | A string of values that contains the fields rendered inside grid.json. All the parameters should be exposed as a result of executing the Mapnik layer `SQL` query.

**Note:** `interactivity` is not compatible with the Mapnik layer `geom_type` option. For example, you cannot create a layergroup instance with a raster layer by defining the `geom_type=raster`.

*You must specify this value as part of the Mapnik layer `SQL` configuration. | *Optional +`attributes` | The id and column values returned by the Mapnik attributes service. (This option is disabled if no configuration is defined).

*You must specify this value as part of the Mapnik layer `SQL`configuration.| *Optional +--- | --- + id | The key value used to fetch columns. | Required + columns | A string of values (columns) returned by the Mapnik attribute service. | Required + +#### Example of Mapnik MapConfig + +{% highlight json %} +{ + "type": "mapnik", + "options": { + "sql": "select * from table", + "cartocss": "#layer { marker-placement: point; }", + "cartocss_version": "2.3.0", + "geom_column": "the_geom_webmercator", + "geom_type": "geometry", + "interactivity": [ "column1", "column2", "..."], + "attributes": { + "id": "cartodb_id", + "columns": ["column1", "column2"] + } + } +} +{% endhighlight %} + +### Torque Layer Options + +If you are using Torque as a layer resource, the following configurations are required in your MapConfig file. For more details about Torque layers in general, see the [Torque API]({{ site.baseurl }}/carto-engine/torque/torqueapi/#torque-api) documentation. + +Torque Layer Option | Description | Optional or Required? +--- | --- +`sql` | A string value, the SQL request to the user database that will fetch the rendered data.

**Tip:** The SQL request should include the following Torque layer configurations: `geom_column`, `interactivity`, and `attributes`, as described in this section. | Required +`cartocss` | A string value, specifying the CartoCSS style to render the tiles.

**Note:** The CartoCSS specification is dependent on the layer type. For details, see [Torque cartocss-reference.js](https://github.com/CartoDB/torque/blob/master/lib/torque/cartocss_reference.js).| Required +`cartocss_version` | A string value, specifying the CartoCSS style version of the CartoCSS attribute.

**Note:** The CartoCSS version is specific to the layer type. | Required +`step` | The number of [animation steps]({{ site.baseurl }}/carto-engine/cartocss/properties-for-torque/#torque-frame-count-number) to render when requesting a torque.png tile. The default value is `0`. | Optional +`geom_column` | The name of the column containing the geometry. The default is `the_geom_webmercator`.

*You must specify this value as part of the Torque layer `SQL`configuration. | *Optional +`srid` | The spatial reference identifier for the geometry column. The default is `3857`. | Optional +`affected_tables` | A string of values containing the tables that the Mapnik layer `SQL` configuration is using. This value is used if there is a problem guessing what the affected tables are from the SQL configuration (i.e. when using PL/SQL functions). | Optional +`attributes` | The id and column values returned by the Torque attributes service. (This option is disabled if no configuration is defined).

*You must specify this value as part of the Torque layer `SQL`configuration.| *Optional +--- | --- + id | The key value used to fetch columns. | Required + columns | A string of values (columns) returned by the Torque attribute service. | Required + +#### Example of Torque MapConfig + +{% highlight json %} +{ + "type": "torque", + "options": { + "sql": "select * from table", + "cartocss": "#layer { ... }", + "cartocss_version": "1.0.0", + "geom_column": "the_geom_webmercator" + } +} +{% endhighlight %} + +### HTTP Layer Options + +If you are using an HTTP destination as the resource for a map layer, the following configurations are required in your MapConfig file. + +HTTP Layer Option | Description | Optional or Required? +--- | --- +`urlTemplate` | A string value, end URL, from where the tile data is retrieved. _URLs must be included in the configuration whitelist to be valid._

**Note:** The {String} value includes:

`{z}` as the zoom level

`{x} and {y}` as the tile coordinates

Optionally, the subdomain `{s}` may be included as part of the `urlTemplate` configuration. Otherwise, you can define the `subdomains` separately, as shown below. | Required +`subdomains` | A string of values used to retrieve tiles from different subdomains. The default value is [`a`, `b`, `c`] when `{s}` is defined in the `urlTemplate` configuration. Otherwise, the default value is `[ ]`.

**Note:** The subdomains value will consistently replace the `{s}` value defined in the `urlTemplate`.| Optional +`tms` | A boolean value that specifies whether the tile is using Tile Map Service format. The default value is `false`.

**Note:** If the value is `true`, the TMS inverses the Y axis numbering for tiles. | Optional + +#### Example of HTTP MapConfig + +{% highlight json %} +{ + "type": "http", + "options": { + "urlTemplate": "http://{s}.example.com/{z}/{x}/{y}.png", + "subdomains": ["a", "b", "c"], + "tms": false + } +} +{% endhighlight %} + +### Plain Layer Options + +If you are using plain layer options as your map resource, the following configurations are required in your MapConfig file. + +_**Note:** At least one of the plain layer options (either `color` or `imageUrl`) must be defined. If both options are defined, only `color` is used as part of the plain layer configuration._ + +Plain Layer Option | Description | Optional or Required? +--- | --- +`color` | A string value of numbers that defines the valid colors to include. The default value is `null`. Valid colors include:

- A string value that includes CSS colors (i.e. `blue`) or a hex color string (i.e. `#0000ff`)

- An integer array of r,g,b values (i.e. `[255,0,0]`)

- An integer array of r,g,b,a values (i.e. `[255,0,0,128]`)

* If **only** the `color` value is used for a plain layer, this value is Required.

* If **both** `color` and `imageUrl` are defined, only the `color` value is used for the plain layer configuration.| *Both +`imageUrl` | A string value, end URL, from where the image is retrieved. The default value is `null`.

* If **only** the `imageUrl` value is used for a plain layer, this value is Required.

* If `color` is defined, this `imageUrl` value is ignored. | *Both + +#### Example of Plain MapConfig + +{% highlight json %} +{ + "type": "plain", + "options": { + "color": "blue", + "imageUrl": "http://example.com/background.png" + } +} +{% endhighlight %} + +### Named Map Layer Options + +You can use a [Named Map]({{ site.baseurl }}/carto-engine/maps-api/named-maps/#named-maps) as a map layer. Note the following limitations before referencing the MapConfig options for a Named Map layer. + +_**Limitations:**_ + +- A Named Map will not allow you to have `named` type layers inside of your template layergroup's layers definition +- A `named` layer does not allow Named Maps from other accounts. You can only use Named Maps from the _same_ user account + +If you are using `named` layer options as your map resource, the following configurations are required in your MapConfig file. + +Named Layer Option | Description | Optional or Required? +--- | --- +`name` | A string value, the name for the Named Map to use. | Required +`config` | An object, the replacement values for the Named Map's template placeholders. | Optional +`auth_tokens` | Strings array, the authorized tokens in case the Named Map has auth method set to `token`. | Optional + +#### Example of Named MapConfig + +{% highlight json %} +{ + "type": "named", + "options": { + "name": "world_borders", + "config": { + "color": "#000" + }, + "auth_tokens": ["token1", "token2"] + } +} +{% endhighlight %} + +### Aggregation Options + +The data used to render tiles, or contained in the tiles (for the case of vector tiles), can be spatially [aggregated](https://carto.com/docs/carto-engine/maps-api/named-maps/) under some circumstances. + +An `aggregation` attribute can be used in the layer `options` to control the aggregation. A value of `false` will disable aggregation for the layer. Otherwise, an object can be passed with the following aggregation parameters: + +Parameter|Description|Default value +`placement`|Determines the kind of aggregated geometry generated ("point-sample", "point-grind" or "centroid").|"centroid" +`columns`|Defines aggregated columns; each one by an "aggregate_function" ("sum", "avg", "min, "max", "mode", "count") and "aggregated_column" name.| +`resolution`|Defines the cell-size of the spatial aggregation grid.|1 (for 256x256 cells per tile) +`threshold`|Minimum rows in the dataset to apply aggregation. + +#### Example of Aggregation MapConfig + +{% highlight json %} +{ + "version": "1.7.0", + "extent": [-20037508.5, -20037508.5, 20037508.5, 20037508.5], + "srid": 3857, + "maxzoom": 18, + "minzoom": 3, + "layers": [ + { + "type": "mapnik", + "options": { + "sql": "select * from table", + "cartocss": "#table { marker-width: [total]; marker-fill: ramp(value, (red, green, blue), jenks); }", + "cartocss_version": "2.3.0", + "aggregation": { + "placement": "centroid",s + "columns": { + "value": { + "aggregate_function": "avg", + "aggregated_column": "value" + }, + "total": { + "aggregate_function": "sum", + "aggregated_column": "value" + } + }, + "resolution": 2, // Aggregation cell is 2x2 pixels + "threshold": 500000 + } + } + } + ] +} +{% endhighlight %} + +### MapConfig Requirements + +All of these are MapConfig requirements for [Anonymous Maps]({{ site.baseurl }}/carto-engine/maps-api/anonymous-maps/#retrieve-resources-from-the-layergroup). + +- Identified by `{z}/{x}/{y}` path + +- If applicable, additionally identified by `LAYER_NUMBER` + +- Can be of different formats: + - png + - grid.json + - torque.json + +- Static images/previews + - With a center or a bounding box + +- Attributes + -Identified by LAYER_NUMBER and FEATURE_ID + +**Tip:** The MapConfig file may be extended for specific uses. For example, [Windshaft-CartoDB](https://github.com/CartoDB/Windshaft-cartodb/blob/master/docs/MultiLayer-API.md) defines the addition of a `stat_tag` element in the config. This extension is also covered as part of the [Named Map Layer Options](#named-map-layer-options). \ No newline at end of file diff --git a/docs/guides/08-MapConfig-aggregation-extension.md b/docs/guides/08-MapConfig-aggregation-extension.md new file mode 100644 index 00000000..9dfb4dfa --- /dev/null +++ b/docs/guides/08-MapConfig-aggregation-extension.md @@ -0,0 +1,64 @@ +## MapConfig Aggregation Extension + +### 1. Purpose + +This specification describes an extension for +[MapConfig 1.7.0](https://github.com/CartoDB/Windshaft/blob/master/doc/MapConfig-1.7.0.md) version. + + +### 2. Changes over specification + +This extension introduces a new layer options for aggregated data tile generation. + +#### 2.1 Aggregation options + +The layer options attribute is extended with a new optional `aggregation` attribute. +The value of this attribute can be `false` to explicitly disable aggregation for the layer. + +```javascript +{ + aggregation: { + + // OPTIONAL + // string, defines the placement of aggregated geometries. Can be one of: + // * "point-sample", the default places geometries at a sample point (one of the aggregated geometries) + // * "point-grid" places geometries at the center of the aggregation grid cells + // * "centroid" places geometriea at the average position of the aggregated points + // See https://github.com/CartoDB/Windshaft-cartodb/blob/master/docs/aggregation.md#placement for more details + placement: "point-sample", + + // OPTIONAL + // object, defines the columns of the aggregated datasets. Each property corresponds to a columns name and + // should contain an object with two properties: "aggregate_function" (one of "sum", "max", "min", "avg", "mode" or "count"), + // and "aggregated_column" (the name of a column of the original layer query or "*") + // A column defined as `"_cdb_features_count": {"aggregate_function": "count", aggregated_column: "*"}` + // is always generated in addition to the defined columns. + // The column names `cartodb_id`, `the_geom`, `the_geom_webmercator` and `_cdb_feature_count` cannot be used + // for aggregated columns, as they correspond to columns always present in the result. + // See https://github.com/CartoDB/Windshaft-cartodb/blob/master/docs/aggregation.md#columns for more details + columns: { + "aggregated_column_1": { + "aggregate_function": "sum", + "aggregated_column": "original_column_1" + } + }, + + // OPTIONAL + // Number, defines the cell-size of the spatial aggregation grid as a pixel resolution power of two (1/4, 1/2,... 2, 4, 16) + // to scale from 256x256 pixels; the default is 1 corresponding to 256x256 cells per tile. + // See https://github.com/CartoDB/Windshaft-cartodb/blob/master/docs/aggregation.md#resolution for more details + resolution: 1, + + // OPTIONAL + // Number, the minimum number of (estimated) rows in the dataset (query results) for aggregation to be applied. + // See https://github.com/CartoDB/Windshaft-cartodb/blob/master/docs/aggregation.md#threshold for more details + threshold: 500000 + } +} +``` + +### History + +#### 1.0.0 + + - Initial version diff --git a/docs/guides/09-MapConfig-analyses-extension.md b/docs/guides/09-MapConfig-analyses-extension.md new file mode 100644 index 00000000..53500e32 --- /dev/null +++ b/docs/guides/09-MapConfig-analyses-extension.md @@ -0,0 +1,95 @@ +## MapConfig Analyses Extension + +### 1. Purpose + +This specification describes an extension for +[MapConfig 1.4.0](https://github.com/CartoDB/Windshaft/blob/master/doc/MapConfig-1.4.0.md) version. + + +### 2. Changes over specification + +This extension targets layers with `sql` option, including layer types: `cartodb`, `mapnik`, and `torque`. + +It extends MapConfig with a new attribute: `analyses`. + +#### 2.1 Analyses attribute + +The new analyses attribute must be an array of analyses as per [camshaft](https://github.com/CartoDB/camshaft). Each +analysis must adhere to the [camshaft-reference](https://github.com/CartoDB/camshaft/blob/0.8.0/reference/versions/0.7.0/reference.json) specification. + +Each node can have an id that can be later references to consume the query from MapConfig's layers. + +Basic analyses example: + +```javascript +[ + { + // REQUIRED + // string, `id` free identifier that can be reference from any layer + "id": "HEAD", + // REQUIRED + // string, `type` camshaft's analysis type + "type": "source", + // REQUIRED + // object, `params` will depend on `type`, check camshaft-reference for more information + "params": { + "query": "select * from your_table" + } + } +] +``` + +### 2.2. Integration with layers + +As pointed before an analysis node id can be referenced from layers to consume its output query. + +The layer consuming the output must reference it with the following option: + +``` +{ + "options": { + // REQUIRED + // object, `source` as in the future we might want to have other source options + "source": { + // REQUIRED + // string, `id` the analysis node identifier + "id": "HEAD" + } + } +} +``` + +#### 2.3. Complete example + +``` +{ + "version": "1.4.0", + "layers": [ + { + "type": "cartodb", + "options": { + "source": { + "id": "HEAD" + }, + "cartocss": "...", + "cartocss_version": "2.3.0" + } + } + ], + "analyses": [ + { + "id": "HEAD", + "type": "source", + "params": { + "query": "select * from your_table" + } + } + ] +} +``` + +### History + +#### 1.0.0 + + - Initial version diff --git a/docs/guides/10-MapConfig-dataviews-extension.md b/docs/guides/10-MapConfig-dataviews-extension.md new file mode 100644 index 00000000..7a5ab59a --- /dev/null +++ b/docs/guides/10-MapConfig-dataviews-extension.md @@ -0,0 +1,279 @@ +## MapConfig Dataviews Extension + +### 1. Purpose + +This specification describes an extension for +[MapConfig 1.4.0](https://github.com/CartoDB/Windshaft/blob/master/doc/MapConfig-1.4.0.md) version. + + +### 2. Changes over specification + +This extension depends on Analyses extension. It extends MapConfig with a new attribute: `dataviews`. + +It makes possible to get tabular data from analysis nodes: aggregated lists, aggregations, and histograms. + +#### 2.1. Dataview types + +##### Aggregation + +An aggregation is a list with aggregated results by a column and a given aggregation function. + +Definition +``` +{ + // REQUIRED + // string, `type` the aggregation type + “type”: “aggregation”, + // REQUIRED + // object, `options` dataview params + “options”: { + // REQUIRED + // string, `column` column name to aggregate by + “column”: “country”, + // REQUIRED + // string, `aggregation` operation to perform + “aggregation”: “count” + // OPTIONAL + // string, `aggregationColumn` column value to aggregate + // This param is required when `aggregation` is different than "count" + “aggregationColumn”: “population” + } +} +``` + +Expected output +``` +{ + "type": "aggregation", + "categories": [ + { + "category": "foo", + "value": 100 + }, + { + "category": "bar", + "value": 200 + } + ] +} +``` + +##### Histograms + +Histograms represent the data distribution for a column. + +Definition +``` +{ + // REQUIRED + // string, `type` the histogram type + “type”: “histogram”, + // REQUIRED + // object, `options` dataview params + “options”: { + // REQUIRED + // string, `column` column name to aggregate by + “column”: “name”, + // OPTIONAL + // number, `bins` how many buckets the histogram should use + “bins”: 10 + } +} +``` + +Expected output +``` +{ + "type": "histogram", + "bins": [{"bin": 0, "start": 2, "end": 2, "min": 2, "max": 2, "freq": 1}, null, null, {"bin": 3, "min": 40, "max": 44, "freq": 2}, null], + "width": 10 +} +``` + +##### Formula + +Formulas given a final value representing the whole dataset. + +Definition +``` +{ + // REQUIRED + // string, `type` the formula type + “type”: “formula”, + // REQUIRED + // object, `options` dataview params + “options”: { + // REQUIRED + // string, `column` column name to aggregate by + “column”: “name”, + // REQUIRED + // string, `aggregation` operation to perform + “operation”: “count” + } +} +``` + +Operation must be: “min”, “max”, “count”, “avg”, or “sum”. + +Result +``` +{ + "type": "formula", + "operation": "count", + "result": 1000, + "nulls": 0 +} +``` + + +#### 2.2 Dataviews attribute + +The new dataviews attribute must be a dictionary of dataviews. + +An analysis node id can be referenced from dataviews to consume its output query. + + +The layer consuming the output must reference it with the following option: + +``` +{ + // REQUIRED + // object, `source` as in the future we might want to have other source options + "source": { + // REQUIRED + // string, `id` the analysis node identifier + "id": "HEAD" + } +} +``` + +#### 2.3. Complete example + +``` +{ + "version": "1.4.0", + "layers": [ + { + "type": "cartodb", + "options": { + "source": { + "id": "HEAD" + }, + "cartocss": "...", + "cartocss_version": "2.3.0" + } + } + ], + "dataviews" { + "basic_histogram": { + "source": { + "id": "HEAD" + }, + "type": "histogram", + "options": { + "column": "pop_max" + } + } + }, + "analyses": [ + { + "id": "HEAD", + "type": "source", + "params": { + "query": "select * from your_table" + } + } + ] +} +``` + +#### 3. Filters + +Camshaft's analyses expose a filtering capability and `aggregation` and `histogram` dataviews get them for free with + this extension. Filters are available with the very dataview id, so if you have a "basic_histogram" histogram dataview + you can filter with a range filter with "basic_histogram" name. + + +#### 3.1 Filter types + +##### Category + +Allows to remove results that are not contained within a set of elements. +Initially this filter can be applied to a `numeric` or `text` columns. + +Params + +``` +{ + “accept”: [“Spain”, “Germany”] + “reject”: [“Japan”] +} +``` + +##### Range filter + +Allows to remove results that don’t satisfy numeric min and max values. +Filter is applied to a numeric column. + +Params + +``` +{ + “min”: 0, + “max”: 1000 +} +``` + +#### 3.2. How to apply filters + +Filters must be applied at map instantiation time. + +With :mapconfig as a valid MapConfig and with :filters (a valid JSON) as: + +##### Anonymous map + +`GET /api/v1/map?config=:mapconfig&filters=:filters` + +`POST /api/v1/map?filters=:filters` +with `BODY=:mapconfig` + +If in the future we need to support a bigger filters param and it doesn’t fit in the query string, + we might solve it by accepting: + +`POST /api/v1/map` +with `BODY={“config”: :mapconfig, “filters”: :filters}` + +##### Named map + +Assume :params (a valid JSON) as named maps params, like in: `{“color”: “red”}` + +`GET /api/v1/named/:name/jsonp?config=:params&filters=:filters&callback=cb` + +`POST /api/v1/named/:name?filters=:filters` +with `BODY=:params` + +If, again, in the future we need to support a bigger filters param that doesn’t fit in the query string, + we might solve it by accepting: + +`POST /api/v1/named/:name` +with `BODY={“config”: :params, “filters”: :filters}` + + +#### 3.3 Bounding box special filter + +A bounding box filter allows to remove results that don’t satisfy a geospatial range. + +The bounding box special filter is available per dataview and there is no need to create a bounding box definition as +it’s always possible to apply a bbox filter per dataview. + +A dataview can get its result filtered by bounding box by sending a bbox param in the query string, +param must be in the form `west,south,east,north`. + +So applying a bbox filter to a dataview looks like: +GET /api/v1/map/:layergroupid/dataview/:dataview_name?bbox=-90,-45,90,45 + +### History + +#### 1.0.0-alpha + + - WIP document diff --git a/docs/guides/11-Mapconfig-named-maps-extension.md b/docs/guides/11-Mapconfig-named-maps-extension.md new file mode 100644 index 00000000..001ec847 --- /dev/null +++ b/docs/guides/11-Mapconfig-named-maps-extension.md @@ -0,0 +1,58 @@ +## MapConfig Named Maps Extension + +### 1. Purpose + +This specification describes an extension for +[MapConfig 1.3.0](https://github.com/CartoDB/Windshaft/blob/master/doc/MapConfig-1.3.0.md) version. + + +### 2. Changes over specification + +This extension introduces a new layer type so it's possible to use a Named Map by its name as a layer. + +#### 2.1 Named layers definition + +```javascript +{ + // REQUIRED + // string, `named` is the only supported value + type: "named", + + // REQUIRED + // object, set `named` map layers configuration + options: { + + // REQUIRED + // string, the name for the Named Map to use + name: "world_borders", + + // OPTIONAL + // object, the replacement values for the Named Map's template placeholders + // See https://github.com/CartoDB/Windshaft-cartodb/blob/master/docs/Map-API.md#instantiate-1 for more details + config: { + "color": "#000" + }, + + // OPTIONAL + // string array, the authorized tokens in case the Named Map has auth method set to `token` + // See https://github.com/CartoDB/Windshaft-cartodb/blob/master/docs/Map-API.md#named-maps-1 for more details + auth_tokens: [ + "token1", + "token2" + ] + } +} +``` + +#### 2.2 Limitations + +1. A Named Map will not allow to have `named` type layers inside their templates layergroup's layers definition. +2. A `named` layer does not allow Named Maps form other accounts, it's only possible to use Named Maps from the very +same user account. + + +### History + +#### 1.0.0 + + - Initial version diff --git a/docs/reference/01-routes.md b/docs/reference/01-routes.md new file mode 100644 index 00000000..0ebe732b --- /dev/null +++ b/docs/reference/01-routes.md @@ -0,0 +1,114 @@ +This document list all routes available in Windshaft-cartodb Maps API server. + +## Routes list + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/:token/:z/:x/:y@:scale_factor?x.:format {:user(f),:token(f),:z(f),:x(f),:y(f),:scale_factor(t),:format(f)} (1)` +
Notes: Mapnik retina tiles [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/:token/:z/:x/:y.:format {:user(f),:token(f),:z(f),:x(f),:y(f),:format(f)} (1)` +
Notes: Mapnik tiles [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/:token/:layer/:z/:x/:y.(:format) {:user(f),:token(f),:layer(f),:z(f),:x(f),:y(f),:format(f)} (1)` +
Notes: Per :layer rendering based on :format [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/:token/:layer/attributes/:fid {:user(f),:token(f),:layer(f),:fid(f)} (1)` +
Notes: Endpoint for info windows data, alternative for sql api when tables are private [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/static/center/:token/:z/:lat/:lng/:width/:height.:format {:user(f),:token(f),:z(f),:lat(f),:lng(f),:width(f),:height(f),:format(f)} (1)` +
Notes: Static Maps API [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/static/bbox/:token/:west,:south,:east,:north/:width/:height.:format {:user(f),:token(f),:west(f),:south(f),:east(f),:north(f),:width(f),:height(f),:format(f)} (1)` +
Notes: Static Maps API [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/:token/:layer/widget/:widgetName {:user(f),:token(f),:layer(f),:widgetName(f)} (1)` +
Notes: By :widgetName per :layer widget [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/:token/:layer/widget/:widgetName/search {:user(f),:token(f),:layer(f),:widgetName(f)} (1)` +
Notes: By :widgetName per :layer widget search [0] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup) {:user(f)} (1)` +
Notes: Map instantiation [0] + +1. `POST (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup) {:user(f)} (1)` +
Notes: Map instantiation [0] + +1. `GET (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template)/:template_id/jsonp {:user(f),:template_id(f)} (1)` +
Notes: Named maps JSONP instantiation [1] + +1. `POST (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template)/:template_id {:user(f),:template_id(f)} (1)` +
Notes: Instantiate named map [1] + +1. `OPTIONS (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup) {:user(f)} (1)` +
Notes: CORS [0] + +1. `GET (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template)/:template_id/:layer/:z/:x/:y.(:format) {:user(f),:template_id(f),:layer(f),:z(f),:x(f),:y(f),:0(f),:format(f)} (1)` +
Notes: Per :layer fixed URL named map tiles [1] + +1. `GET (?:/api/v1/map|/user/:user/api/v1/map|/tiles/layergroup)/static/named/:template_id/:width/:height.:format {:user(f),:template_id(f),:width(f),:height(f),:format(f)} (1)` +
Notes: Static map for named maps [1] + +1. `POST (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template) {:user(f)} (1)` +
Notes: Create named map (w/ API KEY) [1] + +1. `PUT (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template)/:template_id {:user(f),:template_id(f)} (1)` +
Notes: Update a named map (w/ API KEY) [1] + +1. `GET (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template)/:template_id {:user(f),:template_id(f)} (1)` +
Notes: Named map retrieval (w/ API KEY) [1] + +1. `DELETE (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template)/:template_id {:user(f),:template_id(f)} (1)` +
Notes: Delete named map (w/ API KEY) [1] + +1. `GET (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template) {:user(f)} (1)` +
Notes: List named maps (w/ API KEY) [1] + +1. `OPTIONS (?:/api/v1/map/named|/user/:user/api/v1/map/named|/tiles/template)/:template_id {:user(f),:template_id(f)} (1)` +
Notes: CORS [1] + +1. `GET /health {} (1)` +
Notes: Health check + +1. `GET / {} (1)` +
Notes: Welcome message + +1. `GET /version {} (1)` +
Notes: Return relevant module versions: mapnik, grainstore, etc + + +## Optional deprecated routes + +- [0] `/tiles/layergroup` is deprecated and `/api/v1/map` should be used but we keep it for now. +- [1] `/tiles/template` is deprecated and `/api/v1/map/named` should be used but we keep it for now. + +## How to generate the list of routes + +Something like the following patch should do the trick + +```javascript +diff --git a/lib/cartodb/server.js b/lib/cartodb/server.js +index 5f62850..bca377d 100644 +--- a/lib/cartodb/server.js ++++ b/lib/cartodb/server.js +@@ -215,6 +215,20 @@ module.exports = function(serverOptions) { + * END Routing + ******************************************************************************************************************/ + ++ var format = require('util').format; ++ var routesNotes = app._router.stack ++ .filter(function(handler) { return !!handler.route; }) ++ .map(function(handler) { ++ return format("\n1. `%s %s {%s} (1)`\n
Notes: [DEPRECATED]? ", ++ Object.keys(handler.route.methods)[0].toUpperCase(), ++ handler.route.path, ++ handler.keys.map(function(k) { ++ return format(':%s(%s)', k.name, k.optional ? 't' : 'f'); ++ }).join(',') ++ ); ++ }); ++ console.log(routesNotes.join('\n')); ++ + return app; + }; + + +``` diff --git a/docs/support/01-metrics.md b/docs/support/01-metrics.md new file mode 100644 index 00000000..bf845b7a --- /dev/null +++ b/docs/support/01-metrics.md @@ -0,0 +1,43 @@ +## Metrics + +Windshaft-cartodb metrics +========================= +See [Windshaft metrics documentation](https://github.com/CartoDB/Windshaft/blob/master/doc/metrics.md) to understand the full picture. + +The next list includes the API endpoints, each endpoint may have several inner timers, some of them are displayed within this list as subitems. Find the description for them in the Inner timers section. +## Timers +- **windshaft-cartodb.flush_cache**: time to flush the tile and sql cache +- **windshaft-cartodb.get_template**: time to retrieve an specific template +- **windshaft-cartodb.delete_template**: time to delete an specific template +- **windshaft-cartodb.get_template_list**: time to retrieve the list of owned templates +- **windshaft-cartodb.instance_template_post**: time to create a template via HTTP POST +- **windshaft-cartodb.instance_template_get**: time to create a template via HTTP GET + + TemplateMaps_instance + + createLayergroup + +There are some endpoints that are not being tracked: +- Adding a template +- Updating a template + +### Inner timers +Again, each inner timer may have several inner timers. + +- **addCacheChannel**: time to add X-Cache-Channel header based on table last modifications +- **LZMA decompress**: time to decompress request params with LZMA +- **TemplateMaps_instance**: time to retrieve a map template instance, see *getTemplate* and *authorizedByCert* +- **affectedTables**: time to check what are the affected tables for adding the cache channel, see *addCacheChannel* +- **authorize**: time to authorize a request, see *authorizedByAPIKey*, *authorizedByCert*, *authorizedBySigner* +- **authorizedByCert**: time to authorize a template instantiation +- **findLastUpdated**: time to retrieve the last update time for a list of tables, see *affectedTables* +- **generateCacheChannel**: time to generate the headers for the cache channel based on the request, see *addCacheChannel* +- **getSignerMapKey**: time to retrieve from redis the authorized user for a template map +- **getTablePrivacy**: time to retrieve from redis the privacy of a table +- **getTemplate**: time to retrieve from redis the template for a map +- **getUserMapKey**: time to retrieve from redis the user key for a map +- **incMapviewCount**: time to incremenent in redis the map views +- **mapStore_load**: time to retrieve from redis a map configuration +- **req2params.setup**: time to prepare the params from a request, see *req2params* in Windshaft documentation +- **setDBAuth**: time to retrieve from redis and set db user and db password from a user +- **setDBConn**: time to retrieve from redis and set db host and db name from a user +- **setDBParams**: time to prepare all db params to be able to connect/query a database, see *setDBAuth* and *setDBConn* +- **tablePrivacy_getUserDBName**: time to retrieve from redis the database for a user