diff --git a/doc/CartoDB-user-table.md b/doc/CartoDB-user-table.md deleted file mode 100644 index 88d84cc..0000000 --- a/doc/CartoDB-user-table.md +++ /dev/null @@ -1,29 +0,0 @@ -A "cartodb" user table is a table with a well-known set of fields and a well-known set of triggers attached on. - -The fields are: - - - `cartodb_id`, a numerical primary key of serial type - - `created_at`, timestamp with timezone not null default now() - - `updated_at`, timestamp with timezone not null default now() - - `the_geom`, geometry, GiST indexed, constrained (see below) - - `the_geom_webmercator`, geometry, GiST indexed, constrained (see below) - -The values of "the_geom" and "the_geom_webmercator" must match these constraints: - - - Only POINT, MULTILINE, MULTIPOLYGON types ? Maybe UNCONSTRAINED - - Only 2 dimensions ? Maybe UNCONSTRAINED - - SRID=4326 for the_geom and SRID=3857 for the_geom_webmercator - -The triggers are: - - - `track_updates` after modifying statement updates cdb_tablemetadata - - `test_quota` before changing statement to forbid if overquota - - `test_quota_per_row` before changing row to forbod if overquota (checked on a probabilistic basis) - - `update_the_geom_webmercator` before insert or update row to maintain the_geom_webmercator - - `update_updated_at_trigger` before update row to maintain updated_at - -Some conversions will be attempted to perform upon cartodbfication when certain fields appear: - - - `cartodb_id`: If found type TEXT will be attempted to cast - - `created_at`: If found type TEXT will be attempted to cast - - `updated_at`: If found type TEXT will be attempted to cast \ No newline at end of file diff --git a/doc/CartoDB-user-table.rst b/doc/CartoDB-user-table.rst new file mode 100644 index 0000000..c1bb569 --- /dev/null +++ b/doc/CartoDB-user-table.rst @@ -0,0 +1,68 @@ +CartoDB User Table +================== + +Introduction +---------- +A CartoDB user table is a table with a well-known set of columns and a well-known set of triggers attached on. + +Columns +---------- +The required columns of a CartoDB table are: + +- ``cartodb_id`` + - This column will be used as the primary key of the table and it has a sequence as default value + - Its values must be integer, non-zero, non-null and unique + - B-Tree indexed +- ``the_geom`` + - This column stores the main geometric features of a table + - The type of the column in the Postgres database is ``geometry(Geometry,4326)``` + - GiST indexed + - geometry, GiST indexed, constrained (see below) +- ``the_geom_webmercator`` + - This column stores the geometries used for rendering purposes + - The type of the column in the Postgres database is ``geometry(Geometry,3857)`` + - GiST indexed + - This column is automatically updated by the system when the ``the_geom`` column is updated or when there is an insertion of a new row into the table (See triggers below) + +The values of ``the_geom`` and ``the_geom_webmercator`` must be two-dimensional Points, MultiLineStrings or MultiPolygons. Different geometric types in a CartoDB table are not supported. + +Described table example +^^^^^^^^^^ +:: + + Column | Type | Modifiers + ----------------------+-------------------------+-------------------------------------------------------- + cartodb_id | bigint | not null default nextval('t_cartodb_id_seq'::regclass) + the_geom | geometry(Geometry,4326) | + the_geom_webmercator | geometry(Geometry,3857) | + Indexes: + "table_name_pkey" PRIMARY KEY, btree (cartodb_id) + "table_name_the_geom_idx" gist (the_geom) + "table_name_the_geom_webmercator_idx" gist (the_geom_webmercator) + +Triggers +---------- +The triggers generated in each CartoDB table are: + +- ``track_updates`` after modifying statement updates ``cdb_tablemetadata`` +- ``test_quota`` before changing statement to forbid if overquota +- ``test_quota_per_row`` before insert ot update row to forbid if overquota (checked on a probabilistic basis) +- ``update_the_geom_webmercator`` before insert or update row to maintain the ``the_geom_webmercator`` updated with the contents in ``the_geom`` + +Described triggers example +^^^^^^^^^^ +:: + + test_quota BEFORE INSERT OR UPDATE ON t FOR EACH STATEMENT EXECUTE PROCEDURE cdb_checkquota('0.1', '-1', 'public') + test_quota_per_row BEFORE INSERT OR UPDATE ON t FOR EACH ROW EXECUTE PROCEDURE cdb_checkquota('0.001', '-1', 'public') + track_updates AFTER INSERT OR DELETE OR UPDATE OR TRUNCATE ON t FOR EACH STATEMENT EXECUTE PROCEDURE cdb_tablemetadata_trigger() + update_the_geom_webmercator_trigger BEFORE INSERT OR UPDATE OF the_geom ON t FOR EACH ROW EXECUTE PROCEDURE _cdb_update_the_geom_webmercator() + + +Further details +---------- + +Some conversions will be attempted to perform upon cartodbfication when certain fields appear: + +- ``cartodb_id``: If found type TEXT will be attempted to cast to integer. If not casteable, an eror will be raised. +- ``the_geom``: If found type TEXT will be attempted to cast to geometry(Geometry,4326). diff --git a/doc/cartodbfy-requirements.rst b/doc/cartodbfy-requirements.rst index 0743699..81b6b11 100644 --- a/doc/cartodbfy-requirements.rst +++ b/doc/cartodbfy-requirements.rst @@ -1,59 +1,63 @@ +CartoDBfy Requirements +====================== + Introduction -============ +------------ -This document aims at describing what cartodbfy is and what its formal requirements are, with the following goals in mind: +This document aims at describing what the CartoDBfication is and what its formal requirements are, with the following goals in mind: -- clarify what are the expectations of the "cartodbfycation process". -- define an important part of what should be a stable, public API -- allow for better testing, which should in turn... +- Clarify what are the expectations of the "cartodbfycation process". +- Define an important part of what should be a stable, public API +- Allow for better testing, which should in turn... - ...ease modifications and increase quality of the code +What is the CartoDBfycation +--------------------------- -What is the cartodbfycation -=========================== - -The cartodbfycation is the process of converting an arbitrary postgres table into a valid CartoDB table, and register it in the system so that it can be used in the CartoDB editor and platform to generate maps and analysis. - +The CartoDBfycation is the process of converting an arbitrary postgres table into a valid CartoDB table, and register it in the system so that it can be used in the CartoDB editor and platform to generate maps and analysis. +It is performed by running the function ``CDB_CartodbfyTable(reloid REGCLASS)`` over a target table. Valid CartoDB tables -==================== +-------------------- A valid CartoDB table shall meet the following conditions: -- Have a ``cartodb_id`` integer column as primary key with a sequence as default value +- Have a ``cartodb_id`` column with integer, unique, non-zero and non-null values as primary key with a sequence as default value - Have a ``the_geom`` column of type ``Geometry`` with SRID 4326 - Have a ``the_geom_webmercator`` column of type ``Geometry`` with SRID 3857 -- The columns ``the_geom`` and ``the_geom_webmercator`` shall be in sync +- The columns ``the_geom`` and ``the_geom_webmercator`` shall be in sync (task of the ``update_the_geom_webmercator`` trigger) Additionally, a CartoDB table can contain other columns. +See the `CartoDB User Table documentation`_ +.. _CartoDB User Table documentation: https://github.com/CartoDB/cartodb-postgresql/blob/master/doc/CartoDB-user-table.md +for further information. High level requirements -======================= +----------------------- Here is a list of high level requirments for the public function ``CDB_CartodbfyTable()``: -- A call to ``CDB_CartodbfyTable()`` shall modify/rewrite the table and produce a valid CartoDB table with the same name. -- A call to ``CDB_CartodbfyTable()`` shall cause the registration of the table into the platform -- It shall be idempotent, meaning that successive calls to ``CDB_CartodbfyTable()`` shall not produce any visible effect in the system. +- A call to the function shall modify/rewrite the table and produce a valid CartoDB table with the same name. +- A call to the function shall cause the registration of the table into the platform. +- It shall be idempotent, meaning that successive calls to the function shall not produce any visible effect in the system. - If there's a column containing a geometry, it shall be used to generate ``the_geom`` and the ``the_geom_webmercator`` columns. - Exporting and re-importing the same table in CartoDB shall produce equivalent tables, with the same features associated to the same ``cartodb_id``'s. -Note that there should be only one feature per row in the source table. If there's more than one, then which one is used for ``the_geom`` and ``the_geom_webmercator`` fields is not determined. +Note that there should be only one geometry per row in the source table. If there's more than one, then which one is used for ``the_geom`` and ``the_geom_webmercator`` fields is not determined. Low-level requirements -====================== +---------------------- -- If the original table contains a valid (unique and not null) ``cartodb_id`` column, it shall be used -- If the original table contains a ``the_geom`` column or a ``the_geom_webmercator`` column in the expected projection (EPSG 4326 and EPSG 3857, respectively) they shall be used. +- If the original table contains a valid (integer, unique, non-zero and not null) ``cartodb_id`` column, it shall be used +- If the original table contains a ``the_geom`` column or a ``the_geom_webmercator`` geometric column in the expected projection (EPSG 4326 and EPSG 3857, respectively) they shall be used. - A modification of a cartodbfy'ed table shall insert or update a row in ``CDB_TableMetadata`` - A cartodbfy'ed table shall have a ``btree`` index on ``cartodb_id`` - A cartodbfy'ed table shall have ``gist`` indices on ``the_geom`` and ``the_geom_webmercator`` -- Cartodbfy shall deal with text columns for imports, regarding CartoDB columns - +- Cartodbfy shall deal with text columns for imports, regarding CartoDB columns (``cartodb_id``, ``the_geom``, ``the_geom_webmercator``)