From 9f414938ba7ee7daa5469ff4cf08057416e804e6 Mon Sep 17 00:00:00 2001 From: Javier Goizueta Date: Mon, 1 Jul 2019 16:41:40 +0200 Subject: [PATCH 1/4] Document CDB_SyncTable --- doc/CDB_SyncTable.md | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 doc/CDB_SyncTable.md diff --git a/doc/CDB_SyncTable.md b/doc/CDB_SyncTable.md new file mode 100644 index 0000000..7069747 --- /dev/null +++ b/doc/CDB_SyncTable.md @@ -0,0 +1,42 @@ +Synchronize two tables. This function will synchronize a *destination* table with a *source* table. +The idea is that the *destination* is a replica of *source* and *source* has been subject to +modifications that are to be applied to *destination*. + +This will be achieved by deleting the rows in the destination not present +in the source, inserting rows of the source not in the destination and updating modified rows. +If the destination table does not exist it will be created and all the rows of the source inserted into it. + +Both tables must have a consistent `cartodb_id` primary key column which will be used to match +the source and destination rows. + +Note that both tables do not necessaryly become identical after the synchronization, since additional columns +may have been added to the destination; those columns will not be altered by the synchronization. + +In addition some source columns may be skipped by listing them in the optional last argument; such columns +will not be updated in the destination, so if they are present in it their values won't be altered. + + +#### Using the function + +Import some data using COPY FROM into a temporary table, then synchronize a table with the data and +finally delete the temporary table. This could be used import and update some data periodically while +allowing to add columns to the data that will be preserved across updates. + + +```sql +CREATE tmp_pois(cartodb_id int, name text, type text, longitude double precision, latitude double precision); +COPY tmp_pois FROM '/tmp/pois.csv'; +SELECT CDB_SyncTable('tmp_pois', 'public', 'pois'); +DROP TABLE tmp_pois; +``` + +#### Arguments + +``` +CDB_SyncTable(src_table, dst_schema, dst_table, skip_cols) +``` + +* **src_table** REGCLASS the source data for the synchronization +* **dst_scgena** REGNAMESPACE the destination schema +* **dst_table** NAME the destination table to be updated +* **skip_cols** NAME[] an array of column names, empty by default, which will be skipped From f2dae651b3733d21d7a720b430f4f449fa92059c Mon Sep 17 00:00:00 2001 From: Javier Goizueta Date: Mon, 1 Jul 2019 16:58:24 +0200 Subject: [PATCH 2/4] More complete sync table example --- doc/CDB_SyncTable.md | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/doc/CDB_SyncTable.md b/doc/CDB_SyncTable.md index 7069747..08d7bbe 100644 --- a/doc/CDB_SyncTable.md +++ b/doc/CDB_SyncTable.md @@ -22,14 +22,28 @@ Import some data using COPY FROM into a temporary table, then synchronize a tabl finally delete the temporary table. This could be used import and update some data periodically while allowing to add columns to the data that will be preserved across updates. - ```sql -CREATE tmp_pois(cartodb_id int, name text, type text, longitude double precision, latitude double precision); +CREATE tmp_pois(cartodb_id int, name text, type text, longitude double precision, latitude double precision, rank int); COPY tmp_pois FROM '/tmp/pois.csv'; SELECT CDB_SyncTable('tmp_pois', 'public', 'pois'); DROP TABLE tmp_pois; ``` +Now we could perform some changes to the `pois` to maintain our own ranking: + +```sql +UPDATE pois SET rank = random()*4 + 1; +``` + +Then, if the source were updated at `/tmp/pois.csv` we could synchronize with it while maintaining our `rank` values with: + +```sql +CREATE tmp_pois(cartodb_id int, name text, type text, longitude double precision, latitude double precision, rank int); +COPY tmp_pois FROM '/tmp/pois.csv'; +SELECT CDB_SyncTable('tmp_pois', 'public', 'pois', '{rank}'); +DROP TABLE tmp_pois; +``` + #### Arguments ``` From 0e1aeb0a76eca616b49428b33b390063eaaa3802 Mon Sep 17 00:00:00 2001 From: Javier Goizueta Date: Mon, 1 Jul 2019 17:00:22 +0200 Subject: [PATCH 3/4] Minor copy edit --- doc/CDB_SyncTable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/CDB_SyncTable.md b/doc/CDB_SyncTable.md index 08d7bbe..f1b5696 100644 --- a/doc/CDB_SyncTable.md +++ b/doc/CDB_SyncTable.md @@ -35,7 +35,7 @@ Now we could perform some changes to the `pois` to maintain our own ranking: UPDATE pois SET rank = random()*4 + 1; ``` -Then, if the source were updated at `/tmp/pois.csv` we could synchronize with it while maintaining our `rank` values with: +Then, if the source were updated at `/tmp/pois.csv` we could synchronize with it while preserving our `rank` values with: ```sql CREATE tmp_pois(cartodb_id int, name text, type text, longitude double precision, latitude double precision, rank int); From 057695361d64ed61e7d140063becdc286a85eaa3 Mon Sep 17 00:00:00 2001 From: Javier Goizueta Date: Mon, 1 Jul 2019 17:49:42 +0200 Subject: [PATCH 4/4] Fix typo --- doc/CDB_SyncTable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/CDB_SyncTable.md b/doc/CDB_SyncTable.md index f1b5696..9447dd0 100644 --- a/doc/CDB_SyncTable.md +++ b/doc/CDB_SyncTable.md @@ -9,7 +9,7 @@ If the destination table does not exist it will be created and all the rows of t Both tables must have a consistent `cartodb_id` primary key column which will be used to match the source and destination rows. -Note that both tables do not necessaryly become identical after the synchronization, since additional columns +Note that both tables do not necessarily become identical after the synchronization, since additional columns may have been added to the destination; those columns will not be altered by the synchronization. In addition some source columns may be skipped by listing them in the optional last argument; such columns