cartodb4/observatory-extension
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'release_v1_api_functions' into release_v1_api_functions_aug_use

Browse Source
This commit is contained in:
John Krauss
2016-04-25 16:25:11 -04:00
parent a8829e76da 0274337ded
commit 108fa9cc56
16 changed files with 1305 additions and 543 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
doc/methods.md
View File

@@ -18,27 +18,24 @@ normalize | for measures that are are **sums** (e.g. population) the default nor
#### Returns
A JSON object containing the following properties
A NUMERIC value containing the following properties
Key | Description
--- | ---
value | the raw or normalized measure
name | the human readable name of the measure
description | a brief description of the measure
type | the data type (text, number, boolean)
#### Example
Add a Measure to an empty column based on point locations in your table
```SQL
UPDATE tablename SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population') -> 'value'
UPDATE tablename SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population')
```
Get a measure at a single point location
```SQL
SELECT * FROM json_each(OBS_GetUSCensusMeasure(CDB_LatLng(40.7, -73.9), 'Male Population'))
SELECT OBS_GetUSCensusMeasure(CDB_LatLng(40.7, -73.9), 'Male Population')
```
<!--
@@ -48,7 +45,7 @@ Should add the SQL API call here too
## OBS_GetUSCensusMeasure(polygon_geometry, measure_name);
The ```OBS_GetUSCensusMeasure(point_geometry, measure_name)``` method returns a measure based on a subset of the US Census variables within a given polygon. The ```OBS_GetUSCensusMeasure``` method is limited to only a subset of all measures that are available in the Data Observatory, to access the full list, use the ```OBS_GetMeasure``` method below.
The ```OBS_GetUSCensusMeasure(point_geometry, measure_name)``` method returns a measure based on a subset of the US Census variables within a given polygon. The ```OBS_GetUSCensusMeasure``` method is limited to only a subset of all measures that are available in the Data Observatory, to access the full list, use the ```OBS_GetUSCensusMeasure``` method below.
#### Arguments
@@ -60,27 +57,61 @@ normalize | for measures that are are **sums** (e.g. population) the default nor
#### Returns
A JSON object containing the following properties
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
name | the human readable name of the measure
description | a brief description of the measure
type | the data type (text, number, boolean)
#### Example
Add a Measure to an empty column based on polygons in your table
```SQL
UPDATE tablename SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population') -> 'value'
UPDATE tablename SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population')
```
Get a measure at a single polygon
```SQL
SELECT * FROM json_each(OBS_GetMeasure(ST_Buffer(CDB_LatLng(40.7, -73.9),0.001), 'Male Population'))
SELECT OBS_GetMeasure(ST_Buffer(CDB_LatLng(40.7, -73.9),0.001), 'Male Population')
```
<!--
Should add the SQL API call here too
-->
## OBS_GetUSCensusCategory(point_geometry, measure_name);
The ```OBS_GetUSCensusCategory(point_geometry, category_name)``` method returns a categorical measure based on a subset of the US Census variables at a point location. It requires a different function from ```OBS_GetUSCensusMeasure``` because this function will always return TEXT, whereas ```OBS_GetUSCensusMeasure``` will always returna NUMERIC value.
#### Arguments
Name |Description
--- | ---
point_geometry | a WGS84 point geometry (the_geom)
measure_name | a human readable string name of a US Census variable. The glossary of measure_names is [available below]('measure_name table').
#### Returns
A NUMERIC value containing the following properties
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a Measure to an empty column based on point locations in your table
```SQL
UPDATE tablename SET local_male_population = OBS_GetUSCensusCategory(the_geom, 'Spielman Singleton Category 10')
```
Get a measure at a single point location
```SQL
SELECT OBS_GetUSCensusCategory(CDB_LatLng(40.7, -73.9), 'Spielman Singleton Category 10')
```
<!--
@@ -101,27 +132,24 @@ normalize | for measures that are are **sums** (e.g. population) the default nor
#### Returns
A JSON object containing the following properties
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
name | the human readable name of the measure
description | a brief description of the measure
type | the data type (text, number, boolean)
#### Example
Add a Measure to an empty column based on point locations in your table
```SQL
UPDATE tablename SET local_male_population = OBS_GetMeasure(the_geom, '"us.census.acs".B08134006') -> 'value'
UPDATE tablename SET local_male_population = OBS_GetMeasure(the_geom, '"us.census.acs".B08134006')
```
Get a measure at a single point location
```SQL
SELECT * FROM json_each(OBS_GetMeasure(CDB_LatLng(40.7, -73.9), '"us.census.acs".B08134006'))
SELECT OBS_GetMeasure(CDB_LatLng(40.7, -73.9), '"us.census.acs".B08134006')
```
<!--
@@ -143,27 +171,26 @@ normalize | for measures that are are **sums** (e.g. population) the default nor
#### Returns
A JSON object containing the following properties
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
name | the human readable name of the measure
description | a brief description of the measure
type | the data type (text, number, boolean)
#### Example
Add a Measure to an empty column based on polygons in your table
```SQL
UPDATE tablename SET local_male_population = OBS_GetMeasure(the_geom, '"us.census.acs".B08134006') -> 'value'
UPDATE tablename SET local_male_population = OBS_GetMeasure(the_geom, '"us.census.acs".B08134006')
```
Get a measure within a polygon
```SQL
SELECT * FROM json_each(OBS_GetMeasure(ST_Buffer(CDB_LatLng(40.7, -73.9),0.001), '"us.census.acs".B08134006'))
SELECT OBS_GetMeasure(ST_Buffer(CDB_LatLng(40.7, -73.9),0.001), '"us.census.acs".B08134006')
```
<!--

32
src/pg/sql/40_observatory_utility.sql
View File

@@ -33,8 +33,6 @@ BEGIN
END;
$$ LANGUAGE plpgsql;
-- A type for use with the OBS_GetColumnData function
CREATE TYPE cdb_observatory.OBS_ColumnData AS (colname text, tablename text, aggregate text);
-- A function that gets the column data for multiple columns
@@ -44,11 +42,10 @@ CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetColumnData(
column_ids text[],
timespan text
)
RETURNS cdb_observatory.OBS_ColumnData[]
RETURNS SETOF JSON
AS $$
DECLARE
result cdb_observatory.OBS_ColumnData[];
BEGIN
RETURN QUERY
EXECUTE '
WITH geomref AS (
SELECT t.table_id id
@@ -60,17 +57,24 @@ BEGIN
column_ids as (
select row_number() over () as no, a.column_id as column_id from (select unnest($2) as column_id) a
)
SELECT array_agg(ROW(colname, tablename, aggregate)::cdb_observatory.OBS_ColumnData order by column_ids.no)
FROM column_ids, observatory.OBS_column c, observatory.OBS_column_table ct, observatory.OBS_table t
WHERE column_ids.column_id = c.id
AND c.id = ct.column_id
AND t.id = ct.table_id
AND t.timespan = $3
AND t.id in (SELECT id FROM geomref)
SELECT row_to_json(a) from (
select colname,
tablename,
aggregate,
name,
type,
c.description
FROM column_ids, observatory.OBS_column c, observatory.OBS_column_table ct, observatory.OBS_table t
WHERE column_ids.column_id = c.id
AND c.id = ct.column_id
AND t.id = ct.table_id
AND t.timespan = $3
AND t.id in (SELECT id FROM geomref)
order by column_ids.no
) a
'
USING geometry_id, column_ids, timespan
INTO result;
RETURN result;
RETURN;
END;
$$ LANGUAGE plpgsql;

775
src/pg/sql/41_observatory_augmentation.sql
View File

@@ -23,216 +23,313 @@
-- Creates a table of demographic snapshot
CREATE OR REPLACE FUNCTION cdb_observatory.OBS_GetDemographicSnapshot(geom geometry, time_span text default '2009 - 2013', geometry_level text default '"us.census.tiger".block_group')
RETURNS json
RETURNS SETOF JSON
AS $$
DECLARE
target_cols text[];
BEGIN
RETURN row_to_json(cdb_observatory._OBS_GetDemographicSnapshot(geom, time_span, geometry_level));
target_cols := Array['total_pop',
'male_pop',
'female_pop',
'median_age',
'white_pop',
'black_pop',
'asian_pop',
'hispanic_pop',
'amerindian_pop',
'other_race_pop',
'two_or_more_races_pop',
'not_hispanic_pop',
--'not_us_citizen_pop',
--'workers_16_and_over',
--'commuters_by_car_truck_van',
--'commuters_drove_alone',
--'commuters_by_carpool',
--'commuters_by_public_transportation',
--'commuters_by_bus',
--'commuters_by_subway_or_elevated',
--'walked_to_work',
--'worked_at_home',
--'children',
'households',
--'population_3_years_over',
--'in_school',
--'in_grades_1_to_4',
--'in_grades_5_to_8',
--'in_grades_9_to_12',
--'in_undergrad_college',
'pop_25_years_over',
'high_school_diploma',
'less_one_year_college',
'one_year_more_college',
'associates_degree',
'bachelors_degree',
'masters_degree',
--'pop_5_years_over',
--'speak_only_english_at_home',
--'speak_spanish_at_home',
--'pop_determined_poverty_status',
--'poverty',
'median_income',
'gini_index',
'income_per_capita',
'housing_units',
'vacant_housing_units',
'vacant_housing_units_for_rent',
'vacant_housing_units_for_sale',
'median_rent',
'percent_income_spent_on_rent',
'owner_occupied_housing_units',
'million_dollar_housing_units',
'mortgaged_housing_units',
--'pop_15_and_over',
--'pop_never_married',
--'pop_now_married',
--'pop_separated',
--'pop_widowed',
--'pop_divorced',
'commuters_16_over',
'commute_less_10_mins',
'commute_10_14_mins',
'commute_15_19_mins',
'commute_20_24_mins',
'commute_25_29_mins',
'commute_30_34_mins',
'commute_35_44_mins',
'commute_45_59_mins',
'commute_60_more_mins',
'aggregate_travel_time_to_work',
'income_less_10000',
'income_10000_14999',
'income_15000_19999',
'income_20000_24999',
'income_25000_29999',
'income_30000_34999',
'income_35000_39999',
'income_40000_44999',
'income_45000_49999',
'income_50000_59999',
'income_60000_74999',
'income_75000_99999',
'income_100000_124999',
'income_125000_149999',
'income_150000_199999',
'income_200000_or_more',
'land_area'];
RETURN QUERY
EXECUTE
'select * from cdb_observatory._OBS_GetCensus($1, $2 )'
USING geom, target_cols
RETURN;
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetDemographicSnapshot(geom geometry, time_span text default '2009 - 2013', geometry_level text default '"us.census.tiger".block_group' )
RETURNS TABLE(
total_pop NUMERIC,
male_pop NUMERIC,
female_pop NUMERIC,
median_age NUMERIC,
white_pop NUMERIC,
black_pop NUMERIC,
asian_pop NUMERIC,
hispanic_pop NUMERIC,
amerindian_pop NUMERIC,
other_race_pop NUMERIC,
two_or_more_races_pop NUMERIC,
not_hispanic_pop NUMERIC,
--not_us_citizen_pop NUMERIC,
--workers_16_and_over NUMERIC,
--commuters_by_car_truck_van NUMERIC,
--commuters_drove_alone NUMERIC,
--commuters_by_carpool NUMERIC,
--commuters_by_public_transportation NUMERIC,
--commuters_by_bus NUMERIC,
--commuters_by_subway_or_elevated NUMERIC,
--walked_to_work NUMERIC,
--worked_at_home NUMERIC,
--children NUMERIC, -- TODO we should be able to get this at BG
households NUMERIC,
--population_3_years_over NUMERIC,
--in_school NUMERIC,
--in_grades_1_to_4 NUMERIC,
--in_grades_5_to_8 NUMERIC,
--in_grades_9_to_12 NUMERIC,
--in_undergrad_college NUMERIC,
pop_25_years_over NUMERIC,
high_school_diploma NUMERIC,
less_one_year_college NUMERIC,
one_year_more_college NUMERIC,
associates_degree NUMERIC,
bachelors_degree NUMERIC,
masters_degree NUMERIC,
--pop_5_years_over NUMERIC,
--speak_only_english_at_home NUMERIC,
--speak_spanish_at_home NUMERIC,
--pop_determined_poverty_status NUMERIC,
--poverty NUMERIC,
median_income NUMERIC,
gini_index NUMERIC,
income_per_capita NUMERIC,
housing_units NUMERIC,
vacant_housing_units NUMERIC,
vacant_housing_units_for_rent NUMERIC,
vacant_housing_units_for_sale NUMERIC,
median_rent NUMERIC,
percent_income_spent_on_rent NUMERIC,
owner_occupied_housing_units NUMERIC,
million_dollar_housing_units NUMERIC,
mortgaged_housing_units NUMERIC,
--pop_15_and_over NUMERIC,
--pop_never_married NUMERIC,
--pop_now_married NUMERIC,
--pop_separated NUMERIC,
--pop_widowed NUMERIC,
--pop_divorced NUMERIC,
commuters_16_over NUMERIC,
commute_less_10_mins NUMERIC,
commute_10_14_mins NUMERIC,
commute_15_19_mins NUMERIC,
commute_20_24_mins NUMERIC,
commute_25_29_mins NUMERIC,
commute_30_34_mins NUMERIC,
commute_35_44_mins NUMERIC,
commute_45_59_mins NUMERIC,
commute_60_more_mins NUMERIC,
aggregate_travel_time_to_work NUMERIC,
income_less_10000 NUMERIC,
income_10000_14999 NUMERIC,
income_15000_19999 NUMERIC,
income_20000_24999 NUMERIC,
income_25000_29999 NUMERIC,
income_30000_34999 NUMERIC,
income_35000_39999 NUMERIC,
income_40000_44999 NUMERIC,
income_45000_49999 NUMERIC,
income_50000_59999 NUMERIC,
income_60000_74999 NUMERIC,
income_75000_99999 NUMERIC,
income_100000_124999 NUMERIC,
income_125000_149999 NUMERIC,
income_150000_199999 NUMERIC,
income_200000_or_more NUMERIC,
land_area NUMERIC)
AS $$
DECLARE
target_cols text[];
names text[];
vals NUMERIC[];
q text;
BEGIN
target_cols := Array['total_pop',
'male_pop',
'female_pop',
'median_age',
'white_pop',
'black_pop',
'asian_pop',
'hispanic_pop',
'amerindian_pop',
'other_race_pop',
'two_or_more_races_pop',
'not_hispanic_pop',
--'not_us_citizen_pop',
--'workers_16_and_over',
--'commuters_by_car_truck_van',
--'commuters_drove_alone',
--'commuters_by_carpool',
--'commuters_by_public_transportation',
--'commuters_by_bus',
--'commuters_by_subway_or_elevated',
--'walked_to_work',
--'worked_at_home',
--'children',
'households',
--'population_3_years_over',
--'in_school',
--'in_grades_1_to_4',
--'in_grades_5_to_8',
--'in_grades_9_to_12',
--'in_undergrad_college',
'pop_25_years_over',
'high_school_diploma',
'less_one_year_college',
'one_year_more_college',
'associates_degree',
'bachelors_degree',
'masters_degree',
--'pop_5_years_over',
--'speak_only_english_at_home',
--'speak_spanish_at_home',
--'pop_determined_poverty_status',
--'poverty',
'median_income',
'gini_index',
'income_per_capita',
'housing_units',
'vacant_housing_units',
'vacant_housing_units_for_rent',
'vacant_housing_units_for_sale',
'median_rent',
'percent_income_spent_on_rent',
'owner_occupied_housing_units',
'million_dollar_housing_units',
'mortgaged_housing_units',
--'pop_15_and_over',
--'pop_never_married',
--'pop_now_married',
--'pop_separated',
--'pop_widowed',
--'pop_divorced',
'commuters_16_over',
'commute_less_10_mins',
'commute_10_14_mins',
'commute_15_19_mins',
'commute_20_24_mins',
'commute_25_29_mins',
'commute_30_34_mins',
'commute_35_44_mins',
'commute_45_59_mins',
'commute_60_more_mins',
'aggregate_travel_time_to_work',
'income_less_10000',
'income_10000_14999',
'income_15000_19999',
'income_20000_24999',
'income_25000_29999',
'income_30000_34999',
'income_35000_39999',
'income_40000_44999',
'income_45000_49999',
'income_50000_59999',
'income_60000_74999',
'income_75000_99999',
'income_100000_124999',
'income_125000_149999',
'income_150000_199999',
'income_200000_or_more',
'land_area'];
q := 'WITH a As (
SELECT
dimension As names,
dimension_value As vals
FROM cdb_observatory._OBS_GetCensus($1,$2,$3,$4)
)' ||
cdb_observatory._OBS_BuildSnapshotQuery(target_cols) ||
' FROM a';
RETURN QUERY
EXECUTE
q
USING geom, target_cols, time_span, geometry_level;
RETURN;
END;
$$ LANGUAGE plpgsql;
-- CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetDemographicSnapshot(geom geometry, time_span text default '2009 - 2013', geometry_level text default '"us.census.tiger".block_group' )
-- RETURNS TABLE(
-- total_pop NUMERIC,
-- male_pop NUMERIC,
-- female_pop NUMERIC,
-- median_age NUMERIC,
-- white_pop NUMERIC,
-- black_pop NUMERIC,
-- asian_pop NUMERIC,
-- hispanic_pop NUMERIC,
-- amerindian_pop NUMERIC,
-- other_race_pop NUMERIC,
-- two_or_more_races_pop NUMERIC,
-- not_hispanic_pop NUMERIC,
-- --not_us_citizen_pop NUMERIC,
-- --workers_16_and_over NUMERIC,
-- --commuters_by_car_truck_van NUMERIC,
-- --commuters_drove_alone NUMERIC,
-- --commuters_by_carpool NUMERIC,
-- --commuters_by_public_transportation NUMERIC,
-- --commuters_by_bus NUMERIC,
-- --commuters_by_subway_or_elevated NUMERIC,
-- --walked_to_work NUMERIC,
-- --worked_at_home NUMERIC,
-- --children NUMERIC, -- TODO we should be able to get this at BG
-- households NUMERIC,
-- --population_3_years_over NUMERIC,
-- --in_school NUMERIC,
-- --in_grades_1_to_4 NUMERIC,
-- --in_grades_5_to_8 NUMERIC,
-- --in_grades_9_to_12 NUMERIC,
-- --in_undergrad_college NUMERIC,
-- pop_25_years_over NUMERIC,
-- high_school_diploma NUMERIC,
-- less_one_year_college NUMERIC,
-- one_year_more_college NUMERIC,
-- associates_degree NUMERIC,
-- bachelors_degree NUMERIC,
-- masters_degree NUMERIC,
-- --pop_5_years_over NUMERIC,
-- --speak_only_english_at_home NUMERIC,
-- --speak_spanish_at_home NUMERIC,
-- --pop_determined_poverty_status NUMERIC,
-- --poverty NUMERIC,
-- median_income NUMERIC,
-- gini_index NUMERIC,
-- income_per_capita NUMERIC,
-- housing_units NUMERIC,
-- vacant_housing_units NUMERIC,
-- vacant_housing_units_for_rent NUMERIC,
-- vacant_housing_units_for_sale NUMERIC,
-- median_rent NUMERIC,
-- percent_income_spent_on_rent NUMERIC,
-- owner_occupied_housing_units NUMERIC,
-- million_dollar_housing_units NUMERIC,
-- mortgaged_housing_units NUMERIC,
-- --pop_15_and_over NUMERIC,
-- --pop_never_married NUMERIC,
-- --pop_now_married NUMERIC,
-- --pop_separated NUMERIC,
-- --pop_widowed NUMERIC,
-- --pop_divorced NUMERIC,
-- commuters_16_over NUMERIC,
-- commute_less_10_mins NUMERIC,
-- commute_10_14_mins NUMERIC,
-- commute_15_19_mins NUMERIC,
-- commute_20_24_mins NUMERIC,
-- commute_25_29_mins NUMERIC,
-- commute_30_34_mins NUMERIC,
-- commute_35_44_mins NUMERIC,
-- commute_45_59_mins NUMERIC,
-- commute_60_more_mins NUMERIC,
-- aggregate_travel_time_to_work NUMERIC,
-- income_less_10000 NUMERIC,
-- income_10000_14999 NUMERIC,
-- income_15000_19999 NUMERIC,
-- income_20000_24999 NUMERIC,
-- income_25000_29999 NUMERIC,
-- income_30000_34999 NUMERIC,
-- income_35000_39999 NUMERIC,
-- income_40000_44999 NUMERIC,
-- income_45000_49999 NUMERIC,
-- income_50000_59999 NUMERIC,
-- income_60000_74999 NUMERIC,
-- income_75000_99999 NUMERIC,
-- income_100000_124999 NUMERIC,
-- income_125000_149999 NUMERIC,
-- income_150000_199999 NUMERIC,
-- income_200000_or_more NUMERIC,
-- land_area NUMERIC)
-- AS $$
-- DECLARE
-- target_cols text[];
-- names text[];
-- vals NUMERIC[];
-- q text;
-- BEGIN
-- target_cols := Array['total_pop',
-- 'male_pop',
-- 'female_pop',
-- 'median_age',
-- 'white_pop',
-- 'black_pop',
-- 'asian_pop',
-- 'hispanic_pop',
-- 'amerindian_pop',
-- 'other_race_pop',
-- 'two_or_more_races_pop',
-- 'not_hispanic_pop',
-- --'not_us_citizen_pop',
-- --'workers_16_and_over',
-- --'commuters_by_car_truck_van',
-- --'commuters_drove_alone',
-- --'commuters_by_carpool',
-- --'commuters_by_public_transportation',
-- --'commuters_by_bus',
-- --'commuters_by_subway_or_elevated',
-- --'walked_to_work',
-- --'worked_at_home',
-- --'children',
-- 'households',
-- --'population_3_years_over',
-- --'in_school',
-- --'in_grades_1_to_4',
-- --'in_grades_5_to_8',
-- --'in_grades_9_to_12',
-- --'in_undergrad_college',
-- 'pop_25_years_over',
-- 'high_school_diploma',
-- 'less_one_year_college',
-- 'one_year_more_college',
-- 'associates_degree',
-- 'bachelors_degree',
-- 'masters_degree',
-- --'pop_5_years_over',
-- --'speak_only_english_at_home',
-- --'speak_spanish_at_home',
-- --'pop_determined_poverty_status',
-- --'poverty',
-- 'median_income',
-- 'gini_index',
-- 'income_per_capita',
-- 'housing_units',
-- 'vacant_housing_units',
-- 'vacant_housing_units_for_rent',
-- 'vacant_housing_units_for_sale',
-- 'median_rent',
-- 'percent_income_spent_on_rent',
-- 'owner_occupied_housing_units',
-- 'million_dollar_housing_units',
-- 'mortgaged_housing_units',
-- --'pop_15_and_over',
-- --'pop_never_married',
-- --'pop_now_married',
-- --'pop_separated',
-- --'pop_widowed',
-- --'pop_divorced',
-- 'commuters_16_over',
-- 'commute_less_10_mins',
-- 'commute_10_14_mins',
-- 'commute_15_19_mins',
-- 'commute_20_24_mins',
-- 'commute_25_29_mins',
-- 'commute_30_34_mins',
-- 'commute_35_44_mins',
-- 'commute_45_59_mins',
-- 'commute_60_more_mins',
-- 'aggregate_travel_time_to_work',
-- 'income_less_10000',
-- 'income_10000_14999',
-- 'income_15000_19999',
-- 'income_20000_24999',
-- 'income_25000_29999',
-- 'income_30000_34999',
-- 'income_35000_39999',
-- 'income_40000_44999',
-- 'income_45000_49999',
-- 'income_50000_59999',
-- 'income_60000_74999',
-- 'income_75000_99999',
-- 'income_100000_124999',
-- 'income_125000_149999',
-- 'income_150000_199999',
-- 'income_200000_or_more',
-- 'land_area'];
--
-- q :=
-- $query$
-- WITH a As (
-- SELECT
-- array_agg(_OBS_GetCensusJ->>'name') As names,
-- array_agg(_OBS_GetCensusJ->>'value') As vals
-- FROM cdb_observatory._OBS_GetCensusJ($1,$2,$3,$4)
-- )$query$ ||
-- cdb_observatory._OBS_BuildSnapshotQuery(target_cols) ||
-- ' FROM a'
-- ;
--
-- RETURN QUERY
-- EXECUTE
-- q
-- USING geom, target_cols, time_span, geometry_level;
--
-- RETURN;
-- END;
-- $$ LANGUAGE plpgsql;
--Base functions for performing augmentation
@@ -247,7 +344,7 @@ CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetCensus(
time_span text DEFAULT '2009 - 2013',
geometry_level text DEFAULT '"us.census.tiger".block_group'
)
RETURNS TABLE(dimension text[], dimension_value NUMERIC[])
RETURNS SETOF JSON
AS $$
DECLARE
ids text[];
@@ -256,10 +353,35 @@ BEGIN
ids := cdb_observatory._OBS_LookupCensusHuman(dimension_names);
RETURN QUERY
SELECT names, vals FROM cdb_observatory._OBS_Get(geom, ids, time_span, geometry_level);
SELECT * FROM cdb_observatory._OBS_Get(geom, ids, time_span, geometry_level);
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetCensus(
geom geometry,
dimension_name text,
time_span text DEFAULT '2009 - 2013',
geometry_level text DEFAULT '"us.census.tiger".block_group'
)
RETURNS NUMERIC
AS $$
DECLARE
ids Text[];
result_json json;
result Numeric;
BEGIN
ids := cdb_observatory._OBS_LookupCensusHuman(Array[dimension_name]);
result_json := (SELECT a FROM cdb_observatory._OBS_Get(geom, ids, time_span, geometry_level) as a limit 1);
EXECUTE
format('select $1::numeric as "%s"', result_json->>'name')
INTO result
USING
result_json->>'value';
return result;
END;
$$ LANGUAGE plpgsql;
-- Base augmentation fucntion.
@@ -269,14 +391,14 @@ CREATE OR REPLACE FUNCTION cdb_observatory._OBS_Get(
time_span text,
geometry_level text
)
RETURNS TABLE(names text[], vals NUMERIC[])
RETURNS SETOF JSON
AS $$
DECLARE
results NUMERIC[];
results json[];
geom_table_name text;
names text[];
query text;
data_table_info cdb_observatory.OBS_ColumnData[];
data_table_info json[];
BEGIN
geom_table_name := cdb_observatory._OBS_GeomTable(geom, geometry_level);
@@ -287,13 +409,13 @@ BEGIN
RETURN QUERY SELECT '{}'::text[], '{}'::NUMERIC[];
END IF;
data_table_info := cdb_observatory._OBS_GetColumnData(geometry_level,
column_ids,
time_span);
names := (SELECT array_agg((d).colname)
FROM unnest(data_table_info) As d);
execute'
select array_agg( _obs_getcolumndata) from cdb_observatory._OBS_GetColumnData($1,
$2,
$3);'
INTO data_table_info
using geometry_level, column_ids, time_span;
IF ST_GeometryType(geom) = 'ST_Point'
THEN
results := cdb_observatory._OBS_GetPoints(geom,
@@ -302,17 +424,19 @@ BEGIN
ELSIF ST_GeometryType(geom) IN ('ST_Polygon', 'ST_MultiPolygon')
THEN
-- RAISE EXCEPTION 'polygons not supported for now';
results := cdb_observatory._OBS_GetPolygons(geom,
geom_table_name,
data_table_info);
END IF;
IF results IS NULL
THEN
results := Array[]::numeric[];
END IF;
RETURN QUERY SELECT names, results;
RETURN QUERY
EXECUTE
$query$
SELECT unnest($1)
$query$
USING results;
END;
$$ LANGUAGE plpgsql;
@@ -322,12 +446,13 @@ $$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetPoints(
geom geometry,
geom_table_name text,
data_table_info cdb_observatory.OBS_ColumnData[]
data_table_info json[]
)
RETURNS NUMERIC[]
RETURNS json[]
AS $$
DECLARE
result NUMERIC[];
json_result json[];
query text;
i int;
geoid text;
@@ -365,14 +490,14 @@ BEGIN
THEN
-- give back null values
query := query || format('NULL::numeric ');
ELSIF ((data_table_info)[i]).aggregate != 'sum'
ELSIF ((data_table_info)[i])->>'aggregate' != 'sum'
THEN
-- give back full variable
query := query || format('%I ', ((data_table_info)[i]).colname);
query := query || format('%I ', ((data_table_info)[i])->>'colname');
ELSE
-- give back variable normalized by area of geography
query := query || format('%I/%s ',
((data_table_info)[i]).colname,
((data_table_info)[i])->>'colname',
area);
END IF;
@@ -386,17 +511,32 @@ BEGIN
FROM observatory.%I
WHERE %I.geoid = %L
',
((data_table_info)[1]).tablename,
((data_table_info)[1]).tablename,
((data_table_info)[1])->>'tablename',
((data_table_info)[1])->>'tablename',
geoid
);
EXECUTE
query
INTO result
USING geom;
RETURN result;
EXECUTE
$query$
select array_agg(row_to_json(t)) from(
select values as value,
meta->>'name' as name,
meta->>'tablename' as tablename,
meta->>'aggregate' as aggregate,
meta->>'type' as type,
meta->>'description' as description
from (select unnest($1) as values, unnest($2) as meta) b
) t
$query$
INTO json_result
USING result, data_table_info;
RETURN json_result;
END;
$$ LANGUAGE plpgsql;
@@ -411,10 +551,7 @@ CREATE OR REPLACE FUNCTION cdb_observatory.OBS_GetMeasure(
RETURNS NUMERIC
AS $$
DECLARE
names TEXT[];
measure_ids TEXT[];
denominator_id TEXT;
vals NUMERIC[];
result NUMERIC;
BEGIN
IF boundary_id IS NULL THEN
@@ -587,12 +724,13 @@ $$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetPolygons(
geom geometry,
geom_table_name text,
data_table_info cdb_observatory.OBS_ColumnData[]
data_table_info json[]
)
RETURNS NUMERIC[]
RETURNS json[]
AS $$
DECLARE
result NUMERIC[];
result numeric[];
json_result json[];
q_select text;
q_sum text;
q text;
@@ -604,11 +742,11 @@ BEGIN
FOR i IN 1..array_upper(data_table_info, 1)
LOOP
q_select := q_select || format( '%I ', ((data_table_info)[i]).colname);
q_select := q_select || format( '%I ', ((data_table_info)[i])->>'colname');
IF ((data_table_info)[i]).aggregate ='sum'
IF ((data_table_info)[i])->>'aggregate' ='sum'
THEN
q_sum := q_sum || format('sum(overlap_fraction * COALESCE(%I, 0)) ',((data_table_info)[i]).colname,((data_table_info)[i]).colname);
q_sum := q_sum || format('sum(overlap_fraction * COALESCE(%I, 0)) ',((data_table_info)[i])->>'colname',((data_table_info)[i])->>'colname');
ELSE
q_sum := q_sum || ' NULL::numeric ';
END IF;
@@ -632,85 +770,50 @@ BEGIN
values As (
', geom_table_name);
q := q || q_select || format('FROM observatory.%I ', ((data_table_info)[1].tablename));
q := q || q_select || format('FROM observatory.%I ', ((data_table_info)[1]->>'tablename'));
q := q || ' ) ' || q_sum || ' ]::numeric[] FROM _overlaps, values
WHERE values.geoid = _overlaps.geoid';
EXECUTE
q
INTO result
USING geom;
RETURN result;
EXECUTE
$query$
select array_agg(row_to_json(t)) from(
select values as value,
meta->>'name' as name,
meta->>'tablename' as tablename,
meta->>'aggregate' as aggregate,
meta->>'type' as type,
meta->>'description' as description
from (select unnest($1) as values, unnest($2) as meta) b
) t
$query$
INTO json_result
USING result, data_table_info;
RETURN json_result;
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION OBS_GetSegmentSnapshot(geom geometry, geometry_level text default '"us.census.tiger".census_tract')
RETURNS json
AS $$
BEGIN
RETURN row_to_json(cdb_observatory._OBS_GetSegmentSnapshot(geom, geometry_level));
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION _OBS_GetSegmentSnapshot(
CREATE OR REPLACE FUNCTION cdb_observatory.OBS_GetSegmentSnapshot(
geom geometry,
geometry_level text DEFAULT '"us.census.tiger".census_tract'
)
RETURNS TABLE(
segment_name TEXT,
total_pop_quantile NUMERIC,
male_pop_quantile NUMERIC,
female_pop_quantile NUMERIC,
median_age_quantile NUMERIC,
white_pop_quantile NUMERIC,
black_pop_quantile NUMERIC,
asian_pop_quantile NUMERIC,
hispanic_pop_quantile NUMERIC,
not_us_citizen_pop_quantile NUMERIC,
workers_16_and_over_quantile NUMERIC,
commuters_by_car_truck_van_quantile NUMERIC,
commuters_by_public_transportation_quantile NUMERIC,
commuters_by_bus_quantile NUMERIC,
commuters_by_subway_or_elevated_quantile NUMERIC,
walked_to_work_quantile NUMERIC,
worked_at_home_quantile NUMERIC,
children_quantile NUMERIC,
households_quantile NUMERIC,
population_3_years_over_quantile NUMERIC,
in_school_quantile NUMERIC,
in_grades_1_to_4_quantile NUMERIC,
in_grades_5_to_8_quantile NUMERIC,
in_grades_9_to_12_quantile NUMERIC,
in_undergrad_college_quantile NUMERIC,
pop_25_years_over_quantile NUMERIC,
high_school_diploma_quantile NUMERIC,
bachelors_degree_quantile NUMERIC,
masters_degree_quantile NUMERIC,
pop_5_years_over_quantile NUMERIC,
speak_only_english_at_home_quantile NUMERIC,
speak_spanish_at_home_quantile NUMERIC,
pop_determined_poverty_status_quantile NUMERIC,
poverty_quantile NUMERIC,
median_income_quantile NUMERIC,
gini_index_quantile NUMERIC,
income_per_capita_quantile NUMERIC,
housing_units_quantile NUMERIC,
vacant_housing_units_quantile NUMERIC,
vacant_housing_units_for_rent_quantile NUMERIC,
vacant_housing_units_for_sale_quantile NUMERIC,
median_rent_quantile NUMERIC,
percent_income_spent_on_rent_quantile NUMERIC,
owner_occupied_housing_units_quantile NUMERIC,
million_dollar_housing_units_quantile NUMERIC
)
)
RETURNS JSON
AS $$
DECLARE
target_cols text[];
seg_name Text;
geom_id Text;
q Text;
result json;
seg_name Text;
geom_id Text;
q Text;
segment_name Text;
BEGIN
target_cols := Array[
'"us.census.acs".B01001001_quantile',
@@ -761,7 +864,7 @@ target_cols := Array[
EXECUTE
$query$
SELECT (categories)[1]
SELECT (_OBS_GetCategories)->>'name'
FROM cdb_observatory._OBS_GetCategories(
$1,
Array['"us.census.spielman_singleton_segments".X10'],
@@ -775,8 +878,8 @@ target_cols := Array[
format($query$
WITH a As (
SELECT
names As names,
vals As vals
array_agg(_OBS_GET->>'name') As names,
array_agg(_OBS_GET->>'value') As vals
FROM cdb_observatory._OBS_Get($1,
$2,
'2009 - 2013',
@@ -785,14 +888,18 @@ target_cols := Array[
), percentiles As (
%s
FROM a)
SELECT $4, percentiles.*
FROM percentiles
$query$, cdb_observatory._OBS_BuildSnapshotQuery(target_cols));
SELECT row_to_json(r) FROM
( SELECT $4 as segment_name, percentiles.*
FROM percentiles) r
$query$, cdb_observatory._OBS_BuildSnapshotQuery(target_cols)) results;
RETURN QUERY
EXECUTE
q
into result
USING geom, target_cols, geometry_level, segment_name;
return result;
END;
$$ LANGUAGE plpgsql;
@@ -805,14 +912,14 @@ CREATE OR REPLACE FUNCTION cdb_observatory._OBS_GetCategories(
geometry_level text DEFAULT '"us.census.tiger".block_group',
time_span text DEFAULT '2009 - 2013'
)
RETURNS TABLE(names text[], categories text[]) as $$
RETURNS SETOF JSON as $$
DECLARE
geom_table_name text;
geoid text;
names text[];
results text[];
query text;
data_table_info cdb_observatory.OBS_ColumnData[];
data_table_info json[];
BEGIN
geom_table_name := cdb_observatory._OBS_GeomTable(geom, geometry_level);
@@ -823,13 +930,12 @@ BEGIN
RETURN QUERY SELECT '{}'::text[], '{}'::text[];
END IF;
data_table_info := cdb_observatory._OBS_GetColumnData(geometry_level,
dimension_names,
time_span);
names := (SELECT array_agg((d).colname)
FROM unnest(data_table_info) As d);
execute'
select array_agg( _obs_getcolumndata) from cdb_observatory._OBS_GetColumnData($1,
$2,
$3);'
INTO data_table_info
using geometry_level, dimension_names, time_span;
EXECUTE
@@ -843,7 +949,7 @@ BEGIN
query := 'SELECT ARRAY[';
FOR i IN 1..array_upper(data_table_info, 1)
LOOP
query = query || format('%I ', lower(((data_table_info)[i]).colname));
query = query || format('%I ', lower(((data_table_info)[i])->>'colname'));
IF i < array_upper(data_table_info, 1)
THEN
query := query || ',';
@@ -854,8 +960,8 @@ BEGIN
FROM observatory.%I
WHERE %I.geoid = %L
',
((data_table_info)[1]).tablename,
((data_table_info)[1]).tablename,
((data_table_info)[1])->>'tablename',
((data_table_info)[1])->>'tablename',
geoid
);
@@ -863,11 +969,22 @@ BEGIN
query
INTO results
USING geom;
RETURN QUERY
SELECT names,results
EXECUTE
$query$
select row_to_json(t) from(
select categories as category,
meta->>'name' as name,
meta->>'tablename' as tablename,
meta->>'aggregate' as aggregate,
meta->>'type' as type,
meta->>'description' as description
from (select unnest($1) as categories, unnest($2) as meta) b
) t
$query$
USING results, data_table_info;
RETURN;
END;
$$ LANGUAGE plpgsql;

44
src/pg/sql/42_observatory_exploration.sql
View File

@@ -1,3 +1,47 @@
-- return a table that contains a string match based on input
-- TODO: implement search for timespan
CREATE OR REPLACE FUNCTION cdb_observatory._OBS_SearchTables(
search_term text,
time_span text DEFAULT NULL
)
RETURNS table(tablename text, timespan text)
As $$
DECLARE
out_var text[];
BEGIN
IF time_span IS NULL
THEN
RETURN QUERY
EXECUTE
'SELECT tablename, timespan
FROM observatory.obs_table t
JOIN observatory.obs_column_table ct
ON ct.table_id = t.id
JOIN observatory.obs_column c
ON ct.column_id = c.id
WHERE c.type ILIKE ''geometry''
AND c.id = $1'
USING search_term;
ELSE
RETURN QUERY
EXECUTE
'SELECT tablename, timespan
FROM observatory.obs_table t
JOIN observatory.obs_column_table ct
ON ct.table_id = t.id
JOIN observatory.obs_column c
ON ct.column_id = c.id
WHERE c.type ILIKE ''geometry''
AND c.id = $1
AND t.timespan = $2'
USING search_term, time_span;
END IF;
END;
$$ LANGUAGE plpgsql IMMUTABLE;
-- Functions used to search the observatory for measures
--------------------------------------------------------------------------------
-- TODO allow the user to specify the boundary to search for measures

244
src/pg/sql/44_observatory_geometries.sql Normal file
View File

@@ -0,0 +1,244 @@
-- Data Observatory -- Welcome to the Future
-- These Data Observatory functions provide access to boundary polyons (and
-- their ids) such as those available through the US Census Tiger, Who's on
-- First, the Spanish Census, and so on
-- OBS_GetGeometry
--
-- Returns the boundary polygon(s) that overlap with the input point geometry.
-- From an input point geometry, find the boundary which intersects with the
-- centroid of the input geometry
-- Inputs:
-- geom geometry: input point geometry
-- boundary_id text: source id of boundaries
-- see function OBS_ListGeomColumns for all avaiable
-- boundary ids
-- time_span text: time span that the geometries were collected (optional)
--
-- Output:
-- boundary geometry: geometry boundary that intersects with geom, is at the
-- resolution requested with boundary_id, and time_span
--
CREATE OR REPLACE FUNCTION cdb_observatory.OBS_GetGeometry(
geom geometry(Geometry, 4326),
boundary_id text,
time_span text DEFAULT NULL)
RETURNS geometry(Geometry, 4326)
AS $$
DECLARE
boundary geometry(Geometry, 4326);
target_table text;
BEGIN
-- TODO: Check if SRID = 4326, if not transform?
-- if not a point, raise error
IF ST_GeometryType(geom) != 'ST_Point'
THEN
RAISE EXCEPTION 'Invalid geometry type (%), expecting ''ST_Point''', ST_GeometryType(geom);
END IF;
-- choose appropriate table based on time_span
IF time_span IS NULL
THEN
SELECT x.target_tables INTO target_table
FROM cdb_observatory._OBS_SearchTables(boundary_id,
time_span) As x(target_tables,
timespans)
ORDER BY x.timespans DESC
LIMIT 1;
ELSE
-- TODO: modify for only one table returned instead of arbitrarily choosing
-- one with LIMIT 1 (could be conflict between clipped vs non-clipped
-- boundaries in the metadata tables)
SELECT x.target_tables INTO target_table
FROM cdb_observatory._OBS_SearchTables(boundary_id,
time_span) As x(target_tables,
timespans)
WHERE x.timespans = time_span
LIMIT 1;
END IF;
-- if no tables are found, raise notice and return null
IF target_table IS NULL
THEN
RAISE NOTICE 'No boundaries found for ''%'' in ''%''', ST_AsText(geom), boundary_id;
RETURN NULL::geometry;
END IF;
RAISE NOTICE 'target_table: %', target_table;
-- return the first boundary in intersections
EXECUTE format(
'SELECT t.the_geom
FROM observatory.%s As t
WHERE ST_Intersects($1, t.the_geom)
LIMIT 1', target_table)
INTO boundary
USING geom;
RETURN boundary;
END;
$$ LANGUAGE plpgsql;
-- OBS_GetGeometryId
--
-- retrieves the boundary identifier (e.g., '36047' = Kings County/Brooklyn, NY)
-- corresponding to the location geom and boundary types (e.g.,
-- us.census.tiger.county)
-- Inputs:
-- geom geometry: location where the boundary is requested to overlap with
-- boundary_id text: source id of boundaries (e.g., us.census.tiger.county)
-- see function OBS_ListGeomColumns for all avaiable
-- boundary ids
-- time_span text: time span that the geometries were collected (optional)
--
-- Output:
-- geometry_id text: identifier of the geometry which overlaps with the input
-- point geom in the table corresponding to boundary_id and
-- time_span
--
CREATE OR REPLACE FUNCTION cdb_observatory.OBS_GetGeometryId(
geom geometry(Geometry, 4326),
boundary_id text,
time_span text DEFAULT NULL
)
RETURNS text
AS $$
DECLARE
output_id text;
target_table text;
BEGIN
-- If not point, raise error
IF ST_GeometryType(geom) != 'ST_Point'
THEN
RAISE EXCEPTION 'Error: Invalid geometry type (%), expecting ''ST_Point''', ST_GeometryType(geom);
END IF;
-- choose appropriate table based on time_span
IF time_span IS NULL
THEN
SELECT x.target_tables INTO target_table
FROM cdb_observatory._OBS_SearchTables(boundary_id,
time_span) As x(target_tables,
timespans)
ORDER BY x.timespans DESC
LIMIT 1;
ELSE
SELECT x.target_tables INTO target_table
FROM cdb_observatory._OBS_SearchTables(boundary_id,
time_span) As x(target_tables,
timespans)
WHERE x.timespans = time_span
LIMIT 1;
END IF;
-- if no tables are found, raise error
IF target_table IS NULL
THEN
RAISE NOTICE 'Error: No boundaries found for ''%''', boundary_id;
RETURN NULL::text;
END IF;
RAISE NOTICE 'target_table: %', target_table;
-- return name of geometry id column
EXECUTE format(
'SELECT t.geoid
FROM observatory.%s As t
WHERE ST_Intersects($1, t.the_geom)
LIMIT 1', target_table)
INTO output_id
USING geom;
RETURN output_id;
END;
$$ LANGUAGE plpgsql;
-- OBS_GetGeometryById
--
-- Given a geometry reference (e.g., geoid for US Census), and it's geometry
-- level (see OBS_ListGeomColumns() for all available boundary ids), give back
-- the boundary that corresponds to that geometry_id, boundary_id, and
-- time_span
-- Inputs:
-- geometry_id text: geometry id of the requested boundary
-- boundary_id text: source id of boundaries (e.g., us.census.tiger.county)
-- see function OBS_ListGeomColumns for all avaiable
-- boundary ids
-- time_span text: time span that the geometries were collected (optional)
--
-- Output:
-- boundary geometry: geometry boundary that matches geometry_id, is at the
-- resolution requested with boundary_id, and time_span
--
CREATE OR REPLACE FUNCTION cdb_observatory.OBS_GetGeometryById(
geometry_id text, -- ex: '36047'
boundary_id text, -- ex: '"us.census.tiger".county'
time_span text DEFAULT NULL --ex: '2009'
)
RETURNS geometry(geometry, 4326)
AS $$
DECLARE
boundary geometry(geometry, 4326);
target_table text;
geoid_colname text;
geom_colname text;
BEGIN
EXECUTE
format(
$string$
SELECT geoid_ct.colname As geoid_colname,
tablename,
geom_ct.colname As geom_colname
FROM observatory.obs_column_table As geoid_ct,
observatory.obs_table As geom_t,
observatory.obs_column_table As geom_ct,
observatory.obs_column As geom_c
WHERE geoid_ct.column_id
IN (
SELECT source_id
FROM observatory.obs_column_to_column
WHERE reltype = 'geom_ref'
AND target_id = '%s'
)
AND geoid_ct.table_id = geom_t.id and
geom_t.id = geom_ct.table_id and
geom_ct.column_id = geom_c.id and
geom_c.type ILIKE 'geometry'
$string$, boundary_id
) INTO geoid_colname, target_table, geom_colname;
RAISE NOTICE '%', target_table;
IF target_table IS NULL
THEN
RAISE NOTICE 'No geometries found';
RETURN NULL::geometry;
END IF;
-- retrieve boundary
EXECUTE
format(
'SELECT t.%s
FROM observatory.%I As t
WHERE t.%s = $1
LIMIT 1', geom_colname, target_table, geoid_colname)
INTO boundary
USING geometry_id;
RETURN boundary;
END;
$$ LANGUAGE plpgsql;

45
src/pg/test/expected/40_observatory_utility_test.out
View File

@@ -25,6 +25,8 @@ Loading obs_11ee8b82c877c073438bc935a91d3dfccef875d1.sql fixture file...
Done.
Loading obs_d34555209878e8c4b37cf0b2b3d072ff129ec470.sql fixture file...
Done.
Loading obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4.sql fixture file...
Done.
-- OBS_GeomTable
-- get table with known geometry_id
-- should give back a table like obs_{hex hash}
@@ -62,32 +64,35 @@ SELECT
-- -----------|-----------------|-----------
-- geoid | obs_{hex table} | null
-- total_pop | obs_{hex table} | sum
WITH result as (
SELECT
(unnest(cdb_observatory._OBS_GetColumnData(
array_agg(a) expected from cdb_observatory._OBS_GetColumnData(
'"us.census.tiger".census_tract',
Array['"us.census.tiger".census_tract_geoid', '"us.census.acs".B01001001'],
'2009 - 2013'
))).*
ORDER BY colname, tablename ASC;
colname | tablename | aggregate
-----------+----------------------------------------------+-----------
geoid | obs_11ee8b82c877c073438bc935a91d3dfccef875d1 |
geoid | obs_ab038198aaab3f3cb055758638ee4de28ad70146 |
geoid | obs_d34555209878e8c4b37cf0b2b3d072ff129ec470 |
total_pop | obs_ab038198aaab3f3cb055758638ee4de28ad70146 | sum
(4 rows)
'2009 - 2013') a
)
select (expected)[1]::text = '{"colname":"geoid","tablename":"obs_d34555209878e8c4b37cf0b2b3d072ff129ec470","aggregate":null,"name":"US Census Tract Geoids","type":"Text","description":""}' as test_get_obs_column_with_geoid_and_census_1,
(expected)[2]::text = '{"colname":"geoid","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":null,"name":"US Census Tract Geoids","type":"Text","description":""}' as test_get_obs_column_with_geoid_and_census_2
from result;
test_get_obs_column_with_geoid_and_census_1 | test_get_obs_column_with_geoid_and_census_2
---------------------------------------------+---------------------------------------------
t | t
(1 row)
-- should be null-valued
WITH result as (
SELECT
(unnest(cdb_observatory._OBS_GetColumnData(
array_agg(a) expected from cdb_observatory._OBS_GetColumnData(
'"us.census.tiger".census_tract',
Array['"us.census.tiger".baloney'], -- entry not in catalog
'2009 - 2013'
))).*
ORDER BY 1 ASC;
colname | tablename | aggregate
---------+-----------+-----------
(0 rows)
Array['"us.census.tiger".baloney'],
'2009 - 2013') a
)
select expected is null as OBS_GetColumnData_missing_measure
from result;
obs_getcolumndata_missing_measure
-----------------------------------
t
(1 row)
-- OBS_LookupCensusHuman
-- should give back: {"\"us.census.acs\".B19083001"}
@@ -178,3 +183,5 @@ Dropping obs_11ee8b82c877c073438bc935a91d3dfccef875d1 fixture table...
Done.
Dropping obs_d34555209878e8c4b37cf0b2b3d072ff129ec470 fixture table...
Done.
Dropping obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4 fixture table...
Done.

89
src/pg/test/expected/41_observatory_augmentation_test.out
View File

@@ -25,74 +25,76 @@ Loading obs_11ee8b82c877c073438bc935a91d3dfccef875d1.sql fixture file...
Done.
Loading obs_d34555209878e8c4b37cf0b2b3d072ff129ec470.sql fixture file...
Done.
total_pop | male_pop | female_pop | median_age | white_pop | black_pop | asian_pop | hispanic_pop | amerindian_pop | other_race_pop | two_or_more_races_pop | not_hispanic_pop | households | pop_25_years_over | high_school_diploma | less_one_year_college | one_year_more_college | associates_degree | bachelors_degree | masters_degree | median_income | gini_index | income_per_capita | housing_units | vacant_housing_units | vacant_housing_units_for_rent | vacant_housing_units_for_sale | median_rent | percent_income_spent_on_rent | owner_occupied_housing_units | million_dollar_housing_units | mortgaged_housing_units | commuters_16_over | commute_less_10_mins | commute_10_14_mins | commute_15_19_mins | commute_20_24_mins | commute_25_29_mins | commute_30_34_mins | commute_35_44_mins | commute_45_59_mins | commute_60_more_mins | aggregate_travel_time_to_work | income_less_10000 | income_10000_14999 | income_15000_19999 | income_20000_24999 | income_25000_29999 | income_30000_34999 | income_35000_39999 | income_40000_44999 | income_45000_49999 | income_50000_59999 | income_60000_74999 | income_75000_99999 | income_100000_124999 | income_125000_149999 | income_150000_199999 | income_200000_or_more | land_area
------------------+------------------+------------------+------------+------------------+------------------+------------------+------------------+----------------+----------------+-----------------------+------------------+------------------+-------------------+---------------------+-----------------------+-----------------------+-------------------+------------------+------------------+---------------+------------+-------------------+------------------+----------------------+-------------------------------+-------------------------------+-------------+------------------------------+------------------------------+------------------------------+-------------------------+-------------------+----------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+----------------------+-------------------------------+-------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+----------------------+----------------------+----------------------+-----------------------+-----------
9516.27915900609 | 6152.51885204623 | 3363.76030695986 | 28.8 | 5301.51624447348 | 149.500458087105 | 230.000704749392 | 3835.26175169611 | 0 | 0 | 0 | 5681.01740730998 | 3323.51018362871 | 7107.02177675621 | 1040.753188991 | 69.0002114248176 | 793.502431385402 | 327.751004267883 | 2742.7584041365 | 931.502854235037 | 66304 | 0.3494 | 28291 | 3662.76122313407 | 339.251039505353 | 120.750369993431 | 0 | 1764 | 35.3 | 339.251039505353 | 0 | 224.250687130657 | 6549.27006773893 | 327.751004267883 | 28.750088093674 | 201.250616655718 | 621.001902823358 | 373.751145217762 | 1851.5056732326 | 1414.50433420876 | 1115.50341803455 | 615.251885204623 | | 57.500176187348 | 0 | 212.750651893187 | 408.251250930171 | 0 | 155.25047570584 | 109.250334755961 | 92.0002818997568 | 63.2501938060828 | 184.000563799514 | 621.001902823358 | 552.001691398541 | 327.751004267883 | 333.501021886618 | 126.500387612166 | |
Loading obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4.sql fixture file...
Done.
obs_getdemographicsnapshot_test_no_returns
--------------------------------------------
t
(1 row)
dimension | dimension_value
----------------------+-------------------------------------
{total_pop,male_pop} | {9516.27915900609,6152.51885204623}
test_obsgetcensuswithtestpointand2variables
---------------------------------------------
t
(1 row)
dimension | dimension_value
-----------------------+-----------------
{female_pop,male_pop} | {NULL,NULL}
test_obsgetcensuswithnullislandarea
-------------------------------------
t
(1 row)
dimension | dimension_value
-----------------------+-----------------
{female_pop,male_pop} | {}
test_obsgetcensuswithnullisland
---------------------------------
t
(1 row)
names | vals
--------------+----------
{gini_index} | {0.3494}
obs_get_gini_index_at_test_point
----------------------------------
t
(1 row)
names | vals
--------------+------
{gini_index} | {}
obs_get_gini_index_at_null_island
-----------------------------------
t
(1 row)
_obs_getpoints
--------------------
{4809.33511352425}
obs_getpoints_for_test_point
------------------------------
t
(1 row)
_obs_getpoints
----------------
obs_getpoints_for_null_island
-------------------------------
t
(1 row)
_obs_getpolygons
--------------------
{1570.72353789469}
obs_getpolygons_for_test_point
--------------------------------
t
(1 row)
_obs_getpolygons
------------------
{NULL}
obs_getpolygons_for_null_island
---------------------------------
t
(1 row)
segment_name | total_pop_quantile | male_pop_quantile | female_pop_quantile | median_age_quantile | white_pop_quantile | black_pop_quantile | asian_pop_quantile | hispanic_pop_quantile | not_us_citizen_pop_quantile | workers_16_and_over_quantile | commuters_by_car_truck_van_quantile | commuters_by_public_transportation_quantile | commuters_by_bus_quantile | commuters_by_subway_or_elevated_quantile | walked_to_work_quantile | worked_at_home_quantile | children_quantile | households_quantile | population_3_years_over_quantile | in_school_quantile | in_grades_1_to_4_quantile | in_grades_5_to_8_quantile | in_grades_9_to_12_quantile | in_undergrad_college_quantile | pop_25_years_over_quantile | high_school_diploma_quantile | bachelors_degree_quantile | masters_degree_quantile | pop_5_years_over_quantile | speak_only_english_at_home_quantile | speak_spanish_at_home_quantile | pop_determined_poverty_status_quantile | poverty_quantile | median_income_quantile | gini_index_quantile | income_per_capita_quantile | housing_units_quantile | vacant_housing_units_quantile | vacant_housing_units_for_rent_quantile | vacant_housing_units_for_sale_quantile | median_rent_quantile | percent_income_spent_on_rent_quantile | owner_occupied_housing_units_quantile | million_dollar_housing_units_quantile
-----------------------------+--------------------+-------------------+---------------------+---------------------+--------------------+--------------------+--------------------+-----------------------+-----------------------------+------------------------------+-------------------------------------+---------------------------------------------+---------------------------+------------------------------------------+-------------------------+-------------------------+--------------------+---------------------+----------------------------------+--------------------+---------------------------+---------------------------+----------------------------+-------------------------------+----------------------------+------------------------------+---------------------------+-------------------------+---------------------------+-------------------------------------+--------------------------------+----------------------------------------+-------------------+------------------------+---------------------+----------------------------+------------------------+-------------------------------+----------------------------------------+----------------------------------------+----------------------+---------------------------------------+---------------------------------------+---------------------------------------
Wealthy, urban without Kids | 0.234783783783784 | 0.422405405405405 | 0.0987567567567568 | 0.0715 | 0.295310810810811 | 0.407189189189189 | 0.625608108108108 | 0.795202702702703 | 0.703797297297297 | 0.59227027027027 | 0.0180540540540541 | 0.993756756756757 | 0.728162162162162 | 0.995972972972973 | 0.929135135135135 | 0.625432432432432 | 0.0386081081081081 | 0.157121621621622 | 0.241878378378378 | 0.173783783783784 | 0.0380675675675676 | 0.0308108108108108 | 0.0486216216216216 | 0.479743243243243 | 0.297675675675676 | 0.190351351351351 | 0.802513513513514 | 0.757148648648649 | 0.255405405405405 | 0.196094594594595 | 0.816851351351351 | 0.252513513513514 | 0.560054054054054 | 0.777472972972973 | 0.336932432432432 | 0.655378378378378 | 0.141810810810811 | 0.362824324324324 | 0.463837837837838 | 0 | 0.939040540540541 | 0.419445945945946 | 0.0387972972972973 | 0
test_point_segmentation
-------------------------
t
(1 row)
segment_name | total_pop_quantile | male_pop_quantile | female_pop_quantile | median_age_quantile | white_pop_quantile | black_pop_quantile | asian_pop_quantile | hispanic_pop_quantile | not_us_citizen_pop_quantile | workers_16_and_over_quantile | commuters_by_car_truck_van_quantile | commuters_by_public_transportation_quantile | commuters_by_bus_quantile | commuters_by_subway_or_elevated_quantile | walked_to_work_quantile | worked_at_home_quantile | children_quantile | households_quantile | population_3_years_over_quantile | in_school_quantile | in_grades_1_to_4_quantile | in_grades_5_to_8_quantile | in_grades_9_to_12_quantile | in_undergrad_college_quantile | pop_25_years_over_quantile | high_school_diploma_quantile | bachelors_degree_quantile | masters_degree_quantile | pop_5_years_over_quantile | speak_only_english_at_home_quantile | speak_spanish_at_home_quantile | pop_determined_poverty_status_quantile | poverty_quantile | median_income_quantile | gini_index_quantile | income_per_capita_quantile | housing_units_quantile | vacant_housing_units_quantile | vacant_housing_units_for_rent_quantile | vacant_housing_units_for_sale_quantile | median_rent_quantile | percent_income_spent_on_rent_quantile | owner_occupied_housing_units_quantile | million_dollar_housing_units_quantile
--------------+--------------------+-------------------+---------------------+---------------------+--------------------+--------------------+--------------------+-----------------------+-----------------------------+------------------------------+-------------------------------------+---------------------------------------------+---------------------------+------------------------------------------+-------------------------+-------------------------+-------------------+---------------------+----------------------------------+--------------------+---------------------------+---------------------------+----------------------------+-------------------------------+----------------------------+------------------------------+---------------------------+-------------------------+---------------------------+-------------------------------------+--------------------------------+----------------------------------------+------------------+------------------------+---------------------+----------------------------+------------------------+-------------------------------+----------------------------------------+----------------------------------------+----------------------+---------------------------------------+---------------------------------------+---------------------------------------
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
null_island_segmentation
--------------------------
t
(1 row)
names | categories
-------+---------------------------------
{X10} | {"Wealthy, urban without Kids"}
getcategories_at_test_point_1 | getcategories_at_test_point_2
-------------------------------+-------------------------------
t | t
(1 row)
names | categories
-------+------------
{X10} |
getcategories_at_null_island
------------------------------
t
(1 row)
obs_getmeasure
@@ -138,7 +140,6 @@ Done.
obs_getuscensusmeasure
------------------------
6152.51885204623
(1 row)
obs_getuscensusmeasure
------------------------
@@ -179,3 +180,5 @@ Dropping obs_11ee8b82c877c073438bc935a91d3dfccef875d1 fixture table...
Done.
Dropping obs_d34555209878e8c4b37cf0b2b3d072ff129ec470 fixture table...
Done.
Dropping obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4 fixture table...
Done.

109
src/pg/test/expected/42_observatory_exploration_test.out
View File

@@ -25,46 +25,73 @@ Loading obs_11ee8b82c877c073438bc935a91d3dfccef875d1.sql fixture file...
Done.
Loading obs_d34555209878e8c4b37cf0b2b3d072ff129ec470.sql fixture file...
Done.
obs_search
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
("""es.ine"".total_pop","The total number of all people living in a geographic area.","Total Population",sum,es.ine)
("""us.census.acs"".B01001001","The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.","Total Population",sum,us.census.acs)
("""us.census.acs"".B01001001_quantile","The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.","Quantile:Total Population",quantile,us.census.acs)
Loading obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4.sql fixture file...
Done.
_obs_searchtables_tables_match|_obs_searchtables_timespan_matches
t|t
t|t
(2 rows)
_obs_searchtables_timespan_does_not_match
t
(1 row)
obs_search
("""es.ine"".total_pop","The total number of all people living in a geographic area.","Total Population",sum,es.ine)
("""us.census.acs"".B01001001","The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.","Total Population",sum,us.census.acs)
("""us.census.acs"".B01001001_quantile","The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.","Quantile:Total Population",quantile,us.census.acs)
(3 rows)
boundary_id | description | time_span | tablename
--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+----------------------------------------------
"us.census.tiger".block_group | Block groups (BGs) are statistical divisions of census tracts, are generally defined to contain between 600 and 3,000 people, and are used to present data and control block numbering. A block group consists of clusters of blocks within the same census tract that have the same first digit of their four-digit census block number. For example, blocks 3001, 3002, 3003, ..., 3999 in census tract 1210.02 belong to BG 3 in that census tract. Most BGs were delineated by local participants in the Census Bureau\u2019s Participant Statistical Areas Program. The Census Bureau delineated BGs only where a local or tribal government declined to participate, and a regional organization or State Data Center was not available to participate.\r +| 2013 | obs_85328201013baa14e8e8a4a57a01e6f6fbc5f9b1
| \r +| |
| A BG usually covers a contiguous area. Each census tract contains at least one BG, and BGs are uniquely numbered within the census tract. Within the standard census geographic hierarchy, BGs never cross state, county, or census tract boundaries but may cross the boundaries of any other geographic entity. Tribal census tracts and tribal BGs are separate and unique geographic areas defined within federally recognized American Indian reservations and can cross state and county boundaries (see \u201cTribal Census Tract\u201d and \u201cTribal Block Group\u201d). The tribal census tracts and tribal block groups may be completely different from the census tracts and block groups defined by state and county. | |
"us.census.tiger".census_tract | Census tracts are identified by an up to four-digit integer number and may have an optional two-digit suffix; for example 1457.02 or 23. The census tract codes consist of six digits with an implied decimal between the fourth and fifth digit corresponding to the basic census tract number but with leading zeroes and trailing zeroes for census tracts without a suffix. The tract number examples above would have codes of 145702 and 002300, respectively.\r +| 2013 | obs_a92e1111ad3177676471d66bb8036e6d057f271b
| \r +| |
| Some ranges of census tract numbers in the 2010 Census are used to identify distinctive types of census tracts. The code range in the 9400s is used for those census tracts with a majority of population, housing, or land area associated with an American Indian area and matches the numbering used in Census 2000. The code range in the 9800s is new for 2010 and is used to specifically identify special land-use census tracts; that is, census tracts defined to encompass a large area with little or no residential population with special characteristics, such as large parks or employment areas. The range of census tracts in the 9900s represents census tracts delineated specifically to cover large bodies of water. This is different from Census 2000 when water-only census tracts were assigned codes of all zeroes (000000); 000000 is no longer used as a census tract code for the 2010 Census.\r +| |
| \r +| |
| The Census Bureau uses suffixes to help identify census tract changes for comparison purposes. Census tract suffixes may range from .01 to .98. As part of local review of existing census tracts before each census, some census tracts may have grown enough in population size to qualify as more than one census tract. When a census tract is split, the split parts usually retain the basic number but receive different suffixes. For example, if census tract 14 is split, the new tract numbers would be 14.01 and 14.02. In a few counties, local participants request major changes to, and renumbering of, the census tracts; however, this is generally discouraged. Changes to individual census tract boundaries usually do not result in census tract numbering changes.\r +| |
| \r +| |
| The Census Bureau introduced the concept of tribal census tracts for the first time for Census 2000. Tribal census tracts for that census consisted of the standard county-based census tracts tabulated within American Indian areas, thus allowing for the tracts to ignore state and county boundaries for tabulation. The Census Bureau assigned the 9400 range of numbers to identify specific tribal census tracts; however, not all tribal census tracts used this numbering scheme. For the 2010 Census, tribal census tracts no longer are tied to or numbered in the same way as the county-based census tracts (see \u201cTribal Census Tract\u201d). | |
"us.census.tiger".state | States and Equivalent Entities are the primary governmental divisions of the United States. In addition to the 50 states, the Census Bureau treats the District of Columbia, Puerto Rico, American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands as the statistical equivalents of states for the purpose of data presentation. | 2013 | obs_f3f0912fe24bc0c976e837b5a116d0c803cc01ce
"us.census.tiger".puma | PUMAs are geographic areas for which the Census Bureau provides selected extracts of raw data from a small sample of census records that are screened to protect confidentiality. These extracts are referred to as public use microdata sample (PUMS) files.\r +| 2013 | obs_0008b162b516c295d7204c9ba043ab5dbc67c59c
| \r +| |
| For the 2010 Census, each state, the District of Columbia, Puerto Rico, and some Island Area participants delineated PUMAs for use in presenting PUMS data based on a 5 percent sample of decennial census or American Community Survey data. These areas are required to contain at least 100,000 people. This is different from Census 2000 when two types of PUMAs were defined: a 5 percent PUMA as for 2010 and an additional super-PUMA designed to provide a 1 percent sample. The PUMAs are identified by a five-digit census code unique within state. | |
"us.census.tiger".zcta5 | ZCTAs are approximate area representations of U.S. Postal Service (USPS) five-digit ZIP Code service areas that the Census Bureau creates using whole blocks to present statistical data from censuses and surveys. The Census Bureau defines ZCTAs by allocating each block that contains addresses to a single ZCTA, usually to the ZCTA that reflects the most frequently occurring ZIP Code for the addresses within that tabulation block. Blocks that do not contain addresses but are completely surrounded by a single ZCTA (enclaves) are assigned to the surrounding ZCTA; those surrounded by multiple ZCTAs will be added to a single ZCTA based on limited buffering performed between multiple ZCTAs. The Census Bureau identifies five-digit ZCTAs using a five-character numeric code that represents the most frequently occurring USPS ZIP Code within that ZCTA, and this code may contain leading zeros.\r +| 2013 | obs_d483723c5cc76c107d9e0af279d1e7056df3c2be
| \r +| |
| There are significant changes to the 2010 ZCTA delineation from that used in 2000. Coverage was extended to include the Island Areas for 2010 so that the United States, Puerto Rico, and the Island Areas have ZCTAs. Unlike 2000, when areas that could not be assigned to a ZCTA were given a generic code ending in \u201cXX\u201d (land area) or \u201cHH\u201d (water area), for 2010 there is no universal coverage by ZCTAs, and only legitimate five-digit areas are defined. The 2010 ZCTAs will better represent the actual Zip Code service areas because the Census Bureau initiated a process before creation of 2010 blocks to add block boundaries that split polygons with large numbers of addresses using different Zip Codes.\r +| |
| \r +| |
| Data users should not use ZCTAs to identify the official USPS ZIP Code for mail delivery. The USPS makes periodic changes to ZIP Codes to support more efficient mail delivery. The ZCTAs process used primarily residential addresses and was biased towards Zip Codes used for city-style mail delivery, thus there may be Zip Codes that are primarily nonresidential or boxes only that may not have a corresponding ZCTA. | |
"us.census.tiger".county | The primary legal divisions of most states are termed counties. In Louisiana, these divisions are known as parishes. In Alaska, which has no counties, the equivalent entities are the organized boroughs, city and boroughs, municipalities, and census areas; the latter of which are delineated cooperatively for statistical purposes by the state of Alaska and the Census Bureau. In four states (Maryland, Missouri, Nevada, and Virginia), there are one or more incorporated places that are independent of any county organization and thus constitute primary divisions of their states. These incorporated places are known as independent cities and are treated as equivalent entities for purposes of data presentation. The District of Columbia and Guam have no primary divisions, and each area is considered an equivalent entity for purposes of data presentation. All of the counties in Connecticut and Rhode Island and nine counties in Massachusetts were dissolved as functioning governmental entities; however, the Census Bureau continues to present data for these historical entities in order to provide comparable geographic units at the county level of the geographic hierarchy for these states and represents them as nonfunctioning legal entities in data products. The Census Bureau treats the following entities as equivalents of counties for purposes of data presentation: municipios in Puerto Rico, districts and islands in American Samoa, municipalities in the Commonwealth of the Northern Mariana Islands, and islands in the U.S. Virgin Islands. Each county or statistically equivalent entity is assigned a three-character numeric Federal Information Processing Series (FIPS) code based on alphabetical sequence that is unique within state and an eight-digit National Standard feature identifier. | 2013 | obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4
"us.census.tiger".state | States and Equivalent Entities are the primary governmental divisions of the United States. In addition to the 50 states, the Census Bureau treats the District of Columbia, Puerto Rico, American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands as the statistical equivalents of states for the purpose of data presentation. | 2013 | obs_a20f5260b618a2fe2eb95fc1e23febe0db7db096
"us.census.tiger".county | The primary legal divisions of most states are termed counties. In Louisiana, these divisions are known as parishes. In Alaska, which has no counties, the equivalent entities are the organized boroughs, city and boroughs, municipalities, and census areas; the latter of which are delineated cooperatively for statistical purposes by the state of Alaska and the Census Bureau. In four states (Maryland, Missouri, Nevada, and Virginia), there are one or more incorporated places that are independent of any county organization and thus constitute primary divisions of their states. These incorporated places are known as independent cities and are treated as equivalent entities for purposes of data presentation. The District of Columbia and Guam have no primary divisions, and each area is considered an equivalent entity for purposes of data presentation. All of the counties in Connecticut and Rhode Island and nine counties in Massachusetts were dissolved as functioning governmental entities; however, the Census Bureau continues to present data for these historical entities in order to provide comparable geographic units at the county level of the geographic hierarchy for these states and represents them as nonfunctioning legal entities in data products. The Census Bureau treats the following entities as equivalents of counties for purposes of data presentation: municipios in Puerto Rico, districts and islands in American Samoa, municipalities in the Commonwealth of the Northern Mariana Islands, and islands in the U.S. Virgin Islands. Each county or statistically equivalent entity is assigned a three-character numeric Federal Information Processing Series (FIPS) code based on alphabetical sequence that is unique within state and an eight-digit National Standard feature identifier. | 2013 | obs_23da37d4e66e9de2f525572967f8618bde99a8c0
"us.census.tiger".census_tract | Census tracts are identified by an up to four-digit integer number and may have an optional two-digit suffix; for example 1457.02 or 23. The census tract codes consist of six digits with an implied decimal between the fourth and fifth digit corresponding to the basic census tract number but with leading zeroes and trailing zeroes for census tracts without a suffix. The tract number examples above would have codes of 145702 and 002300, respectively.\r +| 2013 | obs_d125aeef87aaa23287a40b454519ece22ee25acf
| \r +| |
| Some ranges of census tract numbers in the 2010 Census are used to identify distinctive types of census tracts. The code range in the 9400s is used for those census tracts with a majority of population, housing, or land area associated with an American Indian area and matches the numbering used in Census 2000. The code range in the 9800s is new for 2010 and is used to specifically identify special land-use census tracts; that is, census tracts defined to encompass a large area with little or no residential population with special characteristics, such as large parks or employment areas. The range of census tracts in the 9900s represents census tracts delineated specifically to cover large bodies of water. This is different from Census 2000 when water-only census tracts were assigned codes of all zeroes (000000); 000000 is no longer used as a census tract code for the 2010 Census.\r +| |
| \r +| |
| The Census Bureau uses suffixes to help identify census tract changes for comparison purposes. Census tract suffixes may range from .01 to .98. As part of local review of existing census tracts before each census, some census tracts may have grown enough in population size to qualify as more than one census tract. When a census tract is split, the split parts usually retain the basic number but receive different suffixes. For example, if census tract 14 is split, the new tract numbers would be 14.01 and 14.02. In a few counties, local participants request major changes to, and renumbering of, the census tracts; however, this is generally discouraged. Changes to individual census tract boundaries usually do not result in census tract numbering changes.\r +| |
| \r +| |
| The Census Bureau introduced the concept of tribal census tracts for the first time for Census 2000. Tribal census tracts for that census consisted of the standard county-based census tracts tabulated within American Indian areas, thus allowing for the tracts to ignore state and county boundaries for tabulation. The Census Bureau assigned the 9400 range of numbers to identify specific tribal census tracts; however, not all tribal census tracts used this numbering scheme. For the 2010 Census, tribal census tracts no longer are tied to or numbered in the same way as the county-based census tracts (see \u201cTribal Census Tract\u201d). | |
"us.census.tiger".block_group | Block groups (BGs) are statistical divisions of census tracts, are generally defined to contain between 600 and 3,000 people, and are used to present data and control block numbering. A block group consists of clusters of blocks within the same census tract that have the same first digit of their four-digit census block number. For example, blocks 3001, 3002, 3003, ..., 3999 in census tract 1210.02 belong to BG 3 in that census tract. Most BGs were delineated by local participants in the Census Bureau\u2019s Participant Statistical Areas Program. The Census Bureau delineated BGs only where a local or tribal government declined to participate, and a regional organization or State Data Center was not available to participate.\r +| 2013 | obs_d610cb3225f282693b8d4dcd98d2c2e2078354c6
| \r +| |
| A BG usually covers a contiguous area. Each census tract contains at least one BG, and BGs are uniquely numbered within the census tract. Within the standard census geographic hierarchy, BGs never cross state, county, or census tract boundaries but may cross the boundaries of any other geographic entity. Tribal census tracts and tribal BGs are separate and unique geographic areas defined within federally recognized American Indian reservations and can cross state and county boundaries (see \u201cTribal Census Tract\u201d and \u201cTribal Block Group\u201d). The tribal census tracts and tribal block groups may be completely different from the census tracts and block groups defined by state and county. | |
boundary_id|description|time_span|tablename
"us.census.tiger".block_group|Block groups (BGs) are statistical divisions of census tracts, are generally defined to contain between 600 and 3,000 people, and are used to present data and control block numbering. A block group consists of clusters of blocks within the same census tract that have the same first digit of their four-digit census block number. For example, blocks 3001, 3002, 3003, ..., 3999 in census tract 1210.02 belong to BG 3 in that census tract. Most BGs were delineated by local participants in the Census Bureau\u2019s Participant Statistical Areas Program. The Census Bureau delineated BGs only where a local or tribal government declined to participate, and a regional organization or State Data Center was not available to participate.
A BG usually covers a contiguous area. Each census tract contains at least one BG, and BGs are uniquely numbered within the census tract. Within the standard census geographic hierarchy, BGs never cross state, county, or census tract boundaries but may cross the boundaries of any other geographic entity. Tribal census tracts and tribal BGs are separate and unique geographic areas defined within federally recognized American Indian reservations and can cross state and county boundaries (see \u201cTribal Census Tract\u201d and \u201cTribal Block Group\u201d). The tribal census tracts and tribal block groups may be completely different from the census tracts and block groups defined by state and county.|2013|obs_85328201013baa14e8e8a4a57a01e6f6fbc5f9b1
"us.census.tiger".census_tract|Census tracts are identified by an up to four-digit integer number and may have an optional two-digit suffix; for example 1457.02 or 23. The census tract codes consist of six digits with an implied decimal between the fourth and fifth digit corresponding to the basic census tract number but with leading zeroes and trailing zeroes for census tracts without a suffix. The tract number examples above would have codes of 145702 and 002300, respectively.
Some ranges of census tract numbers in the 2010 Census are used to identify distinctive types of census tracts. The code range in the 9400s is used for those census tracts with a majority of population, housing, or land area associated with an American Indian area and matches the numbering used in Census 2000. The code range in the 9800s is new for 2010 and is used to specifically identify special land-use census tracts; that is, census tracts defined to encompass a large area with little or no residential population with special characteristics, such as large parks or employment areas. The range of census tracts in the 9900s represents census tracts delineated specifically to cover large bodies of water. This is different from Census 2000 when water-only census tracts were assigned codes of all zeroes (000000); 000000 is no longer used as a census tract code for the 2010 Census.
The Census Bureau uses suffixes to help identify census tract changes for comparison purposes. Census tract suffixes may range from .01 to .98. As part of local review of existing census tracts before each census, some census tracts may have grown enough in population size to qualify as more than one census tract. When a census tract is split, the split parts usually retain the basic number but receive different suffixes. For example, if census tract 14 is split, the new tract numbers would be 14.01 and 14.02. In a few counties, local participants request major changes to, and renumbering of, the census tracts; however, this is generally discouraged. Changes to individual census tract boundaries usually do not result in census tract numbering changes.
The Census Bureau introduced the concept of tribal census tracts for the first time for Census 2000. Tribal census tracts for that census consisted of the standard county-based census tracts tabulated within American Indian areas, thus allowing for the tracts to ignore state and county boundaries for tabulation. The Census Bureau assigned the 9400 range of numbers to identify specific tribal census tracts; however, not all tribal census tracts used this numbering scheme. For the 2010 Census, tribal census tracts no longer are tied to or numbered in the same way as the county-based census tracts (see \u201cTribal Census Tract\u201d).|2013|obs_a92e1111ad3177676471d66bb8036e6d057f271b
"us.census.tiger".state|States and Equivalent Entities are the primary governmental divisions of the United States. In addition to the 50 states, the Census Bureau treats the District of Columbia, Puerto Rico, American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands as the statistical equivalents of states for the purpose of data presentation.|2013|obs_f3f0912fe24bc0c976e837b5a116d0c803cc01ce
"us.census.tiger".puma|PUMAs are geographic areas for which the Census Bureau provides selected extracts of raw data from a small sample of census records that are screened to protect confidentiality. These extracts are referred to as public use microdata sample (PUMS) files.
For the 2010 Census, each state, the District of Columbia, Puerto Rico, and some Island Area participants delineated PUMAs for use in presenting PUMS data based on a 5 percent sample of decennial census or American Community Survey data. These areas are required to contain at least 100,000 people. This is different from Census 2000 when two types of PUMAs were defined: a 5 percent PUMA as for 2010 and an additional super-PUMA designed to provide a 1 percent sample. The PUMAs are identified by a five-digit census code unique within state.|2013|obs_0008b162b516c295d7204c9ba043ab5dbc67c59c
"us.census.tiger".zcta5|ZCTAs are approximate area representations of U.S. Postal Service (USPS) five-digit ZIP Code service areas that the Census Bureau creates using whole blocks to present statistical data from censuses and surveys. The Census Bureau defines ZCTAs by allocating each block that contains addresses to a single ZCTA, usually to the ZCTA that reflects the most frequently occurring ZIP Code for the addresses within that tabulation block. Blocks that do not contain addresses but are completely surrounded by a single ZCTA (enclaves) are assigned to the surrounding ZCTA; those surrounded by multiple ZCTAs will be added to a single ZCTA based on limited buffering performed between multiple ZCTAs. The Census Bureau identifies five-digit ZCTAs using a five-character numeric code that represents the most frequently occurring USPS ZIP Code within that ZCTA, and this code may contain leading zeros.
There are significant changes to the 2010 ZCTA delineation from that used in 2000. Coverage was extended to include the Island Areas for 2010 so that the United States, Puerto Rico, and the Island Areas have ZCTAs. Unlike 2000, when areas that could not be assigned to a ZCTA were given a generic code ending in \u201cXX\u201d (land area) or \u201cHH\u201d (water area), for 2010 there is no universal coverage by ZCTAs, and only legitimate five-digit areas are defined. The 2010 ZCTAs will better represent the actual Zip Code service areas because the Census Bureau initiated a process before creation of 2010 blocks to add block boundaries that split polygons with large numbers of addresses using different Zip Codes.
Data users should not use ZCTAs to identify the official USPS ZIP Code for mail delivery. The USPS makes periodic changes to ZIP Codes to support more efficient mail delivery. The ZCTAs process used primarily residential addresses and was biased towards Zip Codes used for city-style mail delivery, thus there may be Zip Codes that are primarily nonresidential or boxes only that may not have a corresponding ZCTA.|2013|obs_d483723c5cc76c107d9e0af279d1e7056df3c2be
"us.census.tiger".county|The primary legal divisions of most states are termed counties. In Louisiana, these divisions are known as parishes. In Alaska, which has no counties, the equivalent entities are the organized boroughs, city and boroughs, municipalities, and census areas; the latter of which are delineated cooperatively for statistical purposes by the state of Alaska and the Census Bureau. In four states (Maryland, Missouri, Nevada, and Virginia), there are one or more incorporated places that are independent of any county organization and thus constitute primary divisions of their states. These incorporated places are known as independent cities and are treated as equivalent entities for purposes of data presentation. The District of Columbia and Guam have no primary divisions, and each area is considered an equivalent entity for purposes of data presentation. All of the counties in Connecticut and Rhode Island and nine counties in Massachusetts were dissolved as functioning governmental entities; however, the Census Bureau continues to present data for these historical entities in order to provide comparable geographic units at the county level of the geographic hierarchy for these states and represents them as nonfunctioning legal entities in data products. The Census Bureau treats the following entities as equivalents of counties for purposes of data presentation: municipios in Puerto Rico, districts and islands in American Samoa, municipalities in the Commonwealth of the Northern Mariana Islands, and islands in the U.S. Virgin Islands. Each county or statistically equivalent entity is assigned a three-character numeric Federal Information Processing Series (FIPS) code based on alphabetical sequence that is unique within state and an eight-digit National Standard feature identifier.|2013|obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4
"us.census.tiger".state|States and Equivalent Entities are the primary governmental divisions of the United States. In addition to the 50 states, the Census Bureau treats the District of Columbia, Puerto Rico, American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands as the statistical equivalents of states for the purpose of data presentation.|2013|obs_a20f5260b618a2fe2eb95fc1e23febe0db7db096
"us.census.tiger".county|The primary legal divisions of most states are termed counties. In Louisiana, these divisions are known as parishes. In Alaska, which has no counties, the equivalent entities are the organized boroughs, city and boroughs, municipalities, and census areas; the latter of which are delineated cooperatively for statistical purposes by the state of Alaska and the Census Bureau. In four states (Maryland, Missouri, Nevada, and Virginia), there are one or more incorporated places that are independent of any county organization and thus constitute primary divisions of their states. These incorporated places are known as independent cities and are treated as equivalent entities for purposes of data presentation. The District of Columbia and Guam have no primary divisions, and each area is considered an equivalent entity for purposes of data presentation. All of the counties in Connecticut and Rhode Island and nine counties in Massachusetts were dissolved as functioning governmental entities; however, the Census Bureau continues to present data for these historical entities in order to provide comparable geographic units at the county level of the geographic hierarchy for these states and represents them as nonfunctioning legal entities in data products. The Census Bureau treats the following entities as equivalents of counties for purposes of data presentation: municipios in Puerto Rico, districts and islands in American Samoa, municipalities in the Commonwealth of the Northern Mariana Islands, and islands in the U.S. Virgin Islands. Each county or statistically equivalent entity is assigned a three-character numeric Federal Information Processing Series (FIPS) code based on alphabetical sequence that is unique within state and an eight-digit National Standard feature identifier.|2013|obs_23da37d4e66e9de2f525572967f8618bde99a8c0
"us.census.tiger".census_tract|Census tracts are identified by an up to four-digit integer number and may have an optional two-digit suffix; for example 1457.02 or 23. The census tract codes consist of six digits with an implied decimal between the fourth and fifth digit corresponding to the basic census tract number but with leading zeroes and trailing zeroes for census tracts without a suffix. The tract number examples above would have codes of 145702 and 002300, respectively.
Some ranges of census tract numbers in the 2010 Census are used to identify distinctive types of census tracts. The code range in the 9400s is used for those census tracts with a majority of population, housing, or land area associated with an American Indian area and matches the numbering used in Census 2000. The code range in the 9800s is new for 2010 and is used to specifically identify special land-use census tracts; that is, census tracts defined to encompass a large area with little or no residential population with special characteristics, such as large parks or employment areas. The range of census tracts in the 9900s represents census tracts delineated specifically to cover large bodies of water. This is different from Census 2000 when water-only census tracts were assigned codes of all zeroes (000000); 000000 is no longer used as a census tract code for the 2010 Census.
The Census Bureau uses suffixes to help identify census tract changes for comparison purposes. Census tract suffixes may range from .01 to .98. As part of local review of existing census tracts before each census, some census tracts may have grown enough in population size to qualify as more than one census tract. When a census tract is split, the split parts usually retain the basic number but receive different suffixes. For example, if census tract 14 is split, the new tract numbers would be 14.01 and 14.02. In a few counties, local participants request major changes to, and renumbering of, the census tracts; however, this is generally discouraged. Changes to individual census tract boundaries usually do not result in census tract numbering changes.
The Census Bureau introduced the concept of tribal census tracts for the first time for Census 2000. Tribal census tracts for that census consisted of the standard county-based census tracts tabulated within American Indian areas, thus allowing for the tracts to ignore state and county boundaries for tabulation. The Census Bureau assigned the 9400 range of numbers to identify specific tribal census tracts; however, not all tribal census tracts used this numbering scheme. For the 2010 Census, tribal census tracts no longer are tied to or numbered in the same way as the county-based census tracts (see \u201cTribal Census Tract\u201d).|2013|obs_d125aeef87aaa23287a40b454519ece22ee25acf
"us.census.tiger".block_group|Block groups (BGs) are statistical divisions of census tracts, are generally defined to contain between 600 and 3,000 people, and are used to present data and control block numbering. A block group consists of clusters of blocks within the same census tract that have the same first digit of their four-digit census block number. For example, blocks 3001, 3002, 3003, ..., 3999 in census tract 1210.02 belong to BG 3 in that census tract. Most BGs were delineated by local participants in the Census Bureau\u2019s Participant Statistical Areas Program. The Census Bureau delineated BGs only where a local or tribal government declined to participate, and a regional organization or State Data Center was not available to participate.
A BG usually covers a contiguous area. Each census tract contains at least one BG, and BGs are uniquely numbered within the census tract. Within the standard census geographic hierarchy, BGs never cross state, county, or census tract boundaries but may cross the boundaries of any other geographic entity. Tribal census tracts and tribal BGs are separate and unique geographic areas defined within federally recognized American Indian reservations and can cross state and county boundaries (see \u201cTribal Census Tract\u201d and \u201cTribal Block Group\u201d). The tribal census tracts and tribal block groups may be completely different from the census tracts and block groups defined by state and county.|2013|obs_d610cb3225f282693b8d4dcd98d2c2e2078354c6
(10 rows)
Dropping obs_table.sql fixture table...
Done.
Dropping obs_column.sql fixture table...
Done.
Dropping obs_column_table.sql fixture table...
Done.
Dropping obs_column_to_column.sql fixture table...
Done.
Dropping obs_85328201013baa14e8e8a4a57a01e6f6fbc5f9b1 fixture table...
Done.
Dropping obs_3e7cc9cfd403b912c57b42d5f9195af9ce2f3cdb fixture table...
Done.
Dropping obs_ab038198aaab3f3cb055758638ee4de28ad70146 fixture table...
Done.
Dropping obs_a92e1111ad3177676471d66bb8036e6d057f271b fixture table...
Done.
Dropping obs_11ee8b82c877c073438bc935a91d3dfccef875d1 fixture table...
Done.
Dropping obs_d34555209878e8c4b37cf0b2b3d072ff129ec470 fixture table...
Done.
Dropping obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4 fixture table...
Done.

89
src/pg/test/expected/44_observatory_geometries_test.out Normal file
View File

@@ -0,0 +1,89 @@
\i test/sql/load_fixtures.sql
SET client_min_messages TO WARNING;
\set ECHO none
Loading obs_table.sql fixture file...
Done.
Loading obs_column.sql fixture file...
Done.
Loading obs_column_table.sql fixture file...
Done.
Loading obs_column_to_column.sql fixture file...
Done.
Loading obs_85328201013baa14e8e8a4a57a01e6f6fbc5f9b1.sql fixture file...
Done.
Loading obs_3e7cc9cfd403b912c57b42d5f9195af9ce2f3cdb.sql fixture file...
Done.
Loading obs_ab038198aaab3f3cb055758638ee4de28ad70146.sql fixture file...
Done.
Loading obs_a92e1111ad3177676471d66bb8036e6d057f271b.sql fixture file...
Done.
Loading obs_11ee8b82c877c073438bc935a91d3dfccef875d1.sql fixture file...
Done.
Loading obs_d34555209878e8c4b37cf0b2b3d072ff129ec470.sql fixture file...
Done.
Loading obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4.sql fixture file...
Done.
obs_getgeometry_cartodb_census_tract
t
(1 row)
obs_getgeometry_cartodb_county
t
(1 row)
obs_getgeometry_non_existent_boundary_id
t
(1 row)
obs_getgeometry_null_island_census_tract
t
(1 row)
obs_getgeometry_year_census_tract
t
(1 row)
obs_getgeometry_unlisted_year
t
(1 row)
obs_getgeometryid_cartodb_census_tract
t
(1 row)
obs_getgeometryid_cartodb_census_tract_with_year
t
(1 row)
obs_getgeometryid_cartodb_county_with_year
t
(1 row)
obs_getgeometryid_null_island
t
(1 row)
obs_getgeometrybyid_cartodb_county
t
(1 row)
obs_getgeometrybyid_compared_with_obs_getgeometry
t
(1 row)
obs_getgeometrybyid_boundary_id_mismatch_geom_id
t
(1 row)
obs_getgeometrybyid_boundary_id_mismatch_geom_id
t
(1 row)
Dropping obs_table.sql fixture table...
Done.
Dropping obs_column.sql fixture table...
Done.
Dropping obs_column_table.sql fixture table...
Done.
Dropping obs_column_to_column.sql fixture table...
Done.
Dropping obs_85328201013baa14e8e8a4a57a01e6f6fbc5f9b1 fixture table...
Done.
Dropping obs_3e7cc9cfd403b912c57b42d5f9195af9ce2f3cdb fixture table...
Done.
Dropping obs_ab038198aaab3f3cb055758638ee4de28ad70146 fixture table...
Done.
Dropping obs_a92e1111ad3177676471d66bb8036e6d057f271b fixture table...
Done.
Dropping obs_11ee8b82c877c073438bc935a91d3dfccef875d1 fixture table...
Done.
Dropping obs_d34555209878e8c4b37cf0b2b3d072ff129ec470 fixture table...
Done.
Dropping obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4 fixture table...
Done.

14
src/pg/test/fixtures/obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4.sql vendored Normal file
View File

File diff suppressed because one or more lines are too long

24
src/pg/test/sql/40_observatory_utility_test.sql
View File

@@ -32,22 +32,28 @@ SELECT
-- -----------|-----------------|-----------
-- geoid | obs_{hex table} | null
-- total_pop | obs_{hex table} | sum
WITH result as (
SELECT
(unnest(cdb_observatory._OBS_GetColumnData(
array_agg(a) expected from cdb_observatory._OBS_GetColumnData(
'"us.census.tiger".census_tract',
Array['"us.census.tiger".census_tract_geoid', '"us.census.acs".B01001001'],
'2009 - 2013'
))).*
ORDER BY colname, tablename ASC;
'2009 - 2013') a
)
select (expected)[1]::text = '{"colname":"geoid","tablename":"obs_d34555209878e8c4b37cf0b2b3d072ff129ec470","aggregate":null,"name":"US Census Tract Geoids","type":"Text","description":""}' as test_get_obs_column_with_geoid_and_census_1,
(expected)[2]::text = '{"colname":"geoid","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":null,"name":"US Census Tract Geoids","type":"Text","description":""}' as test_get_obs_column_with_geoid_and_census_2
from result;
-- should be null-valued
WITH result as (
SELECT
(unnest(cdb_observatory._OBS_GetColumnData(
array_agg(a) expected from cdb_observatory._OBS_GetColumnData(
'"us.census.tiger".census_tract',
Array['"us.census.tiger".baloney'], -- entry not in catalog
'2009 - 2013'
))).*
ORDER BY 1 ASC;
Array['"us.census.tiger".baloney'],
'2009 - 2013') a
)
select expected is null as OBS_GetColumnData_missing_measure
from result;
-- OBS_LookupCensusHuman
-- should give back: {"\"us.census.acs\".B19083001"}

156
src/pg/test/sql/41_observatory_augmentation_test.sql
View File

@@ -1,82 +1,103 @@
\i test/sql/load_fixtures.sql
--
SELECT * FROM
cdb_observatory._OBS_GetDemographicSnapshot(
cdb_observatory._TestPoint(),
'2009 - 2013',
'"us.census.tiger".block_group'
) As snapshot;
WITH result as(
Select count(OBS_GetDemographicSnapshot->>'value') expected_columns
FROM cdb_observatory.OBS_GetDemographicSnapshot(cdb_observatory._TestPoint())
) select expected_columns ='58' as OBS_GetDemographicSnapshot_test_no_returns
FROM result;
--
-- dimension | dimension_value
-- ----------|----------------
-- total_pop | 9516.27915900609
-- male_pop | 6152.51885204623
SELECT *
FROM
cdb_observatory._OBS_GetCensus(
cdb_observatory._TestPoint(),
Array['total_pop','male_pop']::text[]
);
WITH result as (
SELECT array_agg(_obs_getcensus->>'value') as b
FROM( select * from
cdb_observatory._OBS_GetCensus(
cdb_observatory._TestPoint(),
Array['total_pop','male_pop']::text[]
)) a
)
select b='{9516.27915900609,6152.51885204623}'
as test_obsGetCensusWithTestPointAnd2Variables
from result;
-- what happens on null island?
-- expect nulls back: {female_pop, male_pop} | {NULL, NULL}
SELECT *
FROM
cdb_observatory._OBS_GetCensus(
ST_Buffer(CDB_LatLng(0, 0)::geography, 5000)::geometry,
Array['female_pop','male_pop']::text[]
);
WITH result as (
SELECT count(vals) non_null
FROM( select _OBS_GetCensus->>'value' vals from
cdb_observatory._OBS_GetCensus(
ST_Buffer(CDB_LatLng(0, 0)::geography, 5000)::geometry,
Array['total_pop','male_pop']::text[]
)) a
)
SELECT non_null = 0 as test_obsGetCensusWithNullIslandArea
FROM result;
-- expect nulls back {female_pop, male_pop} | {NULL, NULL}
SELECT *
FROM
cdb_observatory._OBS_GetCensus(
CDB_LatLng(0, 0),
Array['female_pop', 'male_pop']::text[]
);
WITH result as (
SELECT count(vals) non_null
FROM( select _OBS_GetCensus->>'value' vals from
cdb_observatory._OBS_GetCensus(
CDB_LatLng(0, 0),
Array['total_pop','male_pop']::text[]
)) a
)
SELECT non_null = 0 as test_obsGetCensusWithNullIsland
FROM result;
--
-- names | vals
-- -----------|-------
-- gini_index | 0.3494
SELECT * FROM
WITH result as (
SELECT _OBS_Get::text as expected FROM
cdb_observatory._OBS_Get(
cdb_observatory._TestPoint(),
Array['"us.census.acs".B19083001']::text[],
'2009 - 2013',
'"us.census.tiger".block_group'
);
)
) select expected = '{"value":0.3494,"name":"Gini Index","tablename":"obs_3e7cc9cfd403b912c57b42d5f9195af9ce2f3cdb","aggregate":"","type":"Numeric","description":""}'
as OBS_Get_gini_index_at_test_point
from result;
-- gini index at null island
SELECT * FROM
WITH result as (
SELECT count(_OBS_Get) as expected FROM
cdb_observatory._OBS_Get(
CDB_LatLng(0, 0),
Array['"us.census.acs".B19083001']::text[],
'2009 - 2013',
'"us.census.tiger".block_group'
);
)
) select expected = 0 as OBS_Get_gini_index_at_null_island
from result;
-- OBS_GetPoints
-- obs_getpoints
-- --------------------
-- {4809.33511352425}
SELECT
cdb_observatory._OBS_GetPoints(
(cdb_observatory._OBS_GetPoints(
cdb_observatory._TestPoint(),
'obs_a92e1111ad3177676471d66bb8036e6d057f271b'::text, -- see example in obs_geomtable
Array[('total_pop','obs_ab038198aaab3f3cb055758638ee4de28ad70146','sum')::cdb_observatory.OBS_ColumnData]
);
(Array['{"colname":"total_pop","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":"sum","name":"Total Population","type":"Numeric","description":"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates."}'::json])
))[1]::text = '{"value":4809.33511352425,"name":"Total Population","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":"sum","type":"Numeric","description":"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates."}'
as OBS_GetPoints_for_test_point;
-- what happens at null island
SELECT
cdb_observatory._OBS_GetPoints(
(cdb_observatory._OBS_GetPoints(
CDB_LatLng(0, 0),
'obs_a92e1111ad3177676471d66bb8036e6d057f271b'::text, -- see example in obs_geomtable
Array[('total_pop','obs_ab038198aaab3f3cb055758638ee4de28ad70146','sum')::cdb_observatory.OBS_ColumnData]
);
(Array['{"colname":"total_pop","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":"sum","name":"Total Population","type":"Numeric","description":"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates."}'::json])
))[1]::text is null
as OBS_GetPoints_for_null_island;
-- OBS_GetPolygons
-- obs_getpolygons
@@ -84,46 +105,55 @@ SELECT
-- {12996.8172420752}
SELECT
cdb_observatory._OBS_GetPolygons(
(cdb_observatory._OBS_GetPolygons(
cdb_observatory._TestArea(),
'obs_a92e1111ad3177676471d66bb8036e6d057f271b'::text, -- see example in obs_geomtable
Array[('total_pop','obs_ab038198aaab3f3cb055758638ee4de28ad70146','sum')::cdb_observatory.OBS_ColumnData]
);
Array['{"colname":"total_pop","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":"sum","name":"Total Population","type":"Numeric","description":"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates."}'::json]
))[1]::text = '{"value":12996.8172420752,"name":"Total Population","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":"sum","type":"Numeric","description":"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates."}'
as OBS_GetPolygons_for_test_point;
-- see what happens around null island
SELECT
cdb_observatory._OBS_GetPolygons(
(cdb_observatory._OBS_GetPolygons(
ST_Buffer(CDB_LatLng(0, 0)::geography, 500)::geometry,
'obs_a92e1111ad3177676471d66bb8036e6d057f271b'::text, -- see example in obs_geomtable
Array[('total_pop','obs_ab038198aaab3f3cb055758638ee4de28ad70146','sum')::cdb_observatory.OBS_ColumnData]
);
Array['{"colname":"total_pop","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":"sum","name":"Total Population","type":"Numeric","description":"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates."}'::json])
)[1]->>'value' is null
as OBS_GetPolygons_for_null_island;
SELECT * FROM
cdb_observatory._OBS_GetSegmentSnapshot(
SELECT cdb_observatory.OBS_GetSegmentSnapshot(
cdb_observatory._TestPoint(),
'"us.census.tiger".census_tract'
);
)::text = '{"segment_name":"SS_segment_10_clusters","\"us.census.acs\".B01001001_quantile":"0.234783783783784","\"us.census.acs\".B01001002_quantile":"0.422405405405405","\"us.census.acs\".B01001026_quantile":"0.0987567567567568","\"us.census.acs\".B01002001_quantile":"0.0715","\"us.census.acs\".B03002003_quantile":"0.295310810810811","\"us.census.acs\".B03002004_quantile":"0.407189189189189","\"us.census.acs\".B03002006_quantile":"0.625608108108108","\"us.census.acs\".B03002012_quantile":"0.795202702702703","\"us.census.acs\".B05001006_quantile":"0.703797297297297","\"us.census.acs\".B08006001_quantile":"0.59227027027027","\"us.census.acs\".B08006002_quantile":"0.0180540540540541","\"us.census.acs\".B08006008_quantile":"0.993756756756757","\"us.census.acs\".B08006009_quantile":"0.728162162162162","\"us.census.acs\".B08006011_quantile":"0.995972972972973","\"us.census.acs\".B08006015_quantile":"0.929135135135135","\"us.census.acs\".B08006017_quantile":"0.625432432432432","\"us.census.acs\".B09001001_quantile":"0.0386081081081081","\"us.census.acs\".B11001001_quantile":"0.157121621621622","\"us.census.acs\".B14001001_quantile":"0.241878378378378","\"us.census.acs\".B14001002_quantile":"0.173783783783784","\"us.census.acs\".B14001005_quantile":"0.0380675675675676","\"us.census.acs\".B14001006_quantile":"0.0308108108108108","\"us.census.acs\".B14001007_quantile":"0.0486216216216216","\"us.census.acs\".B14001008_quantile":"0.479743243243243","\"us.census.acs\".B15003001_quantile":"0.297675675675676","\"us.census.acs\".B15003017_quantile":"0.190351351351351","\"us.census.acs\".B15003022_quantile":"0.802513513513514","\"us.census.acs\".B15003023_quantile":"0.757148648648649","\"us.census.acs\".B16001001_quantile":"0.255405405405405","\"us.census.acs\".B16001002_quantile":"0.196094594594595","\"us.census.acs\".B16001003_quantile":"0.816851351351351","\"us.census.acs\".B17001001_quantile":"0.252513513513514","\"us.census.acs\".B17001002_quantile":"0.560054054054054","\"us.census.acs\".B19013001_quantile":"0.777472972972973","\"us.census.acs\".B19083001_quantile":"0.336932432432432","\"us.census.acs\".B19301001_quantile":"0.655378378378378","\"us.census.acs\".B25001001_quantile":"0.141810810810811","\"us.census.acs\".B25002003_quantile":"0.362824324324324","\"us.census.acs\".B25004002_quantile":"0.463837837837838","\"us.census.acs\".B25004004_quantile":"0","\"us.census.acs\".B25058001_quantile":"0.939040540540541","\"us.census.acs\".B25071001_quantile":"0.419445945945946","\"us.census.acs\".B25075001_quantile":"0.0387972972972973","\"us.census.acs\".B25075025_quantile":"0"}' as test_point_segmentation;
-- segmentation around null island
SELECT * FROM
cdb_observatory._OBS_GetSegmentSnapshot(
SELECT cdb_observatory.OBS_GetSegmentSnapshot(
CDB_LatLng(0, 0),
'"us.census.tiger".census_tract'
);
)::text = '{"segment_name":null,"\"us.census.acs\".B01001001_quantile":null,"\"us.census.acs\".B01001002_quantile":null,"\"us.census.acs\".B01001026_quantile":null,"\"us.census.acs\".B01002001_quantile":null,"\"us.census.acs\".B03002003_quantile":null,"\"us.census.acs\".B03002004_quantile":null,"\"us.census.acs\".B03002006_quantile":null,"\"us.census.acs\".B03002012_quantile":null,"\"us.census.acs\".B05001006_quantile":null,"\"us.census.acs\".B08006001_quantile":null,"\"us.census.acs\".B08006002_quantile":null,"\"us.census.acs\".B08006008_quantile":null,"\"us.census.acs\".B08006009_quantile":null,"\"us.census.acs\".B08006011_quantile":null,"\"us.census.acs\".B08006015_quantile":null,"\"us.census.acs\".B08006017_quantile":null,"\"us.census.acs\".B09001001_quantile":null,"\"us.census.acs\".B11001001_quantile":null,"\"us.census.acs\".B14001001_quantile":null,"\"us.census.acs\".B14001002_quantile":null,"\"us.census.acs\".B14001005_quantile":null,"\"us.census.acs\".B14001006_quantile":null,"\"us.census.acs\".B14001007_quantile":null,"\"us.census.acs\".B14001008_quantile":null,"\"us.census.acs\".B15003001_quantile":null,"\"us.census.acs\".B15003017_quantile":null,"\"us.census.acs\".B15003022_quantile":null,"\"us.census.acs\".B15003023_quantile":null,"\"us.census.acs\".B16001001_quantile":null,"\"us.census.acs\".B16001002_quantile":null,"\"us.census.acs\".B16001003_quantile":null,"\"us.census.acs\".B17001001_quantile":null,"\"us.census.acs\".B17001002_quantile":null,"\"us.census.acs\".B19013001_quantile":null,"\"us.census.acs\".B19083001_quantile":null,"\"us.census.acs\".B19301001_quantile":null,"\"us.census.acs\".B25001001_quantile":null,"\"us.census.acs\".B25002003_quantile":null,"\"us.census.acs\".B25004002_quantile":null,"\"us.census.acs\".B25004004_quantile":null,"\"us.census.acs\".B25058001_quantile":null,"\"us.census.acs\".B25071001_quantile":null,"\"us.census.acs\".B25075001_quantile":null,"\"us.census.acs\".B25075025_quantile":null}' as null_island_segmentation;
SELECT * FROM
cdb_observatory._OBS_GetCategories(
cdb_observatory._TestPoint(),
Array['"us.census.spielman_singleton_segments".X10'],
'"us.census.tiger".census_tract'
);
SELECT * FROM
cdb_observatory._OBS_GetCategories(
CDB_LatLng(0, 0),
Array['"us.census.spielman_singleton_segments".X10'],
'"us.census.tiger".census_tract'
);
WITH result as (
SELECT array_agg(_OBS_GetCategories) as expected FROM
cdb_observatory._OBS_GetCategories(
cdb_observatory._TestPoint(),
Array['"us.census.spielman_singleton_segments".X10'],
'"us.census.tiger".census_tract'
)
)
select (expected)[1]::text = '{"category":"Wealthy, urban without Kids","name":"SS_segment_10_clusters","tablename":"obs_65f29658e096ca1485bf683f65fdbc9f05ec3c5d","aggregate":null,"type":"Text","description":"Sociodemographic classes from Spielman and Singleton 2015, 10 clusters"}' as GetCategories_at_test_point_1,
(expected)[2]::text ='{"category":"Wealthy, urban without Kids","name":"SS_segment_10_clusters","tablename":"obs_11ee8b82c877c073438bc935a91d3dfccef875d1","aggregate":null,"type":"Text","description":"Sociodemographic classes from Spielman and Singleton 2015, 10 clusters"}' as GetCategories_at_test_point_2
from result;
WITH result as (
SELECT array_agg(_OBS_GetCategories) as expected FROM
cdb_observatory._OBS_GetCategories(
CDB_LatLng(0,0),
Array['"us.census.spielman_singleton_segments".X10'],
'"us.census.tiger".census_tract'
)
)
select expected is null as GetCategories_at_null_island
from result;
-- Point-based OBS_GetMeasure, default normalization (area)
SELECT * FROM

27
src/pg/test/sql/42_observatory_exploration_test.sql
View File

@@ -1,5 +1,32 @@
\i test/sql/load_fixtures.sql
\pset format unaligned
-- set up variables for use in testing
\set cartodb_census_tract_geometry ''
\set cartodb_county_geometry ''
-- _OBS_SearchTables tests
SELECT
t.table_name IN ('obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4',
'obs_23da37d4e66e9de2f525572967f8618bde99a8c0') As _OBS_SearchTables_tables_match,
t.timespan = '2013' As _OBS_SearchTables_timespan_matches
FROM cdb_observatory._OBS_SearchTables(
'"us.census.tiger".county',
'2013'
) As t(table_name, timespan);
-- _OBS_SearchTables tests
-- should not return tables for year that does not match
SELECT count(*) = 0 As _OBS_SearchTables_timespan_does_not_match
FROM cdb_observatory._OBS_SearchTables(
'"us.census.tiger".county',
'1988' -- year before first tiger data was collected
) As t(table_name, timespan);
SELECT cdb_observatory.OBS_Search('total_pop');
SELECT * from cdb_observatory.OBS_GetAvailableBoundaries(cdb_observatory._TestPoint());
\i test/sql/drop_fixtures.sql

115
src/pg/test/sql/44_observatory_geometries_test.sql Normal file
View File

File diff suppressed because one or more lines are too long

4
src/pg/test/sql/drop_fixtures.sql
View File

@@ -51,4 +51,8 @@ DROP TABLE observatory.obs_11ee8b82c877c073438bc935a91d3dfccef875d1;
DROP TABLE observatory.obs_d34555209878e8c4b37cf0b2b3d072ff129ec470;
\echo Done.
\echo Dropping obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4 fixture table...
DROP TABLE observatory.obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4;
\echo Done.
\unset ECHO

4
src/pg/test/sql/load_fixtures.sql
View File

@@ -51,4 +51,8 @@ SET client_min_messages TO WARNING;
\i test/fixtures/obs_d34555209878e8c4b37cf0b2b3d072ff129ec470.sql
\echo Done.
\echo Loading obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4.sql fixture file...
\i test/fixtures/obs_b0ef6dd68d5faddbf231fd7f02916b3d00ec43c4.sql
\echo Done.
\unset ECHO