syntax

fix error variable name bug, pep8 updates

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,6 +2,9 @@
 - [ ] All declared geometries are `geometry(Geometry, 4326)` for general geoms, or `geometry(Point, 4326)`
 - [ ] Existing functions in crankshaft python library called from the extension are kept at least from version N to version N+1 (to avoid breakage during upgrades).
 - [ ] Docs for public-facing functions are written
-- [ ] New functions follow the naming conventions: `CDB_NameOfFunction`. Where internal functions begin with an underscore `_`.
-- [ ] If appropriate, new functions accepts an arbitrary query as an input (see [Crankshaft Issue #6](https://github.com/CartoDB/crankshaft/issues/6) for more information)
- 
+- [ ] New functions follow the naming conventions: `CDB_NameOfFunction`. Where internal functions begin with an underscore
+- [ ] Video explaining the analysis and showing examples 
+- [ ] Analysis Documentation written [template](https://docs.google.com/a/cartodb.com/document/d/1X2KOtaiEBKWNMp8UjwcLB-kE9aIOw09aOjX3oaCjeME/edit?usp=sharing)
+- [ ] Smoke test written
+- [ ] Hand-off document for camshaft node written
+- [ ] If function is in Python, code conforms to [PEP8 Style Guide](https://www.python.org/dev/peps/pep-0008/)

.travis.yml

@@ -35,12 +35,16 @@ before_install:
   - sudo apt-get -y remove --purge postgresql-9.2
   - sudo apt-get -y remove --purge postgresql-9.3
   - sudo apt-get -y remove --purge postgresql-9.4
-  - sudo apt-get -y remove --purge postgis
+  - sudo apt-get -y remove --purge postgresql-9.5
+  - sudo rm -rf /var/lib/postgresql/
+  - sudo rm -rf /var/log/postgresql/
+  - sudo rm -rf /etc/postgresql/
+  - sudo apt-get -y remove --purge postgis-2.2
   - sudo apt-get -y autoremove
 
-  - sudo apt-get -y install postgresql-9.5=9.5.2-3cdb1
-  - sudo apt-get -y install postgresql-server-dev-9.5=9.5.2-2ubuntu1
-  - sudo apt-get -y install postgresql-plpython-9.5=9.5.2-2ubuntu1
+  - sudo apt-get -y install postgresql-9.5=9.5.2-3cdb2
+  - sudo apt-get -y install postgresql-server-dev-9.5=9.5.2-3cdb2
+  - sudo apt-get -y install postgresql-plpython-9.5=9.5.2-3cdb2
   - sudo apt-get -y install postgresql-9.5-postgis-scripts=2.2.2.0-cdb2
   - sudo apt-get -y install postgresql-9.5-postgis-2.2=2.2.2.0-cdb2
 

doc/16_getis_ord_gstar.md

@@ -0,0 +1,40 @@
+## Getis-Ord's G\*
+
+Getis-Ord's G\* is a geo-statistical measurement of the intensity of clustering of high or low values. The clustering of high values can be referred to as "hotspots" because these are areas of high activity or large (relative to the global mean) measurement values. Coldspots are clustered areas with low activity or small measurement values.
+
+### CDB_GetisOrdsG(subquery text, column_name text)
+
+#### Arguments
+
+| Name | Type | Description |
+|------|------|-------------|
+| subquery | text | A query of the data you want to pass to the function. It must include `column_name`, a geometry column (usually `the_geom`) and an id column (usually `cartodb_id`) |
+| column_name | text | This is the column of interest for performing this analysis on. This column should be a numeric type. |
+| w_type (optional) | text | Type of weight to use when finding neighbors. Currently available options are 'knn' (default) and 'queen'. Read more about weight types in [PySAL's weights documentation.](https://pysal.readthedocs.io/en/v1.11.0/users/tutorials/weights.html) |
+| num_ngbrs (optional) | integer | Default: 5. If `knn` is chosen, this will set the number of neighbors. If `knn` is not chosen, any entered value will be ignored. Use `NULL` if not choosing `knn`. |
+| permutations (optional) | integer | The number of permutations for calculating p-values. Default: 999 |
+| geom_col (optional) | text | The column where the geometry information is stored. The format must be PostGIS Geometry type (SRID 4326). Default: `the_geom`. |
+| id_col (optional) | text | The column that has the unique row identifier. |
+
+### Returns
+
+Returns a table with the following columns.
+
+| Name | Type | Description |
+|------|------|-------------|
+| z_score | numeric | z-score, a measure of the intensity of clustering of high values (hotspots) or low values (coldspots). Positive values represent 'hotspots', while negative values represent 'coldspots'. |
+| p_value | numeric  | p-value, a measure of the significance of the intensity of clustering |
+| p_z_sim | numeric | p-value based on standard normal approximation from permutations |
+| rowid | integer | The original `id_col` that can be used to associate the outputs with the original geometry and inputs |
+
+#### Example Usage
+
+The following query returns the original table augmented with the values calculated from the Getis-Ord's G\* analysis.
+
+```sql
+SELECT i.*, m.z_score, m.p_value
+  FROM cdb_crankshaft.CDB_GetisOrdsG('SELECT * FROM incident_reports_clustered',
+                                     'num_incidents') As m
+  JOIN incident_reports_clustered As i
+    ON i.cartodb_id = m.rowid;
+```

doc/18_outliers.md

@@ -0,0 +1,163 @@
+## Outlier Detection
+
+This set of functions detects the presence of outliers. There are three functions for finding outliers from non-spatial data:
+
+1. Static Outliers
+1. Percentage Outliers
+1. Standard Deviation Outliers
+
+### CDB_StaticOutlier(column_value numeric, threshold numeric)
+
+#### Arguments
+
+| Name | Type | Description |
+|------|------|-------------|
+| column_value | numeric | The column of values on which to apply the threshold |
+| threshold | numeric | The static threshold which is used to indicate whether a `column_value` is an outlier or not |
+
+### Returns
+
+Returns a boolean (true/false) depending on whether a value is above or below (or equal to) the threshold
+
+| Name | Type | Description |
+|------|------|-------------|
+| outlier | boolean | classification of whether a row is an outlier or not |
+
+#### Example Usage
+
+With a table `website_visits` and a column of the number of website visits in units of 10,000 visits:
+
+```
+| id | visits_10k |
+|----|------------|
+| 1  | 1 |
+| 2  | 3 |
+| 3  | 5 |
+| 4  | 1 |
+| 5  | 32 |
+| 6  | 3 |
+| 7  | 57 |
+| 8  | 2 |
+```
+
+```sql
+SELECT
+  id,
+  CDB_StaticOutlier(visits_10k, 11.0) As outlier,
+  visits_10k
+FROM website_visits
+```
+
+```
+| id | outlier | visits_10k |
+|----|---------|------------|
+| 1  | f | 1 |
+| 2  | f | 3 |
+| 3  | f | 5 |
+| 4  | f | 1 |
+| 5  | t | 32 |
+| 6  | f | 3 |
+| 7  | t | 57 |
+| 8  | f | 2 |
+```
+
+### CDB_PercentOutlier(column_values numeric[], outlier_fraction numeric, ids int[])
+
+`CDB_PercentOutlier` calculates whether or not a value falls above a given threshold based on a percentage above the mean value of the input values.
+
+#### Arguments
+
+| Name | Type | Description |
+|------|------|-------------|
+| column_values | numeric[] | An array of the values to calculate the outlier classification on |
+| outlier_fraction | numeric | The threshold above which a column value divided by the mean of all values is considered an outlier |
+| ids | int[] | An array of the unique row ids of the input data (usually `cartodb_id`) |
+
+### Returns
+
+Returns a table of the outlier classification with the following columns
+
+| Name | Type | Description |
+|------|------|-------------|
+| is_outlier | boolean  | classification of whether a row is an outlier or not |
+| rowid | int | original row id (e.g., input `cartodb_id`) of the row which has the outlier classification |
+
+#### Example Usage
+
+This example find outliers which are more than 100% larger than the average (that is, more than 2.0 times larger).
+
+```sql
+WITH cte As (
+  SELECT
+    unnest(Array[1,2,3,4,5,6,7,8]) As id,
+    unnest(Array[1,3,5,1,32,3,57,2]) As visits_10k
+  )
+SELECT
+  (CDB_PercentOutlier(array_agg(visits_10k), 2.0, array_agg(id))).*
+FROM cte;
+```
+
+Output
+```
+| outlier | rowid |
+|---------+-------|
+| f | 1 |
+| f | 2 |
+| f | 3 |
+| f | 4 |
+| t | 5 |
+| f | 6 |
+| t | 7 |
+| f | 8 |
+```
+
+### CDB_StdDevOutlier(column_values numeric[], num_deviations numeric, ids int[], is_symmetric boolean DEFAULT true)
+
+`CDB_StdDevOutlier` calculates whether or not a value falls above or below a given threshold based on the number of standard deviations from the mean.
+
+#### Arguments
+
+| Name | Type | Description |
+|------|------|-------------|
+| column_values | numeric[] | An array of the values to calculate the outlier classification on |
+| num_deviations | numeric | The threshold in units of standard deviation |
+| ids | int[] | An array of the unique row ids of the input data (usually `cartodb_id`) |
+| is_symmetric (optional) | boolean | Consider outliers that are symmetric about the mean (default: true) |
+
+### Returns
+
+Returns a table of the outlier classification with the following columns
+
+| Name | Type | Description |
+|------|------|-------------|
+| is_outlier | boolean  | classification of whether a row is an outlier or not |
+| rowid | int | original row id (e.g., input `cartodb_id`) of the row which has the outlier classification |
+
+#### Example Usage
+
+This example find outliers which are more than 100% larger than the average (that is, more than 2.0 times larger).
+
+```sql
+WITH cte As (
+  SELECT
+    unnest(Array[1,2,3,4,5,6,7,8]) As id,
+    unnest(Array[1,3,5,1,32,3,57,2]) As visits_10k
+  )
+SELECT
+  (CDB_StdDevOutlier(array_agg(visits_10k), 2.0, array_agg(id))).*
+FROM cte;
+```
+
+Output
+```
+| outlier | rowid |
+|---------+-------|
+| f | 1 |
+| f | 2 |
+| f | 3 |
+| f | 4 |
+| f | 5 |
+| f | 6 |
+| t | 7 |
+| f | 8 |
+```

src/pg/sql/16_getis.sql

@@ -0,0 +1,18 @@
+-- Getis-Ord's G
+-- Hotspot/Coldspot Analysis tool
+CREATE OR REPLACE FUNCTION
+  CDB_GetisOrdsG(
+      subquery TEXT,
+      column_name TEXT,
+      w_type TEXT DEFAULT 'knn',
+      num_ngbrs INT DEFAULT 5,
+      permutations INT DEFAULT 999,
+      geom_col TEXT DEFAULT 'the_geom',
+      id_col TEXT DEFAULT 'cartodb_id')
+RETURNS TABLE (z_score NUMERIC, p_value NUMERIC, p_z_sim NUMERIC, rowid BIGINT)
+AS $$
+  from crankshaft.clustering import getis_ord
+  return getis_ord(subquery, column_name, w_type, num_ngbrs, permutations, geom_col, id_col)
+$$ LANGUAGE plpythonu;
+
+-- TODO: make a version that accepts the values as arrays

src/pg/sql/18_outliers.sql

@@ -0,0 +1,75 @@
+
+-- Find outliers using a static threshold
+--
+CREATE OR REPLACE FUNCTION CDB_StaticOutlier(column_value numeric, threshold numeric)
+RETURNS boolean
+AS $$
+BEGIN
+
+  RETURN column_value > threshold;
+
+END;
+$$ LANGUAGE plpgsql;
+
+-- Find outliers by a percentage above the threshold
+-- TODO: add symmetric option? `is_symmetric boolean DEFAULT false`
+
+CREATE OR REPLACE FUNCTION CDB_PercentOutlier(column_values numeric[], outlier_fraction numeric, ids int[])
+RETURNS TABLE(is_outlier boolean, rowid int)
+AS $$
+DECLARE
+  avg_val numeric;
+  out_vals boolean[];
+BEGIN
+
+  SELECT avg(i) INTO avg_val
+    FROM unnest(column_values) As x(i);
+
+  IF avg_val = 0 THEN
+    RAISE EXCEPTION 'Mean value is zero. Try another outlier method.';
+  END IF;
+
+  SELECT array_agg(
+           outlier_fraction < i / avg_val) INTO out_vals
+    FROM unnest(column_values) As x(i);
+
+  RETURN QUERY
+  SELECT unnest(out_vals) As is_outlier,
+         unnest(ids) As rowid;
+
+END;
+$$ LANGUAGE plpgsql;
+
+-- Find outliers above a given number of standard deviations from the mean
+
+CREATE OR REPLACE FUNCTION CDB_StdDevOutlier(column_values numeric[], num_deviations numeric, ids int[], is_symmetric boolean DEFAULT true)
+RETURNS TABLE(is_outlier boolean, rowid int)
+AS $$
+DECLARE
+  stddev_val numeric;
+  avg_val numeric;
+  out_vals boolean[];
+BEGIN
+
+  SELECT stddev(i), avg(i) INTO stddev_val, avg_val
+    FROM unnest(column_values) As x(i);
+
+  IF stddev_val = 0 THEN
+    RAISE EXCEPTION 'Standard deviation of input data is zero';
+  END IF;
+
+  IF is_symmetric THEN
+    SELECT array_agg(
+             abs(i - avg_val) / stddev_val > num_deviations) INTO out_vals
+      FROM unnest(column_values) As x(i);
+  ELSE
+    SELECT array_agg(
+             (i - avg_val) / stddev_val > num_deviations) INTO out_vals
+      FROM unnest(column_values) As x(i);
+  END IF;
+
+  RETURN QUERY
+  SELECT unnest(out_vals) As is_outlier,
+         unnest(ids) As rowid;
+END;
+$$ LANGUAGE plpgsql;

src/pg/sql/23_hungarian.sql

@@ -1,870 +0,0 @@
--- https://github.com/esa606/hungarian_algorithm/
-
-create or replace package hungarian_algorithm is
-
-  -- Author  : esa606
-  -- Created : 3/16/2015 11:23:12 AM
-  -- Purpose : PL/SQL implementation of the Hungarian/Kuhn-Munkres Algorithm
-  --           found at http://csclab.murraystate.edu/bob.pilgrim/445/munkres.html
-  --           on March 16, 2015.
-
-  /*This software is released under a BSD license, adapted from <http://opensource.org/licenses/bsd-license.php>
-
-  Copyright (c) 2015 esa606. All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without modification,
-  are permitted provided that the following conditions are met:
-
-      Redistributions of source code must retain the above copyright notice,
-      this list of conditions and the following disclaimer.
-      Redistributions in binary form must reproduce the above copyright notice,
-      this list of conditions and the following disclaimer in the documentation and/or
-      other materials provided with the distribution.
-      Neither the name “esa606” nor the names of its contributors may be used to
-      endorse or promote products derived from this software without specific prior
-      written permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS”
-  AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
-  FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-  CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-  OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.*/
-
-  --The main function call for this implementation.
-  --input table: name of table with the price data.  Prices must be nonnegative integers.
-  --row_name_colname: Name of the input_table column holding the price matrix row labels
-  --col_name_colname: Name of the input_table column holding the price matrix column labels
-  --price_colname: Name of the input_table column holding the price matrix prices
-  --logging_mode: 1 for logging, 0 for no logging
-  procedure hungarian_main (
-    input_table in varchar2,
-    row_name_colname in varchar2 default 'row_idx',
-    col_name_colname in varchar2 default 'col_idx',
-    price_colname in varchar2 default 'price',
-    logging_mode in integer default 0
-  );
-
-  --A testing function that allows steps of the algorithm to be called according to a string.
-  --E.g. hungarian_strstep('123444', ...) would call steps 1, 2, 3, and then 4 three times.
-  --May return errors if the specified steps are not algorithmically correct.
-  --stepstr: the string specifying the order of the steps
-  --Other arguments as above.
-  procedure hungarian_strstep (
-    stepstr in varchar2,
-    input_table in varchar2,
-    row_name_colname in varchar2 default 'row_idx',
-    col_name_colname in varchar2 default 'col_idx',
-    price_colname in varchar2 default 'price',
-    logging_mode in integer default 0
-  );
-
-end hungarian_algorithm;
-/
-create or replace package body hungarian_algorithm is
-
-    /*This software is released under a BSD license, adapted from <http://opensource.org/licenses/bsd-license.php>
-
-  Copyright (c) 2015 esa606. All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without modification,
-  are permitted provided that the following conditions are met:
-
-      Redistributions of source code must retain the above copyright notice,
-      this list of conditions and the following disclaimer.
-      Redistributions in binary form must reproduce the above copyright notice,
-      this list of conditions and the following disclaimer in the documentation and/or
-      other materials provided with the distribution.
-      Neither the name “esa606” nor the names of its contributors may be used to
-      endorse or promote products derived from this software without specific prior
-      written permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS”
-  AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
-  FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-  CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-  OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.*/
-
-  --Logs calls to the Hungarian Algorithm step-by-step.
-  --Typically called by if-statement with variable step_logging_mode.
-  --If step_logging_mode = 0, no logging
-  --If step_logging_mode = 1, logs with first_call as 0
-  --If step_logging_mode = 2, logs with first_call as 1
-  procedure hungarian_logging (
-    curr_step in integer,
-    next_step in integer,
-    n in integer,
-    first_call in integer default 0
-  ) is
-    log_cnt integer;
-    call_no integer;
-  begin
-    --Check the initial highest call_no.  Increment it if this
-    --is the first call, or re-use if not.
-    select count(*)
-    into log_cnt
-    from hungarian_log
-    where rownum = 1;
-
-    if log_cnt = 0 then
-      call_no := 1;
-    else
-      if first_call = 1 then
-        select max(call_no) + 1
-        into call_no
-        from hungarian_log;
-      else
-       select max(call_no)
-        into call_no
-        from hungarian_log;
-      end if;
-    end if;
-
-    insert into hungarian_log
-    values (
-      call_no,
-      current_timestamp,
-      curr_step, next_step,
-      (select sum(starred) from esa_hungarian_base),
-      (select sum(row_covered) from esa_hungarian_base),
-      (select sum(col_covered) from esa_hungarian_base),
-      (select sum(prime_sequence) from esa_hungarian_base),
-      (select sum(z_sequence) from esa_hungarian_base),
-      n
-    );
-    commit;
-  end hungarian_logging;
-
-
-  --A: Transfer initial conditions to base table
-  --B: Error-checks initial conditions
-  --C: Pads the rows or columns if initial price matrix is not square
-  --D: Update elt_idx
-  --Returns the matrix size n.
-  function hungarian_step1_check_setup (
-    input_table in varchar2,
-    row_name_colname in varchar2,
-    col_name_colname in varchar2,
-    price_colname in varchar2,
-    step_logging_mode in integer
-  ) return integer is
-    select_statement varchar2(4000);
-    max_price number;
-    nonunique_indices_cnt integer;
-    num_rows integer;
-    num_cols integer;
-    first_call integer;
-  begin
-    --A: This makes all possible combinations of row and column from the input table
-    --and then fills in price for the combinations in the input table.
-    --If the combo doesn't exist, fills in max_price.
-    select_statement := 'select max(' || price_colname || ') from ' || input_table;
-    execute immediate select_statement into max_price;
-
-    execute immediate 'truncate table hungarian_base';
-
-    execute immediate
-    'insert into hungarian_base
-    with t as (
-      select
-        dense_rank() over (order by ' || row_name_colname || ' asc) as row_idx,
-        dense_rank() over (order by ' || col_name_colname || ' asc) as col_idx,
-        ' || price_colname  || ' as price
-      from ' || input_table || '
-    )
-    select
-      null as elt_idx,
-      s.row_idx, s.col_idx,
-      case
-        when t.price is null then ' || max_price || ' + 1
-        else t.price
-      end as price,
-      0 as starred,
-      0 as row_covered,
-      0 as col_covered,
-      0 as prime_sequence,
-      0 as z_sequence
-    from (
-      select a.row_idx, b. col_idx
-      from (select distinct row_idx from t) a
-      join (select distinct col_idx from t) b
-        on 1 = 1
-    ) s
-
-    left join t
-      on s.row_idx = t.row_idx
-      and s.col_idx = t.col_idx
-
-    order by s.row_idx, s.col_idx';
-    commit;
-
-    --B: Error-checking
-    select count(*)
-    into nonunique_indices_cnt
-    from (
-      select row_idx, col_idx
-      from hungarian_base
-      group by row_idx, col_idx
-      having count(*) > 1
-    );
-
-    if nonunique_indices_cnt > 0 then
-      raise_application_error(-20000, 'At least one task/agent combo is not unique.  Check your identifiers.');
-    end if;
-
-    --D: Padding out rows or columns with max-price if necessary
-    select_statement := 'select count(distinct ' || row_name_colname || ') from ' || input_table;
-    execute immediate select_statement into num_rows;
-
-    select_statement := 'select count(distinct ' || col_name_colname || ') from ' || input_table;
-    execute immediate select_statement into num_cols;
-
-    if num_rows > num_cols then
-      for i in 1..num_rows loop
-        for j in (num_cols+1)..num_rows loop
-          insert into hungarian_base
-          values (null, i, j, max_price + 1, 0, 0, 0, 0, 0);
-        end loop;
-      end loop;
-    elsif num_rows < num_cols then
-      for i in (num_rows+1)..num_cols loop
-        for j in 1..num_cols loop
-          insert into hungarian_base
-          values (null, i, j, max_price + 1, 0, 0, 0, 0, 0);
-        end loop;
-      end loop;
-    end if;
-
-    --D: Correct elt_idx.  Since the initial matrix is likely to be incomplete
-    --either due to missing links or needing padding, I just wait
-    --to do this until that's all filled in.
-    update hungarian_base
-    set elt_idx = (row_idx - 1)*greatest(num_rows, num_cols) + col_idx;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(1, 2, greatest(num_rows, num_cols), first_call);
-    end if;
-
-    commit;
-    return greatest(num_rows, num_cols);
-
-  end hungarian_step1_check_setup;
-
-
-  --For each row of the matrix, find the smallest element and
-  --subtract it from every element in its row.  Go to Step 3.
-  procedure hungarian_step2_reduce (n in integer, step_logging_mode in integer) is
-    row_min number;
-    first_call integer;
-  begin
-    for i in 1..n loop
-
-      select min(price)
-      into row_min
-      from hungarian_base
-      where row_idx = i;
-
-      update hungarian_base
-      set price = price - row_min
-      where row_idx = i;
-
-    end loop;
-    commit;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(2, 3, null, first_call);
-    end if;
-  end hungarian_step2_reduce;
-
-
-  --Find a zero (Z) in the resulting matrix.  If there is no starred zero
-  --in its row or column, star Z. Repeat for each element in the matrix.
-  --Go to Step 4.
-  procedure hungarian_step3_initstar (n in integer, step_logging_mode in integer) is
-    ij_price number;
-    ij_covered integer;
-    first_call integer;
-  begin
-    for i in 1..n loop
-      for j in 1..n loop
-
-        select price
-        into ij_price
-        from hungarian_base
-        where row_idx = i
-          and col_idx = j;
-
-        if ij_price = 0 then
-
-          select count(*)
-          into ij_covered
-          from hungarian_base
-          where row_idx = i
-            and col_idx = j
-            and (row_covered = 1 or col_covered = 1);
-
-          if ij_covered = 0 then
-
-            update hungarian_base
-            set starred = 1
-            where row_idx = i
-              and col_idx = j;
-
-            update hungarian_base
-            set row_covered = 1
-            where row_idx = i;
-
-            update hungarian_base
-            set col_covered = 1
-            where col_idx = j;
-
-          end if;
-        end if;
-      end loop;
-    end loop;
-
-    update hungarian_base
-    set row_covered = 0,
-      col_covered = 0;
-    commit;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(3, 4, null, first_call);
-    end if;
-  end hungarian_step3_initstar;
-
-
-  --Cover each column containing a starred zero.  If n columns
-  --are covered, the starred zeros describe a complete set of unique assignments.
-  --In this case, Go to Step 8 for finishing touches.
-  --Otherwise, Go to Step 5.
-  --Returns the step to goto
-  function hungarian_step4_coverstarred (n in integer, step_logging_mode in integer) return integer is
-    j_starred integer;
-    covered_cnt integer;
-    next_step integer;
-    first_call integer;
-  begin
-    for j in 1..n loop
-
-      select max(starred)
-      into j_starred
-      from hungarian_base
-      where col_idx = j;
-
-      if j_starred = 1 then
-
-        update hungarian_base
-        set col_covered = 1
-        where col_idx = j;
-
-      end if;
-    end loop;
-
-    select count(distinct col_idx)
-    into covered_cnt
-    from hungarian_base
-    where starred = 1;
-
-    commit;
-
-    if covered_cnt = n then
-      next_step := 8; --This is the only place to exit the algorithm, right here
-    else
-      next_step := 5;
-    end if;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(4, next_step, null, first_call);
-    end if;
-    return next_step;
-  end hungarian_step4_coverstarred;
-
-  --A: If there are no uncovered zeroes at all, go to Step 7.
-  --B: Find a noncovered zero and prime it.
-  --C: If there is no starred zero in the row containing this primed zero, go to Step 6.
-  --D: Otherwise, cover this row and uncover the column containing the starred zero.
-  --E: Return to this step to continue in this manner until there are no uncovered zeros left.
-  --Returns the step to goto
-  function hungarian_step5_prime (n in integer, step_logging_mode in integer) return integer is
-    uncovered_zero_cnt integer;
-    min_uncovered_zero integer;
-    new_prime_sequence integer;
-    most_recently_primed_row integer;
-    starred_in_row_cnt integer;
-    starred_in_row_col integer;
-    next_step integer;
-    first_call integer;
-  begin
-    --Part A
-    select count(*)
-    into uncovered_zero_cnt
-    from hungarian_base
-    where row_covered = 0
-      and col_covered = 0
-      and price = 0;
-
-    if uncovered_zero_cnt = 0 then
-      next_step := 7;
-    else
-      --Part B
-      select min(elt_idx)
-      into min_uncovered_zero
-      from hungarian_base
-      where row_covered = 0
-        and col_covered = 0
-        and price = 0;
-
-      select max(prime_sequence) + 1
-      into new_prime_sequence
-      from hungarian_base;
-
-      update hungarian_base
-      set prime_sequence = new_prime_sequence
-      where elt_idx = min_uncovered_zero;
-
-      --Part C
-      most_recently_primed_row := ceil(min_uncovered_zero/n);
-
-      select count(*)
-      into starred_in_row_cnt
-      from hungarian_base
-      where row_idx = most_recently_primed_row
-        and starred = 1;
-
-      if starred_in_row_cnt = 0 then
-        next_step := 6;
-      else
-        --Part D
-        update hungarian_base
-        set row_covered = 1
-        where row_idx = most_recently_primed_row;
-
-        select col_idx
-        into starred_in_row_col
-        from hungarian_base
-        where row_idx = most_recently_primed_row
-          and starred = 1;
-
-        update hungarian_base
-        set col_covered = 0
-        where col_idx = starred_in_row_col;
-
-        --Part E
-        next_step := 5;
-      end if;
-    end if;
-
-    commit;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(5, next_step, null, first_call);
-    end if;
-    return next_step;
-  end hungarian_step5_prime;
-
-
-  --Construct a series of alternating primed and starred zeros as follows:
-  --Let Z0 represent the uncovered primed zero found in Step 5.
-  --Let Z1 denote the starred zero in the column of Z0 (if any).
-  --Let Z2 denote the primed zero in the row of Z1 (there will always be one).
-  --Continue until the series terminates at a primed zero that has no starred
-  --zero in its column.
-  --The above is accomplished by repeated calls to this step.
-  --Then unstar each starred zero of the series, star each primed
-  --zero of the series, erase all primes and uncover every line in the matrix.
-  --Now return to Step 4.
-  --Returns the step to goto
-  function hungarian_step6_zsequence (n in integer, step_logging_mode in integer) return integer is
-    begin_max_z_sequence integer;
-    begin_prime integer;
-    begin_prime_col integer;
-    starred_zeros_in_prime_col_cnt integer;
-    next_step integer;
-    first_call integer;
-  begin
-    --Part A: Add the prime to the sequence.
-    --If the sequence is so far empty, add the most recent prime.
-    --If the sequence is not empty, add the prime in the row of
-    --the most recently added element.
-    select max(z_sequence)
-    into begin_max_z_sequence
-    from hungarian_base;
-
-    if begin_max_z_sequence = 0 then
-
-      select elt_idx
-      into begin_prime
-      from hungarian_base
-      where prime_sequence = (
-        select max(prime_sequence)
-        from hungarian_base
-        );
-
-    else
-
-      select elt_idx
-      into begin_prime
-      from hungarian_base
-      where prime_sequence != 0
-        and row_idx = (
-          select row_idx
-          from hungarian_base
-          where z_sequence = begin_max_z_sequence
-        );
-
-    end if;
-
-    update hungarian_base
-    set z_sequence = begin_max_z_sequence + 1
-    where elt_idx = begin_prime;
-
-    --Part B: If there is a starred zero in the newly-added prime's column,
-    --add it and repeat.  Otherwise, change markings and return to Step 4.
-    begin_prime_col := begin_prime + (1 - ceil(begin_prime/n))*n;
-
-    select count(*)
-    into starred_zeros_in_prime_col_cnt
-    from hungarian_base
-    where price = 0
-      and starred = 1
-      and col_idx = begin_prime_col;
-
-    if starred_zeros_in_prime_col_cnt > 0 then
-
-      update hungarian_base
-      set z_sequence = begin_max_z_sequence + 2
-      where col_idx = begin_prime_col
-        and starred = 1;
-
-      next_step := 6;
-
-    else
-
-      update hungarian_base
-      set starred = 0
-      where z_sequence != 0
-        and starred = 1;
-
-      update hungarian_base
-      set starred = 1
-      where z_sequence != 0
-        and prime_sequence != 0;
-
-      update hungarian_base
-      set row_covered = 0,
-        col_covered = 0,
-        prime_sequence = 0,
-        z_sequence = 0;
-
-      next_step := 4;
-    end if;
-
-    commit;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(6, next_step, null, first_call);
-    end if;
-
-    return next_step;
-  end hungarian_step6_zsequence;
-
-
-  --Add the minimum uncovered price to every element of each covered row,
-  --and subtract it from every element of each uncovered column.
-  --Return to Step 5 without altering any stars, primes, or covered lines.
-  --Returns the step to goto
-  function hungarian_step7_addsubtract (step_logging_mode in integer) return integer is
-    min_uncovered_price number;
-    first_call integer;
-  begin
-    select min(price)
-    into min_uncovered_price
-    from hungarian_base
-    where row_covered = 0
-      and col_covered = 0;
-
-    update hungarian_base
-    set price = price + min_uncovered_price*row_covered - min_uncovered_price*(1-col_covered);
-    commit;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(7, 5, null, first_call);
-    end if;
-
-    return 5;
-
-  end hungarian_step7_addsubtract;
-
-
-  --Creates table hungarian_results with the same row and column names as the
-  --input table, showing only the assigned pairs.
-  procedure hungarian_step8_results (
-    input_table in varchar2,
-    row_name_colname in varchar2,
-    col_name_colname in varchar2,
-    price_colname in varchar2,
-    step_logging_mode in integer
-  ) is
-    table_exists integer;
-    row_idx_select varchar2(4000);
-    b_row_idx_name varchar2(4000);
-    col_idx_select varchar2(4000);
-    b_col_idx_name varchar2(4000);
-    first_call integer;
-  begin
-    select count(*)
-    into table_exists
-    from tab
-    where tname = upper('hungarian_results');
-
-    if table_exists = 1 then
-      execute immediate 'drop table hungarian_results';
-    end if;
-
-    if row_name_colname = 'row_idx' then
-      row_idx_select := '';
-      b_row_idx_name := 'b_row_idx';
-    else
-      row_idx_select := 'a.row_idx,';
-      b_row_idx_name := 'row_idx';
-    end if;
-
-    if col_name_colname = 'col_idx' then
-      col_idx_select := '';
-      b_col_idx_name := 'b_col_idx';
-    else
-      col_idx_select := 'a.col_idx,';
-      b_col_idx_name := 'col_idx';
-    end if;
-
-    execute immediate
-    'create table hungarian_results as
-    select
-      b.' || row_name_colname || ',
-      b.' || col_name_colname || ',
-      ' || row_idx_select || '
-      ' || col_idx_select || '
-      b.' || price_colname || '
-    from (
-      select row_idx, col_idx, elt_idx
-      from hungarian_base
-      where starred = 1
-    ) a
-
-    --Only allows row/column combos that existed in the original
-    --to go through.
-    join (
-      select
-        ' || row_name_colname || ',
-        ' || col_name_colname || ',
-        dense_rank() over (order by ' || row_name_colname || ' asc) as ' || b_row_idx_name || ',
-        dense_rank() over (order by ' || col_name_colname || ' asc) as ' || b_col_idx_name || ',
-        ' || price_colname  || '
-      from ' || input_table || '
-    ) b
-    on a.row_idx = b.' || b_row_idx_name || '
-    and a.col_idx = b.' || b_col_idx_name || '
-
-    order by a.elt_idx';
-
-    commit;
-
-    if step_logging_mode > 0 then
-      if step_logging_mode = 1 then
-        first_call := 0;
-      else
-        first_call := 1;
-      end if;
-      hungarian_algorithm.hungarian_logging(8, null, null, first_call);
-    end if;
-
-  end hungarian_step8_results;
-
-
-  --Steps 4-7 can run in variable order, with repeated calls to each depending on
-  --the previous calls.  They output the next step to go to.
-  --This function interprets that output and calls the appropriate next step.
-  function hungarian_varstep_interpreter (
-    stepno in integer,
-    n in integer,
-    step_logging_mode in integer default 0
-  ) return integer is
-    next_step integer;
-  begin
-    if stepno = 4 then
-      next_step := hungarian_algorithm.hungarian_step4_coverstarred(n, step_logging_mode);
-    elsif stepno = 5 then
-      next_step := hungarian_algorithm.hungarian_step5_prime(n, step_logging_mode);
-    elsif stepno = 6 then
-      next_step := hungarian_algorithm.hungarian_step6_zsequence(n, step_logging_mode);
-    elsif stepno = 7 then
-      next_step := hungarian_algorithm.hungarian_step7_addsubtract(step_logging_mode);
-    end if;
-
-    return next_step;
-
-  end hungarian_varstep_interpreter;
-
-
-  --The main function call for this implementation.
-  --input table: name of table with the price data.  Prices must be nonnegative integers.
-  --row_name_colname: Name of the input_table column holding the price matrix row labels
-  --col_name_colname: Name of the input_table column holding the price matrix column labels
-  --price_colname: Name of the input_table column holding the price matrix prices
-  --logging_mode: 1 for logging, 0 for no logging
-  procedure hungarian_main (
-    input_table in varchar2,
-    row_name_colname in varchar2 default 'row_idx',
-    col_name_colname in varchar2 default 'col_idx',
-    price_colname in varchar2 default 'price',
-    logging_mode in integer default 0
-  ) is
-    step_logging_mode integer;
-    n integer;
-    next_step integer;
-  begin
-    if logging_mode = 1 then
-      step_logging_mode := 2;
-    else
-      step_logging_mode := 0;
-    end if;
-
-    n := hungarian_algorithm.hungarian_step1_check_setup(
-      input_table, row_name_colname, col_name_colname, price_colname, step_logging_mode
-    );
-
-    hungarian_algorithm.hungarian_step2_reduce(n, logging_mode);
-    hungarian_algorithm.hungarian_step3_initstar(n, logging_mode);
-
-    next_step := 4;
-
-    while next_step < 8 loop
-      next_step := hungarian_varstep_interpreter(next_step, n, logging_mode);
-    end loop;
-
-    hungarian_algorithm.hungarian_step8_results(
-      input_table, row_name_colname, col_name_colname, price_colname, logging_mode
-    );
-
-  end hungarian_main;
-
-
-  --A testing function that allows steps to be called according to a string.
-  --E.g. hungarian_strstep('123444') would call steps 1, 2, 3, and then 4 three times.
-  --May return errors if the specified steps are not algorithmically correct.
-  procedure hungarian_strstep (
-    stepstr in varchar2,
-    input_table in varchar2,
-    row_name_colname in varchar2 default 'row_idx',
-    col_name_colname in varchar2 default 'col_idx',
-    price_colname in varchar2 default 'price',
-    logging_mode in integer default 0
-  ) is
-    step_logging_mode integer;
-    n integer;
-    strlen integer;
-    num_steps integer;
-    eight_last integer;
-    stepno integer;
-    throwaway integer; --since varstep is a function
-  begin
-    if substr(stepstr, 1, 3) = '123' then
-      if logging_mode = 1 then
-        step_logging_mode := 2;
-      else
-        step_logging_mode := 0;
-      end if;
-      n := hungarian_algorithm.hungarian_step1_check_setup(
-        input_table, row_name_colname, col_name_colname,
-        price_colname, step_logging_mode
-      );
-      step_logging_mode := logging_mode;
-      hungarian_algorithm.hungarian_step2_reduce(n, logging_mode);
-      hungarian_algorithm.hungarian_step3_initstar(n, logging_mode);
-    else
-      if logging_mode = 1 then
-        step_logging_mode := 2;
-      else
-        step_logging_mode := 0;
-      end if;
-    end if;
-
-    strlen := length(stepstr);
-
-    if substr(stepstr, strlen, 1) = '8' then
-      num_steps := strlen - 1;
-      eight_last := 1;
-    else
-      num_steps := strlen;
-      eight_last := 0;
-    end if;
-
-    for i in 4..num_steps loop
-      stepno := to_number(substr(stepstr, i, 1));
-      if stepno between 4 and 7 then
-        throwaway := hungarian_algorithm.hungarian_varstep_interpreter(stepno, n, step_logging_mode);
-        step_logging_mode := logging_mode;
-      else
-        raise_application_error(-20000, 'Intermediate steps must be 4, 5, 6, or 7.');
-      end if;
-    end loop;
-
-    if eight_last = 1 then
-      --In this case Step 8 is also the first step
-      if strlen = 1 and logging_mode = 1 then
-        step_logging_mode := 2;
-      end if;
-      hungarian_algorithm.hungarian_step8_results(
-        input_table, row_name_colname, col_name_colname, price_colname, step_logging_mode
-      );
-    end if;
-
-  end hungarian_strstep;
-
-end hungarian_algorithm;
-/

src/pg/sql/30_dot_density.sql

@@ -10,45 +10,192 @@
 -- misses per point the funciton accepts before giving up.
 --
 -- Returns: Multipoint with the requested points
-CREATE OR REPLACE FUNCTION cdb_dot_density(geom geometry , no_points Integer, max_iter_per_point Integer DEFAULT 1000)
-RETURNS GEOMETRY AS $$
-DECLARE
-  extent GEOMETRY;
-  test_point Geometry;
-  width                NUMERIC;
-  height               NUMERIC;
-  x0                   NUMERIC;
-  y0                   NUMERIC;
-  xp                   NUMERIC;
-  yp                   NUMERIC;
-  no_left              INTEGER;
-  remaining_iterations INTEGER;
-  points               GEOMETRY[];
-  bbox_line            GEOMETRY;
-  intersection_line    GEOMETRY;
-BEGIN
-  extent  := ST_Envelope(geom);
-  width   := ST_XMax(extent) - ST_XMIN(extent);
-  height  := ST_YMax(extent) - ST_YMIN(extent);
-  x0 	  := ST_XMin(extent);
-  y0 	  := ST_YMin(extent);
-  no_left := no_points;
 
-  LOOP
-    if(no_left=0) THEN
-      EXIT;
-    END IF;
-    yp = y0 + height*random();
-    bbox_line  = ST_MakeLine(
-      ST_SetSRID(ST_MakePoint(yp, x0),4326),
-      ST_SetSRID(ST_MakePoint(yp, x0+width),4326)
-    );
-    intersection_line = ST_Intersection(bbox_line,geom);
-  	test_point = ST_LineInterpolatePoint(st_makeline(st_linemerge(intersection_line)),random());
-	  points := points || test_point;
-	  no_left = no_left - 1 ;
-  END LOOP;
-  RETURN ST_Collect(points);
+CREATE OR REPLACE FUNCTION CDB_DotDensity(g geometry(Polygon, 4326), no_points integer, max_iter integer DEFAULT 1000)
+ RETURNS SETOF geometry(Point, 4326)
+AS $$
+ DECLARE
+     extent GEOMETRY;
+     eq_area_geom GEOMETRY;
+     test_point Geometry;
+     iter     NUMERIC;
+     width    NUMERIC;
+     height   NUMERIC;
+     x0       NUMERIC;
+     y0       NUMERIC;
+     no_left  INTEGER;
+     sample_points GEOMETRY[];
+     points   GEOMETRY[];
+ BEGIN
+   eq_area_geom := ST_TRANSFORM(g, 2163);
+   extent   := ST_Envelope(eq_area_geom);
+   iter := 0;
+   width    := ST_XMax(extent) - ST_XMIN(extent);
+   height   := ST_YMax(extent) - ST_YMIN(extent);
+   x0       := ST_XMin(extent);
+   y0       := ST_YMin(extent);
+   no_left  := no_points;
+
+   LOOP
+     IF(no_left <= 0 or iter >= max_iter) THEN
+       RETURN;
+     END IF;
+
+
+    with random_points as(
+        SELECT ST_SetSRID(ST_MAKEPOINT( x0 + width*random(), y0 + height*random()), 2163) as p
+        FROM generate_series(1,no_left)
+     )
+     SELECT array_agg(p) from random_points
+     WHERE ST_WITHIN(p, eq_area_geom)
+     into sample_points;
+
+     RETURN QUERY select ST_TRANSFORM(a, 4326) from unnest(sample_points) as a;
+
+     IF sample_points IS NOT null THEN
+        no_left := no_left - array_length(sample_points, 1);
+     END IF;
+     iter = iter + 1;
+   END LOOP;
+
+   RETURN;
+ END;
+$$ LANGUAGE plpgsql;
+
+-- DEPRECATED
+
+CREATE OR REPLACE FUNCTION cdb_dot_density(geom geometry, no_points Integer, max_iter_per_point Integer DEFAULT 1000)
+  RETURNS GEOMETRY
+AS $$
+DECLARE
+final_points GEOMETRY;
+
+BEGIN
+
+  with new_points as(
+    SELECT * FROM CDB_DotDensity(geom, no_points, max_iter_per_point) as a
+  )
+  SELECT ST_Collect(a) FROM new_points
+  into final_points;
+  RETURN final_points;
+
 END;
+$$ LANGUAGE plpgsql;
+
+--
+-- Creates N points randomly distributed in the specified secondary polygons
+--
+-- @param g - array of the geometries to be turned in to points
+--
+-- @param no_points - the number of points to generate
+--
+-- @params max_iter_per_point - the function generates points in the polygon's bounding box
+-- and discards points which don't lie in the polygon. max_iter_per_point specifies how many
+-- misses per point the funciton accepts before giving up.
+--
+-- Returns: Multipoint with the requested points
+
+
+
+--
+-- Generate a random response based on the weights given
+--
+-- @param array_ids an array of ids representing the category to return
+--
+-- @param weights an array of weights for each category
+--
+-- Returns : The randomly selected ID.
+
+CREATE OR REPLACE function _cdb_SelectRandomWeights(array_ids numeric[], weights numeric[]) returns NUMERIC
+ as $$
+ DECLARE
+     result NUMERIC;
+ BEGIN
+
+     WITH idw as (
+         select unnest(array_ids) as id, unnest(weights) as percent
+     ),
+     CTE AS (
+         SELECT random() * (SELECT SUM(percent) FROM idw) R
+     )
+     SELECT *
+     FROM (
+         SELECT id, SUM(percent) OVER (ORDER BY id) S, R
+         FROM idw as percent CROSS JOIN CTE
+     ) Q
+     WHERE S >= R
+     ORDER BY id
+     LIMIT 1
+     into result;
+     return result;
+ END
+ $$ LANGUAGE plpgsql;
+
+--
+-- Weighted Dot Density
+--
+-- @param no_points the number of points to generate
+--
+-- @param geoms the target geometries to place the points in
+--
+-- @param weights the weight for each of the target polygons
+--
+-- RETURNS set of points
+
+CREATE OR REPLACE FUNCTION _cdb_WeightedDD(no_points numeric, geoms geometry[], weights numeric[])
+ RETURNS SETOF geometry
+AS $$
+DECLARE
+ i NUMERIC;
+ ids NUMERIC[];
+ perGeom NUMERIC[];
+ selected_poly NUMERIC;
+BEGIN
+ with idseries as (
+     select generate_series(1,array_upper(geoms,1)) as id
+ )
+ select array_agg(id) from idseries into ids;
+
+ FOR i in 1..no_points
+ LOOP
+    select cdb_crankshaft._cdb_SelectRandomWeights(ids, weights) INTO selected_poly;
+    perGeom[selected_poly] = coalesce(perGeom[selected_poly] + 1, 0 );
+ END LOOP;
+
+ raise notice 'pergeom %', perGeom;
+
+ FOR i in 1..array_length(ids,1)
+ LOOP
+     return QUERY
+     select cdb_crankshaft.CDB_DotDensity(geoms[i], coalesce(perGeom[i],0)::INTEGER);
+ END LOOP;
+END
 $$
-LANGUAGE plpgsql VOLATILE;
+LANGUAGE plpgsql;
+
+
+--
+-- Daysymetric Dot Density
+--
+-- @param geom: the geometry that has the
+--
+-- @param no_points: the total number of points to create
+--
+-- @param targetGeoms: the geometry that has the
+--
+-- @param weights: targetGeom weights
+--
+-- RETURNS setof points
+
+CREATE OR REPLACE FUNCTION CDB_DasymetricDotDensity(geom GEOMETRY, no_points NUMERIC, targetGeoms GEOMETRY[], weights numeric [])
+RETURNS setof GEOMETRY
+AS $$
+  BEGIN
+    RAISE NOTICE 'running Dasymetric';
+    RETURN QUERY
+        SELECT cdb_crankshaft._CDB_WeightedDD(no_points, array_agg( ST_INTERSECTION(geom,g)), array_agg(ST_AREA(ST_INTERSECTION(geom,g))*w)::NUMERIC[])
+        FROM unnest(targetGeoms) as g , unnest(weights) as w
+        WHERE geom && g;
+  END
+$$
+LANGUAGE plpgsql;

src/pg/test/expected/02_moran_test.out

@@ -149,135 +149,135 @@ _cdb_random_seeds
 
 (1 row)
 code|quads
-01|LL
-02|LH
-03|HH
-04|HH
-05|LL
-06|HH
-07|LL
-08|LL
-09|LL
-10|HH
-11|HH
-12|HL
-13|LL
-14|HH
+01|HH
+02|HL
+03|LL
+04|LL
+05|LH
+06|LL
+07|HH
+08|HH
+09|HH
+10|LL
+11|LL
+12|LL
+13|HL
+14|LL
 15|LL
-16|LL
-17|LL
-18|LH
-19|LL
-20|LL
-21|HH
-22|LL
-23|HL
+16|HH
+17|HH
+18|LL
+19|HH
+20|HH
+21|LL
+22|HH
+23|LL
 24|LL
-25|LL
-26|LL
+25|HH
+26|HH
 27|LL
-28|LL
-29|LH
-30|HH
-31|LL
+28|HH
+29|LL
+30|LL
+31|HH
 32|LL
-33|LL
-34|LL
-35|LH
-36|HL
-37|LH
-38|LH
-39|LL
-40|LL
-41|LH
-42|HL
-43|LL
-44|HL
-45|LL
-46|HL
+33|HL
+34|LH
+35|LL
+36|LL
+37|HL
+38|HL
+39|HH
+40|HH
+41|HL
+42|LH
+43|LH
+44|LL
+45|LH
+46|LL
 47|LL
-48|LL
-49|HL
-50|LL
-51|HH
-(51 rows)
+48|HH
+49|LH
+50|HH
+51|LL
+52|LL
+(52 rows)
 _cdb_random_seeds
 
 (1 row)
 code|quads
-03|HH
-04|HH
-06|HH
-10|HH
-11|HH
-12|HL
-14|HH
-21|HH
-23|HL
-30|HH
-36|HL
-42|HL
-44|HL
-46|HL
-49|HL
-51|HH
-(16 rows)
+01|HH
+02|HL
+07|HH
+08|HH
+09|HH
+13|HL
+16|HH
+17|HH
+19|HH
+20|HH
+22|HH
+25|HH
+26|HH
+28|HH
+31|HH
+33|HL
+37|HL
+38|HL
+39|HH
+40|HH
+41|HL
+48|HH
+50|HH
+(23 rows)
 _cdb_random_seeds
 
 (1 row)
 code|quads
-01|LL
-02|LH
-05|LL
-07|LL
-08|LL
-09|LL
-13|LL
+03|LL
+04|LL
+05|LH
+06|LL
+10|LL
+11|LL
+12|LL
+14|LL
 15|LL
-16|LL
-17|LL
-18|LH
-19|LL
-20|LL
-22|LL
+18|LL
+21|LL
+23|LL
 24|LL
-25|LL
-26|LL
 27|LL
-28|LL
-29|LH
-31|LL
+29|LL
+30|LL
 32|LL
-33|LL
-34|LL
-35|LH
-37|LH
-38|LH
-39|LL
-40|LL
-41|LH
-43|LL
-45|LL
+34|LH
+35|LL
+36|LL
+42|LH
+43|LH
+44|LL
+45|LH
+46|LL
 47|LL
-48|LL
-50|LL
-(35 rows)
+49|LH
+51|LL
+52|LL
+(29 rows)
 _cdb_random_seeds
 
 (1 row)
 code|quads
-02|LH
-12|HL
-18|LH
-23|HL
-29|LH
-35|LH
-36|HL
-37|LH
-38|LH
-41|LH
-42|HL
-44|HL
-46|HL
-49|HL
-(14 rows)
+02|HL
+05|LH
+13|HL
+33|HL
+34|LH
+37|HL
+38|HL
+41|HL
+42|LH
+43|LH
+45|LH
+49|LH
+(12 rows)

src/pg/test/expected/04_dot_density_test.out

@@ -1,16 +0,0 @@
-WITH g AS (
-  SELECT ST_Buffer(ST_SetSRID(ST_MakePoint(0,0),4326)::geometry, 1000)::geometry AS g
-),
-points AS(
-  SELECT (
-    ST_Dump(
-      cdb_crankshaft.cdb_dot_density(g.g, 100)
-    )
-  ).geom AS p FROM g
-)
-SELECT count(*), sum(CASE WHEN ST_Contains(g,p) THEN 1 ELSE 0 END) FROM points, g
- count | sum 
--------+-----
-   100 | 100
-(1 row)
-

src/pg/test/expected/16_getis_test.out

@@ -0,0 +1,21 @@
+\pset format unaligned
+\set ECHO all
+\i test/fixtures/getis_data.sql
+SET client_min_messages TO WARNING;
+\set ECHO none
+_cdb_random_seeds
+
+(1 row)
+rowid|z_score|p_value
+9|-0.7862|0.0500
+22|-0.3955|0.0330
+33|2.7045|0.0050
+35|1.9524|0.0130
+36|-1.2056|0.0170
+37|3.4785|0.0020
+38|-1.4622|0.0020
+40|5.7098|0.0030
+46|3.4704|0.0120
+47|-0.9994|0.0320
+48|-1.3650|0.0340
+(11 rows)

src/pg/test/expected/18_outliers_test.out

@@ -0,0 +1,23 @@
+SET client_min_messages TO WARNING;
+\set ECHO none
+is_outlier|rowid
+t|11
+t|16
+t|17
+(3 rows)
+is_outlier|rowid
+t|16
+t|17
+(2 rows)
+ERROR:  Standard deviation of input data is zero
+is_outlier|rowid
+t|8
+t|11
+t|16
+(3 rows)
+is_outlier|rowid
+t|8
+t|9
+t|11
+t|16
+(4 rows)

src/pg/test/fixtures/getis_data.sql

@@ -0,0 +1,98 @@
+SET client_min_messages TO WARNING;
+\set ECHO none
+
+--
+-- Getis-Ord's G* test dataset, subsetted from PySAL examples:
+--   https://github.com/pysal/pysal/tree/952ea04029165048a774d9a1846cf86ad000c096/pysal/examples/stl
+--
+
+
+CREATE TABLE getis_data (
+    cartodb_id integer,
+    the_geom geometry(Geometry,4326),
+    hr8893 numeric
+);
+
+COPY getis_data (cartodb_id, the_geom, hr8893) FROM stdin;
+22	0106000020E61000000100000001030000000100000007000000000000E0B10056C0000000C0B8964340FFFFFFFF4C1756C00000002054964340000000A00F1E56C00000004072964340000000C02D1E56C0000000A0439B434000000060381E56C00000000036B04340000000E0E20056C0000000608CB04340000000E0B10056C0000000C0B8964340	10.8557430000000004
+32	0106000020E6100000010000000103000000010000000B000000FFFFFF1FC26656C0FFFFFFBFE25E4340000000A0D86656C0000000E0976F4340000000A03A6956C0000000C0966F434000000020526956C0000000E08A7F4340000000E0F26556C000000000C87F4340000000E0066656C0000000209C834340000000407F5056C0000000803C83434000000020635056C0000000E016814340000000A0F45056C0000000A0F980434000000060D25056C000000060FA5E4340FFFFFF1FC26656C0FFFFFFBFE25E4340	9.92424500000000087
+10	0106000020E610000001000000010300000001000000170000000000000002CD56C000000080CDCC4340000000A054D456C000000020CCD74340000000607ED756C000000000C1DC434000000020E6D756C00000006071E143400000004007BB56C00000000007E2434000000080FABA56C0000000A079EC43400000000040B856C0000000E0D6EB4340FFFFFF3FEEA456C0000000A037EC4340000000C0A9A556C0000000A0ADE7434000000040F3A656C0000000E09FE543400000004063A956C0000000E034DA4340FFFFFF9F04A956C00000008005D74340000000402FA756C00000008069D243400000004046A556C00000002068C84340000000009EA556C0FFFFFF7F3CC34340000000C0C3A756C000000080BCB543400000006082A756C00000004051B2434000000040AABC56C00000006046B343400000006053C256C0FFFFFF7FE2B84340000000E01EC456C000000080ABBC4340000000C0FDC556C0000000E0B3C3434000000000FFC956C000000060BBC643400000000002CD56C000000080CDCC4340	3.79060700000000006
+43	0106000020E6100000010000000103000000010000000F0000000000004025D856C000000020FA1A43400000008092E256C000000060481B434000000060BCE256C0000000C023144340000000A0D7E856C0FFFFFF1F1B14434000000020BEE956C0000000C030144340FFFFFF9FB0E956C0FFFFFF1F1425434000000000D4E956C0000000C00D5A4340000000A0D3E956C0000000202C5A43400000000004E656C00000004066574340000000E0EEE356C0000000A0E35643400000008099DF56C0000000601B5A43400000000033DB56C0000000804A5B43400000004001D856C00000006079594340000000E0A7D756C0000000E0553543400000004025D856C000000020FA1A4340	5.93309800000000021
+6	0106000020E6100000010000000103000000010000000F000000000000A00F4256C0000000E008D4434000000000674956C00000004015D44340000000608C4956C00000004098E64340FFFFFFBF434C56C0FFFFFF3F77E84340000000004C4E56C000000020E5E74340000000C0624E56C0FFFFFF3F97F5434000000020B44956C000000000AFF54340000000E0C64956C00000004009074440FFFFFF1F523056C0FFFFFFDF91074440000000C0EB2F56C000000040BBE54340000000E0B93056C0000000E09FE54340000000E0D63056C0000000007DDE4340000000E0213456C0000000005ADE4340000000802E3456C000000020F7D34340000000A00F4256C0000000E008D44340	9.04867300000000085
+16	0106000020E6100000010000000103000000010000001500000000000020D73356C000000060729B4340000000201F4956C000000000BE9B4340000000A0E34856C000000060CCAC434000000040094256C0FFFFFFDFB1AC4340000000A00F4256C0000000E008D44340000000802E3456C000000020F7D34340000000E0083456C0000000E0ADCA4340000000801D2E56C0000000A06FCA4340FFFFFF7F132E56C00000000079C34340000000607F2956C0000000402EC3434000000080652956C0000000A0EAC04340FFFFFF5FF22756C000000060E5C0434000000080F52756C000000080DDBE434000000020B32656C0000000E0E7BE434000000000AC2656C0FFFFFFFF38BD4340FFFFFFDFC12556C0000000C026BD434000000060C72556C000000060BAB9434000000040441E56C000000020C9B9434000000060381E56C00000000036B04340000000C02D1E56C0000000A0439B434000000020D73356C000000060729B4340	0.74509000000000003
+29	0106000020E610000001000000010300000001000000080000000000002025FD55C0FFFFFF1F7F6D434000000080C61056C0000000A04C6D4340000000A0631756C000000000D56D4340000000C05E1756C0FFFFFF5F24754340FFFFFFFF4C1756C00000002054964340000000E0B10056C0000000C0B89643400000006029FD55C000000080C09643400000002025FD55C0FFFFFF1F7F6D4340	3.12759000000000009
+54	0106000020E6100000010000000103000000010000000F000000FFFFFF1F090C57C0000000202E024340000000E0AA0B57C00000004000154340000000A0C10C57C0000000802015434000000060850C57C000000080EC244340FFFFFF9FB0E956C0FFFFFF1F1425434000000020BEE956C0000000C030144340000000A0D7E856C0FFFFFF1F1B144340FFFFFF3FF2E856C0000000A0A3064340FFFFFFDFD2F956C000000040E2064340000000C01EFB56C0000000201406434000000060BBFB56C0000000809A044340000000605AFD56C0000000E059054340000000405FFE56C0000000E06B014340000000A0F60157C0000000C081014340FFFFFF1F090C57C0000000202E024340	2.06432400000000005
+13	0106000020E6100000010000000103000000010000000F000000000000009EA556C0FFFFFF7F3CC343400000004046A556C00000002068C84340000000402FA756C00000008069D24340FFFFFF9F04A956C00000008005D743400000004063A956C0000000E034DA434000000040F3A656C0000000E09FE54340000000E0F09E56C0000000005AE54340FFFFFF3F069F56C000000040B5E04340FFFFFF1F0E9856C000000020A3E0434000000060109856C0000000E07ED5434000000040E29556C0000000A077D54340FFFFFFDFDC9556C0000000A0EDD1434000000080399356C0000000A0D8D1434000000020699356C0000000E029C34340000000009EA556C0FFFFFF7F3CC34340	0
+28	0106000020E61000000100000001030000000100000008000000000000C05E1756C0FFFFFF5F24754340000000C0DD2C56C0000000407A75434000000080DE3356C0000000406375434000000020D73356C000000060729B4340000000C02D1E56C0000000A0439B4340000000A00F1E56C00000004072964340FFFFFFFF4C1756C00000002054964340000000C05E1756C0FFFFFF5F24754340	1.57115800000000005
+36	0106000020E6100000010000000103000000010000000D00000000000000EE2C56C000000060424E434000000040F72C56C000000000486A4340000000C0DD2C56C0000000407A754340000000C05E1756C0FFFFFF5F24754340000000A0631756C000000000D56D434000000080C61056C0000000A04C6D4340000000C0BE1056C0000000A0065F4340000000407C1256C0FFFFFF7FA75E434000000000BD1156C000000020A954434000000040D01256C0000000605C524340000000404F1256C000000040734F434000000040011156C000000000AF4D434000000000EE2C56C000000060424E4340	0
+68	0106000020E61000000100000001030000000100000006000000000000809F2D56C00000002078CD4240000000E0F64256C0FFFFFFDF38CD424000000000CE4956C00000004053CD4240000000C0E94956C000000080CDEE424000000020682D56C0FFFFFF7F00EF4240000000809F2D56C00000002078CD4240	3.1461920000000001
+27	0106000020E6100000010000000103000000010000000D000000000000407F5056C0000000803C83434000000060615056C000000040A99B4340000000201F4956C000000000BE9B434000000020D73356C000000060729B434000000080DE3356C00000004063754340000000C0DD2C56C0000000407A75434000000040F72C56C000000000486A4340000000202B4956C000000080AC69434000000040454956C000000040E25E434000000060D25056C000000060FA5E4340000000A0F45056C0000000A0F980434000000020635056C0000000E016814340000000407F5056C0000000803C834340	1.5920399999999999
+40	0106000020E6100000010000000103000000010000000F00000000000000B89056C0000000C03144434000000000419456C0000000A03D4A434000000020609356C000000020CC55434000000020578F56C0000000A0725D4340000000004D8C56C0000000C05E614340000000E0F48A56C000000020B164434000000060CA8756C00000002077664340000000C0A68856C0000000C08A64434000000020758A56C0000000E0F4624340000000A0948C56C0000000E0AA5C4340FFFFFF7FF18C56C000000080A5594340000000C0BF8B56C0000000A052544340000000E0C18B56C0000000601D4E4340000000A06B8F56C0000000000A48434000000000B89056C0000000C031444340	45.9054059999999993
+52	0106000020E61000000100000001030000000100000008000000000000E0562D56C0000000208910434000000020864856C00000000054104340FFFFFF9FB84956C0000000409110434000000080A84956C0000000A05B1B4340000000803D4956C0FFFFFF9FB63C434000000080062D56C0FFFFFF7FAE3C434000000060492D56C0000000603E214340000000E0562D56C00000002089104340	4.94153299999999973
+59	0106000020E6100000010000000103000000010000000F0000000000006010C756C000000020A7DE4240FFFFFF9FF7C956C0FFFFFF7F75DE424000000060FBC956C0000000A01ED94240000000C080CA56C0000000001BD94240000000C086D456C000000040CED942400000000065D456C00000004041E442400000008044E256C0000000E0BEE44240FFFFFFFF04E256C0FFFFFFBFD013434000000060BCE256C0000000C0231443400000008092E256C000000060481B43400000004025D856C000000020FA1A4340FFFFFF5F2CD856C0000000E0381A43400000000009D656C0FFFFFFDF1C1A4340000000A08DC656C000000000241A43400000006010C756C000000020A7DE4240	6.82879400000000025
+1	0106000020E61000000100000001030000000100000010000000FFFFFF3F746556C00000002049FD434000000080316556C0000000A0240C4440000000C0A76656C000000000330C444000000000C76656C0000000606C11444000000000106756C000000000A0294440000000C0375156C0000000402F2A444000000020305156C0000000E08D244440000000E0E54956C00000004094244440000000E0C64956C0000000400907444000000020B44956C000000000AFF54340000000C0624E56C0FFFFFF3F97F54340000000E05B5A56C00000006082F5434000000060655A56C00000002040F7434000000060B15F56C0000000807CF74340000000008A5F56C0000000C076FD4340FFFFFF3F746556C00000002049FD4340	1.62445799999999996
+2	0106000020E6100000010000000103000000010000001000000000000080FABA56C0000000A079EC43400000004007BB56C00000000007E2434000000020E6D756C00000006071E14340000000006ED856C000000000E2E64340FFFFFF7FBFDC56C00000006078EE434000000000DDDC56C0000000A04FF1434000000080C7DB56C0000000206FF34340000000808BDB56C0000000C0FEF54340000000A09FDC56C0000000A018F94340000000C02FDF56C000000080BC004440000000A041E056C0000000008A084440FFFFFF3F08E156C0000000C038114440000000406BE056C0000000A0A8194440FFFFFFBFF5BA56C0000000E02A1944400000004014BB56C000000060E70D444000000080FABA56C0000000A079EC4340	2.25549199999999983
+3	0106000020E61000000100000001030000000100000018000000000000A0D87F56C000000080FEF3434000000020048056C0FFFFFF7FB00E444000000080CB7D56C0000000E0F411444000000080237B56C0000000800812444000000060227956C0000000400F114440000000A04A7856C00000002084124440000000C0667756C0000000E0B810444000000080187656C0000000A001124440000000A0037556C0FFFFFF5F36104440000000005C7356C0000000206510444000000080D27156C00000004081114440000000A0657056C000000040EE104440000000A0EF6D56C000000060AE12444000000000BE6C56C0000000E073124440000000802A6A56C0000000402115444000000000C76656C0000000606C114440000000C0A76656C000000000330C444000000080316556C0000000A0240C4440FFFFFF3F746556C00000002049FD4340000000602C6D56C00000004009FD4340FFFFFF5F786D56C0FFFFFFDFC7F54340000000E0447156C000000080A6F54340000000A09C7156C0FFFFFF9F41F44340000000A0D87F56C000000080FEF34340	1.46788999999999992
+4	0106000020E610000001000000010300000001000000160000000000008076A556C000000020BCF0434000000060BCA256C0FFFFFF5F7BF5434000000060F5A056C000000040AEFE4340000000204C9C56C00000008095024440000000A0609B56C000000060EC084440FFFFFFFFC49956C0000000C00C0A444000000000E19856C000000040490F444000000040A79656C0000000805C10444000000080199456C0000000E0120E4440FFFFFFBF879256C00000006094084440000000A0699156C00000006012084440000000807E9056C000000000F408444000000060378F56C00000000049074440000000A0E98C56C0FFFFFF5FE4074440000000E0F48B56C0000000201009444000000020618856C000000040ED08444000000020A88556C000000060CF0A4440FFFFFFDF0C8456C000000080250D444000000020048056C0FFFFFF7FB00E4440000000A0D87F56C000000080FEF34340FFFFFF3FDF7F56C0000000C047F043400000008076A556C000000020BCF04340	2.4842559999999998
+5	0106000020E61000000100000001030000000100000012000000FFFFFF3FEEA456C0000000A037EC43400000000040B856C0000000E0D6EB434000000080FABA56C0000000A079EC43400000004014BB56C000000060E70D444000000020B9AC56C0000000409F0D4440000000A094AD56C0000000005B0A4440000000A07FAB56C0000000A0C1094440FFFFFFBFA8AB56C000000060DD054440000000E04BA956C0FFFFFFFFB30344400000002042A756C0FFFFFFBF900344400000002005A756C0000000409502444000000060D2A756C000000040EE014440000000C0D9A656C0000000008E004440000000E0C0A656C0000000E0F1FD434000000060F5A056C000000040AEFE434000000060BCA256C0FFFFFF5F7BF543400000008076A556C000000020BCF04340FFFFFF3FEEA456C0000000A037EC4340	0
+15	0106000020E6100000010000000103000000010000000B000000000000806CDC56C00000006020A9434000000080A4ED56C000000020E3A94340000000A050EE56C0FFFFFFDF8BAB434000000000C4ED56C00000000058CD4340000000A039EE56C0000000C062CD4340000000C041EE56C00000006068D44340000000E031EE56C000000080D5D74340000000A054D456C000000020CCD743400000000002CD56C000000080CDCC434000000040CDDD56C0000000407EBA4340000000806CDC56C00000006020A94340	1.9498120000000001
+7	0106000020E6100000010000000103000000010000001C000000000000C00D6D56C0000000A096C34340000000E0847B56C00000000075C3434000000060747B56C00000002076C74340000000601B7F56C0000000203CDA4340FFFFFF3FDF7F56C0000000C047F04340000000A0D87F56C000000080FEF34340000000A09C7156C0FFFFFF9F41F44340000000E0447156C000000080A6F54340FFFFFF5F786D56C0FFFFFFDFC7F54340000000602C6D56C00000004009FD4340FFFFFF3F746556C00000002049FD4340000000008A5F56C0000000C076FD434000000060B15F56C0000000807CF7434000000060655A56C00000002040F74340000000E05B5A56C00000006082F54340000000C0624E56C0FFFFFF3F97F54340000000004C4E56C000000020E5E7434000000080B55056C0000000C010E94340FFFFFFDF215256C0000000C096E5434000000020475556C0000000A0D3E14340000000A0245A56C0000000200DDF434000000060E75B56C000000000C0DF434000000000F25B56C0000000C0DBD74340FFFFFF5F635F56C0FFFFFFFF91D74340000000E04F5F56C000000020C1D24340FFFFFF7F996256C00000004094D24340000000A08E6256C0000000C0AAC34340000000C00D6D56C0000000A096C34340	6.02948899999999988
+8	0106000020E6100000010000000103000000010000000E000000FFFFFF1F72F656C000000080A1D443400000004048F656C0000000008EF94340000000A09FDC56C0000000A018F94340000000808BDB56C0000000C0FEF5434000000080C7DB56C0000000206FF3434000000000DDDC56C0000000A04FF14340FFFFFF7FBFDC56C00000006078EE4340000000006ED856C000000000E2E6434000000020E6D756C00000006071E14340000000607ED756C000000000C1DC4340000000A054D456C000000020CCD74340000000E031EE56C000000080D5D74340000000C041EE56C00000006068D44340FFFFFF1F72F656C000000080A1D44340	1.8003849999999999
+9	0106000020E6100000010000000103000000010000001200000000000040E88956C00000004046C3434000000020699356C0000000E029C3434000000080399356C0000000A0D8D14340FFFFFFDFDC9556C0000000A0EDD1434000000040E29556C0000000A077D5434000000060109856C0000000E07ED54340FFFFFF1F0E9856C000000020A3E04340FFFFFF3F069F56C000000040B5E04340000000E0F09E56C0000000005AE5434000000040F3A656C0000000E09FE54340000000C0A9A556C0000000A0ADE74340FFFFFF3FEEA456C0000000A037EC43400000008076A556C000000020BCF04340FFFFFF3FDF7F56C0000000C047F04340000000601B7F56C0000000203CDA434000000060747B56C00000002076C74340000000E0847B56C00000000075C3434000000040E88956C00000004046C34340	4.58125099999999996
+30	0106000020E6100000010000000103000000010000000D0000000000000033DB56C0000000804A5B43400000008099DF56C0000000601B5A4340000000E0EEE356C0000000A0E35643400000000004E656C00000004066574340000000A0D3E956C0000000202C5A4340000000A02EE956C0FFFFFF1F09884340000000A0D9E856C0FFFFFFDF03934340000000A0B8DA56C000000060E592434000000080F4D056C0000000605F9243400000000038D156C0000000A0817F434000000000A0D156C0000000A0FF6B4340000000E0C8DA56C000000080986C43400000000033DB56C0000000804A5B4340	4.41689600000000038
+11	0106000020E61000000100000001030000000100000013000000000000A08E6256C0000000C0AAC34340FFFFFF7F996256C00000004094D24340000000E04F5F56C000000020C1D24340FFFFFF5F635F56C0FFFFFFFF91D7434000000000F25B56C0000000C0DBD7434000000060E75B56C000000000C0DF4340000000A0245A56C0000000200DDF434000000020475556C0000000A0D3E14340FFFFFFDF215256C0000000C096E5434000000080B55056C0000000C010E94340000000004C4E56C000000020E5E74340FFFFFFBF434C56C0FFFFFF3F77E84340000000608C4956C00000004098E6434000000000674956C00000004015D44340000000A00F4256C0000000E008D4434000000040094256C0FFFFFFDFB1AC4340000000A0E34856C000000060CCAC434000000020666256C0FFFFFF9FBFAC4340000000A08E6256C0000000C0AAC34340	1.44743599999999994
+12	0106000020E6100000010000000103000000010000001400000000000040441E56C000000020C9B9434000000060C72556C000000060BAB94340FFFFFFDFC12556C0000000C026BD434000000000AC2656C0FFFFFFFF38BD434000000020B32656C0000000E0E7BE434000000080F52756C000000080DDBE4340FFFFFF5FF22756C000000060E5C0434000000080652956C0000000A0EAC04340000000607F2956C0000000402EC34340FFFFFF7F132E56C00000000079C34340000000801D2E56C0000000A06FCA4340000000E0083456C0000000E0ADCA4340000000802E3456C000000020F7D34340000000E0213456C0000000005ADE4340000000E0D63056C0000000007DDE4340000000E0B93056C0000000E09FE54340000000C0EB2F56C000000040BBE54340FFFFFF5F741E56C00000008040E54340000000605A1E56C0000000E042D3434000000040441E56C000000020C9B94340	1.19196600000000008
+14	0106000020E6100000010000000103000000010000000A000000000000E0E20056C0000000608CB0434000000060381E56C00000000036B0434000000040441E56C000000020C9B94340000000605A1E56C0000000E042D3434000000040BB0356C00000000030D4434000000060CF0356C000000080C9D74340FFFFFFBFFDFD55C0000000E02BD84340000000A0C5FD55C0000000400DBE434000000040E60056C0FFFFFF9F20BE4340000000E0E20056C0000000608CB04340	1.60801700000000003
+17	0106000020E61000000100000001030000000100000011000000000000A0B8DA56C000000060E5924340000000806CDC56C00000006020A9434000000040CDDD56C0000000407EBA43400000000002CD56C000000080CDCC434000000000FFC956C000000060BBC64340000000C0FDC556C0000000E0B3C34340000000E01EC456C000000080ABBC43400000006053C256C0FFFFFF7FE2B8434000000040AABC56C00000006046B34340000000806EB656C0000000A0DBAC4340000000C0E0B156C0000000A0FDA54340000000C03CAF56C000000040B89F4340000000A0DEAE56C0FFFFFF7FC49C4340000000E0F3CB56C0FFFFFF9F039D4340000000205CCC56C0000000805392434000000080F4D056C0000000605F924340000000A0B8DA56C000000060E5924340	4.17331800000000008
+18	0106000020E6100000010000000103000000010000000E000000FFFFFF3FF56C56C000000020977F4340000000A07A6D56C0000000605DAD4340000000800F6D56C0000000A06CAD4340000000C00D6D56C0000000A096C34340000000A08E6256C0000000C0AAC3434000000020666256C0FFFFFF9FBFAC4340000000A0E34856C000000060CCAC4340000000201F4956C000000000BE9B434000000060615056C000000040A99B4340000000407F5056C0000000803C834340000000E0066656C0000000209C834340000000E0F26556C000000000C87F434000000020526956C0000000E08A7F4340FFFFFF3FF56C56C000000020977F4340	3.78325200000000006
+19	0106000020E61000000100000001030000000100000009000000000000A0B78956C0000000A0BD7F4340000000C0C08956C0000000000BA1434000000040E88956C00000004046C34340000000E0847B56C00000000075C34340000000C00D6D56C0000000A096C34340000000800F6D56C0000000A06CAD4340000000A07A6D56C0000000605DAD4340FFFFFF3FF56C56C000000020977F4340000000A0B78956C0000000A0BD7F4340	2.08513599999999988
+20	0106000020E61000000100000001030000000100000015000000000000C0C08956C0000000000BA1434000000060188D56C00000008040A04340000000C0278D56C0000000E0E29C4340000000E04C9456C000000080CC9C4340000000A0579456C0000000E0B4964340FFFFFFFF819F56C0000000806D9643400000006068A056C0FFFFFF1FBB944340000000203AA156C0000000E0CA9743400000002061A456C000000080AE974340000000C08CA556C0000000A0BB964340FFFFFF9F44A556C0000000609794434000000040EFA656C0000000C00C8F43400000004050A756C0FFFFFFBFEA9343400000000062A656C0FFFFFFFF6A9B434000000080DCA756C0000000208AAE43400000006082A756C00000004051B24340000000C0C3A756C000000080BCB54340000000009EA556C0FFFFFF7F3CC3434000000020699356C0000000E029C3434000000040E88956C00000004046C34340000000C0C08956C0000000000BA14340	2.17630200000000018
+21	0106000020E6100000010000000103000000010000001B00000000000040EFA656C0000000C00C8F4340FFFFFF5F1DA756C000000020C58D434000000020E1A456C0000000E00F84434000000020D1A456C0000000A0B2804340FFFFFFDF3AA356C000000040247C4340000000800BA256C0FFFFFF5F987A434000000020409F56C000000000CD7B4340000000E0119E56C000000060C67A434000000080F2A156C000000040207243400000004080A456C0000000A0876F43400000004024A856C0000000E0BD704340FFFFFFDFCEAA56C000000060B67743400000004030AD56C000000060D6844340FFFFFF1F49AD56C00000006072874340000000802FAC56C000000060FE8B434000000000DFAD56C00000008075924340FFFFFFDFF6AD56C00000006012994340000000A0DEAE56C0FFFFFF7FC49C4340000000C03CAF56C000000040B89F4340000000C0E0B156C0000000A0FDA54340000000806EB656C0000000A0DBAC434000000040AABC56C00000006046B343400000006082A756C00000004051B2434000000080DCA756C0000000208AAE43400000000062A656C0FFFFFFFF6A9B43400000004050A756C0FFFFFFBFEA93434000000040EFA656C0000000C00C8F4340	6.30934699999999982
+23	0106000020E6100000010000000103000000010000000B000000FFFFFF1F491457C000000060E99F4340FFFFFF5F1C1457C00000000075AC4340000000A050EE56C0FFFFFFDF8BAB434000000080A4ED56C000000020E3A94340000000806CDC56C00000006020A94340000000A0B8DA56C000000060E5924340000000A0D9E856C0FFFFFFDF03934340000000A02EE956C0FFFFFF1F0988434000000020290757C0FFFFFF3F92884340000000A0B00657C000000080FE9E4340FFFFFF1F491457C000000060E99F4340	4.21135400000000004
+24	0106000020E61000000100000001030000000100000019000000000000A0B78956C0000000A0BD7F4340FFFFFF1FDD9156C000000060B47F434000000000DA9156C0000000205D764340000000A0769456C00000006063764340000000C06F9A56C0000000A02D7B4340000000E0119E56C000000060C67A434000000020409F56C000000000CD7B4340000000800BA256C0FFFFFF5F987A4340FFFFFFDF3AA356C000000040247C434000000020D1A456C0000000A0B280434000000020E1A456C0000000E00F844340FFFFFF5F1DA756C000000020C58D434000000040EFA656C0000000C00C8F4340FFFFFF9F44A556C00000006097944340000000C08CA556C0000000A0BB9643400000002061A456C000000080AE974340000000203AA156C0000000E0CA9743400000006068A056C0FFFFFF1FBB944340FFFFFFFF819F56C0000000806D964340000000A0579456C0000000E0B4964340000000E04C9456C000000080CC9C4340000000C0278D56C0000000E0E29C434000000060188D56C00000008040A04340000000C0C08956C0000000000BA14340000000A0B78956C0000000A0BD7F4340	0.804810000000000025
+47	0106000020E61000000100000001030000000100000009000000FFFFFF9FD60956C0000000003521434000000020F21756C000000040DA20434000000060492D56C0000000603E21434000000080062D56C0FFFFFF7FAE3C434000000000EE2C56C000000060424E434000000040011156C000000000AF4D434000000040780956C0000000005A4D4340000000808A0956C000000080C1494340FFFFFF9FD60956C00000000035214340	0.969790999999999959
+25	0106000020E6100000010000000103000000010000001E000000000000A0491957C0FFFFFF9F355E434000000020381A57C0000000A05B61434000000000201957C0000000803D654340000000802D1957C000000040E867434000000060B41B57C0000000C078694340000000A0611E57C000000040A06E4340000000C0FF1F57C00000006083754340000000803C2457C0000000E0ED7B4340000000A0332457C00000004047804340FFFFFF9FB41B57C0FFFFFF5F08A04340FFFFFF1F491457C000000060E99F4340000000A0B00657C000000080FE9E434000000020290757C0FFFFFF3F92884340FFFFFFBFC70857C000000020EC874340000000C0F20957C000000020DE764340000000C0E50A57C000000040E172434000000000260A57C0000000A03A71434000000000750A57C000000080796F4340FFFFFF7F310957C00000000064684340FFFFFF3FFF0B57C000000020765E434000000000060E57C000000000B45B434000000000640E57C00000000022594340000000A0C40C57C0000000801D57434000000040600C57C0000000C049544340FFFFFF5F100E57C000000000B851434000000060091157C0000000E05353434000000020B21257C0FFFFFFDF48554340FFFFFF3FC11657C00000004060564340000000005F1657C0FFFFFF5FD95B4340000000A0491957C0FFFFFF9F355E4340	3.21533099999999994
+26	0106000020E6100000010000000103000000010000001D000000000000008BBD56C0000000608A6F43400000006075C756C0000000A0EE6F43400000000093C756C0FFFFFF3FF3764340FFFFFFBFD7CC56C0FFFFFF7F7277434000000020BECC56C0FFFFFFFF1B7F43400000000038D156C0000000A0817F434000000080F4D056C0000000605F924340000000205CCC56C00000008053924340000000E0F3CB56C0FFFFFF9F039D4340000000A0DEAE56C0FFFFFF7FC49C4340FFFFFFDFF6AD56C0000000601299434000000000DFAD56C00000008075924340000000802FAC56C000000060FE8B4340FFFFFF1F49AD56C000000060728743400000004030AD56C000000060D6844340FFFFFFDFCEAA56C000000060B6774340FFFFFFDF6EAC56C000000060557743400000002088AC56C000000060887543400000000071AD56C000000080B07543400000004047AD56C0000000C04C744340000000C0FDAE56C0000000406D754340000000C0DAAE56C0000000802B774340000000C070B256C0000000A0FE754340FFFFFFBF86B356C000000040A3744340000000A00EB456C0000000A0907043400000008058B856C000000000FB7143400000004020BC56C0000000C09A714340000000C058BD56C0000000C09E724340000000008BBD56C0000000608A6F4340	2.83366400000000018
+38	0106000020E6100000010000000103000000010000000E000000000000605FFD55C000000020A5494340000000808A0956C000000080C149434000000040780956C0000000005A4D434000000040011156C000000000AF4D4340000000404F1256C000000040734F434000000040D01256C0000000605C52434000000000BD1156C000000020A9544340000000407C1256C0FFFFFF7FA75E4340000000C0BE1056C0000000A0065F434000000080C61056C0000000A04C6D43400000002025FD55C0FFFFFF1F7F6D434000000080B9FA55C0000000607E6D434000000020A9FA55C00000008093494340000000605FFD55C000000020A5494340	1.00387500000000007
+31	0106000020E6100000010000000103000000010000001B000000FFFFFF5F100E57C000000000B851434000000040600C57C0000000C049544340000000A0C40C57C0000000801D57434000000000640E57C0000000002259434000000000060E57C000000000B45B4340FFFFFF3FFF0B57C000000020765E4340FFFFFF7F310957C0000000006468434000000000750A57C000000080796F434000000000260A57C0000000A03A714340000000C0E50A57C000000040E1724340000000C0F20957C000000020DE764340FFFFFFBFC70857C000000020EC87434000000020290757C0FFFFFF3F92884340000000A02EE956C0FFFFFF1F09884340000000A0D3E956C0000000202C5A434000000000D4E956C0000000C00D5A4340FFFFFF7F6BEF56C0000000A0605A4340FFFFFFFFACF056C000000040975843400000008054F356C0FFFFFFDFFB56434000000060C1F656C0000000E0725643400000000014FD56C000000000744C4340000000E005FF56C000000020904B434000000040220257C0000000405C484340000000E0880657C0FFFFFFDFF147434000000000EA0A57C0000000407C4A4340000000E0B10C57C000000020054D4340FFFFFF5F100E57C000000000B8514340	3.01748599999999989
+33	0106000020E6100000010000000103000000010000001400000000000060CA8756C00000002077664340000000603D8756C0000000C04C6A434000000000808856C0FFFFFF1F306D4340000000809C8F56C0000000A00E75434000000000DA9156C0000000205D764340FFFFFF1FDD9156C000000060B47F4340000000A0B78956C0000000A0BD7F4340FFFFFF3FF56C56C000000020977F434000000020526956C0000000E08A7F4340000000A03A6956C0000000C0966F4340000000A0D86656C0000000E0976F4340FFFFFF1FC26656C0FFFFFFBFE25E434000000000AD6656C0000000408054434000000060BA6D56C00000006031544340000000C0BF8B56C0000000A052544340FFFFFF7FF18C56C000000080A5594340000000A0948C56C0000000E0AA5C434000000020758A56C0000000E0F4624340000000C0A68856C0000000C08A64434000000060CA8756C00000002077664340	7.97395700000000041
+34	0106000020E61000000100000001030000000100000015000000FFFFFF5FFEBD56C0000000C0ED454340000000C0E0C056C0000000401048434000000040E0C356C0000000C0AC4D434000000060B0C556C000000000094E4340000000201BC956C0000000E0D74C4340FFFFFF3F1ECD56C0000000204D4E4340000000806BCE56C0000000600150434000000020DDCF56C0000000A015544340000000E0F7D256C0000000801D584340000000E067D556C000000080E65943400000004001D856C000000060795943400000000033DB56C0000000804A5B4340000000E0C8DA56C000000080986C434000000000A0D156C0000000A0FF6B43400000000038D156C0000000A0817F434000000020BECC56C0FFFFFFFF1B7F4340FFFFFFBFD7CC56C0FFFFFF7F727743400000000093C756C0FFFFFF3FF37643400000006075C756C0000000A0EE6F4340000000008BBD56C0000000608A6F4340FFFFFF5FFEBD56C0000000C0ED454340	5.00546399999999991
+35	0106000020E61000000100000001030000000100000031000000FFFFFF9F36AF56C0000000E037514340000000602AB256C0FFFFFFDFE74943400000008057B356C000000020CF4A434000000060A8B456C0000000C07D4A4340000000005FBA56C00000006033454340FFFFFF5FFEBD56C0000000C0ED454340000000008BBD56C0000000608A6F4340000000C058BD56C0000000C09E7243400000004020BC56C0000000C09A7143400000008058B856C000000000FB714340000000A00EB456C0000000A090704340FFFFFFBF86B356C000000040A3744340000000C070B256C0000000A0FE754340000000C0DAAE56C0000000802B774340000000C0FDAE56C0000000406D7543400000004047AD56C0000000C04C7443400000000071AD56C000000080B07543400000002088AC56C00000006088754340FFFFFFDF6EAC56C00000006055774340FFFFFFDFCEAA56C000000060B67743400000004024A856C0000000E0BD7043400000004080A456C0000000A0876F434000000080F2A156C00000004020724340000000E0119E56C000000060C67A4340000000C06F9A56C0000000A02D7B4340000000A0769456C0000000606376434000000000DA9156C0000000205D764340000000809C8F56C0000000A00E75434000000000808856C0FFFFFF1F306D4340000000603D8756C0000000C04C6A434000000060CA8756C0000000207766434000000060A28856C0000000A04C69434000000020CB8C56C0000000409A69434000000080AB9056C0000000608F6D4340FFFFFFFF6D9256C0000000204F714340000000C05C9456C000000040DD714340000000E0A39556C0000000A065704340000000400B9756C000000040A26A434000000040CC9956C0000000A0AA694340FFFFFFFFBA9B56C000000060646A4340000000A0EF9C56C0FFFFFF1FCB694340000000605F9F56C0000000202B614340000000402CA256C000000000995C4340000000C00CA356C00000006094584340000000A08CA656C0000000E04C57434000000000F0A856C000000020B5584340000000A086AB56C0FFFFFF5FCD5643400000004012AC56C0000000004C544340FFFFFF9F36AF56C0000000E037514340	2.46389099999999983
+37	0106000020E6100000010000000103000000010000002A0000000000006054AF56C0000000C0573B4340FFFFFF9F36AF56C0000000E0375143400000004012AC56C0000000004C544340000000A086AB56C0FFFFFF5FCD56434000000000F0A856C000000020B5584340000000A08CA656C0000000E04C574340000000C00CA356C00000006094584340000000402CA256C000000000995C4340000000605F9F56C0000000202B614340000000A0EF9C56C0FFFFFF1FCB694340FFFFFFFFBA9B56C000000060646A434000000040CC9956C0000000A0AA694340000000400B9756C000000040A26A4340000000E0A39556C0000000A065704340000000C05C9456C000000040DD714340FFFFFFFF6D9256C0000000204F71434000000080AB9056C0000000608F6D434000000020CB8C56C0000000409A69434000000060A28856C0000000A04C69434000000060CA8756C00000002077664340000000E0F48A56C000000020B1644340000000004D8C56C0000000C05E61434000000020578F56C0000000A0725D434000000020609356C000000020CC55434000000000419456C0000000A03D4A434000000000B89056C0000000C031444340000000A0029156C00000006064424340FFFFFF5F519356C0000000A0B336434000000020BC9556C00000004007324340000000406F9656C00000008005364340FFFFFF1FAA9556C0000000A07439434000000040129A56C0000000C07E3A4340FFFFFF1FE19A56C000000040623D4340FFFFFF3F2B9A56C000000000263E4340000000E0379A56C00000002001404340FFFFFFFFF5A556C0000000801A404340000000C049A756C0000000A07B3C4340000000801EAA56C000000020A63D4340FFFFFFDFD4AA56C000000060A538434000000060CFAB56C000000040BE384340000000202CAC56C000000040A33B43400000006054AF56C0000000C0573B4340	7.37797400000000003
+39	0106000020E61000000100000001030000000100000008000000000000A04E4956C0000000E03140434000000040454956C000000040E25E4340000000202B4956C000000080AC69434000000040F72C56C000000000486A434000000000EE2C56C000000060424E434000000080062D56C0FFFFFF7FAE3C4340000000803D4956C0FFFFFF9FB63C4340000000A04E4956C0000000E031404340	3.19004699999999985
+60	0106000020E6100000010000000103000000010000000A0000000000006098A956C0000000C019DE42400000006010C756C000000020A7DE4240000000A08DC656C000000000241A43400000002017B256C0000000209F1A434000000080FEAB56C0000000C0710E434000000080D3AB56C0FFFFFF5F2F0C434000000080FFA956C000000080EA0C43400000006018AA56C0000000E0FF0A434000000060E4A856C000000080440A43400000006098A956C0000000C019DE4240	3.26394699999999993
+41	0106000020E6100000010000000103000000010000001700000000000040454956C000000040E25E4340000000A04E4956C0000000E031404340000000A0FB5056C0000000001A414340000000E00D5356C0000000004C404340000000A0DA5656C000000040A4414340000000A07D5956C0000000A0833E4340000000608F5B56C0000000C0273F4340000000E0405D56C0000000803A3E4340000000E0CB5E56C000000040AD3B4340000000E08A6156C0000000A0793D434000000060B56256C0000000A09A3C4340000000A0A06456C0000000A0963D434000000020996756C000000060AB3B4340000000E0156856C0000000A085394340FFFFFF3F536956C0000000C06A384340FFFFFF5F876A56C0000000809638434000000080DD6A56C000000020BB364340000000806B6D56C0000000E09E35434000000060BA6D56C0000000603154434000000000AD6656C00000004080544340FFFFFF1FC26656C0FFFFFFBFE25E434000000060D25056C000000060FA5E434000000040454956C000000040E25E4340	2.44759700000000002
+42	0106000020E610000001000000010300000001000000210000000000000014FD56C000000000744C4340000000C0D70157C000000020CE464340000000E0CC0057C0000000A0ED404340FFFFFF1F3C0257C0000000A0CE3C434000000060C80457C0000000E02D3C4340000000A0020757C000000000763A4340000000602E0957C000000060EF3B434000000020B90A57C000000060DE3B434000000060340A57C0000000402D38434000000060310857C00000004012354340000000E0F50757C0000000E095324340000000E0D40857C000000080DD304340000000E0690B57C0FFFFFF3F3B30434000000000520C57C000000060732E4340000000208D0C57C000000000AB2A434000000060D80E57C000000080CC2A4340000000C05C1057C0000000209229434000000040131257C000000040C52A4340000000C0281A57C000000080262B4340000000E0D81957C000000000F6354340FFFFFF9FB91F57C0FFFFFFDF7D364340000000A0491957C0FFFFFF9F355E4340000000005F1657C0FFFFFF5FD95B4340FFFFFF3FC11657C0000000406056434000000020B21257C0FFFFFFDF4855434000000060091157C0000000E053534340FFFFFF5F100E57C000000000B8514340000000E0B10C57C000000020054D434000000000EA0A57C0000000407C4A4340000000E0880657C0FFFFFFDFF147434000000040220257C0000000405C484340000000E005FF56C000000020904B43400000000014FD56C000000000744C4340	1.29495800000000005
+44	0106000020E6100000010000000103000000010000001700000000000060850C57C000000080EC244340000000208D0C57C000000000AB2A434000000000520C57C000000060732E4340000000E0690B57C0FFFFFF3F3B304340000000E0D40857C000000080DD304340000000E0F50757C0000000E09532434000000060310857C0000000401235434000000060340A57C0000000402D38434000000020B90A57C000000060DE3B4340000000602E0957C000000060EF3B4340000000A0020757C000000000763A434000000060C80457C0000000E02D3C4340FFFFFF1F3C0257C0000000A0CE3C4340000000E0CC0057C0000000A0ED404340000000C0D70157C000000020CE4643400000000014FD56C000000000744C434000000060C1F656C0000000E0725643400000008054F356C0FFFFFFDFFB564340FFFFFFFFACF056C00000004097584340FFFFFF7F6BEF56C0000000A0605A434000000000D4E956C0000000C00D5A4340FFFFFF9FB0E956C0FFFFFF1F1425434000000060850C57C000000080EC244340	4.13399699999999992
+45	0106000020E610000001000000010300000001000000190000000000002017B256C0000000209F1A4340000000A08DC656C000000000241A43400000000009D656C0FFFFFFDF1C1A4340FFFFFF5F2CD856C0000000E0381A43400000004025D856C000000020FA1A4340000000E0A7D756C0000000E0553543400000004001D856C00000006079594340000000E067D556C000000080E6594340000000E0F7D256C0000000801D58434000000020DDCF56C0000000A015544340000000806BCE56C00000006001504340FFFFFF3F1ECD56C0000000204D4E4340000000201BC956C0000000E0D74C434000000060B0C556C000000000094E434000000040E0C356C0000000C0AC4D4340000000C0E0C056C00000004010484340FFFFFF5FFEBD56C0000000C0ED454340000000005FBA56C0000000603345434000000060A8B456C0000000C07D4A43400000008057B356C000000020CF4A4340000000602AB256C0FFFFFFDFE7494340FFFFFF9F36AF56C0000000E0375143400000006054AF56C0000000C0573B43400000006067AF56C000000040593243400000002017B256C0000000209F1A4340	4.29831099999999999
+46	0106000020E6100000010000000103000000010000001200000000000060DE7956C0000000E08D1C4340000000C0897B56C0000000806323434000000060177B56C0000000807E244340000000204D7A56C0000000A0CD234340FFFFFF7FB97A56C0000000409227434000000000028256C0000000E0EB27434000000020048256C0000000002F2A4340000000A04E8956C00000002053344340000000C05A8956C000000020A5364340000000A0029156C0000000606442434000000000B89056C0000000C031444340000000A06B8F56C0000000000A484340000000E0C18B56C0000000601D4E4340000000C0BF8B56C0000000A05254434000000060BA6D56C00000006031544340000000806B6D56C0000000E09E35434000000080B86D56C0FFFFFFFF081C434000000060DE7956C0000000E08D1C4340	27.4838270000000016
+48	0106000020E6100000010000000103000000010000000E000000FFFFFF9FD60956C00000000035214340000000808A0956C000000080C1494340000000605FFD55C000000020A549434000000060C8FC55C0FFFFFF5FDF444340000000C0AAFD55C0000000808E3F434000000080F9FC55C000000060DA394340000000E068FD55C000000060D537434000000000E6FC55C0000000C05E364340000000E0A8FE55C0000000C056334340FFFFFFFFA6FE55C00000004061304340000000E0ACFD55C0000000E0D02C4340000000407CFD55C000000040D92543400000006042FF55C00000004040214340FFFFFF9FD60956C00000000035214340	0
+49	0106000020E6100000010000000103000000010000001600000000000060DE7956C0000000E08D1C434000000020918256C0000000809A1C4340000000207D8256C0000000008611434000000020488D56C000000040610B434000000080429056C000000040A30F434000000060899256C0000000405A154340000000C08C9556C0000000C02718434000000060589756C000000080FD1D434000000060A39756C0000000606A294340FFFFFFBFF49656C0FFFFFF1FC32E434000000020BC9556C00000004007324340FFFFFF5F519356C0000000A0B3364340000000A0029156C00000006064424340000000C05A8956C000000020A5364340000000A04E8956C0000000205334434000000020048256C0000000002F2A434000000000028256C0000000E0EB274340FFFFFF7FB97A56C00000004092274340000000204D7A56C0000000A0CD23434000000060177B56C0000000807E244340000000C0897B56C0000000806323434000000060DE7956C0000000E08D1C4340	2.93446600000000002
+50	0106000020E6100000010000000103000000010000001600000000000020396656C0000000A0C31B434000000080B86D56C0FFFFFFFF081C4340000000806B6D56C0000000E09E35434000000080DD6A56C000000020BB364340FFFFFF5F876A56C00000008096384340FFFFFF3F536956C0000000C06A384340000000E0156856C0000000A08539434000000020996756C000000060AB3B4340000000A0A06456C0000000A0963D434000000060B56256C0000000A09A3C4340000000E08A6156C0000000A0793D4340000000E0CB5E56C000000040AD3B4340000000E0405D56C0000000803A3E4340000000608F5B56C0000000C0273F4340000000A07D5956C0000000A0833E4340000000A0DA5656C000000040A4414340000000E00D5356C0000000004C404340000000A0FB5056C0000000001A414340000000A04E4956C0000000E031404340000000803D4956C0FFFFFF9FB63C434000000080A84956C0000000A05B1B434000000020396656C0000000A0C31B4340	4.45642699999999969
+51	0106000020E6100000010000000103000000010000002400000000000060E4A856C000000080440A43400000006018AA56C0000000E0FF0A434000000080FFA956C000000080EA0C434000000080D3AB56C0FFFFFF5F2F0C434000000080FEAB56C0000000C0710E43400000002017B256C0000000209F1A43400000006067AF56C000000040593243400000006054AF56C0000000C0573B4340000000202CAC56C000000040A33B434000000060CFAB56C000000040BE384340FFFFFFDFD4AA56C000000060A5384340000000801EAA56C000000020A63D4340000000C049A756C0000000A07B3C4340FFFFFFFFF5A556C0000000801A404340000000E0379A56C00000002001404340FFFFFF3F2B9A56C000000000263E4340FFFFFF1FE19A56C000000040623D434000000040129A56C0000000C07E3A4340FFFFFF1FAA9556C0000000A074394340000000406F9656C0000000800536434000000020BC9556C00000004007324340FFFFFFBFF49656C0FFFFFF1FC32E434000000060A39756C0000000606A29434000000060589756C000000080FD1D4340000000C08C9556C0000000C02718434000000060899256C0000000405A15434000000080429056C000000040A30F434000000060029356C000000020C60B4340FFFFFFDF129556C000000060CA0C4340000000E0919A56C0000000E0CE054340FFFFFF7F92A656C0FFFFFFBF5400434000000060F6A756C0000000E03B014340FFFFFF9F39A756C0000000E0A3024340000000E08CA756C0000000A00E064340000000A003A756C0000000C05F09434000000060E4A856C000000080440A4340	4.62926400000000005
+53	0106000020E6100000010000000103000000010000001F00000000000040E21957C0000000A0A6024340000000E0092157C0000000E02C03434000000000242157C0000000608104434000000060892357C0000000405C064340FFFFFF7FC12357C0000000A0EA074340000000A0D82457C00000000027084340FFFFFF7FDB2457C0000000E0450C434000000080C22557C0000000A0750C434000000020B82557C000000040160E434000000020652657C0000000402B0E434000000020632657C0000000A05C114340FFFFFF9F102757C00000006080114340000000A0F62657C0FFFFFFDF91154340000000C0FE2857C0FFFFFF3FEE154340000000A0342957C000000080811A4340FFFFFFDFD72C57C0000000A03B1C4340000000C0662C57C000000040292C434000000060CC2857C0FFFFFFBF582C4340000000800C2857C0FFFFFFBF01374340FFFFFF9FB91F57C0FFFFFFDF7D364340000000E0D81957C000000000F6354340000000C0281A57C000000080262B434000000040131257C000000040C52A4340000000C05C1057C0000000209229434000000060D80E57C000000080CC2A4340000000208D0C57C000000000AB2A434000000060850C57C000000080EC244340000000A0C10C57C00000008020154340000000E0AA0B57C00000004000154340FFFFFF1F090C57C0000000202E02434000000040E21957C0000000A0A6024340	3.99004100000000017
+55	0106000020E6100000010000000103000000010000002200000000000000181856C000000020FEF4424000000020F21756C000000040DA204340FFFFFF9FD60956C000000000352143400000006042FF55C00000004040214340FFFFFF9FB8FE55C0000000E0DB1E4340000000C01AFF55C0000000600E1E43400000006096FE55C0FFFFFFFFB0194340000000A0AAFB55C0000000A0E7154340FFFFFFBFA5FB55C0000000E02914434000000020D6FC55C00000006086114340000000E04DFE55C000000080DD104340000000E02F0156C000000000390D434000000000CA0056C000000000D20B434000000060C0FD55C000000040620C4340000000406BFE55C0000000206209434000000000390256C000000040EC06434000000000C20256C000000080C605434000000080A70256C000000020E704434000000080630156C0000000C04A044340000000A0DE0156C0000000E00D014340000000A0630156C0FFFFFF9FCEFC424000000080B80256C0000000E066FA4240FFFFFF5FAC0256C0000000A09DF74240FFFFFFBF220456C00000002003F74240000000600D0556C000000000D5F8424000000040600556C0000000803AF64240000000C0F20156C0000000A073F54240000000A0B30156C0000000E0EFF34240FFFFFF1FDF0256C000000040B0F24240FFFFFFBF670656C0FFFFFF5FFDF34240000000407E0656C00000006099F24240000000A0390956C0000000E0E8F54240000000E0CA0956C0000000201CF5424000000000181856C000000020FEF44240	3.04025299999999987
+56	0106000020E6100000010000000103000000010000000600000000000000181856C000000020FEF4424000000020572D56C0FFFFFF7F74F44240000000E0562D56C0000000208910434000000060492D56C0000000603E21434000000020F21756C000000040DA20434000000000181856C000000020FEF44240	3.90541099999999997
+57	0106000020E6100000010000000103000000010000001700000000000020488D56C000000040610B4340000000207D8256C0000000008611434000000020918256C0000000809A1C434000000060DE7956C0000000E08D1C434000000080B86D56C0FFFFFFFF081C434000000020396656C0000000A0C31B434000000000936656C0000000601DFA424000000040B46A56C0000000607CEB424000000060E56B56C0FFFFFFFF16EA4240000000403A6C56C00000008003E74240000000E09E6E56C0000000A0A5EB424000000080827656C000000020D9F34240FFFFFF5F1B7756C000000000E7F34240FFFFFFDF797756C00000000029F24240FFFFFF9FA27956C0000000A01DF0424000000020067C56C0000000C063F0424000000080A67E56C0000000A0B8F44240000000A0537D56C00000006058FB424000000020B18056C0000000A012FC4240000000E0AE8256C0FFFFFF5F21FF424000000040A38756C00000008021044340000000209F8856C0000000E0E706434000000020488D56C000000040610B4340	4.33283899999999988
+58	0106000020E6100000010000000103000000010000000B00000000000020864856C00000000054104340000000C0FA4856C0FFFFFF3FC70D434000000000E64756C000000060EA0B4340FFFFFFFFF54856C0000000E00F06434000000080504956C0000000C0DEFE4240000000E0744B56C0000000A07CF9424000000000936656C0000000601DFA424000000020396656C0000000A0C31B434000000080A84956C0000000A05B1B4340FFFFFF9FB84956C0000000409110434000000020864856C00000000054104340	3.8941110000000001
+61	0106000020E61000000100000001030000000100000013000000000000A0F60157C0000000C081014340000000405FFE56C0000000E06B014340000000605AFD56C0000000E05905434000000060BBFB56C0000000809A044340000000C01EFB56C00000002014064340FFFFFFDFD2F956C000000040E2064340FFFFFF3FF2E856C0000000A0A3064340000000A0D7E856C0FFFFFF1F1B14434000000060BCE256C0000000C023144340FFFFFFFF04E256C0FFFFFFBFD01343400000008044E256C0000000E0BEE44240000000803CF456C000000040BEE4424000000080DBF356C0FFFFFF9F97DF42400000004063F456C0FFFFFFBF64DB4240000000007EF456C000000000A6CC4240000000A0000257C00000002054CD424000000080730157C0000000C094E3424000000080EA0157C0000000409BE44240000000A0F60157C0000000C081014340	3.28216300000000016
+62	0106000020E6100000010000000103000000010000000C00000000000020572D56C0FFFFFF7F74F4424000000020682D56C0FFFFFF7F00EF4240000000C0E94956C000000080CDEE4240000000E0EC4956C0000000607AF94240000000E0744B56C0000000A07CF9424000000080504956C0000000C0DEFE4240FFFFFFFFF54856C0000000E00F06434000000000E64756C000000060EA0B4340000000C0FA4856C0FFFFFF3FC70D434000000020864856C00000000054104340000000E0562D56C0000000208910434000000020572D56C0FFFFFF7F74F44240	3.29576199999999986
+63	0106000020E61000000100000001030000000100000013000000000000E0718756C00000000011D64240FFFFFF5F418A56C0FFFFFF3F72DA4240FFFFFFDFFD8C56C00000004087D64240000000E0A59D56C0FFFFFF5FA4F0424000000000CE9456C0000000603BFE4240000000E0919A56C0000000E0CE054340FFFFFFDF129556C000000060CA0C434000000060029356C000000020C60B434000000080429056C000000040A30F434000000020488D56C000000040610B4340000000209F8856C0000000E0E706434000000040A38756C00000008021044340000000E0AE8256C0FFFFFF5F21FF424000000020B18056C0000000A012FC4240000000A0537D56C00000006058FB424000000080A67E56C0000000A0B8F4424000000020067C56C0000000C063F04240000000E0798056C0000000E0DEE84240000000E0718756C00000000011D64240	7.24967900000000043
+64	0106000020E6100000010000000103000000010000001100000000000060BC8956C00000004054D24240FFFFFF7F82A256C00000008046D2424000000040D6A956C0000000C024D242400000006098A956C0000000C019DE424000000060E4A856C000000080440A4340000000A003A756C0000000C05F094340000000E08CA756C0000000A00E064340FFFFFF9F39A756C0000000E0A302434000000060F6A756C0000000E03B014340FFFFFF7F92A656C0FFFFFFBF54004340000000E0919A56C0000000E0CE05434000000000CE9456C0000000603BFE4240000000E0A59D56C0FFFFFF5FA4F04240FFFFFFDFFD8C56C00000004087D64240FFFFFF5F418A56C0FFFFFF3F72DA4240000000E0718756C00000000011D6424000000060BC8956C00000004054D24240	3.04184600000000005
+65	0106000020E6100000010000000103000000010000000B000000000000E0F40F57C0000000C0B5CD4240FFFFFF9FF10F57C0000000000DD34240000000403A1A57C0000000E038DB4240000000001E1A57C00000002044EE424000000040E21957C0000000A0A6024340FFFFFF1F090C57C0000000202E024340000000A0F60157C0000000C08101434000000080EA0157C0000000409BE4424000000080730157C0000000C094E34240000000A0000257C00000002054CD4240000000E0F40F57C0000000C0B5CD4240	1.61801799999999996
+66	0106000020E6100000010000000103000000010000001400000000000000CE4956C00000004053CD4240000000C0655D56C0000000A09ECD4240FFFFFF7F825D56C0FFFFFF1FA9CA4240FFFFFF7F835E56C000000080FAC8424000000020996156C0000000E035C9424000000000DB6056C0FFFFFFBFD6CE4240000000403A6156C0000000803FD3424000000020DB6056C00000000005D74240000000A0606156C000000020EFD84240FFFFFF3F366556C0000000A061DA424000000040A76A56C0000000006BDF424000000040416B56C00000002059E44240000000403A6C56C00000008003E7424000000060E56B56C0FFFFFFFF16EA424000000040B46A56C0000000607CEB424000000000936656C0000000601DFA4240000000E0744B56C0000000A07CF94240000000E0EC4956C0000000607AF94240000000C0E94956C000000080CDEE424000000000CE4956C00000004053CD4240	4.91080100000000019
+67	0106000020E6100000010000000103000000010000002000000000000060BC8956C00000004054D24240000000E0718756C00000000011D64240000000E0798056C0000000E0DEE8424000000020067C56C0000000C063F04240FFFFFF9FA27956C0000000A01DF04240FFFFFFDF797756C00000000029F24240FFFFFF5F1B7756C000000000E7F3424000000080827656C000000020D9F34240000000E09E6E56C0000000A0A5EB4240000000403A6C56C00000008003E7424000000040416B56C00000002059E4424000000040A76A56C0000000006BDF4240FFFFFF3F366556C0000000A061DA4240000000A0606156C000000020EFD8424000000020DB6056C00000000005D74240000000403A6156C0000000803FD3424000000000DB6056C0FFFFFFBFD6CE424000000020996156C0000000E035C94240000000A0D56556C0FFFFFF3F7FC9424000000020FD6656C00000006065CC4240000000A08C6856C000000080A3CB424000000020126956C000000000DBCC424000000060DB6B56C0000000801BCB4240000000E03A6C56C00000008035CC424000000020756D56C000000020BACC4240000000A0296E56C000000080E9CB4240000000C0F16E56C0000000E0A9CC4240000000E0817156C0FFFFFF3F6FCB4240000000A0D07356C000000080EECC424000000060627756C000000060F8CC424000000000908956C0000000E0B3CC424000000060BC8956C00000004054D24240	1.99145700000000003
+69	0106000020E6100000010000000103000000010000001100000000000060E6E956C000000040FFB54240000000C0B1F056C00000006063B6424000000060EAF056C0000000E033CC4240000000007EF456C000000000A6CC42400000004063F456C0FFFFFFBF64DB424000000080DBF356C0FFFFFF9F97DF4240000000803CF456C000000040BEE442400000008044E256C0000000E0BEE442400000000065D456C00000004041E44240000000C086D456C000000040CED94240000000C080CA56C0000000001BD942400000008089CA56C00000008095CB4240000000408AD456C000000000C3CB4240000000A067D456C000000060BDC042400000008032CE56C00000006039C04240000000C049CE56C0000000E0D8B4424000000060E6E956C000000040FFB54240	7.26665000000000028
+70	0106000020E61000000100000001030000000100000014000000FFFFFF7F82A256C00000008046D24240FFFFFF1FC4A256C00000000069CC4240000000E078A356C00000002057CC4240000000408CA356C0000000C0F4A74240FFFFFF7FD8A356C0000000A004A34240000000808AAF56C0000000E0CBA24240FFFFFF3F5BB056C000000020F4A24240000000800DB056C00000002043AF424000000040D5B156C00000004066AF424000000080CAB156C0000000600DCD4240FFFFFF7F5EC056C0000000805BCD4240000000C00AC756C00000000081CB42400000008089CA56C00000008095CB4240000000C080CA56C0000000001BD9424000000060FBC956C0000000A01ED94240FFFFFF9FF7C956C0FFFFFF7F75DE42400000006010C756C000000020A7DE42400000006098A956C0000000C019DE424000000040D6A956C0000000C024D24240FFFFFF7F82A256C00000008046D24240	3.11090400000000011
+71	0106000020E6100000010000000103000000010000000900000000000060378E56C0000000A0EBA74240000000408CA356C0000000C0F4A74240000000E078A356C00000002057CC4240FFFFFF1FC4A256C00000000069CC4240FFFFFF7F82A256C00000008046D2424000000060BC8956C00000004054D2424000000000908956C0000000E0B3CC424000000040968956C000000040EAA7424000000060378E56C0000000A0EBA74240	2.98027100000000011
+72	0106000020E6100000010000000103000000010000000D00000000000080CD0557C0000000407487424000000000A11057C000000060E7874240000000A0F00F57C000000080AAA04240FFFFFF7F761057C0000000A0EEA0424000000000221057C0000000C028BD4240000000E0F40F57C0000000C0B5CD4240000000A0000257C00000002054CD4240000000007EF456C000000000A6CC424000000060EAF056C0000000E033CC4240000000C0B1F056C00000006063B6424000000060E6E956C000000040FFB5424000000000B0EA56C0000000004286424000000080CD0557C00000004074874240	3.86676699999999984
+73	0106000020E6100000010000000103000000010000000E000000000000E0FF5D56C0000000C071AB424000000020E35B56C00000002088AD4240000000605D5B56C0000000409CB4424000000020085D56C00000000002BA424000000080AA5F56C0000000E0F0BE424000000020996156C0000000E035C94240FFFFFF7F835E56C000000080FAC84240FFFFFF7F825D56C0FFFFFF1FA9CA4240000000C0655D56C0000000A09ECD424000000000CE4956C00000004053CD4240000000E0F64256C0FFFFFFDF38CD4240000000002E4356C0FFFFFFFF29AB4240000000C0BC4F56C0000000C03CAB4240000000E0FF5D56C0000000C071AB4240	1.86840800000000007
+74	0106000020E61000000100000001030000000100000024000000000000808AAF56C0000000E0CBA24240000000A093AF56C0000000E0549542400000006089B056C00000002032954240000000E0B1B056C0FFFFFFFF0E924240000000202CB256C00000002007924240000000C071B256C000000060B4864240FFFFFFBF2FBE56C0000000A04E874240000000802ABE56C000000080B48C4240FFFFFF7F90C156C0FFFFFF9FCB8C424000000080BAC156C0000000C003924240000000E09DC256C0000000A01B92424000000020C3C256C0000000807895424000000020D4C456C00000008020954240000000C0EAC456C0FFFFFF7F84964240000000E019C656C0000000409A964240000000E035C656C000000080E899424000000000C3C856C000000000D7994240000000A0D9C856C0000000C0899E42400000006055C856C0FFFFFF1FAB9E42400000000047C856C0000000804BA04240000000005BCA56C0000000A0D1A04240000000805DCA56C0000000204EA84240000000E096CB56C0000000E080A8424000000080B0CB56C000000040A1B44240000000C049CE56C0000000E0D8B442400000008032CE56C00000006039C04240000000A067D456C000000060BDC04240000000408AD456C000000000C3CB42400000008089CA56C00000008095CB4240000000C00AC756C00000000081CB4240FFFFFF7F5EC056C0000000805BCD424000000080CAB156C0000000600DCD424000000040D5B156C00000004066AF4240000000800DB056C00000002043AF4240FFFFFF3F5BB056C000000020F4A24240000000808AAF56C0000000E0CBA24240	12.5770339999999994
+75	0106000020E6100000010000000103000000010000000D000000000000A0A47756C0000000C0DA904240000000E07A7D56C000000060D090424000000060B67D56C00000000057884240000000E0BD7F56C00000008017884240000000C0D87F56C0000000805986424000000040568756C00000000039864240000000605B8756C0000000E0188B4240000000E02F8E56C0000000A0058B424000000060378E56C0000000A0EBA7424000000040968956C000000040EAA7424000000000908956C0000000E0B3CC424000000060627756C000000060F8CC4240000000A0A47756C0000000C0DA904240	7.80359900000000017
+76	0106000020E6100000010000000103000000010000002800000000000060627756C000000060F8CC4240000000A0D07356C000000080EECC4240000000E0817156C0FFFFFF3F6FCB4240000000C0F16E56C0000000E0A9CC4240000000A0296E56C000000080E9CB424000000020756D56C000000020BACC4240000000E03A6C56C00000008035CC424000000060DB6B56C0000000801BCB424000000020126956C000000000DBCC4240000000A08C6856C000000080A3CB424000000020FD6656C00000006065CC4240000000A0D56556C0FFFFFF3F7FC9424000000020996156C0000000E035C9424000000080AA5F56C0000000E0F0BE424000000020085D56C00000000002BA4240000000605D5B56C0000000409CB4424000000020E35B56C00000002088AD4240000000E0FF5D56C0000000C071AB424000000080096056C0FFFFFF1F2BAA424000000080E36056C00000000009A7424000000080E36056C0FFFFFF1F61A3424000000080555F56C0000000A0C4A0424000000060D06056C000000000079F4240000000C0F46556C000000000279E424000000000FE6556C0000000A01B9D424000000040236856C000000020EE9C4240FFFFFF7F196856C000000000C59B424000000000CD6956C0000000E06A9B424000000060CC6956C00000008027994240000000A0336C56C00000000008994240FFFFFFFF326C56C0FFFFFF9F2C974240FFFFFF7F876D56C0FFFFFF5FD296424000000020906D56C0000000208094424000000020866E56C0000000806194424000000080986E56C0000000E0FC924240000000A0A07056C000000080BF924240000000A0B27056C000000060B790424000000060BB7156C000000060A7904240000000A0A47756C0000000C0DA90424000000060627756C000000060F8CC4240	3.47149000000000019
+77	0106000020E6100000010000000103000000010000001800000000000000B0EA56C0000000004286424000000060E6E956C000000040FFB54240000000C049CE56C0000000E0D8B4424000000080B0CB56C000000040A1B44240000000E096CB56C0000000E080A84240000000805DCA56C0000000204EA84240000000005BCA56C0000000A0D1A042400000000047C856C0000000804BA042400000006055C856C0FFFFFF1FAB9E4240000000A0D9C856C0000000C0899E424000000000C3C856C000000000D7994240000000E035C656C000000080E8994240000000E019C656C0000000409A964240000000C0EAC456C0FFFFFF7F8496424000000020D4C456C0000000802095424000000020C3C256C00000008078954240000000E09DC256C0000000A01B92424000000080BAC156C0000000C003924240FFFFFF7F90C156C0FFFFFF9FCB8C4240000000E031C756C000000080E88A4240000000806ECE56C0000000E0E18A424000000040AFCE56C000000020DF704240000000C0B7EA56C0000000C06A71424000000000B0EA56C00000000042864240	4.33482199999999995
+78	0106000020E6100000010000000103000000010000001F000000000000E08FAB56C0000000807A76424000000040ABAC56C000000080A176424000000020BFAC56C0000000A0BC7B4240FFFFFFDFF6AD56C000000000E37B424000000000FEAD56C0000000604F7F42400000004052AF56C000000000847F42400000000061AF56C0FFFFFFBF6B864240000000C071B256C000000060B4864240000000202CB256C00000002007924240000000E0B1B056C0FFFFFFFF0E9242400000006089B056C00000002032954240000000A093AF56C0000000E054954240000000808AAF56C0000000E0CBA24240FFFFFF7FD8A356C0000000A004A34240000000408CA356C0000000C0F4A7424000000060378E56C0000000A0EBA74240000000E02F8E56C0000000A0058B4240000000605B8756C0000000E0188B424000000040568756C0000000003986424000000060BC8956C0000000604186424000000060E68956C0000000809C81424000000060D28A56C0000000207C81424000000000F58A56C000000060D07E424000000000108C56C0000000A0A07E424000000060338C56C000000040987C4240FFFFFFFF318D56C000000080777C424000000040438D56C0000000A0217B4240000000607A8E56C000000080E27A424000000000938E56C000000080EC77424000000040D09056C00000004054764240000000E08FAB56C0000000807A764240	8.45153700000000008
+\.
+
+
+CREATE INDEX getis_data_gix ON getis_data USING GIST(the_geom);

src/pg/test/sql/04_dot_density_test.sql

@@ -1,12 +0,0 @@
-WITH g AS (
-  SELECT ST_Buffer(ST_SetSRID(ST_MakePoint(0,0),4326)::geometry, 1000)::geometry AS g
-),
-points AS(
-  SELECT (
-    ST_Dump(
-      cdb_crankshaft.cdb_dot_density(g.g, 100)
-    )
-  ).geom AS p FROM g
-)
-
-SELECT count(*), sum(CASE WHEN ST_Contains(g,p) THEN 1 ELSE 0 END) FROM points, g

src/pg/test/sql/16_getis_test.sql

@@ -0,0 +1,15 @@
+\pset format unaligned
+\set ECHO all
+\i test/fixtures/getis_data.sql
+
+-- set random seed
+SELECT cdb_crankshaft._cdb_random_seeds(1234);
+
+-- test against PySAL example dataset 'stl_hom'
+SELECT rowid, round(z_score, 4) As z_score, round(p_value, 4) As p_value
+FROM cdb_crankshaft.CDB_GetisOrdsG(
+  'select * from getis_data',
+  'hr8893', 'queen', NULL, 999,
+  'the_geom', 'cartodb_id') As t(z_score, p_value, p_z_sim, rowid)
+WHERE round(p_value, 4) <= 0.05
+ORDER BY rowid ASC;

src/pg/test/sql/18_outliers_test.sql

@@ -0,0 +1,85 @@
+SET client_min_messages TO WARNING;
+\set ECHO none
+\pset format unaligned
+
+--
+-- postgres=# select round(avg(i), 3) as avg,
+--                   round(stddev(i), 3) as stddev,
+--                   round(avg(i) + stddev(i), 3) as one_stddev,
+--                   round(avg(i) + 2 * stddev(i), 3) As two_stddev
+--            from unnest(ARRAY[1,3,2,3,5,1,2,32,12,3,57,2,1,4,2,100]) As x(i);
+--   avg   | stddev | one_stddev | two_stddev
+-- --------+--------+------------+------------
+--  14.375 | 27.322 |     41.697 |     69.020
+
+
+-- With an threshold of 1.0 standard deviation, ids 11, 16, and 17 are outliers
+WITH a AS (
+    SELECT
+      ARRAY[1,3,2,3,5,1,2,32,12, 3,57, 2, 1, 4, 2,100,-100]::numeric[] As vals, ARRAY[1,2,3,4,5,6,7, 8, 9,10,11,12,13,14,15, 16,  17]::int[] As ids
+), b As (
+  SELECT
+    (cdb_crankshaft.cdb_StdDevOutlier(vals, 1.0, ids)).*
+  FROM a
+  ORDER BY ids)
+SELECT *
+FROM b
+WHERE is_outlier IS TRUE;
+
+-- With a threshold of 2.0 standard deviations, id 16 is the only outlier
+WITH a AS (
+    SELECT
+      ARRAY[1,3,2,3,5,1,2,32,12, 3,57, 2, 1, 4, 2,100,-100]::numeric[] As vals,
+      ARRAY[1,2,3,4,5,6,7, 8, 9,10,11,12,13,14,15, 16,  17]::int[] As ids
+), b As (
+  SELECT
+    (cdb_crankshaft.CDB_StdDevOutlier(vals, 2.0, ids)).*
+  FROM a
+  ORDER BY ids)
+SELECT *
+FROM b
+WHERE is_outlier IS TRUE;
+
+-- With a Stddev of zero, should throw back error
+-- With a threshold of 2.0 standard deviations, id 16 is the only outlier
+WITH a AS (
+    SELECT
+      ARRAY[5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5]::numeric[] As vals,
+      ARRAY[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]::int[] As ids
+), b As (
+  SELECT
+    (cdb_crankshaft.CDB_StdDevOutlier(vals, 1.0, ids)).*
+  FROM a
+  ORDER BY ids)
+SELECT *
+FROM b
+WHERE is_outlier IS TRUE;
+
+-- With a ratio threshold of 2.0 threshold (100% above or below the mean)
+--  which is greater than ~21, which are values
+WITH a AS (
+    SELECT
+      ARRAY[1,3,2,3,5,1,2,32,12, 3,57, 2, 1, 4, 2,100,-100]::numeric[] As vals,
+      ARRAY[1,2,3,4,5,6,7, 8, 9,10,11,12,13,14,15, 16,  17]::int[] As ids
+), b As (
+  SELECT
+    (cdb_crankshaft.CDB_PercentOutlier(vals, 2.0, ids)).*
+  FROM a
+  ORDER BY ids)
+SELECT *
+  FROM b
+ WHERE is_outlier IS TRUE;
+
+-- With a static threshold of 11, what are the outliers
+WITH a AS (
+     SELECT
+       ARRAY[1,3,2,3,5,1,2,32,12, 3,57, 2, 1, 4, 2,100,-100]::numeric[] As vals,
+       ARRAY[1,2,3,4,5,6,7, 8, 9,10,11,12,13,14,15, 16,  17]::int[] As ids
+ ), b As (
+   SELECT unnest(vals) As v, unnest(ids) as i
+     FROM a
+ )
+SELECT cdb_crankshaft.CDB_StaticOutlier(v, 11.0) As is_outlier, i As rowid
+  FROM b
+WHERE cdb_crankshaft.CDB_StaticOutlier(v, 11.0) is True
+ORDER BY i;

src/py/crankshaft/crankshaft/clustering/__init__.py

@@ -1,3 +1,4 @@
 """Import all functions from for clustering"""
 from moran import *
 from kmeans import *
+from getis import *

src/py/crankshaft/crankshaft/clustering/getis.py

@@ -0,0 +1,52 @@
+"""
+Getis-Ord's G geostatistics (hotspot/coldspot analysis)
+"""
+
+import pysal as ps
+import plpy
+from collections import OrderedDict
+
+# crankshaft module
+import crankshaft.pysal_utils as pu
+
+# High level interface ---------------------------------------
+
+
+def getis_ord(subquery, attr,
+              w_type, num_ngbrs, permutations, geom_col, id_col):
+    """
+    Getis-Ord's G*
+    Implementation building neighbors with a PostGIS database and PySAL's
+      Getis-Ord's G* hotspot/coldspot module.
+    Andy Eschbacher
+    """
+
+    # geometries with attributes that are null are ignored
+    # resulting in a collection of not as near neighbors if kNN is chosen
+
+    qvals = OrderedDict([("id_col", id_col),
+                         ("attr1", attr),
+                         ("geom_col", geom_col),
+                         ("subquery", subquery),
+                         ("num_ngbrs", num_ngbrs)])
+
+    query = pu.construct_neighbor_query(w_type, qvals)
+
+    try:
+        result = plpy.execute(query)
+        # if there are no neighbors, exit
+        if len(result) == 0:
+            return pu.empty_zipped_array(4)
+    except plpy.SPIError, err:
+        plpy.error('Query failed: %s' % err)
+
+    attr_vals = pu.get_attributes(result)
+
+    # build PySAL weight object
+    weight = pu.get_weight(result, w_type, num_ngbrs)
+
+    # calculate Getis-Ord's G* z- and p-values
+    getis = ps.esda.getisord.G_Local(attr_vals, weight,
+                                     star=True, permutations=permutations)
+
+    return zip(getis.z_sim, getis.p_sim, getis.p_z_sim, weight.id_order)

src/py/crankshaft/crankshaft/pysal_utils/pysal_utils.py

@@ -45,8 +45,16 @@ def get_weight(query_res, w_type='knn', num_ngbrs=5):
 def query_attr_select(params):
     """
         Create portion of SELECT statement for attributes inolved in query.
+        Defaults to order in the params
         @param params: dict of information used in query (column names,
                        table name, etc.)
+            Example:
+            OrderedDict([('numerator', 'price'),
+                         ('denominator', 'sq_meters'),
+                         ('subquery', 'SELECT * FROM interesting_data')])
+        Output:
+          "i.\"price\"::numeric As attr1, " \
+          "i.\"sq_meters\"::numeric As attr2, "
     """
 
     attr_string = ""
@@ -64,7 +72,7 @@ def query_attr_select(params):
                  if k not in ('id_col', 'geom_col', 'subquery',
                               'num_ngbrs', 'subquery')]
 
-        for idx, val in enumerate(sorted(attrs)):
+        for idx, val in enumerate(attrs):
             attr_string += template % {"col": params[val],
                                        "alias_num": idx + 1}
 
@@ -80,8 +88,8 @@ def query_attr_where(params):
              'numerator': 'data1',
              'denominator': 'data2',
              '': ...}
-        Output: 'idx_replace."data1" IS NOT NULL AND idx_replace."data2"
-                IS NOT NULL'
+        Output:
+          'idx_replace."data1" IS NOT NULL AND idx_replace."data2" IS NOT NULL'
         Input:
         {'subquery': ...,
          'time_cols': ['time1', 'time2', 'time3'],
@@ -102,15 +110,17 @@ def query_attr_where(params):
         # moran where clauses
 
         # get keys
-        attrs = sorted([k for k in params
-                        if k not in ('id_col', 'geom_col', 'subquery',
-                                     'num_ngbrs', 'subquery')])
+        attrs = [k for k in params
+                 if k not in ('id_col', 'geom_col', 'subquery',
+                              'num_ngbrs', 'subquery')]
+
         # add values to template
         for attr in attrs:
             attr_string.append(template % params[attr])
 
-        if len(attrs) == 2:
-            attr_string.append("idx_replace.\"%s\" <> 0" % params[attrs[1]])
+        if 'denominator' in attrs:
+            attr_string.append(
+              "idx_replace.\"%s\" <> 0" % params['denominator'])
 
     out = " AND ".join(attr_string)
 

src/py/crankshaft/crankshaft/space_time_dynamics/markov.py

@@ -8,12 +8,14 @@ import pysal as ps
 import plpy
 import crankshaft.pysal_utils as pu
 
+
 def spatial_markov_trend(subquery, time_cols, num_classes=7,
                          w_type='knn', num_ngbrs=5, permutations=0,
                          geom_col='the_geom', id_col='cartodb_id'):
     """
         Predict the trends of a unit based on:
-        1. history of its transitions to different classes (e.g., 1st quantile -> 2nd quantile)
+        1. history of its transitions to different classes (e.g., 1st quantile
+             -> 2nd quantile)
         2. average class of its neighbors
 
         Inputs:
@@ -56,16 +58,15 @@ def spatial_markov_trend(subquery, time_cols, num_classes=7,
         )
         if len(query_result) == 0:
             return zip([None], [None], [None], [None], [None])
-    except plpy.SPIError, e:
-        plpy.debug('Query failed with exception %s: %s' % (err, pu.construct_neighbor_query(w_type, qvals)))
-        plpy.error('Analysis failed: %s' % e)
+    except plpy.SPIError, err:
+        plpy.error('Analysis failed: %s' % err)
         return zip([None], [None], [None], [None], [None])
 
-    ## build weight
+    # build weight
     weights = pu.get_weight(query_result, w_type)
     weights.transform = 'r'
 
-    ## prep time data
+    # prep time data
     t_data = get_time_data(query_result, time_cols)
 
     plpy.debug('shape of t_data %d, %d' % t_data.shape)
@@ -78,23 +79,26 @@ def spatial_markov_trend(subquery, time_cols, num_classes=7,
                                          fixed=False,
                                          permutations=permutations)
 
-    ## get lag classes
+    # get lag classes
     lag_classes = ps.Quantiles(
         ps.lag_spatial(weights, t_data[:, -1]),
         k=num_classes).yb
 
-    ## look up probablity distribution for each unit according to class and lag class
+    # look up probablity distribution for each unit according to class and lag
+    #  class
     prob_dist = get_prob_dist(sp_markov_result.P,
                               lag_classes,
                               sp_markov_result.classes[:, -1])
 
-    ## find the ups and down and overall distribution of each cell
-    trend_up, trend_down, trend, volatility = get_prob_stats(prob_dist,
-                                                             sp_markov_result.classes[:, -1])
+    # find the ups and down and overall distribution of each cell
+    trend_up, trend_down, trend, volatility = get_prob_stats(
+      prob_dist,
+      sp_markov_result.classes[:, -1])
 
-    ## output the results
+    # output the results
     return zip(trend, trend_up, trend_down, volatility, weights.id_order)
 
+
 def get_time_data(markov_data, time_cols):
     """
         Extract the time columns and bin appropriately
@@ -103,7 +107,8 @@ def get_time_data(markov_data, time_cols):
     return np.array([[x['attr' + str(i)] for x in markov_data]
                      for i in range(1, num_attrs+1)], dtype=float).transpose()
 
-## not currently used
+
+# not currently used
 def rebin_data(time_data, num_time_per_bin):
     """
         Convert an n x l matrix into an (n/m) x l matrix where the values are
@@ -131,14 +136,16 @@ def rebin_data(time_data, num_time_per_bin):
     """
 
     if time_data.shape[1] % num_time_per_bin == 0:
-        ## if fit is perfect, then use it
+        # if fit is perfect, then use it
         n_max = time_data.shape[1] / num_time_per_bin
     else:
-        ## fit remainders into an additional column
+        # fit remainders into an additional column
         n_max = time_data.shape[1] / num_time_per_bin + 1
 
-    return np.array([time_data[:, num_time_per_bin * i:num_time_per_bin * (i+1)].mean(axis=1)
-                     for i in range(n_max)]).T
+    return np.array(
+      [time_data[:, num_time_per_bin * i:num_time_per_bin * (i+1)].mean(axis=1)
+       for i in range(n_max)]).T
+
 
 def get_prob_dist(transition_matrix, lag_indices, unit_indices):
     """
@@ -157,6 +164,7 @@ def get_prob_dist(transition_matrix, lag_indices, unit_indices):
     return np.array([transition_matrix[(lag_indices[i], unit_indices[i])]
                      for i in range(len(lag_indices))])
 
+
 def get_prob_stats(prob_dist, unit_indices):
     """
         get the statistics of the probability distributions
@@ -179,11 +187,12 @@ def get_prob_stats(prob_dist, unit_indices):
         trend_up[i] = prob_dist[i, (unit_indices[i]+1):].sum()
         trend_down[i] = prob_dist[i, :unit_indices[i]].sum()
         if prob_dist[i, unit_indices[i]] > 0.0:
-            trend[i] = (trend_up[i] - trend_down[i]) / prob_dist[i, unit_indices[i]]
+            trend[i] = ((trend_up[i] - trend_down[i]) /
+                        (prob_dist[i, unit_indices[i]]))
         else:
             trend[i] = None
 
-    ## calculate volatility of distribution
+    # calculate volatility of distribution
     volatility = prob_dist.std(axis=1)
 
     return trend_up, trend_down, trend, volatility

src/py/crankshaft/test/fixtures/getis.json

@@ -0,0 +1 @@
+[[0.004793783909323601, 0.17999999999999999, 0.49808756424021061], [-1.0701189472090842, 0.079000000000000001, 0.14228288580832316], [-0.67867750971877305, 0.42099999999999999, 0.24867110969448558], [-0.67407386707620487, 0.246, 0.25013217644612995], [-0.79495689068870035, 0.33200000000000002, 0.21331928959090596], [-0.49279481022182703, 0.058999999999999997, 0.31107878905057329], [-0.38075627530057132, 0.28399999999999997, 0.35169205342069643], [-0.86710921611314895, 0.23699999999999999, 0.19294108571294855], [-0.78618647240956485, 0.050000000000000003, 0.2158791250244505], [-0.76108527223116984, 0.064000000000000001, 0.22330306830813684], [-0.13340753531942209, 0.247, 0.44693554317763651], [-0.57584545722033043, 0.48999999999999999, 0.28235982246156488], [-0.78882694661192831, 0.433, 0.2151065788731219], [-0.38769767950046219, 0.375, 0.34911988661484239], [-0.56057819488052207, 0.41399999999999998, 0.28754255985169652], [-0.41354017495644935, 0.45500000000000002, 0.339605447117173], [-0.23993577722243081, 0.49099999999999999, 0.40519002230969337], [-0.1389080156677496, 0.40400000000000003, 0.44476141839645233], [-0.25485737510500855, 0.376, 0.39941662953554224], [-0.71218610582902353, 0.17399999999999999, 0.23817476979886087], [-0.54533105995872144, 0.13700000000000001, 0.2927629228714812], [-0.39547917847510977, 0.033000000000000002, 0.34624464252424236], [-0.43052658996257548, 0.35399999999999998, 0.33340631435564982], [-0.37296719193774736, 0.40300000000000002, 0.35458643102865428], [-0.66482612169465694, 0.31900000000000001, 0.25308085650392698], [-0.13772133540823422, 0.34699999999999998, 0.44523032843016275], [-0.6765304487868502, 0.20999999999999999, 0.24935196033890672], [-0.64518763494323472, 0.32200000000000001, 0.25940279912025543], [-0.5078622084312413, 0.41099999999999998, 0.30577498972600159], [-0.12652006733772059, 0.42899999999999999, 0.44966013262301163], [-0.32691133022814595, 0.498, 0.37186747562269029], [0.25533848511500978, 0.42399999999999999, 0.39923083899077472], [2.7045138116476508, 0.0050000000000000001, 0.0034202212972238577], [-0.1551614486076057, 0.44400000000000001, 0.43834701985429037], [1.9524487722567723, 0.012999999999999999, 0.025442473674991528], [-1.2055816465306763, 0.017000000000000001, 0.11398941970467646], [3.478472976017831, 0.002, 0.00025213964072468009], [-1.4621715757903719, 0.002, 0.071847099325659136], [-0.84010307600180256, 0.085000000000000006, 0.20042529779230778], [5.7097646237318243, 0.0030000000000000001, 5.6566262784940591e-09], [1.5082367956567375, 0.065000000000000002, 0.065746966514827365], [-0.58337270103430816, 0.44, 0.27982121546450034], [-0.083271860457022437, 0.45100000000000001, 0.46681768733385554], [-0.46872337815000953, 0.34599999999999997, 0.31963368715684204], [0.18490279849545319, 0.23799999999999999, 0.42665263797981101], [3.470424529947997, 0.012, 0.00025981817437825683], [-0.99942612137154796, 0.032000000000000001, 0.15879415560388499], [-1.3650387953594485, 0.034000000000000002, 0.08612042845912049], [1.8617160516432014, 0.081000000000000003, 0.03132156240215267], [1.1321188945775384, 0.11600000000000001, 0.12879222611766061], [0.064116686050580601, 0.27300000000000002, 0.4744386578180424], [-0.42032194540259099, 0.29999999999999999, 0.33712514016213468], [-0.79581215423980922, 0.123, 0.21307061309098785], [-0.42792753720906046, 0.45600000000000002, 0.33435193892883741], [-1.0629378527428395, 0.051999999999999998, 0.14390506780140866], [-0.54164761752225477, 0.33700000000000002, 0.29403064095211839], [1.0934778886820793, 0.13700000000000001, 0.13709201601893539], [-0.094068785378413719, 0.38200000000000001, 0.46252725802998929], [0.13482026574801856, 0.36799999999999999, 0.44637699118865737], [-0.13976995315653129, 0.34699999999999998, 0.44442087706276601], [-0.051047663924746682, 0.32000000000000001, 0.47964376985626245], [-0.21468297736730158, 0.41699999999999998, 0.41500724761906527], [-0.20873154637330626, 0.38800000000000001, 0.41732890604390893], [-0.32427876152583485, 0.49199999999999999, 0.37286349875557478], [-0.65254842943280977, 0.374, 0.25702372075306734], [-0.48611858196118796, 0.23300000000000001, 0.31344154643990074], [-0.14482354344529477, 0.32600000000000001, 0.44242509660469886], [-0.51052030974200002, 0.439, 0.30484349480873729], [0.56814382285283538, 0.14999999999999999, 0.28496865660103166], [0.58680919931668207, 0.161, 0.27866592887231878], [0.013390357044409013, 0.25800000000000001, 0.49465818005865647], [-0.19050728887961568, 0.41399999999999998, 0.4244558160399462], [-0.60531777422216049, 0.35199999999999998, 0.2724839368239631], [1.0899331115425805, 0.127, 0.13787130480311838], [0.17015055382651084, 0.36899999999999999, 0.43244586845546418], [-0.21738337124409801, 0.40600000000000003, 0.41395479459421991], [1.0329303331079593, 0.079000000000000001, 0.15081825117169467], [1.0218317101096221, 0.104, 0.15343027913308094]]

src/py/crankshaft/test/test_clustering_getis.py

@@ -0,0 +1,74 @@
+import unittest
+import numpy as np
+
+
+# from mock_plpy import MockPlPy
+# plpy = MockPlPy()
+#
+# import sys
+# sys.modules['plpy'] = plpy
+from helper import plpy, fixture_file
+
+import crankshaft.clustering as cc
+import crankshaft.pysal_utils as pu
+from crankshaft import random_seeds
+import json
+
+# Fixture files produced as follows
+#
+# import pysal as ps
+# import numpy as np
+# import random
+#
+# # setup variables
+# f = ps.open(ps.examples.get_path("stl_hom.dbf"))
+# y = np.array(f.by_col['HR8893'])
+# w_queen = ps.queen_from_shapefile(ps.examples.get_path("stl_hom.shp"))
+#
+# out_queen = [{"id": index + 1,
+#               "neighbors": [x+1 for x in w_queen.neighbors[index]],
+#               "value": val} for index, val in enumerate(y)]
+#
+# with open('neighbors_queen_getis.json', 'w') as f:
+#     f.write(str(out_queen))
+#
+# random.seed(1234)
+# np.random.seed(1234)
+# lgstar_queen = ps.esda.getisord.G_Local(y, w_queen, star=True,
+#                                         permutations=999)
+#
+# with open('getis_queen.json', 'w') as f:
+#     f.write(str(zip(lgstar_queen.z_sim,
+#                     lgstar_queen.p_sim, lgstar_queen.p_z_sim)))
+
+
+class GetisTest(unittest.TestCase):
+    """Testing class for Getis-Ord's G* funtion
+       This test replicates the work done in PySAL documentation:
+          https://pysal.readthedocs.io/en/v1.11.0/users/tutorials/autocorrelation.html#local-g-and-g
+    """
+
+    def setUp(self):
+        plpy._reset()
+
+        # load raw data for analysis
+        self.neighbors_data = json.loads(
+          open(fixture_file('neighbors_getis.json')).read())
+
+        # load pre-computed/known values
+        self.getis_data = json.loads(
+          open(fixture_file('getis.json')).read())
+
+    def test_getis_ord(self):
+        """Test Getis-Ord's G*"""
+        data = [{'id': d['id'],
+                 'attr1': d['value'],
+                 'neighbors': d['neighbors']} for d in self.neighbors_data]
+        plpy._define_result('select', data)
+        random_seeds.set_random_seeds(1234)
+        result = cc.getis_ord('subquery', 'value',
+                              'queen', None, 999, 'the_geom', 'cartodb_id')
+        result = [(row[0], row[1]) for row in result]
+        expected = np.array(self.getis_data)[:, 0:2]
+        for ([res_z, res_p], [exp_z, exp_p]) in zip(result, expected):
+            self.assertAlmostEqual(res_z, exp_z, delta=1e-2)

src/py/crankshaft/test/test_pysal_utils.py

@@ -2,18 +2,33 @@ import unittest
 
 import crankshaft.pysal_utils as pu
 from crankshaft import random_seeds
+from collections import OrderedDict
 
 
 class PysalUtilsTest(unittest.TestCase):
     """Testing class for utility functions related to PySAL integrations"""
 
     def setUp(self):
-        self.params = {"id_col": "cartodb_id",
-                       "attr1": "andy",
-                       "attr2": "jay_z",
-                       "subquery": "SELECT * FROM a_list",
-                       "geom_col": "the_geom",
-                       "num_ngbrs": 321}
+        self.params1 = OrderedDict([("id_col", "cartodb_id"),
+                                    ("attr1", "andy"),
+                                    ("attr2", "jay_z"),
+                                    ("subquery", "SELECT * FROM a_list"),
+                                    ("geom_col", "the_geom"),
+                                    ("num_ngbrs", 321)])
+
+        self.params2 = OrderedDict([("id_col", "cartodb_id"),
+                                    ("numerator", "price"),
+                                    ("denominator", "sq_meters"),
+                                    ("subquery", "SELECT * FROM pecan"),
+                                    ("geom_col", "the_geom"),
+                                    ("num_ngbrs", 321)])
+
+        self.params3 = OrderedDict([("id_col", "cartodb_id"),
+                                    ("numerator", "sq_meters"),
+                                    ("denominator", "price"),
+                                    ("subquery", "SELECT * FROM pecan"),
+                                    ("geom_col", "the_geom"),
+                                    ("num_ngbrs", 321)])
 
         self.params_array = {"id_col": "cartodb_id",
                              "time_cols": ["_2013_dec", "_2014_jan", "_2014_feb"],
@@ -24,34 +39,41 @@ class PysalUtilsTest(unittest.TestCase):
     def test_query_attr_select(self):
         """Test query_attr_select"""
 
-        ans = "i.\"andy\"::numeric As attr1, " \
-              "i.\"jay_z\"::numeric As attr2, "
+        ans1 = ("i.\"andy\"::numeric As attr1, "
+                "i.\"jay_z\"::numeric As attr2, ")
 
-        ans_array = "i.\"_2013_dec\"::numeric As attr1, " \
-                    "i.\"_2014_jan\"::numeric As attr2, " \
-                    "i.\"_2014_feb\"::numeric As attr3, "
+        ans2 = ("i.\"price\"::numeric As attr1, "
+                "i.\"sq_meters\"::numeric As attr2, ")
 
-        self.assertEqual(pu.query_attr_select(self.params), ans)
+        ans3 = ("i.\"sq_meters\"::numeric As attr1, "
+                "i.\"price\"::numeric As attr2, ")
+
+        ans_array = ("i.\"_2013_dec\"::numeric As attr1, "
+                     "i.\"_2014_jan\"::numeric As attr2, "
+                     "i.\"_2014_feb\"::numeric As attr3, ")
+
+        self.assertEqual(pu.query_attr_select(self.params1), ans1)
+        self.assertEqual(pu.query_attr_select(self.params2), ans2)
+        self.assertEqual(pu.query_attr_select(self.params3), ans3)
         self.assertEqual(pu.query_attr_select(self.params_array), ans_array)
 
     def test_query_attr_where(self):
         """Test pu.query_attr_where"""
 
-        ans = "idx_replace.\"andy\" IS NOT NULL AND " \
-              "idx_replace.\"jay_z\" IS NOT NULL AND " \
-              "idx_replace.\"jay_z\" <> 0"
+        ans1 = ("idx_replace.\"andy\" IS NOT NULL AND "
+                "idx_replace.\"jay_z\" IS NOT NULL")
 
-        ans_array = "idx_replace.\"_2013_dec\" IS NOT NULL AND " \
-                    "idx_replace.\"_2014_jan\" IS NOT NULL AND " \
-                    "idx_replace.\"_2014_feb\" IS NOT NULL"
+        ans_array = ("idx_replace.\"_2013_dec\" IS NOT NULL AND "
+                     "idx_replace.\"_2014_jan\" IS NOT NULL AND "
+                     "idx_replace.\"_2014_feb\" IS NOT NULL")
 
-        self.assertEqual(pu.query_attr_where(self.params), ans)
+        self.assertEqual(pu.query_attr_where(self.params1), ans1)
         self.assertEqual(pu.query_attr_where(self.params_array), ans_array)
 
     def test_knn(self):
         """Test knn neighbors constructor"""
 
-        ans = "SELECT i.\"cartodb_id\" As id, " \
+        ans1 = "SELECT i.\"cartodb_id\" As id, " \
                      "i.\"andy\"::numeric As attr1, " \
                      "i.\"jay_z\"::numeric As attr2, " \
                      "(SELECT ARRAY(SELECT j.\"cartodb_id\" " \
@@ -59,17 +81,15 @@ class PysalUtilsTest(unittest.TestCase):
                                    "WHERE " \
                                     "i.\"cartodb_id\" <> j.\"cartodb_id\" AND " \
                                     "j.\"andy\" IS NOT NULL AND " \
-                                    "j.\"jay_z\" IS NOT NULL AND " \
-                                    "j.\"jay_z\" <> 0 " \
+                                    "j.\"jay_z\" IS NOT NULL " \
                                    "ORDER BY " \
                                     "j.\"the_geom\" <-> i.\"the_geom\" ASC " \
                       "LIMIT 321)) As neighbors " \
               "FROM (SELECT * FROM a_list) As i " \
               "WHERE i.\"andy\" IS NOT NULL AND " \
-                    "i.\"jay_z\" IS NOT NULL AND " \
-                    "i.\"jay_z\" <> 0 " \
+                    "i.\"jay_z\" IS NOT NULL " \
               "ORDER BY i.\"cartodb_id\" ASC;"
-        
+
         ans_array = "SELECT i.\"cartodb_id\" As id, " \
               "i.\"_2013_dec\"::numeric As attr1, " \
               "i.\"_2014_jan\"::numeric As attr2, " \
@@ -88,13 +108,13 @@ class PysalUtilsTest(unittest.TestCase):
                     "i.\"_2014_feb\" IS NOT NULL "\
               "ORDER BY i.\"cartodb_id\" ASC;"
 
-        self.assertEqual(pu.knn(self.params), ans)
+        self.assertEqual(pu.knn(self.params1), ans1)
         self.assertEqual(pu.knn(self.params_array), ans_array)
 
     def test_queen(self):
         """Test queen neighbors constructor"""
 
-        ans = "SELECT i.\"cartodb_id\" As id, " \
+        ans1 = "SELECT i.\"cartodb_id\" As id, " \
                      "i.\"andy\"::numeric As attr1, " \
                      "i.\"jay_z\"::numeric As attr2, " \
                      "(SELECT ARRAY(SELECT j.\"cartodb_id\" " \
@@ -104,23 +124,21 @@ class PysalUtilsTest(unittest.TestCase):
                                    "ST_Touches(i.\"the_geom\", " \
                                               "j.\"the_geom\") AND " \
                                    "j.\"andy\" IS NOT NULL AND " \
-                                   "j.\"jay_z\" IS NOT NULL AND " \
-                                   "j.\"jay_z\" <> 0)" \
+                                   "j.\"jay_z\" IS NOT NULL)" \
                                   ") As neighbors " \
               "FROM (SELECT * FROM a_list) As i " \
               "WHERE i.\"andy\" IS NOT NULL AND " \
-                    "i.\"jay_z\" IS NOT NULL AND " \
-                    "i.\"jay_z\" <> 0 " \
+                    "i.\"jay_z\" IS NOT NULL " \
               "ORDER BY i.\"cartodb_id\" ASC;"
 
-        self.assertEqual(pu.queen(self.params), ans)
+        self.assertEqual(pu.queen(self.params1), ans1)
 
     def test_construct_neighbor_query(self):
         """Test construct_neighbor_query"""
 
         # Compare to raw knn query
-        self.assertEqual(pu.construct_neighbor_query('knn', self.params),
-                         pu.knn(self.params))
+        self.assertEqual(pu.construct_neighbor_query('knn', self.params1),
+                         pu.knn(self.params1))
 
     def test_get_attributes(self):
         """Test get_attributes"""