jansson 2.1

With this flag enabled, the decoder stops after a valid JSON input and
thus allows extra data after it.

Fixes GH-25.

CHANGES

@@ -1,3 +1,36 @@
+Version 2.1 (in development)
+============================
+
+* New features:
+
+  - `json_loadb()`: Decode a string with a given size, useful if the
+    string is not null terminated.
+
+  - Add ``JSON_ENCODE_ANY`` encoding flag to allow encoding any JSON
+    value. By default, only arrays and objects can be encoded. (#19)
+
+  - Add ``JSON_REJECT_DUPLICATES`` decoding flag to issue a decoding
+    error if any JSON object in the input contins duplicate keys. (#3)
+
+  - Add ``JSON_DISABLE_EOF_CHECK`` decoding flag to stop decoding after a
+    valid JSON input. This allows other data after the JSON data.
+
+* Bug fixes:
+
+  - Fix an additional memory leak when memory allocation fails in
+    `json_object_set()` and friends.
+
+  - Clear errno before calling `strtod()` for better portability. (#27)
+
+* Building:
+
+  - Avoid set-but-not-used warning/error in a test. (#20)
+
+* Other:
+
+  - Minor clarifications to documentation.
+
+
 Version 2.0.1
 =============
 
@@ -10,7 +43,7 @@ Released 2011-03-31
 
   - Fix object key hashing in json_unpack() strict checking mode.
 
-  - Fix the parentheses in JANSSON_VERSION_HEX macro.
+  - Fix the parentheses in ``JANSSON_VERSION_HEX`` macro.
 
   - Fix `json_object_size()` return value.
 

configure.ac

@@ -1,5 +1,5 @@
 AC_PREREQ([2.60])
-AC_INIT([jansson], [2.0.1], [petri@digip.org])
+AC_INIT([jansson], [2.1], [petri@digip.org])
 
 AM_INIT_AUTOMAKE([1.10 foreign])
 

doc/apiref.rst

@@ -563,7 +563,10 @@ Unicode string and the value is any JSON value.
    existing keys. Returns 0 on success or -1 on error.
 
 
-The following functions implement an iteration protocol for objects:
+The following functions implement an iteration protocol for objects,
+allowing to iterate through all key-value pairs in an object. The
+items are not returned in any particular order, as this would require
+sorting due to the internal object representation.
 
 .. function:: void *json_object_iter(json_t *object)
 
@@ -680,8 +683,10 @@ Encoding
 ========
 
 This sections describes the functions that can be used to encode
-values to JSON. Only objects and arrays can be encoded, since they are
-the only valid "root" values of a JSON text.
+values to JSON. By default, only objects and arrays can be encoded
+directly, since they are the only valid *root* values of a JSON text.
+To encode any JSON value, use the ``JSON_ENCODE_ANY`` flag (see
+below).
 
 By default, the output has no newlines, and spaces are used between
 array and object elements for a readable output. This behavior can be
@@ -722,6 +727,19 @@ can be ORed together to obtain *flags*.
    example, decoding a JSON text and then encoding with this flag
    preserves the order of object keys.
 
+``JSON_ENCODE_ANY``
+   Specifying this flag makes it possible to encode any JSON value on
+   its own. Without it, only objects and arrays can be passed as the
+   *root* value to the encoding functions.
+
+   **Note:** Encoding any value may be useful in some scenarios, but
+   it's generally discouraged as it violates strict compatiblity with
+   :rfc:`4627`. If you use this flag, don't expect interoperatibility
+   with other JSON systems. Even Jansson itself doesn't have any means
+   to decode JSON texts whose root value is not object or array.
+
+   .. versionadded:: 2.1
+
 The following functions perform the actual JSON encoding. The result
 is in UTF-8.
 
@@ -762,14 +780,49 @@ See :ref:`rfc-conformance` for a discussion on Jansson's conformance
 to the JSON specification. It explains many design decisions that
 affect especially the behavior of the decoder.
 
+Each function takes a *flags* parameter that can be used to control
+the behavior of the decoder. Its default value is 0. The following
+macros can be ORed together to obtain *flags*.
+
+``JSON_REJECT_DUPLICATES``
+   Issue a decoding error if any JSON object in the input text
+   contains duplicate keys. Without this flag, the value of the last
+   occurence of each key ends up in the result. Key equivalence is
+   checked byte-by-byte, without special Unicode comparison
+   algorithms.
+
+   .. versionadded:: 2.1
+
+``JSON_DISABLE_EOF_CHECK``
+   By default, the decoder expects that its whole input constitutes a
+   valid JSON text, and issues an error if there's extra data after
+   the otherwise valid JSON input. With this flag enabled, the decoder
+   stops after decoding a valid JSON array or object, and thus allows
+   extra data after the JSON text.
+
+   .. versionadded:: 2.1
+
+The following functions perform the actual JSON decoding.
+
 .. function:: json_t *json_loads(const char *input, size_t flags, json_error_t *error)
 
    .. refcounting:: new
 
    Decodes the JSON string *input* and returns the array or object it
    contains, or *NULL* on error, in which case *error* is filled with
-   information about the error. *flags* is currently unused, and
-   should be set to 0.
+   information about the error. *flags* is described above.
+
+.. function:: json_t *json_loadb(const char *buffer, size_t buflen, size_t flags, json_error_t *error)
+
+   .. refcounting:: new
+
+   Decodes the JSON string *buffer*, whose length is *buflen*, and
+   returns the array or object it contains, or *NULL* on error, in
+   which case *error* is filled with information about the error. This
+   is similar to :func:`json_loads()` except that the string doesn't
+   need to be null-terminated. *flags* is described above.
+
+   .. versionadded:: 2.1
 
 .. function:: json_t *json_loadf(FILE *input, size_t flags, json_error_t *error)
 
@@ -777,8 +830,8 @@ affect especially the behavior of the decoder.
 
    Decodes the JSON text in stream *input* and returns the array or
    object it contains, or *NULL* on error, in which case *error* is
-   filled with information about the error. *flags* is currently
-   unused, and should be set to 0.
+   filled with information about the error. *flags* is described
+   above.
 
 .. function:: json_t *json_load_file(const char *path, size_t flags, json_error_t *error)
 
@@ -786,8 +839,8 @@ affect especially the behavior of the decoder.
 
    Decodes the JSON text in file *path* and returns the array or
    object it contains, or *NULL* on error, in which case *error* is
-   filled with information about the error. *flags* is currently
-   unused, and should be set to 0.
+   filled with information about the error. *flags* is described
+   above.
 
 
 .. _apiref-pack:

doc/conf.py

@@ -48,9 +48,9 @@ copyright = u'2009-2011, Petri Lehtinen'
 # built documents.
 #
 # The short X.Y version.
-version = '2.0'
+version = '2.1'
 # The full version, including alpha/beta/rc tags.
-release = '2.0.1'
+release = '2.1'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

doc/conformance.rst

@@ -30,6 +30,12 @@ error::
 
 All other Unicode codepoints U+0001 through U+10FFFF are allowed.
 
+Unicode normalization or any other transformation is never performed
+on any strings (string values or object keys). When checking for
+equivalence of strings or object keys, the comparison is performed
+byte by byte between the original UTF-8 representations of the
+strings.
+
 Numbers
 =======
 

src/Makefile.am

@@ -17,7 +17,7 @@ libjansson_la_SOURCES = \
 	value.c
 libjansson_la_LDFLAGS = \
 	-export-symbols-regex '^json_' \
-	-version-info 4:1:0
+	-version-info 5:0:1
 
 if GCC
 # These flags are gcc specific

src/dump.c

@@ -421,8 +421,10 @@ char *json_dumps(const json_t *json, size_t flags)
     strbuffer_t strbuff;
     char *result;
 
-    if(!json_is_array(json) && !json_is_object(json))
-        return NULL;
+    if(!(flags & JSON_ENCODE_ANY)) {
+        if(!json_is_array(json) && !json_is_object(json))
+           return NULL;
+    }
 
     if(strbuffer_init(&strbuff))
         return NULL;
@@ -440,8 +442,10 @@ char *json_dumps(const json_t *json, size_t flags)
 
 int json_dumpf(const json_t *json, FILE *output, size_t flags)
 {
-    if(!json_is_array(json) && !json_is_object(json))
-        return -1;
+    if(!(flags & JSON_ENCODE_ANY)) {
+        if(!json_is_array(json) && !json_is_object(json))
+           return -1;
+    }
 
     return do_dump(json, flags, 0, dump_to_file, (void *)output);
 }

src/jansson.h

@@ -21,11 +21,11 @@ extern "C" {
 /* version */
 
 #define JANSSON_MAJOR_VERSION  2
-#define JANSSON_MINOR_VERSION  0
-#define JANSSON_MICRO_VERSION  1
+#define JANSSON_MINOR_VERSION  1
+#define JANSSON_MICRO_VERSION  0
 
 /* Micro version is omitted if it's 0 */
-#define JANSSON_VERSION  "2.0.1"
+#define JANSSON_VERSION  "2.1"
 
 /* Version as a 3-byte hex number, e.g. 0x010201 == 1.2.1. Use this
    for numeric comparisons, e.g. #if JANSSON_VERSION_HEX >= ... */
@@ -214,17 +214,25 @@ json_t *json_copy(json_t *value);
 json_t *json_deep_copy(json_t *value);
 
 
-/* loading, printing */
+/* decoding */
+
+#define JSON_REJECT_DUPLICATES 0x1
+#define JSON_DISABLE_EOF_CHECK 0x2
 
 json_t *json_loads(const char *input, size_t flags, json_error_t *error);
+json_t *json_loadb(const char *buffer, size_t buflen, size_t flags, json_error_t *error);
 json_t *json_loadf(FILE *input, size_t flags, json_error_t *error);
 json_t *json_load_file(const char *path, size_t flags, json_error_t *error);
 
+
+/* encoding */
+
 #define JSON_INDENT(n)      (n & 0x1F)
 #define JSON_COMPACT        0x20
 #define JSON_ENSURE_ASCII   0x40
 #define JSON_SORT_KEYS      0x80
 #define JSON_PRESERVE_ORDER 0x100
+#define JSON_ENCODE_ANY     0x200
 
 char *json_dumps(const json_t *json, size_t flags);
 int json_dumpf(const json_t *json, FILE *output, size_t flags);

src/load.c

@@ -525,6 +525,7 @@ static int lex_scan_number(lex_t *lex, int c, json_error_t *error)
     lex_unget_unsave(lex, c);
 
     saved_text = strbuffer_value(&lex->saved_text);
+    errno = 0;
     value = strtod(saved_text, &end);
     assert(end == saved_text + lex->saved_text.length);
 
@@ -642,9 +643,9 @@ static void lex_close(lex_t *lex)
 
 /*** parser ***/
 
-static json_t *parse_value(lex_t *lex, json_error_t *error);
+static json_t *parse_value(lex_t *lex, size_t flags, json_error_t *error);
 
-static json_t *parse_object(lex_t *lex, json_error_t *error)
+static json_t *parse_object(lex_t *lex, size_t flags, json_error_t *error)
 {
     json_t *object = json_object();
     if(!object)
@@ -667,6 +668,14 @@ static json_t *parse_object(lex_t *lex, json_error_t *error)
         if(!key)
             return NULL;
 
+        if(flags & JSON_REJECT_DUPLICATES) {
+            if(json_object_get(object, key)) {
+                jsonp_free(key);
+                error_set(error, lex, "duplicate object key");
+                goto error;
+            }
+        }
+
         lex_scan(lex, error);
         if(lex->token != ':') {
             jsonp_free(key);
@@ -675,7 +684,7 @@ static json_t *parse_object(lex_t *lex, json_error_t *error)
         }
 
         lex_scan(lex, error);
-        value = parse_value(lex, error);
+        value = parse_value(lex, flags, error);
         if(!value) {
             jsonp_free(key);
             goto error;
@@ -709,7 +718,7 @@ error:
     return NULL;
 }
 
-static json_t *parse_array(lex_t *lex, json_error_t *error)
+static json_t *parse_array(lex_t *lex, size_t flags, json_error_t *error)
 {
     json_t *array = json_array();
     if(!array)
@@ -720,7 +729,7 @@ static json_t *parse_array(lex_t *lex, json_error_t *error)
         return array;
 
     while(lex->token) {
-        json_t *elem = parse_value(lex, error);
+        json_t *elem = parse_value(lex, flags, error);
         if(!elem)
             goto error;
 
@@ -749,7 +758,7 @@ error:
     return NULL;
 }
 
-static json_t *parse_value(lex_t *lex, json_error_t *error)
+static json_t *parse_value(lex_t *lex, size_t flags, json_error_t *error)
 {
     json_t *json;
 
@@ -782,11 +791,11 @@ static json_t *parse_value(lex_t *lex, json_error_t *error)
             break;
 
         case '{':
-            json = parse_object(lex, error);
+            json = parse_object(lex, flags, error);
             break;
 
         case '[':
-            json = parse_array(lex, error);
+            json = parse_array(lex, flags, error);
             break;
 
         case TOKEN_INVALID:
@@ -804,15 +813,30 @@ static json_t *parse_value(lex_t *lex, json_error_t *error)
     return json;
 }
 
-static json_t *parse_json(lex_t *lex, json_error_t *error)
+static json_t *parse_json(lex_t *lex, size_t flags, json_error_t *error)
 {
+    json_t *result;
+
     lex_scan(lex, error);
     if(lex->token != '[' && lex->token != '{') {
         error_set(error, lex, "'[' or '{' expected");
         return NULL;
     }
 
-    return parse_value(lex, error);
+    result = parse_value(lex, flags, error);
+    if(!result)
+        return NULL;
+
+    if(!(flags & JSON_DISABLE_EOF_CHECK)) {
+        lex_scan(lex, error);
+        if(lex->token != TOKEN_EOF) {
+            error_set(error, lex, "end of file expected");
+            json_decref(result);
+            result = NULL;
+        }
+    }
+
+    return result;
 }
 
 typedef struct
@@ -841,8 +865,6 @@ json_t *json_loads(const char *string, size_t flags, json_error_t *error)
     json_t *result;
     string_data_t stream_data;
 
-    (void)flags; /* unused */
-
     stream_data.data = string;
     stream_data.pos = 0;
 
@@ -850,19 +872,47 @@ json_t *json_loads(const char *string, size_t flags, json_error_t *error)
         return NULL;
 
     jsonp_error_init(error, "<string>");
+    result = parse_json(&lex, flags, error);
 
-    result = parse_json(&lex, error);
-    if(!result)
-        goto out;
+    lex_close(&lex);
+    return result;
+}
 
-    lex_scan(&lex, error);
-    if(lex.token != TOKEN_EOF) {
-        error_set(error, &lex, "end of file expected");
-        json_decref(result);
-        result = NULL;
-    }
+typedef struct
+{
+    const char *data;
+    size_t len;
+    size_t pos;
+} buffer_data_t;
+
+static int buffer_get(void *data)
+{
+    char c;
+    buffer_data_t *stream = data;
+    if(stream->pos >= stream->len)
+      return EOF;
+
+    c = stream->data[stream->pos];
+    stream->pos++;
+    return (unsigned char)c;
+}
+
+json_t *json_loadb(const char *buffer, size_t buflen, size_t flags, json_error_t *error)
+{
+    lex_t lex;
+    json_t *result;
+    buffer_data_t stream_data;
+
+    stream_data.data = buffer;
+    stream_data.pos = 0;
+    stream_data.len = buflen;
+
+    if(lex_init(&lex, buffer_get, (void *)&stream_data))
+        return NULL;
+
+    jsonp_error_init(error, "<buffer>");
+    result = parse_json(&lex, flags, error);
 
-out:
     lex_close(&lex);
     return result;
 }
@@ -872,7 +922,6 @@ json_t *json_loadf(FILE *input, size_t flags, json_error_t *error)
     lex_t lex;
     const char *source;
     json_t *result;
-    (void)flags; /* unused */
 
     if(lex_init(&lex, (get_func)fgetc, input))
         return NULL;
@@ -883,19 +932,8 @@ json_t *json_loadf(FILE *input, size_t flags, json_error_t *error)
         source = "<stream>";
 
     jsonp_error_init(error, source);
+    result = parse_json(&lex, flags, error);
 
-    result = parse_json(&lex, error);
-    if(!result)
-        goto out;
-
-    lex_scan(&lex, error);
-    if(lex.token != TOKEN_EOF) {
-        error_set(error, &lex, "end of file expected");
-        json_decref(result);
-        result = NULL;
-    }
-
-out:
     lex_close(&lex);
     return result;
 }

src/value.c

@@ -139,7 +139,10 @@ int json_object_set_new_nocheck(json_t *json, const char *key, json_t *value)
        allocated. */
     k = jsonp_malloc(offsetof(object_key_t, key) + strlen(key) + 1);
     if(!k)
+    {
+        json_decref(value);
         return -1;
+    }
 
     k->serial = object->serial++;
     strcpy(k->key, key);

test/.gitignore

@@ -6,6 +6,7 @@ suites/api/test_cpp
 suites/api/test_dump
 suites/api/test_equal
 suites/api/test_load
+suites/api/test_loadb
 suites/api/test_memory_funcs
 suites/api/test_number
 suites/api/test_object

test/suites/api/Makefile.am

@@ -6,6 +6,7 @@ check_PROGRAMS = \
 	test_dump \
 	test_equal \
 	test_load \
+	test_loadb \
 	test_memory_funcs \
 	test_number \
 	test_object \
@@ -17,6 +18,7 @@ test_array_SOURCES = test_array.c util.h
 test_copy_SOURCES = test_copy.c util.h
 test_dump_SOURCES = test_dump.c util.h
 test_load_SOURCES = test_load.c util.h
+test_loadb_SOURCES = test_loadb.c util.h
 test_memory_funcs_SOURCES = test_memory_funcs.c util.h
 test_number_SOURCES = test_number.c util.h
 test_object_SOURCES = test_object.c util.h

test/suites/api/check-exports

@@ -50,6 +50,7 @@ json_dump_file
 json_loads
 json_loadf
 json_load_file
+json_loadb
 json_equal
 json_copy
 json_deep_copy

test/suites/api/test_dump.c

@@ -9,13 +9,13 @@
 #include <string.h>
 #include "util.h"
 
-int main()
+static void encode_twice()
 {
+    /* Encode an empty object/array, add an item, encode again */
+
     json_t *json;
     char *result;
 
-    /* Encode an empty object/array, add an item, encode again */
-
     json = json_object();
     result = json_dumps(json, 0);
     if(!result || strcmp(result, "{}"))
@@ -43,7 +43,10 @@ int main()
     free(result);
 
     json_decref(json);
+}
 
+static void circular_references()
+{
     /* Construct a JSON object/array with a circular reference:
 
        object: {"a": {"b": {"c": <circular reference to $.a>}}}
@@ -51,6 +54,10 @@ int main()
 
        Encode it, remove the circular reference and encode again.
     */
+
+    json_t *json;
+    char *result;
+
     json = json_object();
     json_object_set_new(json, "a", json_object());
     json_object_set_new(json_object_get(json, "a"), "b", json_object());
@@ -86,6 +93,50 @@ int main()
     free(result);
 
     json_decref(json);
+}
 
+static void encode_other_than_array_or_object()
+{
+    /* Encoding anything other than array or object should only
+     * succeed if the JSON_ENCODE_ANY flag is used */
+
+    json_t *json;
+    FILE *fp = NULL;
+    char *result;
+
+    json = json_string("foo");
+    if(json_dumps(json, 0) != NULL)
+        fail("json_dumps encoded a string!");
+    if(json_dumpf(json, fp, 0) == 0)
+        fail("json_dumpf encoded a string!");
+
+    result = json_dumps(json, JSON_ENCODE_ANY);
+    if(!result || strcmp(result, "\"foo\"") != 0)
+        fail("json_dumps failed to encode a string with JSON_ENCODE_ANY");
+
+    free(result);
+    json_decref(json);
+
+    json = json_integer(42);
+    if(json_dumps(json, 0) != NULL)
+        fail("json_dumps encoded an integer!");
+    if(json_dumpf(json, fp, 0) == 0)
+        fail("json_dumpf encoded an integer!");
+
+    result = json_dumps(json, JSON_ENCODE_ANY);
+    if(!result || strcmp(result, "42") != 0)
+        fail("json_dumps failed to encode an integer with JSON_ENCODE_ANY");
+
+    free(result);
+    json_decref(json);
+
+
+}
+
+int main()
+{
+    encode_twice();
+    circular_references();
+    encode_other_than_array_or_object();
     return 0;
 }

test/suites/api/test_load.c

@@ -9,16 +9,52 @@
 #include <string.h>
 #include "util.h"
 
-int main()
+static void file_not_found()
 {
     json_t *json;
     json_error_t error;
 
     json = json_load_file("/path/to/nonexistent/file.json", 0, &error);
+    if(json)
+        fail("json_load_file returned non-NULL for a nonexistent file");
     if(error.line != -1)
         fail("json_load_file returned an invalid line number");
     if(strcmp(error.text, "unable to open /path/to/nonexistent/file.json: No such file or directory") != 0)
         fail("json_load_file returned an invalid error message");
+}
+
+static void reject_duplicates()
+{
+    json_error_t error;
+
+    if(json_loads("{\"foo\": 1, \"foo\": 2}", JSON_REJECT_DUPLICATES, &error))
+        fail("json_loads did not detect a duplicate key");
+    check_error("duplicate object key near '\"foo\"'", "<string>", 1, 16, 16);
+}
+
+static void disable_eof_check()
+{
+    json_error_t error;
+    json_t *json;
+
+    const char *text = "{\"foo\": 1} garbage";
+
+    if(json_loads(text, 0, &error))
+        fail("json_loads did not detect garbage after JSON text");
+    check_error("end of file expected near 'garbage'", "<string>", 1, 18, 18);
+
+    json = json_loads(text, JSON_DISABLE_EOF_CHECK, &error);
+    if(!json)
+        fail("json_loads failed with JSON_DISABLE_EOF_CHECK");
+
+    json_decref(json);
+}
+
+int main()
+{
+    file_not_found();
+    reject_duplicates();
+    disable_eof_check();
 
     return 0;
 }

test/suites/api/test_loadb.c

@@ -0,0 +1,38 @@
+/*
+ * Copyright (c) 2009-2011 Petri Lehtinen <petri@digip.org>
+ *
+ * Jansson is free software; you can redistribute it and/or modify
+ * it under the terms of the MIT license. See LICENSE for details.
+ */
+
+#include <jansson.h>
+#include <string.h>
+#include "util.h"
+
+int main()
+{
+    json_t *json;
+    json_error_t error;
+    const char str[] = "[\"A\", {\"B\": \"C\"}, 1, 2, 3]garbage";
+    size_t len = strlen(str) - strlen("garbage");
+
+    json = json_loadb(str, len, 0, &error);
+    if(!json) {
+        fail("json_loadb failed on a valid JSON buffer");
+    }
+    json_decref(json);
+
+    json = json_loadb(str, len - 1, 0, &error);
+    if (json) {
+        json_decref(json);
+        fail("json_loadb should have failed on an incomplete buffer, but it didn't");
+    }
+    if(error.line != 1) {
+        fail("json_loadb returned an invalid line number on fail");
+    }
+    if(strcmp(error.text, "']' expected near end of file") != 0) {
+        fail("json_loadb returned an invalid error message for an unclosed top-level array");
+    }
+
+    return 0;
+}