new version: 2016.2.1

Since Cygwin has SG_WINDOWS defined but is likely to expose a POSIX or
GNU-like interface, using _WIN32 to decide whether Windows-specific
functions are available seems more appropriate than SG_WINDOWS to me.

3rdparty/CMakeLists.txt

@@ -3,3 +3,7 @@ if (NOT SYSTEM_EXPAT)
 endif()
 
 add_subdirectory(utf8)
+
+if (ENABLE_DNS)
+	add_subdirectory(udns)
+endif()

3rdparty/udns/CMakeLists.txt

@@ -0,0 +1,41 @@
+include (SimGearComponent)
+INCLUDE (CheckFunctionExists)
+INCLUDE (CheckSymbolExists) 
+include (CheckIncludeFile)
+
+
+CHECK_FUNCTION_EXISTS(poll HAVE_POLL)
+CHECK_FUNCTION_EXISTS(getopt HAVE_GETOPT)
+CHECK_FUNCTION_EXISTS(inet_ntop HAVE_INET_PTON_NTOP)
+CHECK_SYMBOL_EXISTS(AF_INET6 "netinet/in.h" HAVE_IPv6) 
+# WINDOWS should be defined by msvc, should it?
+# somehow it is not, if somebody know a better way to define WINDOWS, please fix
+check_include_file(windows.h  WINDOWS)
+
+CONFIGURE_FILE(${CMAKE_CURRENT_SOURCE_DIR}/config.h.in ${CMAKE_BINARY_DIR}/config.h)
+include_directories(${CMAKE_BINARY_DIR})
+
+set(HEADERS
+)
+
+set(SOURCES
+    udns_resolver.c
+    udns_dn.c
+    udns_dntosp.c
+    udns_parse.c
+    udns_resolver.c
+    udns_init.c
+    udns_misc.c
+    udns_rr_a.c
+    udns_rr_ptr.c
+    udns_rr_mx.c
+    udns_rr_txt.c
+    udns_bl.c
+    udns_rr_srv.c
+    udns_rr_naptr.c
+    udns_codes.c
+    udns_jran.c
+    udns_XtoX.c
+)
+
+simgear_component(udns 3rdparty/udns "${SOURCES}" "${HEADERS}")

3rdparty/udns/COPYING.LGPL

@@ -0,0 +1,502 @@
+                  GNU LESSER GENERAL PUBLIC LICENSE
+                       Version 2.1, February 1999
+
+ Copyright (C) 1991, 1999 Free Software Foundation, Inc.
+ 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+[This is the first released version of the Lesser GPL.  It also counts
+ as the successor of the GNU Library Public License, version 2, hence
+ the version number 2.1.]
+
+                            Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+Licenses are intended to guarantee your freedom to share and change
+free software--to make sure the software is free for all its users.
+
+  This license, the Lesser General Public License, applies to some
+specially designated software packages--typically libraries--of the
+Free Software Foundation and other authors who decide to use it.  You
+can use it too, but we suggest you first think carefully about whether
+this license or the ordinary General Public License is the better
+strategy to use in any particular case, based on the explanations below.
+
+  When we speak of free software, we are referring to freedom of use,
+not price.  Our General Public Licenses are designed to make sure that
+you have the freedom to distribute copies of free software (and charge
+for this service if you wish); that you receive source code or can get
+it if you want it; that you can change the software and use pieces of
+it in new free programs; and that you are informed that you can do
+these things.
+
+  To protect your rights, we need to make restrictions that forbid
+distributors to deny you these rights or to ask you to surrender these
+rights.  These restrictions translate to certain responsibilities for
+you if you distribute copies of the library or if you modify it.
+
+  For example, if you distribute copies of the library, whether gratis
+or for a fee, you must give the recipients all the rights that we gave
+you.  You must make sure that they, too, receive or can get the source
+code.  If you link other code with the library, you must provide
+complete object files to the recipients, so that they can relink them
+with the library after making changes to the library and recompiling
+it.  And you must show them these terms so they know their rights.
+
+  We protect your rights with a two-step method: (1) we copyright the
+library, and (2) we offer you this license, which gives you legal
+permission to copy, distribute and/or modify the library.
+
+  To protect each distributor, we want to make it very clear that
+there is no warranty for the free library.  Also, if the library is
+modified by someone else and passed on, the recipients should know
+that what they have is not the original version, so that the original
+author's reputation will not be affected by problems that might be
+introduced by others.
+
+  Finally, software patents pose a constant threat to the existence of
+any free program.  We wish to make sure that a company cannot
+effectively restrict the users of a free program by obtaining a
+restrictive license from a patent holder.  Therefore, we insist that
+any patent license obtained for a version of the library must be
+consistent with the full freedom of use specified in this license.
+
+  Most GNU software, including some libraries, is covered by the
+ordinary GNU General Public License.  This license, the GNU Lesser
+General Public License, applies to certain designated libraries, and
+is quite different from the ordinary General Public License.  We use
+this license for certain libraries in order to permit linking those
+libraries into non-free programs.
+
+  When a program is linked with a library, whether statically or using
+a shared library, the combination of the two is legally speaking a
+combined work, a derivative of the original library.  The ordinary
+General Public License therefore permits such linking only if the
+entire combination fits its criteria of freedom.  The Lesser General
+Public License permits more lax criteria for linking other code with
+the library.
+
+  We call this license the "Lesser" General Public License because it
+does Less to protect the user's freedom than the ordinary General
+Public License.  It also provides other free software developers Less
+of an advantage over competing non-free programs.  These disadvantages
+are the reason we use the ordinary General Public License for many
+libraries.  However, the Lesser license provides advantages in certain
+special circumstances.
+
+  For example, on rare occasions, there may be a special need to
+encourage the widest possible use of a certain library, so that it becomes
+a de-facto standard.  To achieve this, non-free programs must be
+allowed to use the library.  A more frequent case is that a free
+library does the same job as widely used non-free libraries.  In this
+case, there is little to gain by limiting the free library to free
+software only, so we use the Lesser General Public License.
+
+  In other cases, permission to use a particular library in non-free
+programs enables a greater number of people to use a large body of
+free software.  For example, permission to use the GNU C Library in
+non-free programs enables many more people to use the whole GNU
+operating system, as well as its variant, the GNU/Linux operating
+system.
+
+  Although the Lesser General Public License is Less protective of the
+users' freedom, it does ensure that the user of a program that is
+linked with the Library has the freedom and the wherewithal to run
+that program using a modified version of the Library.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.  Pay close attention to the difference between a
+"work based on the library" and a "work that uses the library".  The
+former contains code derived from the library, whereas the latter must
+be combined with the library in order to run.
+
+                  GNU LESSER GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License Agreement applies to any software library or other
+program which contains a notice placed by the copyright holder or
+other authorized party saying it may be distributed under the terms of
+this Lesser General Public License (also called "this License").
+Each licensee is addressed as "you".
+
+  A "library" means a collection of software functions and/or data
+prepared so as to be conveniently linked with application programs
+(which use some of those functions and data) to form executables.
+
+  The "Library", below, refers to any such software library or work
+which has been distributed under these terms.  A "work based on the
+Library" means either the Library or any derivative work under
+copyright law: that is to say, a work containing the Library or a
+portion of it, either verbatim or with modifications and/or translated
+straightforwardly into another language.  (Hereinafter, translation is
+included without limitation in the term "modification".)
+
+  "Source code" for a work means the preferred form of the work for
+making modifications to it.  For a library, complete source code means
+all the source code for all modules it contains, plus any associated
+interface definition files, plus the scripts used to control compilation
+and installation of the library.
+
+  Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running a program using the Library is not restricted, and output from
+such a program is covered only if its contents constitute a work based
+on the Library (independent of the use of the Library in a tool for
+writing it).  Whether that is true depends on what the Library does
+and what the program that uses the Library does.
+
+  1. You may copy and distribute verbatim copies of the Library's
+complete source code as you receive it, in any medium, provided that
+you conspicuously and appropriately publish on each copy an
+appropriate copyright notice and disclaimer of warranty; keep intact
+all the notices that refer to this License and to the absence of any
+warranty; and distribute a copy of this License along with the
+Library.
+
+  You may charge a fee for the physical act of transferring a copy,
+and you may at your option offer warranty protection in exchange for a
+fee.
+
+  2. You may modify your copy or copies of the Library or any portion
+of it, thus forming a work based on the Library, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) The modified work must itself be a software library.
+
+    b) You must cause the files modified to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    c) You must cause the whole of the work to be licensed at no
+    charge to all third parties under the terms of this License.
+
+    d) If a facility in the modified Library refers to a function or a
+    table of data to be supplied by an application program that uses
+    the facility, other than as an argument passed when the facility
+    is invoked, then you must make a good faith effort to ensure that,
+    in the event an application does not supply such function or
+    table, the facility still operates, and performs whatever part of
+    its purpose remains meaningful.
+
+    (For example, a function in a library to compute square roots has
+    a purpose that is entirely well-defined independent of the
+    application.  Therefore, Subsection 2d requires that any
+    application-supplied function or table used by this function must
+    be optional: if the application does not supply it, the square
+    root function must still compute square roots.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Library,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Library, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote
+it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Library.
+
+In addition, mere aggregation of another work not based on the Library
+with the Library (or with a work based on the Library) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may opt to apply the terms of the ordinary GNU General Public
+License instead of this License to a given copy of the Library.  To do
+this, you must alter all the notices that refer to this License, so
+that they refer to the ordinary GNU General Public License, version 2,
+instead of to this License.  (If a newer version than version 2 of the
+ordinary GNU General Public License has appeared, then you can specify
+that version instead if you wish.)  Do not make any other change in
+these notices.
+
+  Once this change is made in a given copy, it is irreversible for
+that copy, so the ordinary GNU General Public License applies to all
+subsequent copies and derivative works made from that copy.
+
+  This option is useful when you wish to copy part of the code of
+the Library into a program that is not a library.
+
+  4. You may copy and distribute the Library (or a portion or
+derivative of it, under Section 2) in object code or executable form
+under the terms of Sections 1 and 2 above provided that you accompany
+it with the complete corresponding machine-readable source code, which
+must be distributed under the terms of Sections 1 and 2 above on a
+medium customarily used for software interchange.
+
+  If distribution of object code is made by offering access to copy
+from a designated place, then offering equivalent access to copy the
+source code from the same place satisfies the requirement to
+distribute the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  5. A program that contains no derivative of any portion of the
+Library, but is designed to work with the Library by being compiled or
+linked with it, is called a "work that uses the Library".  Such a
+work, in isolation, is not a derivative work of the Library, and
+therefore falls outside the scope of this License.
+
+  However, linking a "work that uses the Library" with the Library
+creates an executable that is a derivative of the Library (because it
+contains portions of the Library), rather than a "work that uses the
+library".  The executable is therefore covered by this License.
+Section 6 states terms for distribution of such executables.
+
+  When a "work that uses the Library" uses material from a header file
+that is part of the Library, the object code for the work may be a
+derivative work of the Library even though the source code is not.
+Whether this is true is especially significant if the work can be
+linked without the Library, or if the work is itself a library.  The
+threshold for this to be true is not precisely defined by law.
+
+  If such an object file uses only numerical parameters, data
+structure layouts and accessors, and small macros and small inline
+functions (ten lines or less in length), then the use of the object
+file is unrestricted, regardless of whether it is legally a derivative
+work.  (Executables containing this object code plus portions of the
+Library will still fall under Section 6.)
+
+  Otherwise, if the work is a derivative of the Library, you may
+distribute the object code for the work under the terms of Section 6.
+Any executables containing that work also fall under Section 6,
+whether or not they are linked directly with the Library itself.
+
+  6. As an exception to the Sections above, you may also combine or
+link a "work that uses the Library" with the Library to produce a
+work containing portions of the Library, and distribute that work
+under terms of your choice, provided that the terms permit
+modification of the work for the customer's own use and reverse
+engineering for debugging such modifications.
+
+  You must give prominent notice with each copy of the work that the
+Library is used in it and that the Library and its use are covered by
+this License.  You must supply a copy of this License.  If the work
+during execution displays copyright notices, you must include the
+copyright notice for the Library among them, as well as a reference
+directing the user to the copy of this License.  Also, you must do one
+of these things:
+
+    a) Accompany the work with the complete corresponding
+    machine-readable source code for the Library including whatever
+    changes were used in the work (which must be distributed under
+    Sections 1 and 2 above); and, if the work is an executable linked
+    with the Library, with the complete machine-readable "work that
+    uses the Library", as object code and/or source code, so that the
+    user can modify the Library and then relink to produce a modified
+    executable containing the modified Library.  (It is understood
+    that the user who changes the contents of definitions files in the
+    Library will not necessarily be able to recompile the application
+    to use the modified definitions.)
+
+    b) Use a suitable shared library mechanism for linking with the
+    Library.  A suitable mechanism is one that (1) uses at run time a
+    copy of the library already present on the user's computer system,
+    rather than copying library functions into the executable, and (2)
+    will operate properly with a modified version of the library, if
+    the user installs one, as long as the modified version is
+    interface-compatible with the version that the work was made with.
+
+    c) Accompany the work with a written offer, valid for at
+    least three years, to give the same user the materials
+    specified in Subsection 6a, above, for a charge no more
+    than the cost of performing this distribution.
+
+    d) If distribution of the work is made by offering access to copy
+    from a designated place, offer equivalent access to copy the above
+    specified materials from the same place.
+
+    e) Verify that the user has already received a copy of these
+    materials or that you have already sent this user a copy.
+
+  For an executable, the required form of the "work that uses the
+Library" must include any data and utility programs needed for
+reproducing the executable from it.  However, as a special exception,
+the materials to be distributed need not include anything that is
+normally distributed (in either source or binary form) with the major
+components (compiler, kernel, and so on) of the operating system on
+which the executable runs, unless that component itself accompanies
+the executable.
+
+  It may happen that this requirement contradicts the license
+restrictions of other proprietary libraries that do not normally
+accompany the operating system.  Such a contradiction means you cannot
+use both them and the Library together in an executable that you
+distribute.
+
+  7. You may place library facilities that are a work based on the
+Library side-by-side in a single library together with other library
+facilities not covered by this License, and distribute such a combined
+library, provided that the separate distribution of the work based on
+the Library and of the other library facilities is otherwise
+permitted, and provided that you do these two things:
+
+    a) Accompany the combined library with a copy of the same work
+    based on the Library, uncombined with any other library
+    facilities.  This must be distributed under the terms of the
+    Sections above.
+
+    b) Give prominent notice with the combined library of the fact
+    that part of it is a work based on the Library, and explaining
+    where to find the accompanying uncombined form of the same work.
+
+  8. You may not copy, modify, sublicense, link with, or distribute
+the Library except as expressly provided under this License.  Any
+attempt otherwise to copy, modify, sublicense, link with, or
+distribute the Library is void, and will automatically terminate your
+rights under this License.  However, parties who have received copies,
+or rights, from you under this License will not have their licenses
+terminated so long as such parties remain in full compliance.
+
+  9. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Library or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Library (or any work based on the
+Library), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Library or works based on it.
+
+  10. Each time you redistribute the Library (or any work based on the
+Library), the recipient automatically receives a license from the
+original licensor to copy, distribute, link with or modify the Library
+subject to these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties with
+this License.
+
+  11. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Library at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Library by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Library.
+
+If any portion of this section is held invalid or unenforceable under any
+particular circumstance, the balance of the section is intended to apply,
+and the section as a whole is intended to apply in other circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  12. If the distribution and/or use of the Library is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Library under this License may add
+an explicit geographical distribution limitation excluding those countries,
+so that distribution is permitted only in or among countries not thus
+excluded.  In such case, this License incorporates the limitation as if
+written in the body of this License.
+
+  13. The Free Software Foundation may publish revised and/or new
+versions of the Lesser General Public License from time to time.
+Such new versions will be similar in spirit to the present version,
+but may differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Library
+specifies a version number of this License which applies to it and
+"any later version", you have the option of following the terms and
+conditions either of that version or of any later version published by
+the Free Software Foundation.  If the Library does not specify a
+license version number, you may choose any version ever published by
+the Free Software Foundation.
+
+  14. If you wish to incorporate parts of the Library into other free
+programs whose distribution conditions are incompatible with these,
+write to the author to ask for permission.  For software which is
+copyrighted by the Free Software Foundation, write to the Free
+Software Foundation; we sometimes make exceptions for this.  Our
+decision will be guided by the two goals of preserving the free status
+of all derivatives of our free software and of promoting the sharing
+and reuse of software generally.
+
+                            NO WARRANTY
+
+  15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
+WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
+EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
+OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
+KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
+LIBRARY IS WITH YOU.  SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
+THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
+WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
+AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
+FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
+CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
+LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
+RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
+FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
+SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
+DAMAGES.
+
+                     END OF TERMS AND CONDITIONS
+
+           How to Apply These Terms to Your New Libraries
+
+  If you develop a new library, and you want it to be of the greatest
+possible use to the public, we recommend making it free software that
+everyone can redistribute and change.  You can do so by permitting
+redistribution under these terms (or, alternatively, under the terms of the
+ordinary General Public License).
+
+  To apply these terms, attach the following notices to the library.  It is
+safest to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least the
+"copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the library's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This library is free software; you can redistribute it and/or
+    modify it under the terms of the GNU Lesser General Public
+    License as published by the Free Software Foundation; either
+    version 2.1 of the License, or (at your option) any later version.
+
+    This library is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+    Lesser General Public License for more details.
+
+    You should have received a copy of the GNU Lesser General Public
+    License along with this library; if not, write to the Free Software
+    Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+
+Also add information on how to contact you by electronic and paper mail.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the library, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the
+  library `Frob' (a library for tweaking knobs) written by James Random Hacker.
+
+  <signature of Ty Coon>, 1 April 1990
+  Ty Coon, President of Vice
+
+That's all there is to it!

3rdparty/udns/Makefile.in

@@ -0,0 +1,197 @@
+#! /usr/bin/make -rf
+# Makefile.in
+# libudns Makefile
+#
+# Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+# This file is part of UDNS library, an async DNS stub resolver.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library, in file named COPYING.LGPL; if not,
+# write to the Free Software Foundation, Inc., 59 Temple Place,
+# Suite 330, Boston, MA  02111-1307  USA
+
+NAME   = udns
+VERS   = 0.4
+SOVER  = 0
+
+SRCS   = udns_dn.c udns_dntosp.c udns_parse.c udns_resolver.c udns_init.c \
+	udns_misc.c udns_XtoX.c \
+	udns_rr_a.c udns_rr_ptr.c udns_rr_mx.c udns_rr_txt.c udns_bl.c \
+	udns_rr_srv.c udns_rr_naptr.c udns_codes.c udns_jran.c
+USRCS  = dnsget.c rblcheck.c ex-rdns.c
+DIST   = COPYING.LGPL udns.h udns.3 dnsget.1 rblcheck.1 $(SRCS) $(USRCS) \
+	 NEWS TODO NOTES Makefile.in configure configure.lib \
+	 inet_XtoX.c getopt.c
+
+OBJS   = $(SRCS:.c=.o) $(GEN:.c=.o)
+LIB    = lib$(NAME).a
+LIBFL  = -L. -l$(NAME)
+
+SOBJS  = $(OBJS:.o=.lo)
+SOLIB  = lib$(NAME)_s.so
+SOLIBV = lib$(NAME).so.$(SOVER)
+SOLIBFL= -L. -l$(NAME)_s
+
+UTILS   = $(USRCS:.c=)
+UOBJS   = $(USRCS:.c=.o)
+SOUTILS = $(USRCS:.c=_s)
+
+NAMEPFX = $(NAME)-$(VERS)
+
+CC = @CC@
+CFLAGS = @CFLAGS@
+CDEFS = @CDEFS@
+LD = @LD@
+LDFLAGS = @LDFLAGS@
+LIBS = @LIBS@
+LDSHARED = $(LD) -shared
+PICFLAGS = -fPIC
+AWK = awk
+TAR = tar
+
+all: static
+
+.SUFFIXES: .c .o .lo
+
+static: $(LIB) $(UTILS)
+staticlib: $(LIB)
+$(LIB): $(OBJS)
+	-rm -f $@
+	$(AR) rv $@ $(OBJS)
+.c.o:
+	$(CC) $(CFLAGS) $(CDEFS) -c $<
+
+shared: $(SOLIBV) $(SOUTILS)
+sharedlib: $(SOLIBV)
+
+$(SOLIBV): $(SOBJS)
+	$(LDSHARED) -Wl,--soname,$(SOLIBV) -o $@ $(SOBJS) $(LDFLAGS) $(LIBS)
+$(SOLIB): $(SOLIBV)
+	rm -f $@
+	ln -s $(SOLIBV) $@
+.c.lo:
+	$(CC) $(CFLAGS) $(PICFLAGS) $(CDEFS) -o $@ -c $<
+
+# udns_codes.c is generated from udns.h
+udns_codes.c: udns.h
+	@echo Generating $@
+	@set -e; exec >$@.tmp; \
+	set T type C class R rcode; \
+	echo "/* Automatically generated. */"; \
+	echo "#include \"udns.h\""; \
+	while [ "$$1" ]; do \
+	 echo; \
+	 echo "const struct dns_nameval dns_$${2}tab[] = {"; \
+	 $(AWK) "/^  DNS_$${1}_[A-Z0-9_]+[ 	]*=/ \
+	  { printf \" {%s,\\\"%s\\\"},\\n\", \$$1, substr(\$$1,7) }" \
+	  udns.h ; \
+	 echo " {0,0}};"; \
+	 echo "const char *dns_$${2}name(enum dns_$${2} code) {"; \
+	 echo " static char nm[20];"; \
+	 echo " switch(code) {"; \
+	 $(AWK) "BEGIN{i=0} \
+	   /^  DNS_$${1}_[A-Z0-9_]+[ 	]*=/ \
+	   {printf \" case %s: return dns_$${2}tab[%d].name;\\n\",\$$1,i++}\
+	   " udns.h ; \
+	 echo " }"; \
+	 echo " return _dns_format_code(nm,\"$$2\",code);"; \
+	 echo "}"; \
+	 shift 2; \
+	done
+	@mv $@.tmp $@
+
+udns.3.html: udns.3
+	groff -man -Thtml udns.3 > $@.tmp
+	mv $@.tmp $@
+
+dist: $(NAMEPFX).tar.gz
+$(NAMEPFX).tar.gz: $(DIST)
+	$(TAR) -cv -f $@ -z --transform 's|^|$(NAMEPFX)/|' $(DIST)
+
+subdist:
+	cp -p $(DIST) $(TARGET)/
+
+clean:
+	rm -f $(OBJS)
+	rm -f $(SOBJS)
+	rm -f $(UOBJS)
+	rm -f config.log
+distclean: clean
+	rm -f $(LIB) $(SOLIB) $(SOLIBV) udns.3.html
+	rm -f $(UTILS) $(SOUTILS)
+	rm -f config.status config.h Makefile
+
+
+Makefile: configure configure.lib Makefile.in
+	./configure
+	@echo
+	@echo Please rerun make >&2
+	@exit 1
+
+.PHONY: all static staticlib shared sharedlib dist clean distclean subdist \
+  depend dep deps
+
+depend dep deps: $(SRCS) $(USRC)
+	@echo Generating deps for:
+	@echo \ $(SRCS)
+	@echo \ $(USRCS)
+	@sed '/^# depend/q' Makefile.in > Makefile.tmp
+	@set -e; \
+	for f in $(SRCS) $(USRCS); do \
+	 echo $${f%.c}.o $${f%.c}.lo: $$f \
+	  `sed -n 's/^#[ 	]*include[ 	]*"\(.*\)".*/\1/p' $$f`; \
+	done >> Makefile.tmp; \
+	for f in $(USRCS:.c=.o); do \
+	 echo "$${f%.?}: $$f \$$(LIB)"; \
+	 echo "	\$$(LD) \$$(LDLAGS) -o \$$@ $$f \$$(LIBFL) \$$(LIBS)"; \
+	 echo "$${f%.?}_s: $$f \$$(SOLIB)"; \
+	 echo "	\$$(LD) \$$(LDFLAGS) -o \$$@ $$f \$$(SOLIBFL)"; \
+	done >> Makefile.tmp ; \
+	if cmp Makefile.tmp Makefile.in >/dev/null 2>&1 ; then \
+	 echo Makefile.in unchanged; rm -f Makefile.tmp; \
+	else \
+	 echo Updating Makfile.in; mv -f Makefile.tmp Makefile.in ; \
+	fi
+
+# depend
+udns_dn.o udns_dn.lo: udns_dn.c udns.h
+udns_dntosp.o udns_dntosp.lo: udns_dntosp.c udns.h
+udns_parse.o udns_parse.lo: udns_parse.c udns.h
+udns_resolver.o udns_resolver.lo: udns_resolver.c config.h udns.h
+udns_init.o udns_init.lo: udns_init.c config.h udns.h
+udns_misc.o udns_misc.lo: udns_misc.c udns.h
+udns_XtoX.o udns_XtoX.lo: udns_XtoX.c config.h udns.h inet_XtoX.c
+udns_rr_a.o udns_rr_a.lo: udns_rr_a.c udns.h
+udns_rr_ptr.o udns_rr_ptr.lo: udns_rr_ptr.c udns.h
+udns_rr_mx.o udns_rr_mx.lo: udns_rr_mx.c udns.h
+udns_rr_txt.o udns_rr_txt.lo: udns_rr_txt.c udns.h
+udns_bl.o udns_bl.lo: udns_bl.c udns.h
+udns_rr_srv.o udns_rr_srv.lo: udns_rr_srv.c udns.h
+udns_rr_naptr.o udns_rr_naptr.lo: udns_rr_naptr.c udns.h
+udns_codes.o udns_codes.lo: udns_codes.c udns.h
+udns_jran.o udns_jran.lo: udns_jran.c udns.h
+dnsget.o dnsget.lo: dnsget.c config.h udns.h getopt.c
+rblcheck.o rblcheck.lo: rblcheck.c config.h udns.h getopt.c
+ex-rdns.o ex-rdns.lo: ex-rdns.c udns.h
+dnsget: dnsget.o $(LIB)
+	$(LD) $(LDLAGS) -o $@ dnsget.o $(LIBFL) $(LIBS)
+dnsget_s: dnsget.o $(SOLIB)
+	$(LD) $(LDFLAGS) -o $@ dnsget.o $(SOLIBFL)
+rblcheck: rblcheck.o $(LIB)
+	$(LD) $(LDLAGS) -o $@ rblcheck.o $(LIBFL) $(LIBS)
+rblcheck_s: rblcheck.o $(SOLIB)
+	$(LD) $(LDFLAGS) -o $@ rblcheck.o $(SOLIBFL)
+ex-rdns: ex-rdns.o $(LIB)
+	$(LD) $(LDLAGS) -o $@ ex-rdns.o $(LIBFL) $(LIBS)
+ex-rdns_s: ex-rdns.o $(SOLIB)
+	$(LD) $(LDFLAGS) -o $@ ex-rdns.o $(SOLIBFL)

3rdparty/udns/NEWS

@@ -0,0 +1,136 @@
+NEWS
+User-visible changes in udns library.  Recent changes on top.
+
+0.4 (Jan 2014)
+
+ - bugfix: fix a bug in new list code introduced in 0.3
+ - portability: use $(LD)/$(LDFLAGS)/$(LIBS)
+
+0.3 (Jan 2014)
+
+ - bugfix: refactor double-linked list implementation in udns_resolver.c
+   (internal to the library) to be more strict-aliasing-friendly, because
+   old code were miscompiled by gcc.
+
+ - bugfix: forgotten strdup() in rblcheck
+
+0.2 (Dec 2011)
+
+ - bugfix: SRV RR handling: fix domain name parsing and crash in case
+   if no port is specified on input for SRV record query
+
+ - (trivial api) dns_set_opts() now returns number of unrecognized 
+   options instead of always returning 0
+
+ - dnsget: combine -f and -o options in dnsget (and stop documenting -f),
+   and report unknown/invalid -o options (and error out)
+
+ - dnsget: pretty-print SSHFP RRs
+
+ 0.1 (Dec 2010)
+
+ - bugfix: udns_new(old) - when actually cloning another context -
+   makes the new context referencing memory from old, which leads
+   to crashes when old is modified later
+
+ - use random queue IDs (the 16bit qID) in queries instead of sequentional
+   ones, based on simple pseudo-random RNG by Bob Jenkins (udns_jran.[ch]).
+   Some people believe that this improves security (CVE-2008-1447).  I'm
+   still not convinced (see comments in udns_resolver.c), but it isn't
+   difficult to add after all.
+
+ - deprecate dns_random16() function which was declared in udns.h
+   (not anymore) but never documented.  In order to keep ABI compatible
+   it is still exported.
+
+ - library has a way now to set query flags (DNS_SET_DO; DNS_SET_CD).
+
+ - dnsget now prints non-printable chars in all strings in DNS RRs using
+   decimal escape sequences (\%03u) instead of hexadecimal (\%02x) when
+   before - other DNS software does it like this.
+
+ - recognize a few more record types in dnsget, notable some DNSSEC RRs;
+   add -f option for dnsget to set query flags.
+
+ - udns is not a Debian native package anymore (was a wrong idea)
+
+0.0.9 (16 Jan 2007)
+
+ - incompat: minor API changes in dns_init() &friends.  dns_init()
+   now requires extra `struct dns_ctx *' argument.  Not bumped
+   soversion yet - I only expect one "release" with this change.
+
+ - many small bugfixes, here and there
+
+ - more robust FORMERR replies handling - not only such replies are now
+   recognized, but udns retries queries without EDNS0 extensions if tried
+   with, but server reported FORMERR
+
+ - portability changes, udns now includes getopt() implementation fo
+   the systems lacking it (mostly windows), and dns_ntop()&dns_pton(),
+   which are either just wrappers for system functions or reimplementations.
+
+ - build is now based on autoconf-like configuration
+
+ - NAPTR (RFC3403) RR decoding support
+
+ - new file NOTES which complements TODO somewhat, and includes some
+   important shortcomings
+
+ - many internal cleanups, including some preparations for better error
+   recovery, security and robustness (and thus API changes)
+
+ - removed some #defines which are now unused (like DNS_MAXSRCH)
+
+ - changed WIN32 to WINDOWS everywhere in preprocessor tests,
+   to be able to build it on win64 as well
+
+0.0.8 (12 Sep 2005)
+
+ - added SRV records (rfc2782) parsing,
+   thanks to Thadeu Lima de Souza Cascardo for implementation.
+
+ - bugfixes:
+   o use uninitialized value when no reply, library died with assertion:
+     assert((status < 0 && result == 0) || (status >= 0 && result != 0)).
+   o on some OSes, struct sockaddr_in has additional fields, so
+     memcmp'ing two sockaddresses does not work.
+
+ - rblcheck(.1)
+
+0.0.7 (20 Apr 2005)
+
+ - dnsget.1 manpage and several enhancements to dnsget.
+
+ - allow nameserver names for -n option of dnsget.
+
+ - API change: all dns_submit*() routines now does not expect
+   last `now' argument, since requests aren't sent immediately
+   anymore.
+
+ - API change: different application timer callback mechanism.
+   Udns now uses single per-context timer instead of per-query.
+
+ - don't assume DNS replies only contain backward DN pointers,
+   allow forward pointers too.  Change parsing API.
+
+ - debianize
+
+0.0.6 (08 Apr 2005)
+
+ - use double sorted list for requests (sorted by deadline).
+   This should significantly speed up timeout processing for
+   large number of requests.
+
+ - changed debugging interface, so it is finally useable
+   (still not documented).
+
+ - dnsget routine is now Officially Useable, and sometimes
+   even more useable than `host' from BIND distribution
+   (and sometimes not - dnsget does not have -C option
+   and TCP mode)
+
+ - Debian packaging in debian/ -- udns is now maintained as a
+   native Debian package.
+
+ - alot (and I really mean alot) of code cleanups all over.

3rdparty/udns/NOTES

@@ -0,0 +1,226 @@
+Assorted notes about udns (library).
+
+UDP-only mode
+~~~~~~~~~~~~~
+
+First of all, since udns is (currently) UDP-only, there are some
+shortcomings.
+
+It assumes that a reply will fit into a UDP buffer.  With adoption of EDNS0,
+and general robustness of IP stacks, in most cases it's not an issue.  But
+in some cases there may be problems:
+
+ - if an RRset is "very large" so it does not fit even in buffer of size
+   requested by the library (current default is 4096; some servers limits
+   it further), we will not see the reply, or will only see "damaged"
+   reply (depending on the server).
+
+ - many DNS servers ignores EDNS0 option requests.  In this case, no matter
+   which buffer size udns library will request, such servers reply is limited
+   to 512 bytes (standard pre-EDNS0 DNS packet size).  (Udns falls back to
+   non-EDNO0 query if EDNS0-enabled one received FORMERR or NOTIMPL error).
+
+The problem is that with this, udns currently will not consider replies with
+TC (truncation) bit set, and will treat such replies the same way as it
+treats SERVFAIL replies, thus trying next server, or temp-failing the query
+if no more servers to try.  In other words, if the reply is really large, or
+if the servers you're using don't support EDNS0, your application will be
+unable to resolve a given name.
+
+Yet it's not common situation - in practice, it's very rare.
+
+Implementing TCP mode isn't difficult, but it complicates API significantly.
+Currently udns uses only single UDP socket (or - maybe in the future - two,
+see below), but in case of TCP, it will need to open and close sockets for
+TCP connections left and right, and that have to be integrated into an
+application's event loop in an easy and efficient way.  Plus all the
+timeouts - different for connect(), write, and several stages of read.
+
+IPv6 vs IPv4 usage
+~~~~~~~~~~~~~~~~~~
+
+This is only relevant for nameservers reachable over IPv6, NOT for IPv6
+queries.  I.e., if you've IPv6 addresses in 'nameservers' line in your
+/etc/resolv.conf file.  Even more: if you have BOTH IPv6 AND IPv4 addresses
+there.  Or pass them to udns initialization routines.
+
+Since udns uses a single UDP socket to communicate with all nameservers,
+it should support both v4 and v6 communications.  Most current platforms
+supports this mode - using PF_INET6 socket and V4MAPPED addresses, i.e,
+"tunnelling" IPv4 inside IPv6.  But not all systems supports this.  And
+more, it has been said that such mode is deprecated.
+
+So, list only IPv4 or only IPv6 addresses, but don't mix them, in your
+/etc/resolv.conf.
+
+An alternative is to use two sockets instead of 1 - one for IPv6 and one
+for IPv4.  For now I'm not sure if it's worth the complexity - again, of
+the API, not the library itself (but this will not simplify library either).
+
+Single socket for all queries
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Using single UDP socket for sending queries to all nameservers has obvious
+advantages.  First it's, again, trivial, simple to use API.  And simple
+library too.  Also, after sending queries to all nameservers (in case first
+didn't reply in time), we will be able to receive late reply from first
+nameserver and accept it.
+
+But this mode has disadvantages too.  Most important is that it's much easier
+to send fake reply to us, as the UDP port where we expects the reply to come
+to is constant during the whole lifetime of an application.  More secure
+implementations uses random port for every single query.  While port number
+(16 bits integer) can not hold much randomness, it's still of some help.
+Ok, udns is a stub resolver, so it expects sorta friendly environment, but
+on LAN it's usually much easier to fire an attack, due to the speed of local
+network, where a bad guy can generate alot of packets in a short time.
+
+Spoofing of replies (Kaminsky attack, CVE-2008-1447)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+While udns uses random numbers for query IDs, it uses single UDP port for
+all queries (see previous item).  And even if it used random UDP port for
+each query, the attack described in CVE-2008-1447 is still quite trivial.
+This is not specific to udns library unfortunately - it is inherent property
+of the protocol.  Udns is designed to work in a LAN, it needs full recursive
+resolver nearby, and modern LAN usually uses high-bandwidth equipment which
+makes the Kaminsky attack trivial.  The problem is that even with qID (16
+bits) and random UDP port (about 20 bits available to a regular process)
+combined still can not hold enough randomness, so on a fast network it is
+still easy to flood the target with fake replies and hit the "right" reply
+before real reply comes.  So random qIDs don't add much protection anyway,
+even if this feature is implemented in udns, and using all available
+techniques wont solve it either.
+
+See also long comment in udns_resolver.c, udns_newid().
+
+Assumptions about RRs returned
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Currently udns processes records in the reply it received sequentially.
+This means that order of the records is significant.  For example, if
+we asked for foo.bar A, but the server returned that foo.bar is a CNAME
+(alias) for bar.baz, and bar.baz, in turn, has address 1.2.3.4, when
+the CNAME should come first in reply, followed by A.  While DNS specs
+does not say anything about order of records - it's an rrSET - unordered, -
+I think an implementation which returns the records in "wrong" order is
+somewhat insane...
+
+CNAME recursion
+~~~~~~~~~~~~~~~
+
+Another interesting point is the handling of CNAMEs returned as replies
+to non-CNAME queries.  If we asked for foo.bar A, but it's a CNAME, udns
+expects BOTH the CNAME itself and the target DN to be present in the reply.
+In other words, udns DOES NOT RECURSE CNAMES.  If we asked for foo.bar A,
+but only record in reply was that foo.bar is a CNAME for bar.baz, udns will
+return no records to an application (NXDOMAIN).  Strictly speaking, udns
+should repeat the query asking for bar.baz A, and recurse.  But since it's
+stub resolver, recursive resolver should recurse for us instead.
+
+It's not very difficult to implement, however.  Probably with some (global?)
+flag to en/dis-able the feature.  Provided there's some demand for it.
+
+To clarify: udns handles CNAME recursion in a single reply packet just fine.
+
+Note also that standard gethostbyname() routine does not recurse in this
+situation, too.
+
+Error reporting
+~~~~~~~~~~~~~~~
+
+Too many places in the code (various failure paths) sets generic "TEMPFAIL"
+error condition.  For example, if no nameserver replied to our query, an
+application will get generic TEMPFAIL, instead of something like TIMEDOUT.
+This probably should be fixed, but most applications don't care about the
+exact reasons of failure - 4 common cases are already too much:
+  - query returned some valid data
+  - NXDOMAIN
+  - valid domain but no data of requested type - =NXDOMAIN in most cases
+  - temporary error - this one sometimes (incorrectly!) treated as NXDOMAIN
+    by (naive) applications.
+DNS isn't yes/no, it's at least 3 variants, temp err being the 3rd important
+case!  And adding more variations for the temp error case is complicating things
+even more - again, from an application writer standpoint.  For diagnostics,
+such more specific error cases are of good help.
+
+Planned API changes
+~~~~~~~~~~~~~~~~~~~
+
+At least one thing I want to change for some future version is a way how
+queries are submitted and how replies are handled.
+
+I want to made dns_query object to be owned by an application.  So that instead
+of udns library allocating it for the lifetime of query, it will be pre-
+allocated by an application.  This simplifies and enhances query submitting
+interface, and complicates it a bit too, in simplest cases.
+
+Currently, we have:
+
+dns_submit_dn(dn, cls, typ, flags, parse, cbck, data)
+dns_submit_p(name, cls, typ, flags, parse, cbck, data)
+dns_submit_a4(ctx, name, flags, cbck, data)
+
+and so on -- with many parameters missed for type-specific cases, but generic
+cases being too complex for most common usage.
+
+Instead, with dns_query being owned by an app, we will be able to separately
+set up various parts of the query - domain name (various forms), type&class,
+parser, flags, callback...  and even change them at runtime.  And we will also
+be able to reuse query structures, instead of allocating/freeing them every
+time.  So the whole thing will look something like:
+
+ q = dns_alloc_query();
+ dns_submit(dns_q_flags(dns_q_a4(q, name, cbck), DNS_F_NOSRCH), data);
+
+The idea is to have a set of functions accepting struct dns_query* and
+returning it (so the calls can be "nested" like the above), to set up
+relevant parts of the query - specific type of callback, conversion from
+(type-specific) query parameters into a domain name (this is for type-
+specific query initializers), and setting various flags and options and
+type&class things.
+
+One example where this is almost essential - if we want to support
+per-query set of nameservers (which isn't at all useless: imagine a
+high-volume mail server, were we want to direct DNSBL queries to a separate
+set of nameservers, and rDNS queries to their own set and so on).  Adding
+another argument (set of nameservers to use) to EVERY query submitting
+routine is.. insane.  Especially since in 99% cases it will be set to
+default NULL.  But with such "nesting" of query initializers, it becomes
+trivial.
+
+This change (the way how queries gets submitted) will NOT break API/ABI
+compatibility with old versions, since the new submitting API works in
+parallel with current (and current will use the new one as building
+blocks, instead of doing all work at once).
+
+Another way to do the same is to manipulate query object right after a
+query has been submitted, but before any events processing (during this
+time, query object is allocated and initialized, but no actual network
+packets were sent - it will happen on the next event processing).  But
+this way it become impossible to perform syncronous resolver calls, since
+those calls hide query objects they use internally.
+
+Speaking of replies handling - the planned change is to stop using dynamic
+memory (malloc) inside the library.  That is, instead of allocating a buffer
+for a reply dynamically in a parsing routine (or memdup'ing the raw reply
+packet if no parsing routine is specified), I want udns to return the packet
+buffer it uses internally, and change parsing routines to expect a buffer
+for result.  When parsing, a routine will return true amount of memory it
+will need to place the result, regardless of whenever it has enough room
+or not, so that an application can (re)allocate properly sized buffer and
+call a parsing routine again.
+
+This, in theory, also can be done without breaking current API/ABI, but in
+that case we'll again need a parallel set of routines (parsing included),
+which makes the library more complicated with too many ways of doing the
+same thing.  Still, code reuse is at good level.
+
+Another modification I plan to include is to have an ability to work in
+terms of domain names (DNs) as used with on-wire DNS packets, not only
+with asciiz representations of them.  For this to work, the above two
+changes (query submission and result passing) have to be completed first
+(esp. the query submission part), so that it will be possible to specify
+some additional query flags (for example) to request domain names instead
+of the text strings, and to allow easy query submissions with either DNs
+or text strings.

3rdparty/udns/TODO

@@ -0,0 +1,59 @@
+TODO
+
+The following is mostly an internal, not user-visible stuff.
+
+* rearrange an API to make dns_query object owned by application,
+  so that it'll look like this:
+   struct dns_query *q;
+   q = dns_query_alloc(ctx);
+   dns_query_set(q, options, domain_name, flags, ...);
+   dns_query_submit(ctx, q);
+  For more information see NOTES file, section "Planned API changes".
+
+* allow NULL callbacks?  Or provide separate resolver
+  context list of queries which are done but wich did not
+  have callback, and dns_pick() routine to retrieve results
+  from this query, i.e. allow non-callback usage?  The
+  non-callback usage may be handy sometimes (any *good*
+  example?), but it will be difficult to provide type-safe
+  non-callback interface due to various RR-specific types
+  in use.
+
+* DNS_OPT_FLAGS should be DNS_OPT_ADDFLAGS and DNS_OPT_SETFLAGS.
+  Currently one can't add a single flag bit but preserve
+  existing bits... at least not without retrieving all current
+  flags before, which isn't that bad anyway.
+
+* dns_set_opts() may process flags too (such as aaonly etc)
+
+* a way to disable $NSCACHEIP et al processing?
+  (with now separate dns_init() and dns_reset(), it has finer
+  control, but still no way to init from system files but ignore
+  environment variables and the like)
+
+* initialize/open the context automatically, and be more
+  liberal about initialization in general?
+
+* dns_init(ctx, do_open) - make the parameter opposite, aka
+  dns_init(ctx, skip_open) ?
+
+* allow TCP queue?
+
+* more accurate error reporting.  Currently, udns always returns TEMPFAIL,
+  but don't specify why it happened (ENOMEM, timeout, etc).
+
+* check the error value returned by recvfrom() and
+  sendto() and determine which errors to ignore.
+
+* maybe merge dns_timeouts() and dns_ioevent(), to have
+  only one entry point for everything?  For traditional
+  select-loop-based eventloop it may be easier, but for
+  callback-driven event loops the two should be separate.
+  Provide an option, or a single dns_events() entry point
+  for select-loop approach, or just call dns_ioevent()
+  from within dns_timeouts() (probably after renaming
+  it to be dns_events()) ?
+
+* implement /etc/hosts lookup too, ala [c-]ares??
+
+* sortlist support?

3rdparty/udns/config.h.in

@@ -0,0 +1,5 @@
+#cmakedefine HAVE_POLL
+#cmakedefine HAVE_GETOPT
+#cmakedefine HAVE_INET_PTON_NTOP
+#cmakedefine HAVE_IPv6
+#cmakedefine WINDOWS

3rdparty/udns/configure

@@ -0,0 +1,165 @@
+#! /bin/sh
+# autoconf-style configuration script
+#
+
+set -e
+
+name=udns
+
+if [ -f udns.h -a -f udns_resolver.c ] ; then :
+else
+  echo "configure: error: sources not found at `pwd`" >&2
+  exit 1
+fi
+
+options="ipv6"
+
+for opt in $options; do
+  eval enable_$opt=
+done
+
+if [ -f config.status ]; then
+  . ./config.status
+fi
+
+enable() {
+  opt=`echo "$1" | sed 's/^--[^-]*-//'`
+  case "$opt" in
+    ipv6) ;;
+    *) echo "configure: unrecognized option \`$1'" >&2; exit 1;;
+  esac
+  eval enable_$opt=$2
+}
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --disable-*|--without-*|--no-*) enable "$1" n;;
+    --enable-*|--with-*) enable "$1" y;;
+    --help | --hel | --he | --h | -help | -hel | -he | -h )
+      cat <<EOF
+configure: configure $name package.
+Usage: ./configure [options]
+where options are:
+ --enable-option, --with-option --
+   enable the named option/feature
+ --disable-option, --without-option, --no-option --
+   disable the named option/feature
+ --help - print this help and exit
+Optional features (all enabled by default if system supports a feature):
+  ipv6 - enable/disable IP version 6 (IPv6) support
+EOF
+      exit 0
+      ;;
+    *) echo "configure: unknown option \`$1'" >&2; exit 1 ;;
+  esac
+  shift
+done
+
+. ./configure.lib
+
+ac_msg "configure"
+ac_result "$name package"
+
+ac_prog_c_compiler_v
+ac_prog_ranlib_v
+
+ac_ign ac_yesno "for getopt()" ac_have GETOPT ac_link <<EOF
+#include <stdio.h>
+extern int optind;
+extern char *optarg;
+extern int getopt(int, char **, char *);
+int main(int argc, char **argv) {
+  getopt(argc, argv, "abc");
+  return optarg ? optind : 0;
+}
+EOF
+
+if ac_library_find_v 'socket and connect' "" "-lsocket -lnsl" <<EOF
+int main() { socket(); connect(); return 0; }
+EOF
+then :
+else
+  ac_fatal "cannot find libraries needed for sockets"
+fi
+
+ac_ign \
+ ac_yesno "for inet_pton() && inet_ntop()" \
+ ac_have INET_PTON_NTOP \
+ ac_link <<EOF
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <arpa/inet.h>
+int main() {
+  char buf[64];
+  long x = 0;
+  inet_pton(AF_INET, &x, buf);
+  return inet_ntop(AF_INET, &x, buf, sizeof(buf));
+} 
+EOF
+
+if ac_yesno "for socklen_t" ac_compile <<EOF
+#include <sys/types.h>
+#include <sys/socket.h>
+int foo() { socklen_t len; len = 0; return len; }
+EOF
+then :
+else
+  ac_define socklen_t int
+fi
+
+if [ n != "$enable_ipv6" ]; then
+if ac_yesno "for IPv6" ac_have IPv6 ac_compile <<EOF
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+int main() {
+  struct sockaddr_in6 sa;
+  sa.sin6_family = AF_INET6;
+  return 0;
+}
+EOF
+then :
+elif [ "$enable_ipv6" ]; then
+  ac_fatal "IPv6 is requested but not available"
+fi
+fi # !disable_ipv6?
+
+if ac_yesno "for poll()" ac_have POLL ac_link <<EOF
+#include <sys/types.h>
+#include <sys/poll.h>
+int main() {
+  struct pollfd pfd[2];
+  return poll(pfd, 2, 10);
+}
+EOF
+then :
+else
+  ac_ign ac_yesno "for sys/select.h" ac_have SYS_SELECT_H ac_cpp <<EOF
+#include <sys/types.h>
+#include <sys/select.h>
+EOF
+fi
+
+ac_config_h
+ac_output Makefile
+ac_msg "creating config.status"
+rm -f config.status
+{
+echo "# automatically generated by configure to hold command-line options"
+echo
+found=
+for opt in $options; do
+  eval val=\$enable_$opt
+  if [ -n "$val" ]; then
+    echo enable_$opt=$val
+    found=y
+  fi
+done
+if [ ! "$found" ]; then
+  echo "# (no options encountered)"
+fi
+} > config.status
+ac_result ok
+
+ac_result "all done."
+exit 0

3rdparty/udns/configure.lib

@@ -0,0 +1,268 @@
+# configure.lib
+# a library of shell routines for simple autoconf system
+#
+
+set -e
+ac_substitutes=
+rm -f conftest* config.log
+exec 5>config.log
+cat <<EOF >&5
+This file contains any messages produced by compilers etc while
+running configure, to aid debugging if configure script makes a mistake.
+
+EOF
+
+case `echo "a\c"` in
+  *c*) ac_en=-n ac_ec= ;;
+  *)   ac_en= ac_ec='\c' ;;
+esac
+
+##### Messages
+ac_msg() {
+  echo $ac_en "$*... $ac_ec"
+  echo ">>> $*" >&5
+}
+ac_checking() {
+  echo $ac_en "checking $*... $ac_ec"
+  echo ">>> checking $*" >&5
+}
+ac_result() {
+  echo "$1"
+  echo "=== $1" >&5
+}
+ac_fatal() {
+  echo "configure: fatal: $*" >&2
+  echo "=== FATAL: $*" >&5
+  exit 1
+}
+ac_warning() {
+  echo "configure: warning: $*" >&2
+  echo "=== WARNING: $*" >&5
+}
+ac_ign() {
+  "$@" || :
+}
+
+# ac_run command...
+# captures output in conftest.out
+ac_run() {
+  # apparently UnixWare (for one) /bin/sh optimizes the following "if"
+  # "away", by checking if there's such a command BEFORE redirecting
+  # output.  So error message (like "gcc: command not found") goes
+  # to stderr instead of to conftest.out, and `cat conftest.out' below
+  # fails.
+  if "$@" >conftest.out 2>&1; then
+    return 0
+  else
+    echo "==== Command invocation failed. Command line was:" >&5
+    echo "$*" >&5
+    echo "==== compiler input was:" >&5
+    cat conftest.c >&5
+    echo "==== output was:" >&5
+    cat conftest.out >&5
+    echo "====" >&5
+    return 1
+  fi
+}
+
+# common case for ac_verbose: yes/no result
+ac_yesno() {
+  ac_checking "$1"
+  shift
+  if "$@"; then
+    ac_result yes
+    return 0
+  else
+    ac_result no
+    return 1
+  fi
+}
+
+ac_subst() {
+  ac_substitutes="$ac_substitutes $*"
+}
+
+ac_define() {
+  CDEFS="$CDEFS -D$1=${2:-1}"
+}
+
+ac_have() {
+  ac_what=$1; shift
+  if "$@"; then
+    ac_define HAVE_$ac_what
+    eval ac_have_$ac_what=yes
+    return 0
+  else
+    eval ac_have_$ac_what=no
+    return 1
+  fi
+}
+
+##### Compiling, linking
+
+# run a compiler
+ac_run_compiler() {
+  rm -f conftest*; cat >conftest.c
+  ac_run $CC $CFLAGS $CDEFS "$@" conftest.c
+}
+
+ac_compile() {
+  ac_run_compiler -c
+}
+
+ac_link() {
+  ac_run_compiler -o conftest $LIBS "$@"
+}
+
+ac_cpp() {
+  ac_run_compiler -E "$@"
+}
+
+### check for C compiler.  Set $CC, $CFLAGS etc
+ac_prog_c_compiler_v() {
+  ac_checking "for C compiler"
+  rm -f conftest*
+  echo 'int main(int argc, char **argv) { return 0; }' >conftest.c
+
+  if [ -n "$CC" ]; then
+    if ac_run $CC -o conftest conftest.c && ac_run ./conftest; then
+      ac_result "\$CC ($CC)"
+    else
+      ac_result no
+      ac_fatal "\$CC ($CC) is not a working compiler"
+    fi
+  else
+    for cc in gcc cc ; do
+      if ac_run $cc -o conftest conftest.c && ac_run ./conftest; then
+        ac_result "$cc"
+        CC=$cc
+        break
+      fi
+    done
+    if [ -z "$CC" ]; then
+      ac_result no
+      ac_fatal "no working C compiler found in \$PATH. please set \$CC variable"
+    fi
+  fi
+  if [ -z "$CFLAGS" ]; then
+    if ac_yesno "whenever C compiler ($CC) is GNU CC" \
+        ac_grep_cpp yEs_mAsTeR <<EOF
+#ifdef __GNUC__
+  yEs_mAsTeR;
+#endif
+EOF
+    then
+      CFLAGS="-Wall -W -O2 -pipe"
+    else
+      CFLAGS=-O
+    fi
+  fi
+  cc="$CC $CFLAGS"
+  ccld="$cc"
+  if [ -n "$LDFLAGS" ]; then ccld="$ccld $LDFLAGS"; fi
+  if [ -n "$LIBS" ]; then ccld="$ccld $LIBS"; fi
+  if ac_yesno "whenever the C compiler ($ccld)
+           can produce executables" \
+     ac_compile_run <<EOF
+int main() { return 0; }
+EOF
+  then :
+  else
+    ac_fatal "no working C compiler found"
+  fi
+  LD='$(CC)'
+  [ -n "$AR" ] || AR=ar
+  [ -n "$ARFLAGS" ] || ARFLAGS=rv
+  [ -n "$AWK" ] || AWK=awk
+  ac_substitutes="$ac_substitutes CC CFLAGS CDEFS LD LDFLAGS LIBS AR ARFLAGS AWK"
+}
+
+
+ac_prog_ranlib_v() {
+  ac_checking "for ranlib"
+  if [ -n "$RANLIB" ]; then
+    ac_result "\$RANLIB ($RANLIB)"
+  else
+    ifs="$IFS"
+    IFS=:
+    for dir in $PATH; do
+      [ -n "$dir" ] || dir=.
+      if [ -f $dir/ranlib ]; then
+        RANLIB=ranlib
+        break
+      fi
+    done
+    IFS="$ifs"
+    if [ -z "$RANLIB" ]; then ac_result no; RANLIB=:
+    else ac_result "$RANLIB"
+    fi
+  fi
+  ac_substitutes="$ac_substitutes RANLIB"
+}
+
+ac_library_find_v() {
+  ac_checking "for libraries needed for $1"
+  shift
+  fond=
+  rm -f conftest*; cat >conftest.c
+  for lib in "$@"; do
+    if ac_run $CC $CFLAGS $LDFLAGS conftest.c -o conftest $LIBS $lib; then
+      found=y
+      break
+    fi
+  done
+  if [ ! "$found" ]; then
+    ac_result "not found"
+    return 1
+  fi
+  if [ -z "$lib" ]; then
+    ac_result "ok (none needed)"
+  else
+    ac_result "ok ($lib)"
+    LIBS="$LIBS $lib"
+  fi
+}
+
+ac_compile_run() {
+  ac_link "$@" && ac_run ./conftest
+}
+
+ac_grep_cpp() {
+  pattern="$1"; shift
+  ac_cpp "$@" && grep "$pattern" conftest.out >/dev/null
+}
+
+ac_output() {
+  for var in $ac_substitutes; do
+    eval echo "\"s|@$var@|\$$var|\""
+  done >conftest.sed
+  for file in "$@"; do
+    ac_msg "creating $file"
+    if [ -f $file.in ]; then
+      sed -f conftest.sed $file.in > $file.tmp
+      mv -f $file.tmp $file
+      ac_result ok
+    else
+      ac_result failed
+      ac_fatal "$file.in not found"
+    fi
+  done
+  rm -f conftest*
+}
+
+ac_config_h() {
+  h=${1:-config.h}
+  ac_msg "creating $h"
+  rm -f $1.tmp
+  echo "/* automatically generated by configure. */" > $h.tmp
+  echo "$CDEFS" | tr ' ' '
+' | sed -e 's/^-D/#define /' -e 's/=/ /' >> $h.tmp
+  if [ -f $h ] && cmp -s $h.tmp $h ; then
+    rm -f $h.tmp
+    ac_result unchanged
+  else
+    mv -f $h.tmp $h
+    ac_result ok
+  fi
+  CDEFS=-DHAVE_CONFIG_H
+}

3rdparty/udns/dnsget.1

@@ -0,0 +1,195 @@
+.\" dnsget.1: dnsget manpage
+.\"
+.\" Copyright (C) 2005-2014  Michael Tokarev <mjt+udns@tls.msk.ru>
+.\" This file is part of UDNS library, an async DNS stub resolver.
+.\"
+.\" This library is free software; you can redistribute it and/or
+.\" modify it under the terms of the GNU Lesser General Public
+.\" License as published by the Free Software Foundation; either
+.\" version 2.1 of the License, or (at your option) any later version.
+.\"
+.\" This library is distributed in the hope that it will be useful,
+.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
+.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+.\" Lesser General Public License for more details.
+.\"
+.\" You should have received a copy of the GNU Lesser General Public
+.\" License along with this library, in file named COPYING.LGPL; if not,
+.\" write to the Free Software Foundation, Inc., 59 Temple Place,
+.\" Suite 330, Boston, MA  02111-1307  USA
+
+.TH dnsget 1 "Jan 2014" "User Utilities"
+
+.SH NAME
+dnsget \- DNS lookup utility
+
+.SH SYNOPSYS
+.B dnsget
+.RB [\| \-v \||\| \-q \|]
+.RB [\| \-c
+.IR class \|]
+.RB [\| \-t
+.IR type \|]
+.RB [\| \-o
+.IR opt , opt ,...]
+.IR name \|.\|.\|.
+
+.SH DESCRIPTION
+.B dnsget
+is a simple command-line to perform DNS lookups, similar to
+.BR host (1)
+and
+.BR dig (1).
+It is useable for both interactive/debugging scenarious and
+in scripts.
+The program is implemented using
+.BR udns (3)
+library.
+
+.PP
+By default,
+.B dnsget
+produces a human-readable output, similar to
+.RS
+.nf
+alias.example.com. CNAME www.example.com.
+www.example.com. A 192.168.1.1
+www.example.com. MX 10 mx.example.com.
+.fi
+.RE
+which is just sufficient to see how a given name resolves.
+Output format is controllable with
+.B \-v
+and
+.B \-q
+options -- the former increases verbosity level up to printing
+the whole DNS contents of all packets sent and received, which
+is suitable for debugging DNS problems, while the latter reduces
+the level, making output more quiet, up to bare result with no
+error messages, which is good for scripts.
+
+.SH OPTIONS
+
+The following options are recognized by
+.BR dnsget :
+
+.TP
+.B \-v
+produce more detailed output.  More
+.BR \-v 's
+means more details will be produced.  With single
+.BR \-v , dnsget
+will print contents of all received DNS packets (in a readable format),
+while with
+.BR \-vv ,
+it will output all outgoing DNS packets too.
+
+.TP
+.B \-q
+the opposite for \fB\-v\fR -- produce less detailed output.
+With single
+.BR \-q , dnsget
+will only show (decoded) data from final DNS resource records (RR),
+while
+.B \-qq
+also suppresses error messages.
+
+.TP
+\fB\-t \fItype\fR
+request record(s) of the given type \fItype\fR.  By default,
+.B dnsget
+will ask for IPv4 address (A) record, or for PTR record if the
+argument in question is an IPv4 or IPv6 address.  Recognized
+types include A, AAAA, MX, TXT, CNAME, PTR, NS, SOA, ANY and
+others.
+
+.TP
+\fB\-c \fIclass\fR
+request DNS record(s) of the given class \fIclass\fR.  By
+default
+.B dnsget
+uses IN class.  Valid classes include IN, CH, HS, ANY.
+
+.TP
+.B \-a
+(compatibility option).  Equivalent to setting query type to
+.B ANY
+and increasing verbosity level
+.RB ( \-v ).
+
+.TP
+.B \-C
+(planned)
+
+.TP
+.B \-x
+(planned)
+
+.TP
+\fB\-o \fIopt\fR,\fIopt\fR,...
+(may be specified several times).
+Set resolver options (in a form \fIoption\fR:\fIvalue\fR) as if they
+were set in
+.RB $ RES_OPTIONS
+environment variable, or set query flags:
+.RS
+.TP
+\fBtimeout\fR:\fIsec\fR
+Set initial query timeout to \fIsec\fR.
+.TP
+\fBattempts\fR:\fInum\fR
+(re)try every query \fInum\fR times before failing.
+.TP
+\fBudpbuf\fR:\fIbytes\fR
+set DNS UDP buffer size to \fIbytes\fR bytes.  Valid values
+are from 512 to 65535.  If \fIbytes\fR is greather than 512,
+EDNS0 (RFC 2671) extensions will be used.
+.TP
+\fBport\fR:\fInum\fR
+Use given UDP port number \fInum\fR instead of the default port 53 (domain).
+.TP
+\fBaa\fR
+set AA (auth only) query bit.
+.TP
+\fBnord\fR
+do not set RD (recursion desired) query bit (set by default).
+.TP
+\fBdnssec\fR or \fBdo\fR
+set DNSSEC OK (DO) query flag (\fBdnsget\fR does not verify DNSSEC signatures,
+only displays them; this is set in EDNS RR).
+.TP
+\fBcd\fR
+set CD (checking disabled) query bit.
+.RE
+
+.TP
+\fB\-n \fInameserver\fR
+Use the given nameserver(s) (may be specified more than once)
+instead of the default.  Using this option has the same same effect as 
+.RB $ NSCACHEIP
+or
+.RB $ NAMESERVERS
+environment variables, with the only difference that only IPv4 addresses
+are recognized for now, and it is possible to specify names (which will
+be resolved using default settings) instead of IP addresses.
+
+.TP
+.B \-h
+print short help and exit.
+
+.SH "RETURN VALUE"
+When all names where resovled successefully,
+.B dnsget
+exits with zero exit status.  If at least one name was not found,
+.B dnsget
+will exit with return code 100.  If some other error occured during
+name resolution, it will exit with code 99.  In case of usage or
+initialization error,
+.B dnsget
+will return 1.
+
+.SH "SEE ALSO"
+.BR host (1)
+.BR dig (1)
+.BR resolv.conf (5)
+.BR udns (3).

3rdparty/udns/dnsget.c

@@ -0,0 +1,759 @@
+/* dnsget.c
+   simple host/dig-like application using UDNS library
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+#ifdef WINDOWS
+#include <windows.h>
+#include <winsock2.h>
+#else
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <sys/time.h>
+#include <unistd.h>
+#endif
+#include <time.h>
+#include <stdarg.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include "udns.h"
+
+#ifndef HAVE_GETOPT
+# include "getopt.c"
+#endif
+
+#ifndef AF_INET6
+# define AF_INET6 10
+#endif
+
+static char *progname;
+static int verbose = 1;
+static int errors;
+static int notfound;
+
+/* verbosity level:
+ * <0 - bare result
+ *  0 - bare result and error messages
+ *  1 - readable result
+ *  2 - received packet contents and `trying ...' stuff
+ *  3 - sent and received packet contents
+ */
+
+static void die(int errnum, const char *fmt, ...) {
+  va_list ap;
+  fprintf(stderr, "%s: ", progname);
+  va_start(ap, fmt); vfprintf(stderr, fmt, ap); va_end(ap);
+  if (errnum) fprintf(stderr, ": %s\n", strerror(errnum));
+  else putc('\n', stderr);
+  fflush(stderr);
+  exit(1);
+}
+
+static const char *dns_xntop(int af, const void *src) {
+  static char buf[6*5+4*4];
+  return dns_ntop(af, src, buf, sizeof(buf));
+}
+
+struct query {
+  const char *name;		/* original query string */
+  unsigned char *dn;		/* the DN being looked up */
+  enum dns_type qtyp;		/* type of the query */
+};
+
+static void query_free(struct query *q) {
+  free(q->dn);
+  free(q);
+}
+
+static struct query *
+query_new(const char *name, const unsigned char *dn, enum dns_type qtyp) {
+  struct query *q = malloc(sizeof(*q));
+  unsigned l = dns_dnlen(dn);
+  unsigned char *cdn = malloc(l);
+  if (!q || !cdn) die(0, "out of memory");
+  memcpy(cdn, dn, l);
+  q->name = name;
+  q->dn = cdn;
+  q->qtyp = qtyp;
+  return q;
+}
+
+static enum dns_class qcls = DNS_C_IN;
+
+static void
+dnserror(struct query *q, int errnum) {
+  if (verbose >= 0)
+    fprintf(stderr, "%s: unable to lookup %s record for %s: %s\n", progname,
+            dns_typename(q->qtyp), dns_dntosp(q->dn), dns_strerror(errnum));
+  if (errnum == DNS_E_NXDOMAIN || errnum == DNS_E_NODATA)
+    ++notfound;
+  else
+    ++errors;
+  query_free(q);
+}
+
+static const unsigned char *
+printtxt(const unsigned char *c) {
+  unsigned n = *c++;
+  const unsigned char *e = c + n;
+  if (verbose > 0) while(c < e) {
+    if (*c < ' ' || *c >= 127) printf("\\%03u", *c);
+    else if (*c == '\\' || *c == '"') printf("\\%c", *c);
+    else putchar(*c);
+    ++c;
+  }
+  else
+   fwrite(c, n, 1, stdout);
+  return e;
+}
+
+static void
+printhex(const unsigned char *c, const unsigned char *e) {
+  while(c < e)
+    printf("%02x", *c++);
+}
+
+static unsigned char to_b64[] =
+"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
+
+static void
+printb64(const unsigned char *c, const unsigned char *e) {
+  while(c < e) {
+    putchar(to_b64[c[0] >> 2]);
+    if (c+1 < e) {
+      putchar(to_b64[(c[0] & 0x3) << 4 | c[1] >> 4]);
+      if (c+2 < e) {
+        putchar(to_b64[(c[1] & 0xf) << 2 | c[2] >> 6]);
+        putchar(to_b64[c[2] & 0x3f]);
+      }
+      else {
+        putchar(to_b64[(c[1] & 0xf) << 2]);
+	putchar('=');
+	break;
+      }
+    }
+    else {
+      putchar(to_b64[(c[0] & 0x3) << 4]);
+      putchar('=');
+      putchar('=');
+      break;
+    }
+    c += 3;
+  }
+}
+
+static void
+printdate(time_t time) {
+  struct tm *tm = gmtime(&time);
+  printf("%04d%02d%02d%02d%02d%02d",
+    tm->tm_year + 1900, tm->tm_mon + 1, tm->tm_mday,
+    tm->tm_hour, tm->tm_min, tm->tm_sec);
+}
+
+static void
+printrr(const struct dns_parse *p, struct dns_rr *rr) {
+  const unsigned char *pkt = p->dnsp_pkt;
+  const unsigned char *end = p->dnsp_end;
+  const unsigned char *dptr = rr->dnsrr_dptr;
+  const unsigned char *dend = rr->dnsrr_dend;
+  unsigned char *dn = rr->dnsrr_dn;
+  const unsigned char *c;
+  unsigned n;
+
+  if (verbose > 0) {
+    if (verbose > 1) {
+      if (!p->dnsp_rrl && !rr->dnsrr_dn[0] && rr->dnsrr_typ == DNS_T_OPT) {
+        printf(";EDNS%d OPT record (UDPsize: %d, ERcode: %d, Flags: 0x%02x): %d bytes\n",
+               (rr->dnsrr_ttl>>16) & 0xff,	/* version */
+               rr->dnsrr_cls,			/* udp size */
+               (rr->dnsrr_ttl>>24) & 0xff,	/* extended rcode */
+               rr->dnsrr_ttl & 0xffff,		/* flags */
+               rr->dnsrr_dsz);
+        return;
+      }
+      n = printf("%s.", dns_dntosp(rr->dnsrr_dn));
+      printf("%s%u\t%s\t%s\t",
+             n > 15 ? "\t" : n > 7 ? "\t\t" : "\t\t\t",
+             rr->dnsrr_ttl,
+             dns_classname(rr->dnsrr_cls),
+             dns_typename(rr->dnsrr_typ));
+    }
+    else
+      printf("%s. %s ", dns_dntosp(rr->dnsrr_dn), dns_typename(rr->dnsrr_typ));
+  }
+
+  switch(rr->dnsrr_typ) {
+
+  case DNS_T_CNAME:
+  case DNS_T_PTR:
+  case DNS_T_NS:
+  case DNS_T_MB:
+  case DNS_T_MD:
+  case DNS_T_MF:
+  case DNS_T_MG:
+  case DNS_T_MR:
+    if (dns_getdn(pkt, &dptr, end, dn, DNS_MAXDN) <= 0) goto xperr;
+    printf("%s.", dns_dntosp(dn));
+    break;
+
+  case DNS_T_A:
+    if (rr->dnsrr_dsz != 4) goto xperr;
+    printf("%d.%d.%d.%d", dptr[0], dptr[1], dptr[2], dptr[3]);
+    break;
+
+  case DNS_T_AAAA:
+    if (rr->dnsrr_dsz != 16) goto xperr;
+    printf("%s", dns_xntop(AF_INET6, dptr));
+    break;
+
+  case DNS_T_MX:
+    c = dptr + 2;
+    if (dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0 || c != dend) goto xperr;
+    printf("%d %s.", dns_get16(dptr), dns_dntosp(dn));
+    break;
+
+  case DNS_T_TXT:
+    /* first verify it */
+    for(c = dptr; c < dend; c += n) {
+      n = *c++;
+      if (c + n > dend) goto xperr;
+    }
+    c = dptr; n = 0;
+    while (c < dend) {
+      if (verbose > 0) printf(n++ ? "\" \"":"\"");
+      c = printtxt(c);
+    }
+    if (verbose > 0) putchar('"');
+    break;
+
+  case DNS_T_HINFO:	/* CPU, OS */
+    c = dptr;
+    n = *c++; if ((c += n) >= dend) goto xperr;
+    n = *c++; if ((c += n) != dend) goto xperr;
+    c = dptr;
+    if (verbose > 0) putchar('"');
+    c = printtxt(c);
+    if (verbose > 0) printf("\" \""); else putchar(' ');
+    printtxt(c);
+    if (verbose > 0) putchar('"');
+    break;
+
+  case DNS_T_WKS:
+    c = dptr;
+    if (dptr + 4 + 2 >= end) goto xperr;
+    printf("%s %d", dns_xntop(AF_INET, dptr), dptr[4]);
+    c = dptr + 5;
+    for (n = 0; c < dend; ++c, n += 8) {
+      if (*c) {
+        unsigned b;
+        for (b = 0; b < 8; ++b)
+          if (*c & (1 << (7-b))) printf(" %d", n + b);
+      }
+    }
+    break;
+
+  case DNS_T_SRV:	/* prio weight port targetDN */
+    c = dptr;
+    c += 2 + 2 + 2;
+    if (dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0 || c != dend) goto xperr;
+    c = dptr;
+    printf("%d %d %d %s.",
+           dns_get16(c+0), dns_get16(c+2), dns_get16(c+4),
+           dns_dntosp(dn));
+    break;
+
+  case DNS_T_NAPTR:	/* order pref flags serv regexp repl */
+    c = dptr;
+    c += 4;	/* order, pref */
+    for (n = 0; n < 3; ++n)
+      if (c >= dend) goto xperr;
+      else c += *c + 1;
+    if (dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0 || c != dend) goto xperr;
+    c = dptr;
+    printf("%u %u", dns_get16(c+0), dns_get16(c+2));
+    c += 4;
+    for(n = 0; n < 3; ++n) {
+      putchar(' ');
+      if (verbose > 0) putchar('"');
+      c = printtxt(c);
+      if (verbose > 0) putchar('"');
+    } 
+    printf(" %s.", dns_dntosp(dn));
+    break;
+
+  case DNS_T_KEY:
+  case DNS_T_DNSKEY:
+    /* flags(2) proto(1) algo(1) pubkey */
+  case DNS_T_DS:
+  case DNS_T_DLV:
+    /* ktag(2) proto(1) algo(1) pubkey */
+    c = dptr;
+    if (c + 2 + 1 + 1 > dend) goto xperr;
+    printf("%d %d %d", dns_get16(c), c[2], c[3]);
+    c += 2 + 1 + 1;
+    if (c < dend) {
+      putchar(' ');
+      printb64(c, dend);
+    }
+    break;
+
+  case DNS_T_SIG:
+  case DNS_T_RRSIG:
+    /* type(2) algo(1) labels(1) ottl(4) sexp(4) sinc(4) tag(2) sdn sig */
+    c = dptr;
+    c += 2 + 1 + 1 + 4 + 4 + 4 + 2;
+    if (dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0) goto xperr;
+    printf("%s %u %u %u ",
+           dns_typename(dns_get16(dptr)), dptr[2], dptr[3], dns_get32(dptr+4));
+    printdate(dns_get32(dptr+8));
+    putchar(' ');
+    printdate(dns_get32(dptr+12));
+    printf(" %d %s. ", dns_get16(dptr+10), dns_dntosp(dn));
+    printb64(c, dend);
+    break;
+
+  case DNS_T_SSHFP: /* algo(1), fp type(1), fp... */
+    if (dend < dptr + 3) goto xperr;
+    printf("%u %u ", dptr[0], dptr[1]); /* algo, fp type */
+    printhex(dptr + 2, dend);
+    break;
+
+#if 0	/* unused RR types? */
+  case DNS_T_NSEC: /* nextDN bitmaps */
+    c = dptr;
+    if (dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0) goto xperr;
+    printf("%s.", dns_dntosp(dn));
+    unfinished.
+    break;
+#endif
+
+
+  case DNS_T_SOA:
+    c = dptr;
+    if (dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0 ||
+        dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0 ||
+        c + 4*5 != dend)
+      goto xperr;
+    dns_getdn(pkt, &dptr, end, dn, DNS_MAXDN);
+    printf("%s. ", dns_dntosp(dn));
+    dns_getdn(pkt, &dptr, end, dn, DNS_MAXDN);
+    printf("%s. ", dns_dntosp(dn));
+    printf("%u %u %u %u %u",
+           dns_get32(dptr), dns_get32(dptr+4), dns_get32(dptr+8),
+           dns_get32(dptr+12), dns_get32(dptr+16));
+    break;
+
+  case DNS_T_MINFO:
+    c = dptr;
+    if (dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0 ||
+        dns_getdn(pkt, &c, end, dn, DNS_MAXDN) <= 0 ||
+	c != dend)
+      goto xperr;
+    dns_getdn(pkt, &dptr, end, dn, DNS_MAXDN);
+    printf("%s. ", dns_dntosp(dn));
+    dns_getdn(pkt, &dptr, end, dn, DNS_MAXDN);
+    printf("%s.", dns_dntosp(dn));
+    break;
+
+  case DNS_T_NULL:
+  default:
+    printhex(dptr, dend);
+    break;
+  }
+  putchar('\n');
+  return;
+
+xperr:
+  printf("<parse error>\n");
+  ++errors;
+}
+
+static int
+printsection(struct dns_parse *p, int nrr, const char *sname) {
+  struct dns_rr rr;
+  int r;
+  if (!nrr) return 0;
+  if (verbose > 1) printf("\n;; %s section (%d):\n", sname, nrr);
+
+  p->dnsp_rrl = nrr;
+  while((r = dns_nextrr(p, &rr)) > 0)
+    printrr(p, &rr);
+  if (r < 0) printf("<<ERROR>>\n");
+  return r;
+}
+
+/* dbgcb will only be called if verbose > 1 */
+static void
+dbgcb(int code, const struct sockaddr *sa, unsigned slen,
+      const unsigned char *pkt, int r,
+      const struct dns_query *unused_q, void *unused_data) {
+  struct dns_parse p;
+  const unsigned char *cur, *end;
+  int numqd;
+
+  if (code > 0)	{
+    printf(";; trying %s.\n", dns_dntosp(dns_payload(pkt)));
+    printf(";; sending %d bytes query to ", r);
+  }
+  else
+    printf(";; received %d bytes response from ", r);
+  if (sa->sa_family == AF_INET && slen >= sizeof(struct sockaddr_in))
+    printf("%s port %d\n",
+           dns_xntop(AF_INET, &((struct sockaddr_in*)sa)->sin_addr),
+           htons(((struct sockaddr_in*)sa)->sin_port));
+#ifdef HAVE_IPv6
+  else if (sa->sa_family == AF_INET6 && slen >= sizeof(struct sockaddr_in6))
+    printf("%s port %d\n",
+           dns_xntop(AF_INET6, &((struct sockaddr_in6*)sa)->sin6_addr),
+           htons(((struct sockaddr_in6*)sa)->sin6_port));
+#endif
+  else
+    printf("<<unknown socket type %d>>\n", sa->sa_family);
+  if (code > 0 && verbose < 3) {
+    putchar('\n');
+    return;
+  }
+
+  if (code == -2) printf(";; reply from unexpected source\n");
+  if (code == -5) printf(";; reply to a query we didn't sent (or old)\n");
+  if (r < DNS_HSIZE) {
+    printf(";; short packet (%d bytes)\n", r);
+    return;
+  }
+  if (dns_opcode(pkt) != 0)
+    printf(";; unexpected opcode %d\n", dns_opcode(pkt));
+  if (dns_tc(pkt) != 0)
+    printf(";; warning: TC bit set, probably incomplete reply\n");
+
+  printf(";; ->>HEADER<<- opcode: ");
+  switch(dns_opcode(pkt)) {
+  case 0: printf("QUERY"); break;
+  case 1: printf("IQUERY"); break;
+  case 2: printf("STATUS"); break;
+  default: printf("UNKNOWN(%u)", dns_opcode(pkt)); break;
+  }
+  printf(", status: %s, id: %d, size: %d\n;; flags:",
+         dns_rcodename(dns_rcode(pkt)), dns_qid(pkt), r);
+  if (dns_qr(pkt)) printf(" qr");
+  if (dns_aa(pkt)) printf(" aa");
+  if (dns_tc(pkt)) printf(" tc");
+  if (dns_rd(pkt)) printf(" rd");
+  if (dns_ra(pkt)) printf(" ra");
+  /* if (dns_z(pkt))  printf(" z"); only one reserved bit left */
+  if (dns_ad(pkt)) printf(" ad");
+  if (dns_cd(pkt)) printf(" cd");
+  numqd = dns_numqd(pkt);
+  printf("; QUERY: %d, ANSWER: %d, AUTHORITY: %d, ADDITIONAL: %d\n",
+         numqd, dns_numan(pkt), dns_numns(pkt), dns_numar(pkt));
+  if (numqd != 1)
+    printf(";; unexpected number of entries in QUERY section: %d\n",
+           numqd);
+  printf("\n;; QUERY SECTION (%d):\n", numqd);
+  cur = dns_payload(pkt);
+  end = pkt + r;
+  while(numqd--) {
+    if (dns_getdn(pkt, &cur, end, p.dnsp_dnbuf, DNS_MAXDN) <= 0 ||
+        cur + 4 > end) {
+      printf("; invalid query section\n");
+      return;
+    }
+    r = printf(";%s.", dns_dntosp(p.dnsp_dnbuf));
+    printf("%s%s\t%s\n",
+           r > 23 ? "\t" : r > 15 ? "\t\t" : r > 7 ? "\t\t\t" : "\t\t\t\t",
+           dns_classname(dns_get16(cur+2)), dns_typename(dns_get16(cur)));
+    cur += 4;
+  }
+
+  p.dnsp_pkt = pkt;
+  p.dnsp_cur = p.dnsp_ans = cur;
+  p.dnsp_end = end;
+  p.dnsp_qdn = NULL;
+  p.dnsp_qcls = p.dnsp_qtyp = 0;
+  p.dnsp_ttl = 0xffffffffu;
+  p.dnsp_nrr = 0;
+
+  r = printsection(&p, dns_numan(pkt), "ANSWER");
+  if (r == 0)
+    r = printsection(&p, dns_numns(pkt), "AUTHORITY");
+  if (r == 0)
+    r = printsection(&p, dns_numar(pkt), "ADDITIONAL");
+  putchar('\n');
+}
+
+static void dnscb(struct dns_ctx *ctx, void *result, void *data) {
+  int r = dns_status(ctx);
+  struct query *q = data;
+  struct dns_parse p;
+  struct dns_rr rr;
+  unsigned nrr;
+  unsigned char dn[DNS_MAXDN];
+  const unsigned char *pkt, *cur, *end;
+  if (!result) {
+    dnserror(q, r);
+    return;
+  }
+  pkt = result; end = pkt + r; cur = dns_payload(pkt);
+  dns_getdn(pkt, &cur, end, dn, sizeof(dn));
+  dns_initparse(&p, NULL, pkt, cur, end);
+  p.dnsp_qcls = p.dnsp_qtyp = 0;
+  nrr = 0;
+  while((r = dns_nextrr(&p, &rr)) > 0) {
+    if (!dns_dnequal(dn, rr.dnsrr_dn)) continue;
+    if ((qcls == DNS_C_ANY || qcls == rr.dnsrr_cls) &&
+        (q->qtyp == DNS_T_ANY || q->qtyp == rr.dnsrr_typ))
+      ++nrr;
+    else if (rr.dnsrr_typ == DNS_T_CNAME && !nrr) {
+      if (dns_getdn(pkt, &rr.dnsrr_dptr, end,
+                    p.dnsp_dnbuf, sizeof(p.dnsp_dnbuf)) <= 0 ||
+          rr.dnsrr_dptr != rr.dnsrr_dend) {
+        r = DNS_E_PROTOCOL;
+        break;
+      }
+      else {
+        if (verbose == 1) {
+          printf("%s.", dns_dntosp(dn));
+          printf(" CNAME %s.\n", dns_dntosp(p.dnsp_dnbuf));
+        }
+        dns_dntodn(p.dnsp_dnbuf, dn, sizeof(dn));
+      }
+    }
+  }
+  if (!r && !nrr)
+    r = DNS_E_NODATA;
+  if (r < 0) {
+    dnserror(q, r);
+    free(result);
+    return;
+  }
+  if (verbose < 2) {	/* else it is already printed by dbgfn */
+    dns_rewind(&p, NULL);
+    p.dnsp_qtyp = q->qtyp == DNS_T_ANY ? 0 : q->qtyp;
+    p.dnsp_qcls = qcls == DNS_C_ANY ? 0 : qcls;
+    while(dns_nextrr(&p, &rr))
+      printrr(&p, &rr);
+  }
+  free(result);
+  query_free(q);
+}
+
+int main(int argc, char **argv) {
+  int i;
+  int fd;
+  fd_set fds;
+  struct timeval tv;
+  time_t now;
+  char *ns[DNS_MAXSERV];
+  int nns = 0;
+  struct query *q;
+  enum dns_type qtyp = 0;
+  struct dns_ctx *nctx = NULL;
+  int flags = 0;
+
+  if (!(progname = strrchr(argv[0], '/'))) progname = argv[0];
+  else argv[0] = ++progname;
+
+  if (argc <= 1)
+    die(0, "try `%s -h' for help", progname);
+
+  if (dns_init(NULL, 0) < 0 || !(nctx = dns_new(NULL)))
+    die(errno, "unable to initialize dns library");
+  /* we keep two dns contexts: one may be needed to resolve
+   * nameservers if given as names, using default options.
+   */
+
+  while((i = getopt(argc, argv, "vqt:c:an:o:f:h")) != EOF) switch(i) {
+  case 'v': ++verbose; break;
+  case 'q': --verbose; break;
+  case 't':
+    if (optarg[0] == '*' && !optarg[1])
+      i = DNS_T_ANY;
+    else if ((i = dns_findtypename(optarg)) <= 0)
+      die(0, "unrecognized query type `%s'", optarg);
+    qtyp = i;
+    break;
+  case 'c':
+    if (optarg[0] == '*' && !optarg[1])
+      i = DNS_C_ANY;
+    else if ((i = dns_findclassname(optarg)) < 0)
+      die(0, "unrecognized query class `%s'", optarg);
+    qcls = i;
+    break;
+  case 'a':
+    qtyp = DNS_T_ANY;
+    ++verbose;
+    break;
+  case 'n':
+    if (nns >= DNS_MAXSERV)
+      die(0, "too many nameservers, %d max", DNS_MAXSERV);
+    ns[nns++] = optarg;
+    break;
+  case 'o':
+  case 'f': {
+    char *opt;
+    const char *const delim = " \t,;";
+    for(opt = strtok(optarg, delim); opt != NULL; opt = strtok(NULL, delim)) {
+      if (dns_set_opts(NULL, optarg) == 0)
+        ;
+      else if (strcmp(opt, "aa") == 0) flags |= DNS_AAONLY;
+      else if (strcmp(optarg, "nord") == 0) flags |= DNS_NORD;
+      else if (strcmp(optarg, "dnssec") == 0) flags |= DNS_SET_DO;
+      else if (strcmp(optarg, "do")     == 0) flags |= DNS_SET_DO;
+      else if (strcmp(optarg, "cd") == 0) flags |= DNS_SET_CD;
+      else
+        die(0, "invalid option: `%s'", opt);
+    }
+    break;
+  }
+  case 'h':
+    printf(
+"%s: simple DNS query tool (using udns version %s)\n"
+"Usage: %s [options] domain-name...\n"
+"where options are:\n"
+" -h - print this help and exit\n"
+" -v - be more verbose\n"
+" -q - be less verbose\n"
+" -t type - set query type (A, AAA, PTR etc)\n"
+" -c class - set query class (IN (default), CH, HS, *)\n"
+" -a - equivalent to -t ANY -v\n"
+" -n ns - use given nameserver(s) instead of default\n"
+"  (may be specified multiple times)\n"
+" -o opt,opt,... (comma- or space-separated list,\n"
+"                 may be specified more than once):\n"
+"  set resovler options (the same as setting $RES_OPTIONS):\n"
+"   timeout:sec  - initial query timeout\n"
+"   attempts:num - number of attempt to resovle a query\n"
+"   ndots:num    - if name has more than num dots, lookup it before search\n"
+"   port:num     - port number for queries instead of default 53\n"
+"   udpbuf:num   - size of UDP buffer (use EDNS0 if >512)\n"
+"  or query flags:\n"
+"   aa,nord,dnssec,do,cd - set query flag (auth-only, no recursion,\n"
+"     enable DNSSEC (DNSSEC Ok), check disabled)\n"
+      , progname, dns_version(), progname);
+    return 0;
+  default:
+    die(0, "try `%s -h' for help", progname);
+  }
+
+  argc -= optind; argv += optind;
+  if (!argc)
+    die(0, "no name(s) to query specified");
+
+  if (nns) {
+    /* if nameservers given as names, resolve them.
+     * We only allow IPv4 nameservers as names for now.
+     * Ok, it is easy enouth to try both AAAA and A,
+     * but the question is what to do by default.
+     */
+    struct sockaddr_in sin;
+    int j, r = 0, opened = 0;
+    memset(&sin, 0, sizeof(sin));
+    sin.sin_family = AF_INET;
+    sin.sin_port = htons(dns_set_opt(NULL, DNS_OPT_PORT, -1));
+    dns_add_serv(NULL, NULL);
+    for(i = 0; i < nns; ++i) {
+      if (dns_pton(AF_INET, ns[i], &sin.sin_addr) <= 0) {
+        struct dns_rr_a4 *rr;
+        if (!opened) {
+          if (dns_open(nctx) < 0)
+            die(errno, "unable to initialize dns context");
+          opened = 1;
+        }
+        rr = dns_resolve_a4(nctx, ns[i], 0);
+        if (!rr)
+          die(0, "unable to resolve nameserver %s: %s",
+              ns[i], dns_strerror(dns_status(nctx)));
+        for(j = 0; j < rr->dnsa4_nrr; ++j) {
+          sin.sin_addr = rr->dnsa4_addr[j];
+          if ((r = dns_add_serv_s(NULL, (struct sockaddr *)&sin)) < 0)
+            break;
+        }
+        free(rr);
+      }
+      else
+        r = dns_add_serv_s(NULL, (struct sockaddr *)&sin);
+      if (r < 0)
+        die(errno, "unable to add nameserver %s",
+             dns_xntop(AF_INET, &sin.sin_addr));
+    }
+  }
+  dns_free(nctx);
+
+  fd = dns_open(NULL);
+  if (fd < 0)
+    die(errno, "unable to initialize dns context");
+
+  if (verbose > 1)
+    dns_set_dbgfn(NULL, dbgcb);
+
+  if (flags)
+    dns_set_opt(NULL, DNS_OPT_FLAGS, flags);
+
+  for (i = 0; i < argc; ++i) {
+    char *name = argv[i];
+    union {
+      struct in_addr addr;
+      struct in6_addr addr6;
+    } a;
+    unsigned char dn[DNS_MAXDN];
+    enum dns_type l_qtyp = 0;
+    int abs;
+    if (dns_pton(AF_INET, name, &a.addr) > 0) {
+      dns_a4todn(&a.addr, 0, dn, sizeof(dn));
+      l_qtyp = DNS_T_PTR;
+      abs = 1;
+    }
+#ifdef HAVE_IPv6
+    else if (dns_pton(AF_INET6, name, &a.addr6) > 0) {
+      dns_a6todn(&a.addr6, 0, dn, sizeof(dn));
+      l_qtyp = DNS_T_PTR;
+      abs = 1;
+    }
+#endif
+    else if (!dns_ptodn(name, strlen(name), dn, sizeof(dn), &abs))
+      die(0, "invalid name `%s'\n", name);
+    else
+      l_qtyp = DNS_T_A;
+    if (qtyp) l_qtyp = qtyp;
+    q = query_new(name, dn, l_qtyp);
+    if (abs) abs = DNS_NOSRCH;
+    if (!dns_submit_dn(NULL, dn, qcls, l_qtyp, abs, 0, dnscb, q))
+      dnserror(q, dns_status(NULL));
+  }
+
+  FD_ZERO(&fds);
+  now = 0;
+  while((i = dns_timeouts(NULL, -1, now)) > 0) {
+    FD_SET(fd, &fds);
+    tv.tv_sec = i;
+    tv.tv_usec = 0;
+    i = select(fd+1, &fds, 0, 0, &tv);
+    now = time(NULL);
+    if (i > 0) dns_ioevent(NULL, now);
+  }
+
+  return errors ? 1 : notfound ? 100 : 0;
+}

3rdparty/udns/ex-rdns.c

@@ -0,0 +1,114 @@
+/* ex-rdns.c
+   parallel rDNS resolver example - read IP addresses from stdin,
+   write domain names to stdout
+ 
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <sys/poll.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <time.h>
+#include <stdlib.h>
+#include <string.h>
+#include "udns.h"
+
+static int curq;
+
+static const char *n2ip(const unsigned char *c) {
+  static char b[sizeof("255.255.255.255")];
+  sprintf(b, "%u.%u.%u.%u", c[0], c[1], c[2], c[3]);
+  return b;
+}
+static void dnscb(struct dns_ctx *ctx, struct dns_rr_ptr *rr, void *data) {
+  const char *ip = n2ip((unsigned char *)&data);
+  int i;
+  --curq;
+  if (rr) {
+    printf("%s", ip);
+    for(i = 0; i < rr->dnsptr_nrr; ++i)
+      printf(" %s", rr->dnsptr_ptr[i]);
+    putchar('\n');
+    free(rr);
+  }
+  else
+    fprintf(stderr, "%s: %s\n", ip, dns_strerror(dns_status(ctx)));
+}
+
+int main(int argc, char **argv) {
+  int c;
+  time_t now;
+  int maxq = 10;
+  struct pollfd pfd;
+  char linebuf[1024];
+  char *eol;
+  int eof;
+
+  if (dns_init(NULL, 1) < 0) {
+    fprintf(stderr, "unable to initialize dns library\n");
+    return 1;
+  }
+  while((c = getopt(argc, argv, "m:r")) != EOF) switch(c) {
+  case 'm': maxq = atoi(optarg); break;
+  case 'r':
+     dns_set_opt(0, DNS_OPT_FLAGS,
+                 dns_set_opt(0, DNS_OPT_FLAGS, -1) | DNS_NORD);
+     break;
+  default: return 1;
+  }
+  if (argc != optind) return 1;
+
+  pfd.fd = dns_sock(0);
+  pfd.events = POLLIN;
+  now = time(NULL);
+  c = optind;
+  eof = 0;
+  while(curq || !eof) {
+    if (!eof && curq < maxq) {
+      union { struct in_addr a; void *p; } pa;
+      if (!fgets(linebuf, sizeof(linebuf), stdin)) {
+        eof = 1;
+        continue;
+      }
+      eol = strchr(linebuf, '\n');
+      if (eol) *eol = '\0';
+      if (!linebuf[0]) continue;
+      if (dns_pton(AF_INET, linebuf, &pa.a) <= 0)
+        fprintf(stderr, "%s: invalid address\n", linebuf);
+      else if (dns_submit_a4ptr(0, &pa.a, dnscb, pa.p) == 0)
+        fprintf(stderr, "%s: unable to submit query: %s\n",
+                linebuf, dns_strerror(dns_status(0)));
+      else
+        ++curq;
+      continue;
+    }
+    if (curq) {
+      c = dns_timeouts(0, -1, now);
+      c = poll(&pfd, 1, c < 0 ? -1 : c * 1000);
+      now = time(NULL);
+      if (c)
+        dns_ioevent(0, now);
+    }
+  }
+  return 0;
+}

3rdparty/udns/getopt.c

@@ -0,0 +1,165 @@
+/* getopt.c
+ * Simple getopt() implementation.
+ *
+ * Standard interface:
+ *  extern int getopt(int argc, char *const *argv, const char *opts);
+ *  extern int optind;    current index in argv[]
+ *  extern char *optarg;  argument for the current option
+ *  extern int optopt;    the current option
+ *  extern int opterr;    to control error printing
+ *
+ * Some minor extensions:
+ *  ignores leading `+' sign in opts[] (unemplemented GNU extension)
+ *  handles optional arguments, in form "x::" in opts[]
+ *  if opts[] starts with `:', will return `:' in case of missing required
+ *    argument, instead of '?'.
+ *
+ * Compile with -DGETOPT_NO_OPTERR to never print errors internally.
+ * Compile with -DGETOPT_NO_STDIO to use write() calls instead of fprintf() for
+ *  error reporting (ignored with -DGETOPT_NO_OPTERR).
+ * Compile with -DGETOPT_CLASS=static to get static linkage.
+ * Compile with -DGETOPT_MY to redefine all visible symbols to be prefixed
+ *  with "my_", like my_getopt instead of getopt.
+ * Compile with -DTEST to get a test executable.
+ *
+ * Written by Michael Tokarev.  Public domain.
+ */
+
+#include <string.h>
+
+#ifndef GETOPT_CLASS
+# define GETOPT_CLASS
+#endif
+#ifdef GETOPT_MY
+# define optarg my_optarg
+# define optind my_optind
+# define opterr my_opterr
+# define optopt my_optopt
+# define getopt my_getopt
+#endif
+
+GETOPT_CLASS char *optarg /* = NULL */;
+GETOPT_CLASS int optind = 1;
+GETOPT_CLASS int opterr = 1;
+GETOPT_CLASS int optopt;
+
+static char *nextc /* = NULL */;
+
+#if defined(GETOPT_NO_OPTERR)
+
+#define printerr(argv, msg)
+
+#elif defined(GETOPT_NO_STDIO)
+
+extern int write(int, void *, int);
+
+static void printerr(char *const *argv, const char *msg) {
+  if (opterr) {
+    char buf[64];
+    unsigned pl = strlen(argv[0]);
+    unsigned ml = strlen(msg);
+    char *p;
+    if (pl + /*": "*/2 + ml + /*" -- c\n"*/6 > sizeof(buf)) {
+      write(2, argv[0], pl);
+      p = buf;
+    }
+    else {
+      memcpy(buf, argv[0], ml);
+      p = buf + pl;
+    }
+    *p++ = ':'; *p++ = ' ';
+    memcpy(p, msg, ml); p += ml;
+    *p++ = ' '; *p++ = '-'; *p++ = '-'; *p++ = ' ';
+    *p++ = optopt;
+    *p++ = '\n';
+    write(2, buf, p - buf);
+  }
+}
+
+#else
+
+#include <stdio.h>
+static void printerr(char *const *argv, const char *msg) {
+  if (opterr)
+     fprintf(stderr, "%s: %s -- %c\n", argv[0], msg, optopt);
+}
+
+#endif
+
+GETOPT_CLASS int getopt(int argc, char *const *argv, const char *opts) {
+  char *p;
+
+  optarg = 0;
+  if (*opts == '+') /* GNU extension (permutation) - isn't supported */
+    ++opts;
+
+  if (!optind) {  /* a way to reset things */
+    nextc = 0;
+    optind = 1;
+  }
+
+  if (!nextc || !*nextc) {   /* advance to the next argv element */
+    /* done scanning? */
+    if (optind >= argc)
+      return -1;
+    /* not an optional argument */
+    if (argv[optind][0] != '-')
+      return -1;
+    /* bare `-' */
+    if (argv[optind][1] == '\0')
+      return -1;
+    /* special case `--' argument */
+    if (argv[optind][1] == '-' && argv[optind][2] == '\0') {
+      ++optind;
+      return -1;
+    }
+    nextc = argv[optind] + 1;
+  }
+
+  optopt = *nextc++;
+  if (!*nextc)
+    ++optind;
+  p = strchr(opts, optopt);
+  if (!p || optopt == ':') {
+    printerr(argv, "illegal option");
+    return '?';
+  }
+  if (p[1] == ':') {
+    if (*nextc) {
+      optarg = nextc;
+      nextc = NULL;
+      ++optind;
+    }
+    else if (p[2] != ':') {	/* required argument */
+      if (optind >= argc) {
+        printerr(argv, "option requires an argument");
+        return *opts == ':' ? ':' : '?';
+      }
+      else
+        optarg = argv[optind++];
+    }
+  }
+  return optopt;
+}
+
+#ifdef TEST
+
+#include <stdio.h>
+
+int main(int argc, char **argv) {
+  int c;
+  while((c = getopt(argc, argv, "ab:c::")) != -1) switch(c) {
+  case 'a':
+  case 'b':
+  case 'c':
+    printf("option %c %s\n", c, optarg ? optarg : "(none)");
+    break;
+  default:
+    return -1;
+  }
+  for(c = optind; c < argc; ++c)
+    printf("non-opt: %s\n", argv[c]);
+  return 0;
+}
+
+#endif

3rdparty/udns/inet_XtoX.c

@@ -0,0 +1,327 @@
+/* inet_XtoX.c
+ * Simple implementation of the following functions:
+ *  inet_ntop(), inet_ntoa(), inet_pton(), inet_aton().
+ *
+ * Differences from traditional implementaitons:
+ *  o modifies destination buffers even on error return.
+ *  o no fancy (hex, or 1.2) input support in inet_aton()
+ *  o inet_aton() does not accept junk after an IP address.
+ *  o inet_ntop(AF_INET) requires at least 16 bytes in dest,
+ *    and inet_ntop(AF_INET6) at least 40 bytes
+ *    (traditional inet_ntop() will try to fit anyway)
+ *
+ * Compile with -Dinet_XtoX_prefix=pfx_ to have pfx_*() instead of inet_*()
+ * Compile with -Dinet_XtoX_no_ntop or -Dinet_XtoX_no_pton
+ *  to disable net2str or str2net conversions.
+ *
+ * #define inet_XtoX_prototypes and #include "this_file.c"
+ * to get function prototypes only (but not for inet_ntoa()).
+ * #define inet_XtoX_decl to be `static' for static visibility,
+ * or use __declspec(dllexport) or somesuch...
+ *
+ * Compile with -DTEST to test against stock implementation.
+ *
+ * Written by Michael Tokarev.  Public domain.
+ */
+
+#ifdef inet_XtoX_prototypes
+
+struct in_addr;
+
+#else
+
+#include <errno.h>
+
+#ifdef TEST
+
+# include <netinet/in.h>
+# include <sys/socket.h>
+# include <arpa/inet.h>
+# include <stdio.h>
+# include <stdlib.h>
+# include <unistd.h>
+# include <string.h>
+# undef inet_XtoX_prefix
+# define inet_XtoX_prefix mjt_inet_
+# undef inet_XtoX_no_ntop
+# undef inet_XtoX_no_pton
+
+#else /* !TEST */
+
+struct in_addr {	/* declare it here to avoid messing with headers */
+  unsigned char x[4];
+};
+
+#endif /* TEST */
+
+#endif /* inet_XtoX_prototypes */
+
+#ifndef inet_XtoX_prefix
+# define inet_XtoX_prefix inet_
+#endif
+#ifndef inet_XtoX_decl
+# define inet_XtoX_decl /*empty*/
+#endif
+
+#define cc2_(x,y) cc2__(x,y)
+#define cc2__(x,y) x##y
+#define fn(x) cc2_(inet_XtoX_prefix,x)
+
+#ifndef inet_XtoX_no_ntop
+
+inet_XtoX_decl const char *
+fn(ntop)(int af, const void *src, char *dst, unsigned size);
+
+#ifndef inet_XtoX_prototypes
+
+static int mjt_ntop4(const void *_src, char *dst, int size) {
+  unsigned i, x, r;
+  char *p;
+  const unsigned char *s = _src;
+  if (size < 4*4)	/* for simplicity, disallow non-max-size buffer */
+    return 0;
+  for (i = 0, p = dst; i < 4; ++i) {
+    if (i) *p++ = '.';
+    x = r = s[i];
+    if (x > 99) { *p++ = (char)(r / 100 + '0'); r %= 100; }
+    if (x > 9) { *p++ = (char)(r / 10 + '0'); r %= 10; }
+    *p++ = (char)(r + '0');
+  }
+  *p = '\0';
+  return 1;
+}
+
+static char *hexc(char *p, unsigned x) {
+  static char hex[16] = "0123456789abcdef";
+  if (x > 0x0fff) *p++ = hex[(x >>12) & 15];
+  if (x > 0x00ff) *p++ = hex[(x >> 8) & 15];
+  if (x > 0x000f) *p++ = hex[(x >> 4) & 15];
+  *p++ = hex[x & 15];
+  return p;
+}
+
+static int mjt_ntop6(const void *_src, char *dst, int size) {
+  unsigned i;
+  unsigned short w[8];
+  unsigned bs = 0, cs = 0;
+  unsigned bl = 0, cl = 0;
+  char *p;
+  const unsigned char *s = _src;
+
+  if (size < 40)	/* for simplicity, disallow non-max-size buffer */
+    return 0;
+
+  for(i = 0; i < 8; ++i, s += 2) {
+    w[i] = (((unsigned short)(s[0])) << 8) | s[1];
+    if (!w[i]) {
+      if (!cl++) cs = i;
+    }
+    else {
+      if (cl > bl) bl = cl, bs = cs;
+    }
+  }
+  if (cl > bl) bl = cl, bs = cs;
+  p = dst;
+  if (bl == 1)
+    bl = 0;
+  if (bl) {
+    for(i = 0; i < bs; ++i) {
+      if (i) *p++ = ':';
+      p = hexc(p, w[i]);
+    }
+    *p++ = ':';
+    i += bl;
+    if (i == 8)
+      *p++ = ':';
+  }
+  else
+    i = 0;
+  for(; i < 8; ++i) {
+    if (i) *p++ = ':';
+    if (i == 6 && !bs && (bl == 6 || (bl == 5 && w[5] == 0xffff)))
+      return mjt_ntop4(s - 4, p, size - (p - dst));
+    p = hexc(p, w[i]);
+  }
+  *p = '\0';
+  return 1;
+}
+
+inet_XtoX_decl const char *
+fn(ntop)(int af, const void *src, char *dst, unsigned size) {
+  switch(af) {
+  /* don't use AF_*: don't mess with headers */
+  case 2:  /* AF_INET */  if (mjt_ntop4(src, dst, size)) return dst; break;
+  case 10: /* AF_INET6 */ if (mjt_ntop6(src, dst, size)) return dst; break;
+  default: errno = EAFNOSUPPORT; return (char*)0;
+  }
+  errno = ENOSPC;
+  return (char*)0;
+}
+
+inet_XtoX_decl const char *
+fn(ntoa)(struct in_addr addr) {
+  static char buf[4*4];
+  mjt_ntop4(&addr, buf, sizeof(buf));
+  return buf;
+}
+
+#endif /* inet_XtoX_prototypes */
+#endif /* inet_XtoX_no_ntop */
+
+#ifndef inet_XtoX_no_pton
+
+inet_XtoX_decl int fn(pton)(int af, const char *src, void *dst);
+inet_XtoX_decl int fn(aton)(const char *src, struct in_addr *addr);
+
+#ifndef inet_XtoX_prototypes
+
+static int mjt_pton4(const char *c, void *dst) {
+  unsigned char *a = dst;
+  unsigned n, o;
+  for (n = 0; n < 4; ++n) {
+    if (*c < '0' || *c > '9')
+      return 0;
+    o = *c++ - '0';
+    while(*c >= '0' && *c <= '9')
+      if ((o = o * 10 + (*c++ - '0')) > 255)
+        return 0;
+    if (*c++ != (n == 3 ? '\0' : '.'))
+      return 0;
+    *a++ = (unsigned char)o;
+  }
+  return 1;
+}
+
+static int mjt_pton6(const char *c, void *dst) {
+  unsigned short w[8], *a = w, *z, *i;
+  unsigned v, o;
+  const char *sc;
+  unsigned char *d = dst;
+  if (*c != ':') z = (unsigned short*)0;
+  else if (*++c != ':') return 0;
+  else ++c, z = a;
+  i = 0;
+  for(;;) {
+    v = 0;
+    sc = c;
+    for(;;) {
+      if (*c >= '0' && *c <= '9') o = *c - '0';
+      else if (*c >= 'a' && *c <= 'f') o = *c - 'a' + 10;
+      else if (*c >= 'A' && *c <= 'F') o = *c - 'A' + 10;
+      else break;
+      v = (v << 4) | o;
+      if (v > 0xffff) return 0;
+      ++c;
+    }
+    if (sc == c) {
+      if (z == a && !*c)
+        break;
+      else
+        return 0;
+    }
+    if (*c == ':') {
+      if (a >= w + 8)
+        return 0;
+      *a++ = v;
+      if (*++c == ':') {
+        if (z)
+          return 0;
+        z = a;
+        if (!*++c)
+          break;
+      }
+    }
+    else if (!*c) {
+      if (a >= w + 8)
+        return 0;
+      *a++ = v;
+      break;
+    }
+    else if (*c == '.') {
+      if (a > w + 6)
+        return 0;
+      if (!mjt_pton4(sc, d))
+        return 0;
+      *a++ = ((unsigned)(d[0]) << 8) | d[1];
+      *a++ = ((unsigned)(d[2]) << 8) | d[3];
+      break;
+    }
+    else
+      return 0;
+  }
+  v = w + 8 - a;
+  if ((v && !z) || (!v && z))
+    return 0;
+  for(i = w; ; ++i) {
+    if (i == z)
+      while(v--) { *d++ = '\0'; *d++ = '\0'; }
+    if (i >= a)
+      break;
+    *d++ = (unsigned char)((*i >> 8) & 255);
+    *d++ = (unsigned char)(*i & 255);
+  }
+  return 1;
+}
+
+inet_XtoX_decl int fn(pton)(int af, const char *src, void *dst) {
+  switch(af) {
+  /* don't use AF_*: don't mess with headers */
+  case 2  /* AF_INET  */: return mjt_pton4(src, dst);
+  case 10 /* AF_INET6 */: return mjt_pton6(src, dst);
+  default: errno = EAFNOSUPPORT; return -1;
+  }
+}
+
+inet_XtoX_decl int fn(aton)(const char *src, struct in_addr *addr) {
+  return mjt_pton4(src, addr);
+}
+
+#endif /* inet_XtoX_prototypes */
+
+#endif /* inet_XtoX_no_pton */
+
+#ifdef TEST
+
+int main(int argc, char **argv) {
+  int i;
+  char n0[16], n1[16];
+  char p0[64], p1[64];
+  int af = AF_INET;
+  int pl = sizeof(p0);
+  int r0, r1;
+  const char *s0, *s1;
+
+  while((i = getopt(argc, argv, "46a:p:")) != EOF) switch(i) {
+  case '4': af = AF_INET;  break;
+  case '6': af = AF_INET6; break;
+  case 'a': case 'p': pl = atoi(optarg); break;
+  default: return 1;
+  }
+  for(i = optind; i < argc; ++i) {
+    char *a = argv[i];
+
+    printf("%s:\n", a);
+    r0 = inet_pton(af, a, n0);
+    printf(" p2n stock: %s\n",
+     (r0 < 0 ? "(notsupp)" : !r0 ? "(inval)" : fn(ntop)(af,n0,p0,sizeof(p0))));
+    r1 = fn(pton)(af, a, n1);
+    printf(" p2n this : %s\n",
+     (r1 < 0 ? "(notsupp)" : !r1 ? "(inval)" : fn(ntop)(af,n1,p1,sizeof(p1))));
+
+    if ((r0 > 0) != (r1 > 0) ||
+        (r0 > 0 && r1 > 0 && memcmp(n0, n1, af == AF_INET ? 4 : 16) != 0))
+      printf(" DIFFER!\n");
+
+    s0 = inet_ntop(af, n1, p0, pl);
+    printf(" n2p stock: %s\n", s0 ? s0 : "(inval)");
+    s1 = fn(ntop)(af, n1, p1, pl);
+    printf(" n2p this : %s\n", s1 ? s1 : "(inval)");
+    if ((s0 != 0) != (s1 != 0) ||
+        (s0 && s1 && strcmp(s0, s1) != 0))
+      printf(" DIFFER!\n");
+
+  }
+  return 0;
+}
+
+#endif /* TEST */

3rdparty/udns/rblcheck.1

@@ -0,0 +1,151 @@
+.\" rblcheck.1
+.\" rblckeck manpage
+.\"
+.\" Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+.\" This file is part of UDNS library, an async DNS stub resolver.
+.\"
+.\" This library is free software; you can redistribute it and/or
+.\" modify it under the terms of the GNU Lesser General Public
+.\" License as published by the Free Software Foundation; either
+.\" version 2.1 of the License, or (at your option) any later version.
+.\"
+.\" This library is distributed in the hope that it will be useful,
+.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
+.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+.\" Lesser General Public License for more details.
+.\"
+.\" You should have received a copy of the GNU Lesser General Public
+.\" License along with this library, in file named COPYING.LGPL; if not,
+.\" write to the Free Software Foundation, Inc., 59 Temple Place,
+.\" Suite 330, Boston, MA  02111-1307  USA
+
+.TH rblckeck 1 "Apr 2005" "User Utilities"
+
+.SH NAME
+rblckeck \- DNSBL lookup utility
+
+.SH SYNOPSYS
+.B rblcheck
+.RB [\| \-s
+.IR zone \|]
+.RB [\| \-S
+.IR zone\-file \|]
+.RB [\| \-c \|]
+.RB [\| \-tmvq \|]
+.RB [\| \-n
+.IR nsaddr \|]
+.IR address \|.\|.\|.
+
+.SH DESCRIPTION
+.B rblcheck
+is a simple command-line to perform DNSBL (DNS-based blocklists) lookups.
+For every IP address (or a name, in which case it will be resolved to an
+address first), the utility verifies whenever it is listed in a (list of)
+DNS blocklists specified with
+.B \-s
+or
+.B \-S
+options, optionally obtains text assotiated with the listing (usually it
+is either some description about the reason of the listing or an URL
+referring to such a description), and displays results on standard output.
+.PP
+The program is implemented on top of
+.BR udns (3)
+library.
+
+.SH OPTIONS
+
+The following options are recognized by
+.BR rblcheck :
+
+.TP
+.B \-s \fIzone\fR
+add the given \fIzone\fR DNSBL name to the list of active zones.
+.TP
+.B \-S \fIzone-file\fR
+add list of zones from the named \fIzone-file\fR to the list of
+active zones (the file specifies one zone as the first word on a
+line, empty lines and lines starting with `#' character are ignored).
+.TP
+.B \-c
+reset active zone list.
+.TP
+.B \-v
+be more verbose, produce more detailed output.
+.TP
+.B \-q
+the opposite for \fB\-v\fR -- produce less detailed output.
+.TP
+.B \-t
+obtain text for listed addresses.
+.TP
+.B \-n \fInsaddr\fR
+Use the given nameserver (given as IPv4 or IPv6 address) instead of the
+default.  The same effect may be achieved by setting $NSCACHEIP environment
+variable.
+.TP
+.B \-m
+stop after first hit, ie after the first address which is found to be
+listed.
+
+.TP
+.B \-h
+print short help and exit.
+
+.PP
+If no
+.BR \-s ,
+.BR \-S
+and
+.B \-c
+options are given,
+.B rblcheck
+will try to obtain list of zones using $RBLCHECK_ZONES environment variable,
+or ~/.rblcheckrc, or /etc/rblckechrc files, in that order.  If no zones are
+found, it will exit unsuccessefully.
+
+.SH "RETURN VALUE"
+When no addresses given are listed and no errors occured,
+.B rblcheck
+exits with code 0.  If at least one address is listed,
+.B rblcheck
+returns 100.  In case of DNS errors,
+.B rblcheck
+returns 2.
+
+.SH ENVIRONMENT
+
+.TP
+.B $RBLCHECK_ZONES
+if no
+.BR \-s ,
+.B \-S
+or
+.B \-c
+option is given,
+.B rblcheck
+tries this variable to obtain list of DNSBL zones to check against.
+
+.SH FILES
+
+.TP
+$HOME/.rblcheckrc and /etc/rblcheckrc
+if no
+.BR \-s ,
+.B \-S
+or
+.B \-c
+option is given, and no $RBLCHECK_ZONES environment variable is set,
+.B rblcheck
+will try the two files (the first one that exists) to obtain list of
+DNSBL zones to check against.
+Each line specifies one zone (only first word in each line is used).
+Empty lines and lines starting with `#' character are ignored.
+
+.SH "SEE ALSO"
+.BR dnsget (1)
+.BR resolv.conf (5)
+.BR udns (3).
+
+.SH AUTHOR
+This program and manual pages are written by Michael Tokarev.

3rdparty/udns/rblcheck.c

@@ -0,0 +1,378 @@
+/* rblcheck.c
+   dnsbl (rbl) checker application
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#ifdef WINDOWS
+# include <winsock2.h>
+#else
+# include <unistd.h>
+# include <sys/types.h>
+# include <sys/socket.h>
+# include <netinet/in.h>
+#endif
+#include <time.h>
+#include <errno.h>
+#include <stdarg.h>
+#include "udns.h"
+
+#ifndef HAVE_GETOPT
+# include "getopt.c"
+#endif
+
+static const char *version = "udns-rblcheck 0.4";
+static char *progname;
+
+static void error(int die, const char *fmt, ...) {
+  va_list ap;
+  fprintf(stderr, "%s: ", progname);
+  va_start(ap, fmt); vfprintf(stderr, fmt, ap); va_end(ap);
+  putc('\n', stderr);
+  fflush(stderr);
+  if (die)
+    exit(1);
+}
+
+struct rblookup {
+  struct ipcheck *parent;
+  struct in_addr key;
+  const char *zone;
+  struct dns_rr_a4  *addr;
+  struct dns_rr_txt *txt;
+};
+
+struct ipcheck {
+  const char *name;
+  int naddr;
+  int listed;
+  struct rblookup *lookup;
+};
+
+#define notlisted ((void*)1)
+
+static int nzones, nzalloc;
+static const char **zones;
+
+static int do_txt;
+static int stopfirst;
+static int verbose = 1;
+/* verbosity level:
+ * <0 - only bare As/TXTs
+ * 0 - what RBL result
+ * 1(default) - what is listed by RBL: result
+ * 2          - what is[not ]listed by RBL: result, name lookups
+ */
+
+static int listed;
+static int failures;
+
+static void *ecalloc(int size, int cnt) {
+  void *t = calloc(size, cnt);
+  if (!t)
+    error(1, "out of memory");
+  return t;
+}
+
+static void addzone(const char *zone) {
+  if (nzones >= nzalloc) {
+    const char **zs = (const char**)ecalloc(sizeof(char*), (nzalloc += 16));
+    if (zones) {
+      memcpy(zs, zones, nzones * sizeof(char*));
+      free(zones);
+    }
+    zones = zs;
+  }
+  zones[nzones++] = zone;
+}
+
+static int addzonefile(const char *fname) {
+  FILE *f = fopen(fname, "r");
+  char linebuf[2048];
+  if (!f)
+    return 0;
+  while(fgets(linebuf, sizeof(linebuf), f)) {
+    char *p = linebuf, *e;
+    while(*p == ' ' || *p == '\t') ++p;
+    if (*p == '#' || *p == '\n') continue;
+    e = p;
+    while(*e && *e != ' ' && *e != '\t' && *e != '\n')
+      ++e;
+    *e++ = '\0';
+    p = memcpy(ecalloc(e - p, 1), p, e - p); // strdup
+    addzone(p);
+  }
+  fclose(f);
+  return 1;
+}
+
+static void dnserror(struct rblookup *ipl, const char *what) {
+  char buf[4*4];
+  error(0, "unable to %s for %s (%s): %s",
+          what, dns_ntop(AF_INET, &ipl->key, buf, sizeof(buf)),
+          ipl->zone, dns_strerror(dns_status(0)));
+  ++failures;
+}
+
+static void display_result(struct ipcheck *ipc) {
+  int j;
+  struct rblookup *l, *le;
+  char buf[4*4];
+  if (!ipc->naddr) return;
+  for (l = ipc->lookup, le = l + nzones * ipc->naddr; l < le; ++l) {
+    if (!l->addr) continue;
+    if (verbose < 2 && l->addr == notlisted) continue;
+    if (verbose >= 0) {
+      dns_ntop(AF_INET, &l->key, buf, sizeof(buf));
+      if (ipc->name) printf("%s[%s]", ipc->name, buf);
+      else printf("%s", buf);
+    }
+    if (l->addr == notlisted) {
+      printf(" is NOT listed by %s\n", l->zone);
+      continue;
+    }
+    else if (verbose >= 1)
+      printf(" is listed by %s: ", l->zone);
+    else if (verbose >= 0)
+      printf(" %s ", l->zone);
+    if (verbose >= 1 || !do_txt)
+      for (j = 0; j < l->addr->dnsa4_nrr; ++j)
+        printf("%s%s", j ? " " : "",
+               dns_ntop(AF_INET, &l->addr->dnsa4_addr[j], buf, sizeof(buf)));
+    if (!do_txt) ;
+    else if (l->txt) {
+      for(j = 0; j < l->txt->dnstxt_nrr; ++j) {
+        unsigned char *t = l->txt->dnstxt_txt[j].txt;
+        unsigned char *e = t + l->txt->dnstxt_txt[j].len;
+        printf("%s\"", verbose > 0 ? "\n\t" : j ? " " : "");
+        while(t < e) {
+          if (*t < ' ' || *t >= 127) printf("\\x%02x", *t);
+          else if (*t == '\\' || *t == '"') printf("\\%c", *t);
+          else putchar(*t);
+          ++t;
+        }
+        putchar('"');
+      }
+      free(l->txt);
+    }
+    else
+      printf("%s<no text available>", verbose > 0 ? "\n\t" : "");
+    free(l->addr);
+    putchar('\n');
+  }
+  free(ipc->lookup);
+}
+
+static void txtcb(struct dns_ctx *ctx, struct dns_rr_txt *r, void *data) {
+  struct rblookup *ipl = data;
+  if (r) {
+    ipl->txt = r;
+    ++ipl->parent->listed;
+  }
+  else if (dns_status(ctx) != DNS_E_NXDOMAIN)
+    dnserror(ipl, "lookup DNSBL TXT record");
+}
+
+static void a4cb(struct dns_ctx *ctx, struct dns_rr_a4 *r, void *data) {
+  struct rblookup *ipl = data;
+  if (r) {
+    ipl->addr = r;
+    ++listed;
+    if (do_txt) {
+      if (dns_submit_a4dnsbl_txt(0, &ipl->key, ipl->zone, txtcb, ipl))
+        return;
+      dnserror(ipl, "submit DNSBL TXT record");
+    }
+    ++ipl->parent->listed;
+  }
+  else if (dns_status(ctx) != DNS_E_NXDOMAIN)
+    dnserror(ipl, "lookup DNSBL A record");
+  else
+    ipl->addr = notlisted;
+}
+
+static int
+submit_a_queries(struct ipcheck *ipc,
+                 int naddr, const struct in_addr *addr) {
+  int z, a;
+  struct rblookup *rl = ecalloc(sizeof(*rl), nzones * naddr);
+  ipc->lookup = rl;
+  ipc->naddr = naddr;
+  for(a = 0; a < naddr; ++a) {
+    for(z = 0; z < nzones; ++z) {
+      rl->key = addr[a];
+      rl->zone = zones[z];
+      rl->parent = ipc;
+      if (!dns_submit_a4dnsbl(0, &rl->key, rl->zone, a4cb, rl))
+        dnserror(rl, "submit DNSBL A query");
+      ++rl;
+    }
+  }
+  return 0;
+}
+
+static void namecb(struct dns_ctx *ctx, struct dns_rr_a4 *rr, void *data) {
+  struct ipcheck *ipc = data;
+  if (rr) {
+    submit_a_queries(ipc, rr->dnsa4_nrr, rr->dnsa4_addr);
+    free(rr);
+  }
+  else {
+    error(0, "unable to lookup `%s': %s",
+          ipc->name, dns_strerror(dns_status(ctx)));
+    ++failures;
+  }
+}
+
+static int submit(struct ipcheck *ipc) {
+  struct in_addr addr;
+  if (dns_pton(AF_INET, ipc->name, &addr) > 0) {
+    submit_a_queries(ipc, 1, &addr);
+    ipc->name = NULL;
+  }
+  else if (!dns_submit_a4(0, ipc->name, 0, namecb, ipc)) {
+    error(0, "unable to submit name query for %s: %s\n",
+          ipc->name, dns_strerror(dns_status(0)));
+    ++failures;
+  }
+  return 0;
+}
+
+static void waitdns(struct ipcheck *ipc) {
+  struct timeval tv;
+  fd_set fds;
+  int c;
+  int fd = dns_sock(NULL);
+  time_t now = 0;
+  FD_ZERO(&fds);
+  while((c = dns_timeouts(NULL, -1, now)) > 0) {
+    FD_SET(fd, &fds);
+    tv.tv_sec = c;
+    tv.tv_usec = 0;
+    c = select(fd+1, &fds, NULL, NULL, &tv);
+    now = time(NULL);
+    if (c > 0)
+      dns_ioevent(NULL, now);
+    if (stopfirst && ipc->listed)
+      break;
+  }
+}
+
+int main(int argc, char **argv) {
+  int c;
+  struct ipcheck ipc;
+  char *nameserver = NULL;
+  int zgiven = 0;
+
+  if (!(progname = strrchr(argv[0], '/'))) progname = argv[0];
+  else argv[0] = ++progname;
+
+  while((c = getopt(argc, argv, "hqtvms:S:cn:")) != EOF) switch(c) {
+  case 's': ++zgiven; addzone(optarg); break;
+  case 'S':
+    ++zgiven;
+    if (addzonefile(optarg)) break;
+    error(1, "unable to read zonefile `%s'", optarg);
+  case 'c': ++zgiven; nzones = 0; break;
+  case 'q': --verbose; break;
+  case 'v': ++verbose; break;
+  case 't': do_txt = 1; break;
+  case 'n': nameserver = optarg; break;
+  case 'm': ++stopfirst; break;
+  case 'h':
+    printf("%s: %s (udns library version %s).\n",
+           progname, version, dns_version());
+    printf("Usage is: %s [options] address..\n", progname);
+    printf(
+"Where options are:\n"
+" -h - print this help and exit\n"
+" -s service - add the service (DNSBL zone) to the serice list\n"
+" -S service-file - add the DNSBL zone(s) read from the given file\n"
+" -c - clear service list\n"
+" -v - increase verbosity level (more -vs => more verbose)\n"
+" -q - decrease verbosity level (opposite of -v)\n"
+" -t - obtain and print TXT records if any\n"
+" -m - stop checking after first address match in any list\n"
+" -n ipaddr - use the given nameserver instead of the default\n"
+"(if no -s or -S option is given, use $RBLCHECK_ZONES, ~/.rblcheckrc\n"
+"or /etc/rblcheckrc in that order)\n"
+    );
+    return 0;
+  default:
+    error(1, "use `%s -h' for help", progname);
+  }
+
+  if (!zgiven) {
+    char *s = getenv("RBLCHECK_ZONES");
+    if (s) {
+      char *k;
+      s = strdup(s);
+      for(k = strtok(s, " \t"); k; k = strtok(NULL, " \t"))
+        addzone(k);
+      free(s);
+    }
+    else {	/* probably worthless on windows? */
+      char *path;
+      char *home = getenv("HOME");
+      if (!home) home = ".";
+      path = malloc(strlen(home) + 1 + sizeof(".rblcheckrc"));
+      sprintf(path, "%s/.rblcheckrc", home);
+      if (!addzonefile(path))
+        addzonefile("/etc/rblcheckrc");
+      free(path);
+    }
+  }
+  if (!nzones)
+    error(1, "no service (zone) list specified (-s or -S option)");
+
+  argv += optind;
+  argc -= optind;
+
+  if (!argc)
+    return 0;
+
+  if (dns_init(NULL, 0) < 0)
+    error(1, "unable to initialize DNS library: %s", strerror(errno));
+  if (nameserver) {
+    dns_add_serv(NULL, NULL);
+    if (dns_add_serv(NULL, nameserver) < 0)
+      error(1, "wrong IP address for a nameserver: `%s'", nameserver);
+  }
+  if (dns_open(NULL) < 0)
+    error(1, "unable to initialize DNS library: %s", strerror(errno));
+
+  for (c = 0; c < argc; ++c) {
+    if (c && (verbose > 1 || (verbose == 1 && do_txt))) putchar('\n');
+    memset(&ipc, 0, sizeof(ipc));
+    ipc.name = argv[c];
+    submit(&ipc);
+    waitdns(&ipc);
+    display_result(&ipc);
+    if (stopfirst > 1 && listed) break;
+  }
+
+  return listed ? 100 : failures ? 2 : 0;
+}

3rdparty/udns/udns.h

@@ -0,0 +1,778 @@
+/* udns.h
+   header file for the UDNS library.
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#ifndef UDNS_VERSION	/* include guard */
+
+#define UDNS_VERSION "0.4"
+
+#ifdef WINDOWS
+# ifdef UDNS_DYNAMIC_LIBRARY
+#  ifdef DNS_LIBRARY_BUILD
+#   define UDNS_API __declspec(dllexport)
+#   define UDNS_DATA_API __declspec(dllexport)
+#  else
+#   define UDNS_API __declspec(dllimport)
+#   define UDNS_DATA_API __declspec(dllimport)
+#  endif
+# endif
+#endif
+
+#ifndef UDNS_API
+# define UDNS_API
+#endif
+#ifndef UDNS_DATA_API
+# define UDNS_DATA_API
+#endif
+
+#include <sys/types.h>		/* for time_t */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* forward declarations if sockets stuff isn't #include'd */
+struct in_addr;
+struct in6_addr;
+struct sockaddr;
+
+/**************************************************************************/
+/**************** Common definitions **************************************/
+
+UDNS_API const char *
+dns_version(void);
+
+struct dns_ctx;
+struct dns_query;
+
+/* shorthand for [const] unsigned char */
+typedef unsigned char dnsc_t;
+typedef const unsigned char dnscc_t;
+
+#define DNS_MAXDN	255	/* max DN length */
+#define DNS_DNPAD	1	/* padding for DN buffers */
+#define DNS_MAXLABEL	63	/* max DN label length */
+#define DNS_MAXNAME	1024	/* max asciiz domain name length */
+#define DNS_HSIZE	12	/* DNS packet header size */
+#define DNS_PORT	53	/* default domain port */
+#define DNS_MAXSERV	6	/* max servers to consult */
+#define DNS_MAXPACKET	512	/* max traditional-DNS UDP packet size */
+#define DNS_EDNS0PACKET	4096	/* EDNS0 packet size to use */
+
+enum dns_class {	/* DNS RR Classes */
+  DNS_C_INVALID	= 0,	/* invalid class */
+  DNS_C_IN	= 1,	/* Internet */
+  DNS_C_CH	= 3,	/* CHAOS */
+  DNS_C_HS	= 4,	/* HESIOD */
+  DNS_C_ANY	= 255	/* wildcard */
+};
+
+enum dns_type {		/* DNS RR Types */
+  DNS_T_INVALID		= 0,	/* Cookie. */
+  DNS_T_A		= 1,	/* Host address. */
+  DNS_T_NS		= 2,	/* Authoritative server. */
+  DNS_T_MD		= 3,	/* Mail destination. */
+  DNS_T_MF		= 4,	/* Mail forwarder. */
+  DNS_T_CNAME		= 5,	/* Canonical name. */
+  DNS_T_SOA		= 6,	/* Start of authority zone. */
+  DNS_T_MB		= 7,	/* Mailbox domain name. */
+  DNS_T_MG		= 8,	/* Mail group member. */
+  DNS_T_MR		= 9,	/* Mail rename name. */
+  DNS_T_NULL		= 10,	/* Null resource record. */
+  DNS_T_WKS		= 11,	/* Well known service. */
+  DNS_T_PTR		= 12,	/* Domain name pointer. */
+  DNS_T_HINFO		= 13,	/* Host information. */
+  DNS_T_MINFO		= 14,	/* Mailbox information. */
+  DNS_T_MX		= 15,	/* Mail routing information. */
+  DNS_T_TXT		= 16,	/* Text strings. */
+  DNS_T_RP		= 17,	/* Responsible person. */
+  DNS_T_AFSDB		= 18,	/* AFS cell database. */
+  DNS_T_X25		= 19,	/* X_25 calling address. */
+  DNS_T_ISDN		= 20,	/* ISDN calling address. */
+  DNS_T_RT		= 21,	/* Router. */
+  DNS_T_NSAP		= 22,	/* NSAP address. */
+  DNS_T_NSAP_PTR	= 23,	/* Reverse NSAP lookup (deprecated). */
+  DNS_T_SIG		= 24,	/* Security signature. */
+  DNS_T_KEY		= 25,	/* Security key. */
+  DNS_T_PX		= 26,	/* X.400 mail mapping. */
+  DNS_T_GPOS		= 27,	/* Geographical position (withdrawn). */
+  DNS_T_AAAA		= 28,	/* Ip6 Address. */
+  DNS_T_LOC		= 29,	/* Location Information. */
+  DNS_T_NXT		= 30,	/* Next domain (security). */
+  DNS_T_EID		= 31,	/* Endpoint identifier. */
+  DNS_T_NIMLOC		= 32,	/* Nimrod Locator. */
+  DNS_T_SRV		= 33,	/* Server Selection. */
+  DNS_T_ATMA		= 34,	/* ATM Address */
+  DNS_T_NAPTR		= 35,	/* Naming Authority PoinTeR */
+  DNS_T_KX		= 36,	/* Key Exchange */
+  DNS_T_CERT		= 37,	/* Certification record */
+  DNS_T_A6		= 38,	/* IPv6 address (deprecates AAAA) */
+  DNS_T_DNAME		= 39,	/* Non-terminal DNAME (for IPv6) */
+  DNS_T_SINK		= 40,	/* Kitchen sink (experimentatl) */
+  DNS_T_OPT		= 41,	/* EDNS0 option (meta-RR) */
+  DNS_T_DS		= 43,	/* DNSSEC */
+  DNS_T_SSHFP		= 44,
+  DNS_T_IPSECKEY	= 45,
+  DNS_T_RRSIG		= 46,	/* DNSSEC */
+  DNS_T_NSEC		= 47,	/* DNSSEC */
+  DNS_T_DNSKEY		= 48,
+  DNS_T_DHCID		= 49,
+  DNS_T_NSEC3		= 50,
+  DNS_T_NSEC3PARAMS	= 51,
+  DNS_T_TALINK		= 58, /* draft-ietf-dnsop-trust-history */
+  DNS_T_SPF		= 99,
+  DNS_T_UINFO		= 100,
+  DNS_T_UID		= 101,
+  DNS_T_GID		= 102,
+  DNS_T_UNSPEC		= 103,
+  DNS_T_TSIG		= 250,	/* Transaction signature. */
+  DNS_T_IXFR		= 251,	/* Incremental zone transfer. */
+  DNS_T_AXFR		= 252,	/* Transfer zone of authority. */
+  DNS_T_MAILB		= 253,	/* Transfer mailbox records. */
+  DNS_T_MAILA		= 254,	/* Transfer mail agent records. */
+  DNS_T_ANY		= 255,	/* Wildcard match. */
+  DNS_T_ZXFR		= 256,	/* BIND-specific, nonstandard. */
+  DNS_T_DLV		= 32769, /* RFC 4431, 5074, DNSSEC Lookaside Validation */
+  DNS_T_MAX		= 65536
+};
+
+/**************************************************************************/
+/**************** Domain Names (DNs) **************************************/
+
+/* return length of the DN */
+UDNS_API unsigned
+dns_dnlen(dnscc_t *dn);
+
+/* return #of labels in a DN */
+UDNS_API unsigned
+dns_dnlabels(dnscc_t *dn);
+
+/* lower- and uppercase single DN char */
+#define DNS_DNLC(c) ((c) >= 'A' && (c) <= 'Z' ? (c) - 'A' + 'a' : (c))
+#define DNS_DNUC(c) ((c) >= 'a' && (c) <= 'z' ? (c) - 'a' + 'A' : (c))
+
+/* compare the DNs, return dnlen of equal or 0 if not */
+UDNS_API unsigned
+dns_dnequal(dnscc_t *dn1, dnscc_t *dn2);
+
+/* copy one DN to another, size checking */
+UDNS_API unsigned
+dns_dntodn(dnscc_t *sdn, dnsc_t *ddn, unsigned ddnsiz);
+
+/* convert asciiz string of length namelen (0 to use strlen) to DN */
+UDNS_API int
+dns_ptodn(const char *name, unsigned namelen,
+          dnsc_t *dn, unsigned dnsiz, int *isabs);
+
+/* simpler form of dns_ptodn() */
+#define dns_sptodn(name,dn,dnsiz) dns_ptodn((name),0,(dn),(dnsiz),0)
+
+UDNS_DATA_API extern dnscc_t dns_inaddr_arpa_dn[14];
+#define DNS_A4RSIZE	30
+UDNS_API int
+dns_a4todn(const struct in_addr *addr, dnscc_t *tdn,
+           dnsc_t *dn, unsigned dnsiz);
+UDNS_API int
+dns_a4ptodn(const struct in_addr *addr, const char *tname,
+            dnsc_t *dn, unsigned dnsiz);
+UDNS_API dnsc_t *
+dns_a4todn_(const struct in_addr *addr, dnsc_t *dn, dnsc_t *dne);
+
+UDNS_DATA_API extern dnscc_t dns_ip6_arpa_dn[10];
+#define DNS_A6RSIZE	74
+UDNS_API int
+dns_a6todn(const struct in6_addr *addr, dnscc_t *tdn,
+           dnsc_t *dn, unsigned dnsiz);
+UDNS_API int
+dns_a6ptodn(const struct in6_addr *addr, const char *tname,
+            dnsc_t *dn, unsigned dnsiz);
+UDNS_API dnsc_t *
+dns_a6todn_(const struct in6_addr *addr, dnsc_t *dn, dnsc_t *dne);
+
+/* convert DN into asciiz string */
+UDNS_API int
+dns_dntop(dnscc_t *dn, char *name, unsigned namesiz);
+
+/* convert DN into asciiz string, using static buffer (NOT thread-safe!) */
+UDNS_API const char *
+dns_dntosp(dnscc_t *dn);
+
+/* return buffer size (incl. null byte) required for asciiz form of a DN */
+UDNS_API unsigned
+dns_dntop_size(dnscc_t *dn);
+
+/* either wrappers or reimplementations for inet_ntop() and inet_pton() */
+UDNS_API const char *dns_ntop(int af, const void *src, char *dst, int size);
+UDNS_API int dns_pton(int af, const char *src, void *dst);
+
+/**************************************************************************/
+/**************** DNS raw packet layout ***********************************/
+
+enum dns_rcode {	/* reply codes */
+  DNS_R_NOERROR		= 0,	/* ok, no error */
+  DNS_R_FORMERR		= 1,	/* format error */
+  DNS_R_SERVFAIL	= 2,	/* server failed */
+  DNS_R_NXDOMAIN	= 3,	/* domain does not exists */
+  DNS_R_NOTIMPL		= 4,	/* not implemented */
+  DNS_R_REFUSED		= 5,	/* query refused */
+  /* these are for BIND_UPDATE */
+  DNS_R_YXDOMAIN	= 6,	/* Name exists */
+  DNS_R_YXRRSET		= 7,	/* RRset exists */
+  DNS_R_NXRRSET		= 8,	/* RRset does not exist */
+  DNS_R_NOTAUTH		= 9,	/* Not authoritative for zone */
+  DNS_R_NOTZONE		= 10,	/* Zone of record different from zone section */
+  /*ns_r_max = 11,*/
+  /* The following are TSIG extended errors */
+  DNS_R_BADSIG		= 16,
+  DNS_R_BADKEY		= 17,
+  DNS_R_BADTIME		= 18
+};
+
+static __inline unsigned dns_get16(dnscc_t *s) {
+  return ((unsigned)s[0]<<8) | s[1];
+}
+static __inline unsigned dns_get32(dnscc_t *s) {
+  return ((unsigned)s[0]<<24) | ((unsigned)s[1]<<16)
+        | ((unsigned)s[2]<<8) | s[3];
+}
+static __inline dnsc_t *dns_put16(dnsc_t *d, unsigned n) {
+  *d++ = (dnsc_t)((n >> 8) & 255); *d++ = (dnsc_t)(n & 255); return d;
+}
+static __inline dnsc_t *dns_put32(dnsc_t *d, unsigned n) {
+  *d++ = (dnsc_t)((n >> 24) & 255); *d++ = (dnsc_t)((n >> 16) & 255);
+  *d++ = (dnsc_t)((n >>  8) & 255); *d++ = (dnsc_t)(n & 255);
+  return d;
+}
+
+/* DNS Header layout */
+enum {
+ /* bytes 0:1 - query ID */
+  DNS_H_QID1	= 0,
+  DNS_H_QID2	= 1,
+  DNS_H_QID	= DNS_H_QID1,
+#define dns_qid(pkt)	dns_get16((pkt)+DNS_H_QID)
+ /* byte 2: flags1 */
+  DNS_H_F1	= 2,
+  DNS_HF1_QR	= 0x80,	/* query response flag */
+#define dns_qr(pkt)	((pkt)[DNS_H_F1]&DNS_HF1_QR)
+  DNS_HF1_OPCODE = 0x78, /* opcode, 0 = query */
+#define dns_opcode(pkt)	(((pkt)[DNS_H_F1]&DNS_HF1_OPCODE)>>3)
+  DNS_HF1_AA	= 0x04,	/* auth answer */
+#define dns_aa(pkt)	((pkt)[DNS_H_F1]&DNS_HF1_AA)
+  DNS_HF1_TC	= 0x02,	/* truncation flag */
+#define dns_tc(pkt)	((pkt)[DNS_H_F1]&DNS_HF1_TC)
+  DNS_HF1_RD	= 0x01,	/* recursion desired (may be set in query) */
+#define dns_rd(pkt)	((pkt)[DNS_H_F1]&DNS_HF1_RD)
+ /* byte 3: flags2 */
+  DNS_H_F2	= 3,
+  DNS_HF2_RA	= 0x80,	/* recursion available */
+#define dns_ra(pkt)	((pkt)[DNS_H_F2]&DNS_HF2_RA)
+  DNS_HF2_Z	= 0x40,	/* reserved */
+  DNS_HF2_AD	= 0x20, /* DNSSEC: authentic data */
+#define dns_ad(pkt)	((pkt)[DNS_H_F2]&DNS_HF2_AD)
+  DNS_HF2_CD	= 0x10, /* DNSSEC: checking disabled */
+#define dns_cd(pkt)	((pkt)[DNS_H_F2]&DNS_HF2_CD)
+  DNS_HF2_RCODE	= 0x0f,	/* response code, DNS_R_XXX above */
+#define dns_rcode(pkt)	((pkt)[DNS_H_F2]&DNS_HF2_RCODE)
+ /* bytes 4:5: qdcount, numqueries */
+  DNS_H_QDCNT1	= 4,
+  DNS_H_QDCNT2	= 5,
+  DNS_H_QDCNT	= DNS_H_QDCNT1,
+#define dns_numqd(pkt)	dns_get16((pkt)+4)
+ /* bytes 6:7: ancount, numanswers */
+  DNS_H_ANCNT1	= 6,
+  DNS_H_ANCNT2	= 7,
+  DNS_H_ANCNT	= DNS_H_ANCNT1,
+#define dns_numan(pkt)	dns_get16((pkt)+6)
+ /* bytes 8:9: nscount, numauthority */
+  DNS_H_NSCNT1	= 8,
+  DNS_H_NSCNT2	= 9,
+  DNS_H_NSCNT	= DNS_H_NSCNT1,
+#define dns_numns(pkt)	dns_get16((pkt)+8)
+ /* bytes 10:11: arcount, numadditional */
+  DNS_H_ARCNT1	= 10,
+  DNS_H_ARCNT2	= 11,
+  DNS_H_ARCNT	= DNS_H_ARCNT1,
+#define dns_numar(pkt)	dns_get16((pkt)+10)
+#define dns_payload(pkt) ((pkt)+DNS_HSIZE)
+  /* EDNS0 (OPT RR) flags (Ext. Flags) */
+  DNS_EF1_DO	= 0x80, /* DNSSEC OK */
+};
+
+/* packet buffer: start at pkt, end before pkte, current pos *curp.
+ * extract a DN and set *curp to the next byte after DN in packet.
+ * return -1 on error, 0 if dnsiz is too small, or dnlen on ok.
+ */
+UDNS_API int
+dns_getdn(dnscc_t *pkt, dnscc_t **curp, dnscc_t *end,
+          dnsc_t *dn, unsigned dnsiz);
+
+/* skip the DN at position cur in packet ending before pkte,
+ * return pointer to the next byte after the DN or NULL on error */
+UDNS_API dnscc_t *
+dns_skipdn(dnscc_t *end, dnscc_t *cur);
+
+struct dns_rr {		/* DNS Resource Record */
+  dnsc_t dnsrr_dn[DNS_MAXDN];	/* the DN of the RR */
+  enum dns_class dnsrr_cls;	/* Class */
+  enum dns_type  dnsrr_typ;	/* Type */
+  unsigned dnsrr_ttl;		/* Time-To-Live (TTL) */
+  unsigned dnsrr_dsz;		/* data size */
+  dnscc_t *dnsrr_dptr;		/* pointer to start of data */
+  dnscc_t *dnsrr_dend;		/* past end of data */
+};
+
+struct dns_parse {	/* RR/packet parsing state */
+  dnscc_t *dnsp_pkt;		/* start of the packet */
+  dnscc_t *dnsp_end;		/* end of the packet */
+  dnscc_t *dnsp_cur;		/* current packet position */
+  dnscc_t *dnsp_ans;		/* start of answer section */
+  int dnsp_rrl;			/* number of RRs left to go */
+  int dnsp_nrr;			/* RR count so far */
+  unsigned dnsp_ttl;		/* TTL value so far */
+  dnscc_t *dnsp_qdn;		/* the RR DN we're looking for */
+  enum dns_class dnsp_qcls;	/* RR class we're looking for or 0 */
+  enum dns_type  dnsp_qtyp;	/* RR type we're looking for or 0 */
+  dnsc_t dnsp_dnbuf[DNS_MAXDN];	/* domain buffer */
+};
+
+/* initialize the parse structure */
+UDNS_API void
+dns_initparse(struct dns_parse *p, dnscc_t *qdn,
+              dnscc_t *pkt, dnscc_t *cur, dnscc_t *end);
+
+/* search next RR, <0=error, 0=no more RRs, >0 = found. */
+UDNS_API int
+dns_nextrr(struct dns_parse *p, struct dns_rr *rr);
+
+UDNS_API void
+dns_rewind(struct dns_parse *p, dnscc_t *qdn);
+
+
+/**************************************************************************/
+/**************** Resolver Context ****************************************/
+
+/* default resolver context */
+UDNS_DATA_API extern struct dns_ctx dns_defctx;
+
+/* reset resolver context to default state, close it if open, drop queries */
+UDNS_API void
+dns_reset(struct dns_ctx *ctx);
+
+/* reset resolver context and read in system configuration */
+UDNS_API int
+dns_init(struct dns_ctx *ctx, int do_open);
+
+/* return new resolver context with the same settings as copy */
+UDNS_API struct dns_ctx *
+dns_new(const struct dns_ctx *copy);
+
+/* free resolver context returned by dns_new(); all queries are dropped */
+UDNS_API void
+dns_free(struct dns_ctx *ctx);
+
+/* add nameserver for a resolver context (or reset nslist if serv==NULL) */
+UDNS_API int
+dns_add_serv(struct dns_ctx *ctx, const char *serv);
+
+/* add nameserver using struct sockaddr structure (with ports) */
+UDNS_API int
+dns_add_serv_s(struct dns_ctx *ctx, const struct sockaddr *sa);
+
+/* add search list element for a resolver context (or reset it if srch==NULL) */
+UDNS_API int
+dns_add_srch(struct dns_ctx *ctx, const char *srch);
+
+/* set options for a resolver context */
+UDNS_API int
+dns_set_opts(struct dns_ctx *ctx, const char *opts);
+
+enum dns_opt {		/* options */
+  DNS_OPT_FLAGS,	/* flags, DNS_F_XXX */
+  DNS_OPT_TIMEOUT,	/* timeout in secounds */
+  DNS_OPT_NTRIES,	/* number of retries */
+  DNS_OPT_NDOTS,	/* ndots */
+  DNS_OPT_UDPSIZE,	/* EDNS0 UDP size */
+  DNS_OPT_PORT,		/* port to use */
+};
+
+/* set or get (if val<0) an option */
+UDNS_API int
+dns_set_opt(struct dns_ctx *ctx, enum dns_opt opt, int val);
+
+enum dns_flags {
+  DNS_NOSRCH	= 0x00010000,	/* do not perform search */
+  DNS_NORD	= 0x00020000,	/* request no recursion */
+  DNS_AAONLY	= 0x00040000,	/* set AA flag in queries */
+  DNS_SET_DO	= 0x00080000,	/* set EDNS0 "DO" bit (DNSSEC OK) */
+  DNS_SET_CD	= 0x00100000,	/* set CD bit (DNSSEC: checking disabled) */
+};
+
+/* set the debug function pointer */
+typedef void
+(dns_dbgfn)(int code, const struct sockaddr *sa, unsigned salen,
+            dnscc_t *pkt, int plen,
+            const struct dns_query *q, void *data);
+UDNS_API void
+dns_set_dbgfn(struct dns_ctx *ctx, dns_dbgfn *dbgfn);
+
+/* open and return UDP socket */
+UDNS_API int
+dns_open(struct dns_ctx *ctx);
+
+/* return UDP socket or -1 if not open */
+UDNS_API int
+dns_sock(const struct dns_ctx *ctx);
+
+/* close the UDP socket */
+UDNS_API void
+dns_close(struct dns_ctx *ctx);
+
+/* return number of requests queued */
+UDNS_API int
+dns_active(const struct dns_ctx *ctx);
+
+/* return status of the last operation */
+UDNS_API int
+dns_status(const struct dns_ctx *ctx);
+UDNS_API void
+dns_setstatus(struct dns_ctx *ctx, int status);
+
+/* handle I/O event on UDP socket */
+UDNS_API void
+dns_ioevent(struct dns_ctx *ctx, time_t now);
+
+/* process any timeouts, return time in secounds to the
+ * next timeout (or -1 if none) but not greather than maxwait */
+UDNS_API int
+dns_timeouts(struct dns_ctx *ctx, int maxwait, time_t now);
+
+/* define timer requesting routine to use */
+typedef void dns_utm_fn(struct dns_ctx *ctx, int timeout, void *data);
+UDNS_API void
+dns_set_tmcbck(struct dns_ctx *ctx, dns_utm_fn *fn, void *data);
+
+/**************************************************************************/
+/**************** Making Queries ******************************************/
+
+/* query callback routine */
+typedef void dns_query_fn(struct dns_ctx *ctx, void *result, void *data);
+
+/* query parse routine: raw DNS => application structure */
+typedef int
+dns_parse_fn(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+             void **res);
+
+enum dns_status {
+  DNS_E_NOERROR		= 0,	/* ok, not an error */
+  DNS_E_TEMPFAIL	= -1,	/* timeout, SERVFAIL or similar */
+  DNS_E_PROTOCOL	= -2,	/* got garbled reply */
+  DNS_E_NXDOMAIN	= -3,	/* domain does not exists */
+  DNS_E_NODATA		= -4,	/* domain exists but no data of reqd type */
+  DNS_E_NOMEM		= -5,	/* out of memory while processing */
+  DNS_E_BADQUERY	= -6	/* the query is malformed */
+};
+
+/* submit generic DN query */
+UDNS_API struct dns_query *
+dns_submit_dn(struct dns_ctx *ctx,
+              dnscc_t *dn, int qcls, int qtyp, int flags,
+              dns_parse_fn *parse, dns_query_fn *cbck, void *data);
+/* submit generic name query */
+UDNS_API struct dns_query *
+dns_submit_p(struct dns_ctx *ctx,
+             const char *name, int qcls, int qtyp, int flags,
+             dns_parse_fn *parse, dns_query_fn *cbck, void *data);
+
+/* cancel the given async query in progress */
+UDNS_API int
+dns_cancel(struct dns_ctx *ctx, struct dns_query *q);
+
+/* resolve a generic query, return the answer */
+UDNS_API void *
+dns_resolve_dn(struct dns_ctx *ctx,
+               dnscc_t *qdn, int qcls, int qtyp, int flags,
+               dns_parse_fn *parse);
+UDNS_API void *
+dns_resolve_p(struct dns_ctx *ctx,
+              const char *qname, int qcls, int qtyp, int flags,
+              dns_parse_fn *parse);
+UDNS_API void *
+dns_resolve(struct dns_ctx *ctx, struct dns_query *q);
+
+
+/* Specific RR handlers */
+
+#define dns_rr_common(prefix)						\
+  char *prefix##_cname;		/* canonical name */			\
+  char *prefix##_qname;		/* original query name */		\
+  unsigned prefix##_ttl;	/* TTL value */				\
+  int prefix##_nrr		/* number of records */
+
+struct dns_rr_null {		/* NULL RRset, aka RRset template */
+  dns_rr_common(dnsn);
+};
+
+UDNS_API int
+dns_stdrr_size(const struct dns_parse *p);
+UDNS_API void *
+dns_stdrr_finish(struct dns_rr_null *ret, char *cp, const struct dns_parse *p);
+
+struct dns_rr_a4 {		/* the A RRset */
+  dns_rr_common(dnsa4);
+  struct in_addr *dnsa4_addr;	/* array of addresses, naddr elements */
+};
+
+UDNS_API dns_parse_fn dns_parse_a4;	/* A RR parsing routine */
+typedef void				/* A query callback routine */
+dns_query_a4_fn(struct dns_ctx *ctx, struct dns_rr_a4 *result, void *data);
+
+/* submit A IN query */
+UDNS_API struct dns_query *
+dns_submit_a4(struct dns_ctx *ctx, const char *name, int flags,
+              dns_query_a4_fn *cbck, void *data);
+
+/* resolve A IN query */
+UDNS_API struct dns_rr_a4 *
+dns_resolve_a4(struct dns_ctx *ctx, const char *name, int flags);
+
+
+struct dns_rr_a6 {		/* the AAAA RRset */
+  dns_rr_common(dnsa6);
+  struct in6_addr *dnsa6_addr;	/* array of addresses, naddr elements */
+};
+
+UDNS_API dns_parse_fn dns_parse_a6;	/* A RR parsing routine */
+typedef void				/* A query callback routine */
+dns_query_a6_fn(struct dns_ctx *ctx, struct dns_rr_a6 *result, void *data);
+
+/* submit AAAA IN query */
+UDNS_API struct dns_query *
+dns_submit_a6(struct dns_ctx *ctx, const char *name, int flags,
+              dns_query_a6_fn *cbck, void *data);
+
+/* resolve AAAA IN query */
+UDNS_API struct dns_rr_a6 *
+dns_resolve_a6(struct dns_ctx *ctx, const char *name, int flags);
+
+
+struct dns_rr_ptr {		/* the PTR RRset */
+  dns_rr_common(dnsptr);
+  char **dnsptr_ptr;		/* array of PTRs */
+};
+
+UDNS_API dns_parse_fn dns_parse_ptr;	/* PTR RR parsing routine */
+typedef void				/* PTR query callback */
+dns_query_ptr_fn(struct dns_ctx *ctx, struct dns_rr_ptr *result, void *data);
+/* submit PTR IN in-addr.arpa query */
+UDNS_API struct dns_query *
+dns_submit_a4ptr(struct dns_ctx *ctx, const struct in_addr *addr,
+                 dns_query_ptr_fn *cbck, void *data);
+/* resolve PTR IN in-addr.arpa query */
+UDNS_API struct dns_rr_ptr *
+dns_resolve_a4ptr(struct dns_ctx *ctx, const struct in_addr *addr);
+
+/* the same as above, but for ip6.arpa */
+UDNS_API struct dns_query *
+dns_submit_a6ptr(struct dns_ctx *ctx, const struct in6_addr *addr,
+                 dns_query_ptr_fn *cbck, void *data);
+UDNS_API struct dns_rr_ptr *
+dns_resolve_a6ptr(struct dns_ctx *ctx, const struct in6_addr *addr);
+
+
+struct dns_mx {		/* single MX RR */
+  int priority;		/* MX priority */
+  char *name;		/* MX name */
+};
+struct dns_rr_mx {		/* the MX RRset */
+  dns_rr_common(dnsmx);
+  struct dns_mx *dnsmx_mx;	/* array of MXes */
+};
+UDNS_API dns_parse_fn dns_parse_mx;	/* MX RR parsing routine */
+typedef void				/* MX RR callback */
+dns_query_mx_fn(struct dns_ctx *ctx, struct dns_rr_mx *result, void *data);
+/* submit MX IN query */
+UDNS_API struct dns_query *
+dns_submit_mx(struct dns_ctx *ctx, const char *name, int flags,
+              dns_query_mx_fn *cbck, void *data);
+/* resolve MX IN query */
+UDNS_API struct dns_rr_mx *
+dns_resolve_mx(struct dns_ctx *ctx, const char *name, int flags);
+
+
+struct dns_txt {	/* single TXT record */
+  int len;		/* length of the text */
+  dnsc_t *txt;	/* pointer to text buffer. May contain nulls. */
+};
+struct dns_rr_txt {		/* the TXT RRset */
+  dns_rr_common(dnstxt);
+  struct dns_txt *dnstxt_txt;	/* array of TXT records */
+};
+UDNS_API dns_parse_fn dns_parse_txt;	/* TXT RR parsing routine */
+typedef void				/* TXT RR callback */
+dns_query_txt_fn(struct dns_ctx *ctx, struct dns_rr_txt *result, void *data);
+/* submit TXT query */
+UDNS_API struct dns_query *
+dns_submit_txt(struct dns_ctx *ctx, const char *name, int qcls, int flags,
+               dns_query_txt_fn *cbck, void *data);
+/* resolve TXT query */
+UDNS_API struct dns_rr_txt *
+dns_resolve_txt(struct dns_ctx *ctx, const char *name, int qcls, int flags);
+
+
+struct dns_srv {	/* single SRV RR */
+  int priority;		/* SRV priority */
+  int weight;		/* SRV weight */
+  int port;		/* SRV port */
+  char *name;		/* SRV name */
+};
+struct dns_rr_srv {		/* the SRV RRset */
+  dns_rr_common(dnssrv);
+  struct dns_srv *dnssrv_srv;	/* array of SRVes */
+};
+UDNS_API dns_parse_fn dns_parse_srv;	/* SRV RR parsing routine */
+typedef void				/* SRV RR callback */
+dns_query_srv_fn(struct dns_ctx *ctx, struct dns_rr_srv *result, void *data);
+/* submit SRV IN query */
+UDNS_API struct dns_query *
+dns_submit_srv(struct dns_ctx *ctx,
+               const char *name, const char *srv, const char *proto,
+               int flags, dns_query_srv_fn *cbck, void *data);
+/* resolve SRV IN query */
+UDNS_API struct dns_rr_srv *
+dns_resolve_srv(struct dns_ctx *ctx,
+                const char *name, const char *srv, const char *proto,
+                int flags);
+
+/* NAPTR (RFC3403) RR type */
+struct dns_naptr {	/* single NAPTR RR */
+  int order;		/* NAPTR order */
+  int preference;	/* NAPTR preference */
+  char *flags;		/* NAPTR flags */
+  char *service;	/* NAPTR service */
+  char *regexp;		/* NAPTR regexp */
+  char *replacement;	/* NAPTR replacement */
+};
+
+struct dns_rr_naptr {		/* the NAPTR RRset */
+  dns_rr_common(dnsnaptr);
+  struct dns_naptr *dnsnaptr_naptr;	/* array of NAPTRes */
+};
+UDNS_API dns_parse_fn dns_parse_naptr;	/* NAPTR RR parsing routine */
+typedef void				/* NAPTR RR callback */
+dns_query_naptr_fn(struct dns_ctx *ctx,
+                   struct dns_rr_naptr *result, void *data);
+/* submit NAPTR IN query */
+UDNS_API struct dns_query *
+dns_submit_naptr(struct dns_ctx *ctx, const char *name, int flags,
+                 dns_query_naptr_fn *cbck, void *data);
+/* resolve NAPTR IN query */
+UDNS_API struct dns_rr_naptr *
+dns_resolve_naptr(struct dns_ctx *ctx, const char *name, int flags);
+
+
+UDNS_API struct dns_query *
+dns_submit_a4dnsbl(struct dns_ctx *ctx,
+                   const struct in_addr *addr, const char *dnsbl,
+                   dns_query_a4_fn *cbck, void *data);
+UDNS_API struct dns_query *
+dns_submit_a4dnsbl_txt(struct dns_ctx *ctx,
+                       const struct in_addr *addr, const char *dnsbl,
+                       dns_query_txt_fn *cbck, void *data);
+UDNS_API struct dns_rr_a4 *
+dns_resolve_a4dnsbl(struct dns_ctx *ctx,
+                    const struct in_addr *addr, const char *dnsbl);
+UDNS_API struct dns_rr_txt *
+dns_resolve_a4dnsbl_txt(struct dns_ctx *ctx,
+                        const struct in_addr *addr, const char *dnsbl);
+
+UDNS_API struct dns_query *
+dns_submit_a6dnsbl(struct dns_ctx *ctx,
+                   const struct in6_addr *addr, const char *dnsbl,
+                   dns_query_a4_fn *cbck, void *data);
+UDNS_API struct dns_query *
+dns_submit_a6dnsbl_txt(struct dns_ctx *ctx,
+                       const struct in6_addr *addr, const char *dnsbl,
+                       dns_query_txt_fn *cbck, void *data);
+UDNS_API struct dns_rr_a4 *
+dns_resolve_a6dnsbl(struct dns_ctx *ctx,
+                    const struct in6_addr *addr, const char *dnsbl);
+UDNS_API struct dns_rr_txt *
+dns_resolve_a6dnsbl_txt(struct dns_ctx *ctx,
+                        const struct in6_addr *addr, const char *dnsbl);
+
+UDNS_API struct dns_query *
+dns_submit_rhsbl(struct dns_ctx *ctx,
+                 const char *name, const char *rhsbl,
+                 dns_query_a4_fn *cbck, void *data);
+UDNS_API struct dns_query *
+dns_submit_rhsbl_txt(struct dns_ctx *ctx,
+                     const char *name, const char *rhsbl,
+                     dns_query_txt_fn *cbck, void *data);
+UDNS_API struct dns_rr_a4 *
+dns_resolve_rhsbl(struct dns_ctx *ctx, const char *name, const char *rhsbl);
+UDNS_API struct dns_rr_txt *
+dns_resolve_rhsbl_txt(struct dns_ctx *ctx, const char *name, const char *rhsbl);
+
+/**************************************************************************/
+/**************** Names, Names ********************************************/
+
+struct dns_nameval {
+  int val;
+  const char *name;
+};
+
+UDNS_DATA_API extern const struct dns_nameval dns_classtab[];
+UDNS_DATA_API extern const struct dns_nameval dns_typetab[];
+UDNS_DATA_API extern const struct dns_nameval dns_rcodetab[];
+UDNS_API int
+dns_findname(const struct dns_nameval *nv, const char *name);
+#define dns_findclassname(cls) dns_findname(dns_classtab, (cls))
+#define dns_findtypename(type) dns_findname(dns_typetab, (type))
+#define dns_findrcodename(rcode) dns_findname(dns_rcodetab, (rcode))
+
+UDNS_API const char *dns_classname(enum dns_class cls);
+UDNS_API const char *dns_typename(enum dns_type type);
+UDNS_API const char *dns_rcodename(enum dns_rcode rcode);
+const char *_dns_format_code(char *buf, const char *prefix, int code);
+
+UDNS_API const char *dns_strerror(int errnum);
+
+/* simple pseudo-random number generator, code by Bob Jenkins */
+
+struct udns_jranctx {	/* the context */
+  unsigned a, b, c, d;
+};
+
+/* initialize the RNG with a given seed */
+UDNS_API void
+udns_jraninit(struct udns_jranctx *x, unsigned seed);
+
+/* return next random number.  32bits on most platforms so far. */
+UDNS_API unsigned
+udns_jranval(struct udns_jranctx *x);
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#endif	/* include guard */

3rdparty/udns/udns_XtoX.c

@@ -0,0 +1,50 @@
+/* udns_XtoX.c
+   udns_ntop() and udns_pton() routines, which are either
+     - wrappers for inet_ntop() and inet_pton() or
+     - reimplementations of those routines.
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+#include "udns.h"
+
+#ifdef HAVE_INET_PTON_NTOP
+
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <arpa/inet.h>
+
+const char *dns_ntop(int af, const void *src, char *dst, int size) {
+  return inet_ntop(af, src, dst, size);
+}
+
+int dns_pton(int af, const char *src, void *dst) {
+  return inet_pton(af, src, dst);
+}
+
+#else
+#define inet_XtoX_no_ntop
+#define inet_XtoX_prefix dns_
+#include "inet_XtoX.c"
+
+#endif

3rdparty/udns/udns_bl.c

@@ -0,0 +1,160 @@
+/* udns_bl.c
+   DNSBL stuff
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include "udns.h"
+#ifndef NULL
+# define NULL 0
+#endif
+
+struct dns_query *
+dns_submit_a4dnsbl(struct dns_ctx *ctx,
+                   const struct in_addr *addr, const char *dnsbl,
+                   dns_query_a4_fn *cbck, void *data) {
+  dnsc_t dn[DNS_MAXDN];
+  if (dns_a4ptodn(addr, dnsbl, dn, sizeof(dn)) <= 0) {
+    dns_setstatus(ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_A, DNS_NOSRCH,
+                  dns_parse_a4, (dns_query_fn*)cbck, data);
+}
+
+struct dns_query *
+dns_submit_a4dnsbl_txt(struct dns_ctx *ctx,
+                       const struct in_addr *addr, const char *dnsbl,
+                       dns_query_txt_fn *cbck, void *data) {
+  dnsc_t dn[DNS_MAXDN];
+  if (dns_a4ptodn(addr, dnsbl, dn, sizeof(dn)) <= 0) {
+    dns_setstatus(ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_TXT, DNS_NOSRCH,
+                  dns_parse_txt, (dns_query_fn*)cbck, data);
+}
+
+struct dns_rr_a4 *
+dns_resolve_a4dnsbl(struct dns_ctx *ctx,
+                    const struct in_addr *addr, const char *dnsbl) {
+  return (struct dns_rr_a4 *)
+    dns_resolve(ctx, dns_submit_a4dnsbl(ctx, addr, dnsbl, 0, 0));
+}
+
+struct dns_rr_txt *
+dns_resolve_a4dnsbl_txt(struct dns_ctx *ctx,
+                        const struct in_addr *addr, const char *dnsbl) {
+  return (struct dns_rr_txt *)
+    dns_resolve(ctx, dns_submit_a4dnsbl_txt(ctx, addr, dnsbl, 0, 0));
+}
+
+
+struct dns_query *
+dns_submit_a6dnsbl(struct dns_ctx *ctx,
+                   const struct in6_addr *addr, const char *dnsbl,
+                   dns_query_a4_fn *cbck, void *data) {
+  dnsc_t dn[DNS_MAXDN];
+  if (dns_a6ptodn(addr, dnsbl, dn, sizeof(dn)) <= 0) {
+    dns_setstatus(ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_A, DNS_NOSRCH,
+                  dns_parse_a4, (dns_query_fn*)cbck, data);
+}
+
+struct dns_query *
+dns_submit_a6dnsbl_txt(struct dns_ctx *ctx,
+                       const struct in6_addr *addr, const char *dnsbl,
+                       dns_query_txt_fn *cbck, void *data) {
+  dnsc_t dn[DNS_MAXDN];
+  if (dns_a6ptodn(addr, dnsbl, dn, sizeof(dn)) <= 0) {
+    dns_setstatus(ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_TXT, DNS_NOSRCH,
+                  dns_parse_txt, (dns_query_fn*)cbck, data);
+}
+
+struct dns_rr_a4 *
+dns_resolve_a6dnsbl(struct dns_ctx *ctx,
+                    const struct in6_addr *addr, const char *dnsbl) {
+  return (struct dns_rr_a4 *)
+    dns_resolve(ctx, dns_submit_a6dnsbl(ctx, addr, dnsbl, 0, 0));
+}
+
+struct dns_rr_txt *
+dns_resolve_a6dnsbl_txt(struct dns_ctx *ctx,
+                        const struct in6_addr *addr, const char *dnsbl) {
+  return (struct dns_rr_txt *)
+    dns_resolve(ctx, dns_submit_a6dnsbl_txt(ctx, addr, dnsbl, 0, 0));
+}
+
+static int
+dns_rhsbltodn(const char *name, const char *rhsbl, dnsc_t dn[DNS_MAXDN])
+{
+  int l = dns_sptodn(name, dn, DNS_MAXDN);
+  if (l <= 0) return 0;
+  l = dns_sptodn(rhsbl, dn+l-1, DNS_MAXDN-l+1);
+  if (l <= 0) return 0;
+  return 1;
+}
+
+struct dns_query *
+dns_submit_rhsbl(struct dns_ctx *ctx, const char *name, const char *rhsbl,
+                 dns_query_a4_fn *cbck, void *data) {
+  dnsc_t dn[DNS_MAXDN];
+  if (!dns_rhsbltodn(name, rhsbl, dn)) {
+    dns_setstatus(ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_A, DNS_NOSRCH,
+                  dns_parse_a4, (dns_query_fn*)cbck, data);
+}
+struct dns_query *
+dns_submit_rhsbl_txt(struct dns_ctx *ctx, const char *name, const char *rhsbl,
+                     dns_query_txt_fn *cbck, void *data) {
+  dnsc_t dn[DNS_MAXDN];
+  if (!dns_rhsbltodn(name, rhsbl, dn)) {
+    dns_setstatus(ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_TXT, DNS_NOSRCH,
+                  dns_parse_txt, (dns_query_fn*)cbck, data);
+}
+
+struct dns_rr_a4 *
+dns_resolve_rhsbl(struct dns_ctx *ctx, const char *name, const char *rhsbl) {
+  return (struct dns_rr_a4*)
+    dns_resolve(ctx, dns_submit_rhsbl(ctx, name, rhsbl, 0, 0));
+}
+
+struct dns_rr_txt *
+dns_resolve_rhsbl_txt(struct dns_ctx *ctx, const char *name, const char *rhsbl)
+{
+  return (struct dns_rr_txt*)
+    dns_resolve(ctx, dns_submit_rhsbl_txt(ctx, name, rhsbl, 0, 0));
+}

3rdparty/udns/udns_codes.c

@@ -0,0 +1,199 @@
+/* Automatically generated. */
+#include "udns.h"
+
+const struct dns_nameval dns_typetab[] = {
+ {DNS_T_INVALID,"INVALID"},
+ {DNS_T_A,"A"},
+ {DNS_T_NS,"NS"},
+ {DNS_T_MD,"MD"},
+ {DNS_T_MF,"MF"},
+ {DNS_T_CNAME,"CNAME"},
+ {DNS_T_SOA,"SOA"},
+ {DNS_T_MB,"MB"},
+ {DNS_T_MG,"MG"},
+ {DNS_T_MR,"MR"},
+ {DNS_T_NULL,"NULL"},
+ {DNS_T_WKS,"WKS"},
+ {DNS_T_PTR,"PTR"},
+ {DNS_T_HINFO,"HINFO"},
+ {DNS_T_MINFO,"MINFO"},
+ {DNS_T_MX,"MX"},
+ {DNS_T_TXT,"TXT"},
+ {DNS_T_RP,"RP"},
+ {DNS_T_AFSDB,"AFSDB"},
+ {DNS_T_X25,"X25"},
+ {DNS_T_ISDN,"ISDN"},
+ {DNS_T_RT,"RT"},
+ {DNS_T_NSAP,"NSAP"},
+ {DNS_T_NSAP_PTR,"NSAP_PTR"},
+ {DNS_T_SIG,"SIG"},
+ {DNS_T_KEY,"KEY"},
+ {DNS_T_PX,"PX"},
+ {DNS_T_GPOS,"GPOS"},
+ {DNS_T_AAAA,"AAAA"},
+ {DNS_T_LOC,"LOC"},
+ {DNS_T_NXT,"NXT"},
+ {DNS_T_EID,"EID"},
+ {DNS_T_NIMLOC,"NIMLOC"},
+ {DNS_T_SRV,"SRV"},
+ {DNS_T_ATMA,"ATMA"},
+ {DNS_T_NAPTR,"NAPTR"},
+ {DNS_T_KX,"KX"},
+ {DNS_T_CERT,"CERT"},
+ {DNS_T_A6,"A6"},
+ {DNS_T_DNAME,"DNAME"},
+ {DNS_T_SINK,"SINK"},
+ {DNS_T_OPT,"OPT"},
+ {DNS_T_DS,"DS"},
+ {DNS_T_SSHFP,"SSHFP"},
+ {DNS_T_IPSECKEY,"IPSECKEY"},
+ {DNS_T_RRSIG,"RRSIG"},
+ {DNS_T_NSEC,"NSEC"},
+ {DNS_T_DNSKEY,"DNSKEY"},
+ {DNS_T_DHCID,"DHCID"},
+ {DNS_T_NSEC3,"NSEC3"},
+ {DNS_T_NSEC3PARAMS,"NSEC3PARAMS"},
+ {DNS_T_TALINK,"TALINK"},
+ {DNS_T_SPF,"SPF"},
+ {DNS_T_UINFO,"UINFO"},
+ {DNS_T_UID,"UID"},
+ {DNS_T_GID,"GID"},
+ {DNS_T_UNSPEC,"UNSPEC"},
+ {DNS_T_TSIG,"TSIG"},
+ {DNS_T_IXFR,"IXFR"},
+ {DNS_T_AXFR,"AXFR"},
+ {DNS_T_MAILB,"MAILB"},
+ {DNS_T_MAILA,"MAILA"},
+ {DNS_T_ANY,"ANY"},
+ {DNS_T_ZXFR,"ZXFR"},
+ {DNS_T_DLV,"DLV"},
+ {DNS_T_MAX,"MAX"},
+ {0,0}};
+const char *dns_typename(enum dns_type code) {
+ static char nm[20];
+ switch(code) {
+ case DNS_T_INVALID: return dns_typetab[0].name;
+ case DNS_T_A: return dns_typetab[1].name;
+ case DNS_T_NS: return dns_typetab[2].name;
+ case DNS_T_MD: return dns_typetab[3].name;
+ case DNS_T_MF: return dns_typetab[4].name;
+ case DNS_T_CNAME: return dns_typetab[5].name;
+ case DNS_T_SOA: return dns_typetab[6].name;
+ case DNS_T_MB: return dns_typetab[7].name;
+ case DNS_T_MG: return dns_typetab[8].name;
+ case DNS_T_MR: return dns_typetab[9].name;
+ case DNS_T_NULL: return dns_typetab[10].name;
+ case DNS_T_WKS: return dns_typetab[11].name;
+ case DNS_T_PTR: return dns_typetab[12].name;
+ case DNS_T_HINFO: return dns_typetab[13].name;
+ case DNS_T_MINFO: return dns_typetab[14].name;
+ case DNS_T_MX: return dns_typetab[15].name;
+ case DNS_T_TXT: return dns_typetab[16].name;
+ case DNS_T_RP: return dns_typetab[17].name;
+ case DNS_T_AFSDB: return dns_typetab[18].name;
+ case DNS_T_X25: return dns_typetab[19].name;
+ case DNS_T_ISDN: return dns_typetab[20].name;
+ case DNS_T_RT: return dns_typetab[21].name;
+ case DNS_T_NSAP: return dns_typetab[22].name;
+ case DNS_T_NSAP_PTR: return dns_typetab[23].name;
+ case DNS_T_SIG: return dns_typetab[24].name;
+ case DNS_T_KEY: return dns_typetab[25].name;
+ case DNS_T_PX: return dns_typetab[26].name;
+ case DNS_T_GPOS: return dns_typetab[27].name;
+ case DNS_T_AAAA: return dns_typetab[28].name;
+ case DNS_T_LOC: return dns_typetab[29].name;
+ case DNS_T_NXT: return dns_typetab[30].name;
+ case DNS_T_EID: return dns_typetab[31].name;
+ case DNS_T_NIMLOC: return dns_typetab[32].name;
+ case DNS_T_SRV: return dns_typetab[33].name;
+ case DNS_T_ATMA: return dns_typetab[34].name;
+ case DNS_T_NAPTR: return dns_typetab[35].name;
+ case DNS_T_KX: return dns_typetab[36].name;
+ case DNS_T_CERT: return dns_typetab[37].name;
+ case DNS_T_A6: return dns_typetab[38].name;
+ case DNS_T_DNAME: return dns_typetab[39].name;
+ case DNS_T_SINK: return dns_typetab[40].name;
+ case DNS_T_OPT: return dns_typetab[41].name;
+ case DNS_T_DS: return dns_typetab[42].name;
+ case DNS_T_SSHFP: return dns_typetab[43].name;
+ case DNS_T_IPSECKEY: return dns_typetab[44].name;
+ case DNS_T_RRSIG: return dns_typetab[45].name;
+ case DNS_T_NSEC: return dns_typetab[46].name;
+ case DNS_T_DNSKEY: return dns_typetab[47].name;
+ case DNS_T_DHCID: return dns_typetab[48].name;
+ case DNS_T_NSEC3: return dns_typetab[49].name;
+ case DNS_T_NSEC3PARAMS: return dns_typetab[50].name;
+ case DNS_T_TALINK: return dns_typetab[51].name;
+ case DNS_T_SPF: return dns_typetab[52].name;
+ case DNS_T_UINFO: return dns_typetab[53].name;
+ case DNS_T_UID: return dns_typetab[54].name;
+ case DNS_T_GID: return dns_typetab[55].name;
+ case DNS_T_UNSPEC: return dns_typetab[56].name;
+ case DNS_T_TSIG: return dns_typetab[57].name;
+ case DNS_T_IXFR: return dns_typetab[58].name;
+ case DNS_T_AXFR: return dns_typetab[59].name;
+ case DNS_T_MAILB: return dns_typetab[60].name;
+ case DNS_T_MAILA: return dns_typetab[61].name;
+ case DNS_T_ANY: return dns_typetab[62].name;
+ case DNS_T_ZXFR: return dns_typetab[63].name;
+ case DNS_T_DLV: return dns_typetab[64].name;
+ case DNS_T_MAX: return dns_typetab[65].name;
+ }
+ return _dns_format_code(nm,"type",code);
+}
+
+const struct dns_nameval dns_classtab[] = {
+ {DNS_C_INVALID,"INVALID"},
+ {DNS_C_IN,"IN"},
+ {DNS_C_CH,"CH"},
+ {DNS_C_HS,"HS"},
+ {DNS_C_ANY,"ANY"},
+ {0,0}};
+const char *dns_classname(enum dns_class code) {
+ static char nm[20];
+ switch(code) {
+ case DNS_C_INVALID: return dns_classtab[0].name;
+ case DNS_C_IN: return dns_classtab[1].name;
+ case DNS_C_CH: return dns_classtab[2].name;
+ case DNS_C_HS: return dns_classtab[3].name;
+ case DNS_C_ANY: return dns_classtab[4].name;
+ }
+ return _dns_format_code(nm,"class",code);
+}
+
+const struct dns_nameval dns_rcodetab[] = {
+ {DNS_R_NOERROR,"NOERROR"},
+ {DNS_R_FORMERR,"FORMERR"},
+ {DNS_R_SERVFAIL,"SERVFAIL"},
+ {DNS_R_NXDOMAIN,"NXDOMAIN"},
+ {DNS_R_NOTIMPL,"NOTIMPL"},
+ {DNS_R_REFUSED,"REFUSED"},
+ {DNS_R_YXDOMAIN,"YXDOMAIN"},
+ {DNS_R_YXRRSET,"YXRRSET"},
+ {DNS_R_NXRRSET,"NXRRSET"},
+ {DNS_R_NOTAUTH,"NOTAUTH"},
+ {DNS_R_NOTZONE,"NOTZONE"},
+ {DNS_R_BADSIG,"BADSIG"},
+ {DNS_R_BADKEY,"BADKEY"},
+ {DNS_R_BADTIME,"BADTIME"},
+ {0,0}};
+const char *dns_rcodename(enum dns_rcode code) {
+ static char nm[20];
+ switch(code) {
+ case DNS_R_NOERROR: return dns_rcodetab[0].name;
+ case DNS_R_FORMERR: return dns_rcodetab[1].name;
+ case DNS_R_SERVFAIL: return dns_rcodetab[2].name;
+ case DNS_R_NXDOMAIN: return dns_rcodetab[3].name;
+ case DNS_R_NOTIMPL: return dns_rcodetab[4].name;
+ case DNS_R_REFUSED: return dns_rcodetab[5].name;
+ case DNS_R_YXDOMAIN: return dns_rcodetab[6].name;
+ case DNS_R_YXRRSET: return dns_rcodetab[7].name;
+ case DNS_R_NXRRSET: return dns_rcodetab[8].name;
+ case DNS_R_NOTAUTH: return dns_rcodetab[9].name;
+ case DNS_R_NOTZONE: return dns_rcodetab[10].name;
+ case DNS_R_BADSIG: return dns_rcodetab[11].name;
+ case DNS_R_BADKEY: return dns_rcodetab[12].name;
+ case DNS_R_BADTIME: return dns_rcodetab[13].name;
+ }
+ return _dns_format_code(nm,"rcode",code);
+}

3rdparty/udns/udns_dn.c

@@ -0,0 +1,379 @@
+/* udns_dn.c
+   domain names manipulation routines
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include <string.h>
+#include "udns.h"
+
+unsigned dns_dnlen(dnscc_t *dn) {
+  register dnscc_t *d = dn;
+  while(*d)
+    d += 1 + *d;
+  return (unsigned)(d - dn) + 1;
+}
+
+unsigned dns_dnlabels(register dnscc_t *dn) {
+  register unsigned l = 0;
+  while(*dn)
+    ++l, dn += 1 + *dn;
+  return l;
+}
+
+unsigned dns_dnequal(register dnscc_t *dn1, register dnscc_t *dn2) {
+  register unsigned c;
+  dnscc_t *dn = dn1;
+  for(;;) {
+    if ((c = *dn1++) != *dn2++)
+      return 0;
+    if (!c)
+      return (unsigned)(dn1 - dn);
+    while(c--) {
+      if (DNS_DNLC(*dn1) != DNS_DNLC(*dn2))
+        return 0;
+      ++dn1; ++dn2;
+    }
+  }
+}
+
+unsigned
+dns_dntodn(dnscc_t *sdn, dnsc_t *ddn, unsigned ddnsiz) {
+  unsigned sdnlen = dns_dnlen(sdn);
+  if (ddnsiz < sdnlen)
+    return 0;
+  memcpy(ddn, sdn, sdnlen);
+  return sdnlen;
+}
+
+int
+dns_ptodn(const char *name, unsigned namelen,
+          dnsc_t *dn, unsigned dnsiz, int *isabs)
+{
+  dnsc_t *dp;		/* current position in dn (len byte first) */
+  dnsc_t *const de	/* end of dn: last byte that can be filled up */
+      = dn + (dnsiz >= DNS_MAXDN ? DNS_MAXDN : dnsiz) - 1;
+  dnscc_t *np = (dnscc_t *)name;
+  dnscc_t *ne = np + (namelen ? namelen : strlen((char*)np));
+  dnsc_t *llab;		/* start of last label (llab[-1] will be length) */
+  unsigned c;		/* next input character, or length of last label */
+
+  if (!dnsiz)
+    return 0;
+  dp = llab = dn + 1;
+
+  while(np < ne) {
+
+    if (*np == '.') {	/* label delimiter */
+      c = dp - llab;		/* length of the label */
+      if (!c) {			/* empty label */
+        if (np == (dnscc_t *)name && np + 1 == ne) {
+          /* special case for root dn, aka `.' */
+          ++np;
+          break;
+        }
+        return -1;		/* zero label */
+      }
+      if (c > DNS_MAXLABEL)
+        return -1;		/* label too long */
+      llab[-1] = (dnsc_t)c;	/* update len of last label */
+      llab = ++dp; /* start new label, llab[-1] will be len of it */
+      ++np;
+      continue;
+    }
+
+    /* check whenever we may put out one more byte */
+    if (dp >= de) /* too long? */
+      return dnsiz >= DNS_MAXDN ? -1 : 0;
+    if (*np != '\\') { /* non-escape, simple case */
+      *dp++ = *np++;
+      continue;
+    }
+    /* handle \-style escape */
+    /* note that traditionally, domain names (gethostbyname etc)
+     * used decimal \dd notation, not octal \ooo (RFC1035), so
+     * we're following this tradition here.
+     */
+    if (++np == ne)
+      return -1;			/* bad escape */
+    else if (*np >= '0' && *np <= '9') { /* decimal number */
+      /* we allow not only exactly 3 digits as per RFC1035,
+       * but also 2 or 1, for better usability. */
+      c = *np++ - '0';
+      if (np < ne && *np >= '0' && *np <= '9') { /* 2digits */
+        c = c * 10 + *np++ - '0';
+        if (np < ne && *np >= '0' && *np <= '9') {
+          c = c * 10 + *np++ - '0';
+          if (c > 255)
+            return -1;			/* bad escape */
+        }
+      }
+    }
+    else
+      c = *np++;
+    *dp++ = (dnsc_t)c;	/* place next out byte */
+  }
+
+  if ((c = dp - llab) > DNS_MAXLABEL)
+    return -1;				/* label too long */
+  if ((llab[-1] = (dnsc_t)c) != 0) {
+    *dp++ = 0;
+    if (isabs)
+      *isabs = 0;
+  }
+  else if (isabs)
+    *isabs = 1;
+
+  return dp - dn;
+}
+
+dnscc_t dns_inaddr_arpa_dn[14] = "\07in-addr\04arpa";
+
+dnsc_t *
+dns_a4todn_(const struct in_addr *addr, dnsc_t *dn, dnsc_t *dne) {
+  const unsigned char *s = ((const unsigned char *)addr) + 4;
+  while(s > (const unsigned char *)addr) {
+    unsigned n = *--s;
+    dnsc_t *p = dn + 1;
+    if (n > 99) {
+      if (p + 2 > dne) return 0;
+      *p++ = n / 100 + '0';
+      *p++ = (n % 100 / 10) + '0';
+      *p = n % 10 + '0';
+    }
+    else if (n > 9) {
+      if (p + 1 > dne) return 0;
+      *p++ = n / 10 + '0';
+      *p = n % 10 + '0';
+    }
+    else {
+      if (p > dne) return 0;
+      *p = n + '0';
+    }
+    *dn = p - dn;
+    dn = p + 1;
+  }
+  return dn;
+}
+
+int dns_a4todn(const struct in_addr *addr, dnscc_t *tdn,
+               dnsc_t *dn, unsigned dnsiz) {
+  dnsc_t *dne = dn + (dnsiz > DNS_MAXDN ? DNS_MAXDN : dnsiz);
+  dnsc_t *p;
+  unsigned l;
+  p = dns_a4todn_(addr, dn, dne);
+  if (!p) return 0;
+  if (!tdn)
+    tdn = dns_inaddr_arpa_dn;
+  l = dns_dnlen(tdn);
+  if (p + l > dne) return dnsiz >= DNS_MAXDN ? -1 : 0;
+  memcpy(p, tdn, l);
+  return (p + l) - dn;
+}
+
+int dns_a4ptodn(const struct in_addr *addr, const char *tname,
+                dnsc_t *dn, unsigned dnsiz) {
+  dnsc_t *p;
+  int r;
+  if (!tname)
+    return dns_a4todn(addr, NULL, dn, dnsiz);
+  p = dns_a4todn_(addr, dn, dn + dnsiz);
+  if (!p) return 0;
+  r = dns_sptodn(tname, p, dnsiz - (p - dn));
+  return r != 0 ? r : dnsiz >= DNS_MAXDN ? -1 : 0;
+}
+
+dnscc_t dns_ip6_arpa_dn[10] = "\03ip6\04arpa";
+
+dnsc_t *
+dns_a6todn_(const struct in6_addr *addr, dnsc_t *dn, dnsc_t *dne) {
+  const unsigned char *s = ((const unsigned char *)addr) + 16;
+  if (dn + 64 > dne) return 0;
+  while(s > (const unsigned char *)addr) {
+    unsigned n = *--s & 0x0f;
+    *dn++ = 1;
+    *dn++ = n > 9 ? n + 'a' - 10 : n + '0';
+    *dn++ = 1;
+    n = *s >> 4;
+    *dn++ = n > 9 ? n + 'a' - 10 : n + '0';
+  }
+  return dn;
+}
+
+int dns_a6todn(const struct in6_addr *addr, dnscc_t *tdn,
+               dnsc_t *dn, unsigned dnsiz) {
+  dnsc_t *dne = dn + (dnsiz > DNS_MAXDN ? DNS_MAXDN : dnsiz);
+  dnsc_t *p;
+  unsigned l;
+  p = dns_a6todn_(addr, dn, dne);
+  if (!p) return 0;
+  if (!tdn)
+    tdn = dns_ip6_arpa_dn;
+  l = dns_dnlen(tdn);
+  if (p + l > dne) return dnsiz >= DNS_MAXDN ? -1 : 0;
+  memcpy(p, tdn, l);
+  return (p + l) - dn;
+}
+
+int dns_a6ptodn(const struct in6_addr *addr, const char *tname,
+                dnsc_t *dn, unsigned dnsiz) {
+  dnsc_t *p;
+  int r;
+  if (!tname)
+    return dns_a6todn(addr, NULL, dn, dnsiz);
+  p = dns_a6todn_(addr, dn, dn + dnsiz);
+  if (!p) return 0;
+  r = dns_sptodn(tname, p, dnsiz - (p - dn));
+  return r != 0 ? r : dnsiz >= DNS_MAXDN ? -1 : 0;
+}
+
+/* return size of buffer required to convert the dn into asciiz string.
+ * Keep in sync with dns_dntop() below.
+ */
+unsigned dns_dntop_size(dnscc_t *dn) {
+  unsigned size = 0;		/* the size reqd */
+  dnscc_t *le;			/* label end */
+
+  while(*dn) {
+    /* *dn is the length of the next label, non-zero */
+    if (size)
+      ++size;		/* for the dot */
+    le = dn + *dn + 1;
+    ++dn;
+    do {
+      switch(*dn) {
+      case '.':
+      case '\\':
+      /* Special modifiers in zone files. */
+      case '"':
+      case ';':
+      case '@':
+      case '$':
+        size += 2;
+        break;
+      default:
+        if (*dn <= 0x20 || *dn >= 0x7f)
+          /* \ddd decimal notation */
+          size += 4;
+        else
+          size += 1;
+      }
+    } while(++dn < le);
+  }
+  size += 1;	/* zero byte at the end - string terminator */
+  return size > DNS_MAXNAME ? 0 : size;
+}
+
+/* Convert the dn into asciiz string.
+ * Keep in sync with dns_dntop_size() above.
+ */
+int dns_dntop(dnscc_t *dn, char *name, unsigned namesiz) {
+  char *np = name;			/* current name ptr */
+  char *const ne = name + namesiz;	/* end of name */
+  dnscc_t *le;		/* label end */
+
+  while(*dn) {
+    /* *dn is the length of the next label, non-zero */
+    if (np != name) {
+      if (np >= ne) goto toolong;
+      *np++ = '.';
+    }
+    le = dn + *dn + 1;
+    ++dn;
+    do {
+      switch(*dn) {
+      case '.':
+      case '\\':
+      /* Special modifiers in zone files. */
+      case '"':
+      case ';':
+      case '@':
+      case '$':
+        if (np + 2 > ne) goto toolong;
+        *np++ = '\\';
+        *np++ = *dn;
+        break;
+      default:
+        if (*dn <= 0x20 || *dn >= 0x7f) {
+          /* \ddd decimal notation */
+          if (np + 4 >= ne) goto toolong;
+          *np++ = '\\';
+          *np++ = '0' + (*dn / 100);
+          *np++ = '0' + ((*dn % 100) / 10);
+          *np++ = '0' + (*dn % 10);
+        }
+        else {
+          if (np >= ne) goto toolong;
+          *np++ = *dn;
+        }
+      }
+    } while(++dn < le);
+  }
+  if (np >= ne) goto toolong;
+  *np++ = '\0';
+  return np - name;
+toolong:
+  return namesiz >= DNS_MAXNAME ? -1 : 0;
+}
+
+#ifdef TEST
+#include <stdio.h>
+#include <stdlib.h>
+
+int main(int argc, char **argv) {
+  int i;
+  int sz;
+  dnsc_t dn[DNS_MAXDN+10];
+  dnsc_t *dl, *dp;
+  int isabs;
+
+  sz = (argc > 1) ? atoi(argv[1]) : 0;
+
+  for(i = 2; i < argc; ++i) {
+    int r = dns_ptodn(argv[i], 0, dn, sz, &isabs);
+    printf("%s: ", argv[i]);
+    if (r < 0) printf("error\n");
+    else if (!r) printf("buffer too small\n");
+    else {
+      printf("len=%d dnlen=%d size=%d name:",
+             r, dns_dnlen(dn), dns_dntop_size(dn));
+      dl = dn;
+      while(*dl) {
+        printf(" %d=", *dl);
+        dp = dl + 1;
+        dl = dp + *dl;
+        while(dp < dl) {
+          if (*dp <= ' ' || *dp >= 0x7f)
+            printf("\\%03d", *dp);
+          else if (*dp == '.' || *dp == '\\')
+            printf("\\%c", *dp);
+          else
+            putchar(*dp);
+          ++dp;
+        }
+      }
+      if (isabs) putchar('.');
+      putchar('\n');
+    }
+  }
+  return 0;
+}
+
+#endif /* TEST */

3rdparty/udns/udns_dntosp.c

@@ -0,0 +1,30 @@
+/* udns_dntosp.c
+   dns_dntosp() = convert DN to asciiz string using static buffer
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include "udns.h"
+
+static char name[DNS_MAXNAME];
+
+const char *dns_dntosp(dnscc_t *dn) {
+  return dns_dntop(dn, name, sizeof(name)) > 0 ? name : 0;
+}

3rdparty/udns/udns_init.c

@@ -0,0 +1,231 @@
+/* udns_init.c
+   resolver initialisation stuff
+
+   Copyright (C) 2006  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+#ifdef WINDOWS
+# include <winsock2.h>          /* includes <windows.h> */
+# include <iphlpapi.h>		/* for dns server addresses etc */
+#else
+# include <sys/types.h>
+# include <unistd.h>
+# include <fcntl.h>
+#endif	/* !WINDOWS */
+
+#include <stdlib.h>
+#include <string.h>
+#include "udns.h"
+
+#define ISSPACE(x) (x == ' ' || x == '\t' || x == '\r' || x == '\n')
+
+static const char space[] = " \t\r\n";
+
+static void dns_set_serv_internal(struct dns_ctx *ctx, char *serv) {
+  dns_add_serv(ctx, NULL);
+  for(serv = strtok(serv, space); serv; serv = strtok(NULL, space))
+    dns_add_serv(ctx, serv);
+}
+
+static void dns_set_srch_internal(struct dns_ctx *ctx, char *srch) {
+  dns_add_srch(ctx, NULL);
+  for(srch = strtok(srch, space); srch; srch = strtok(NULL, space))
+    dns_add_srch(ctx, srch);
+}
+
+#ifdef WINDOWS
+
+#ifndef NO_IPHLPAPI
+/* Apparently, some systems does not have proper headers for IPHLPAIP to work.
+ * The best is to upgrade headers, but here's another, ugly workaround for
+ * this: compile with -DNO_IPHLPAPI.
+ */
+
+typedef DWORD (WINAPI *GetAdaptersAddressesFunc)(
+  ULONG Family, DWORD Flags, PVOID Reserved,
+  PIP_ADAPTER_ADDRESSES pAdapterAddresses,
+  PULONG pOutBufLen);
+
+static int dns_initns_iphlpapi(struct dns_ctx *ctx) {
+  HANDLE h_iphlpapi;
+  GetAdaptersAddressesFunc pfnGetAdAddrs;
+  PIP_ADAPTER_ADDRESSES pAddr, pAddrBuf;
+  PIP_ADAPTER_DNS_SERVER_ADDRESS pDnsAddr;
+  ULONG ulOutBufLen;
+  DWORD dwRetVal;
+  int ret = -1;
+
+  h_iphlpapi = LoadLibrary("iphlpapi.dll");
+  if (!h_iphlpapi)
+    return -1;
+  pfnGetAdAddrs = (GetAdaptersAddressesFunc)
+    GetProcAddress(h_iphlpapi, "GetAdaptersAddresses");
+  if (!pfnGetAdAddrs) goto freelib;
+  ulOutBufLen = 0;
+  dwRetVal = pfnGetAdAddrs(AF_UNSPEC, 0, NULL, NULL, &ulOutBufLen);
+  if (dwRetVal != ERROR_BUFFER_OVERFLOW) goto freelib;
+  pAddrBuf = malloc(ulOutBufLen);
+  if (!pAddrBuf) goto freelib;
+  dwRetVal = pfnGetAdAddrs(AF_UNSPEC, 0, NULL, pAddrBuf, &ulOutBufLen);
+  if (dwRetVal != ERROR_SUCCESS) goto freemem;
+  for (pAddr = pAddrBuf; pAddr; pAddr = pAddr->Next)
+    for (pDnsAddr = pAddr->FirstDnsServerAddress;
+	 pDnsAddr;
+	 pDnsAddr = pDnsAddr->Next)
+      dns_add_serv_s(ctx, pDnsAddr->Address.lpSockaddr);
+  ret = 0;
+freemem:
+  free(pAddrBuf);
+freelib:
+  FreeLibrary(h_iphlpapi);
+  return ret;
+}
+
+#else /* NO_IPHLPAPI */
+
+#define dns_initns_iphlpapi(ctx) (-1)
+
+#endif /* NO_IPHLPAPI */
+
+static int dns_initns_registry(struct dns_ctx *ctx) {
+  LONG res;
+  HKEY hk;
+  DWORD type = REG_EXPAND_SZ | REG_SZ;
+  DWORD len;
+  char valBuf[1024];
+
+#define REGKEY_WINNT "SYSTEM\\CurrentControlSet\\Services\\Tcpip\\Parameters"
+#define REGKEY_WIN9x "SYSTEM\\CurrentControlSet\\Services\\VxD\\MSTCP"
+  res = RegOpenKeyEx(HKEY_LOCAL_MACHINE, REGKEY_WINNT, 0, KEY_QUERY_VALUE, &hk);
+  if (res != ERROR_SUCCESS)
+    res = RegOpenKeyEx(HKEY_LOCAL_MACHINE, REGKEY_WIN9x,
+                       0, KEY_QUERY_VALUE, &hk);
+  if (res != ERROR_SUCCESS)
+    return -1;
+  len = sizeof(valBuf) - 1;
+  res = RegQueryValueEx(hk, "NameServer", NULL, &type, (BYTE*)valBuf, &len);
+  if (res != ERROR_SUCCESS || !len || !valBuf[0]) {
+    len = sizeof(valBuf) - 1;
+    res = RegQueryValueEx(hk, "DhcpNameServer", NULL, &type,
+                          (BYTE*)valBuf, &len);
+  }
+  RegCloseKey(hk);
+  if (res != ERROR_SUCCESS || !len || !valBuf[0])
+    return -1;
+  valBuf[len] = '\0';
+  /* nameservers are stored as a whitespace-seperate list:
+   * "192.168.1.1 123.21.32.12" */
+  dns_set_serv_internal(ctx, valBuf);
+  return 0;
+}
+
+#else /* !WINDOWS */
+
+static int dns_init_resolvconf(struct dns_ctx *ctx) {
+  char *v;
+  char buf[2049];	/* this buffer is used to hold /etc/resolv.conf */
+  int has_srch = 0;
+
+  /* read resolv.conf... */
+  { int fd = open("/etc/resolv.conf", O_RDONLY);
+    if (fd >= 0) {
+      int l = read(fd, buf, sizeof(buf) - 1);
+      close(fd);
+      buf[l < 0 ? 0 : l] = '\0';
+    }
+    else
+      buf[0] = '\0';
+  }
+  if (buf[0]) {	/* ...and parse it */
+    char *line, *nextline;
+    line = buf;
+    do {
+      nextline = strchr(line, '\n');
+      if (nextline) *nextline++ = '\0';
+      v = line;
+      while(*v && !ISSPACE(*v)) ++v;
+      if (!*v) continue;
+      *v++ = '\0';
+      while(ISSPACE(*v)) ++v;
+      if (!*v) continue;
+      if (strcmp(line, "domain") == 0) {
+        dns_set_srch_internal(ctx, strtok(v, space));
+	has_srch = 1;
+      }
+      else if (strcmp(line, "search") == 0) {
+        dns_set_srch_internal(ctx, v);
+	has_srch = 1;
+      }
+      else if (strcmp(line, "nameserver") == 0)
+        dns_add_serv(ctx, strtok(v, space));
+      else if (strcmp(line, "options") == 0)
+        dns_set_opts(ctx, v);
+    } while((line = nextline) != NULL);
+  }
+
+  buf[sizeof(buf)-1] = '\0';
+
+  /* get list of nameservers from env. vars. */
+  if ((v = getenv("NSCACHEIP")) != NULL ||
+      (v = getenv("NAMESERVERS")) != NULL) {
+    strncpy(buf, v, sizeof(buf) - 1);
+    dns_set_serv_internal(ctx, buf);
+  }
+  /* if $LOCALDOMAIN is set, use it for search list */
+  if ((v = getenv("LOCALDOMAIN")) != NULL) {
+    strncpy(buf, v, sizeof(buf) - 1);
+    dns_set_srch_internal(ctx, buf);
+    has_srch = 1;
+  }
+  if ((v = getenv("RES_OPTIONS")) != NULL)
+    dns_set_opts(ctx, v);
+
+  /* if still no search list, use local domain name */
+  if (has_srch &&
+      gethostname(buf, sizeof(buf) - 1) == 0 &&
+      (v = strchr(buf, '.')) != NULL &&
+      *++v != '\0')
+    dns_add_srch(ctx, v);
+
+  return 0;
+}
+
+#endif /* !WINDOWS */
+
+int dns_init(struct dns_ctx *ctx, int do_open) {
+  if (!ctx)
+    ctx = &dns_defctx;
+  dns_reset(ctx);
+
+#ifdef WINDOWS
+  if (dns_initns_iphlpapi(ctx) != 0)
+    dns_initns_registry(ctx);
+  /*XXX WINDOWS: probably good to get default domain and search list too...
+   * And options.  Something is in registry. */
+  /*XXX WINDOWS: maybe environment variables are also useful? */
+#else
+  dns_init_resolvconf(ctx);
+#endif
+
+  return do_open ? dns_open(ctx) : 0;
+}

3rdparty/udns/udns_jran.c

@@ -0,0 +1,52 @@
+/* udns_jran.c: small non-cryptographic random number generator
+ * taken from http://burtleburtle.net/bob/rand/smallprng.html
+ * by Bob Jenkins, Public domain.
+ */
+
+#include "udns.h"
+
+#define rot32(x,k) (((x) << (k)) | ((x) >> (32-(k))))
+#define rot64(x,k) (((x) << (k)) | ((x) >> (64-(k))))
+#define tr32(x) ((x)&0xffffffffu)
+
+unsigned udns_jranval(struct udns_jranctx *x) {
+  /* This routine can be made to work with either 32 or 64bit words -
+   * if JRAN_32_64 is defined when compiling the file.
+   * We use if() instead of #if since there's no good
+   * portable way to check sizeof() in preprocessor without
+   * introducing some ugly configure-time checks.
+   * Most compilers will optimize the wrong branches away anyway.
+   * By default it assumes 32bit integers
+   */
+#ifdef JRAN_32_64
+  if (sizeof(unsigned) == 4) {
+#endif
+    unsigned e = tr32(x->a - rot32(x->b, 27));
+    x->a = tr32(x->b ^ rot32(x->c, 17));
+    x->b = tr32(x->c + x->d);
+    x->c = tr32(x->d + e);
+    x->d = tr32(e + x->a);
+#ifdef JRAN_32_64
+  }
+  else if (sizeof(unsigned) == 8) { /* assuming it's 64bits */
+    unsigned e = x->a - rot64(x->b, 7);
+    x->a = x->b ^ rot64(x->c, 13);
+    x->b = x->c + rot64(x->d, 37);
+    x->c = x->d + e;
+    x->d = e + x->a;
+  }
+  else {
+    unsigned e = 0;
+    x->d = 1/e; /* bail */
+  }
+#endif
+  return x->d;
+}
+
+void udns_jraninit(struct udns_jranctx *x, unsigned seed) {
+  unsigned i;
+  x->a = 0xf1ea5eed;
+  x->b = x->c = x->d = seed;
+  for (i = 0; i < 20; ++i)
+     (void)udns_jranval(x);
+}

3rdparty/udns/udns_misc.c

@@ -0,0 +1,67 @@
+/* udns_misc.c
+   miscellaneous routines
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include "udns.h"
+
+int dns_findname(const struct dns_nameval *nv, const char *name) {
+  register const char *a, *b;
+  for(; nv->name; ++nv)
+    for(a = name, b = nv->name; ; ++a, ++b)
+      if (DNS_DNUC(*a) != *b) break;
+      else if (!*a) return nv->val;
+  return -1;
+}
+
+const char *_dns_format_code(char *buf, const char *prefix, int code) {
+  char *bp = buf;
+  unsigned c, n;
+  do *bp++ = DNS_DNUC(*prefix);
+  while(*++prefix);
+  *bp++ = '#';
+  if (code < 0) code = -code, *bp++ = '-';
+  n = 0; c = code;
+  do ++n;
+  while((c /= 10));
+  c = code;
+  bp[n--] = '\0';
+  do bp[n--] = c % 10 + '0';
+  while((c /= 10));
+  return buf;
+}
+
+const char *dns_strerror(int err) {
+  if (err >= 0) return "successeful completion";
+  switch(err) {
+  case DNS_E_TEMPFAIL:	return "temporary failure in name resolution";
+  case DNS_E_PROTOCOL:	return "protocol error";
+  case DNS_E_NXDOMAIN:	return "domain name does not exist";
+  case DNS_E_NODATA:	return "valid domain but no data of requested type";
+  case DNS_E_NOMEM:	return "out of memory";
+  case DNS_E_BADQUERY:	return "malformed query";
+  default:		return "unknown error";
+  }
+}
+
+const char *dns_version(void) {
+  return UDNS_VERSION;
+}

3rdparty/udns/udns_parse.c

@@ -0,0 +1,169 @@
+/* udns_parse.c
+   raw DNS packet parsing routines
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include <string.h>
+#include <assert.h>
+#include "udns.h"
+
+dnscc_t *dns_skipdn(dnscc_t *cur, dnscc_t *end) {
+  unsigned c;
+  for(;;) {
+    if (cur >= end)
+      return NULL;
+    c = *cur++;
+    if (!c)
+      return cur;
+    if (c & 192)		/* jump */
+      return cur + 1 >= end ? NULL : cur + 1;
+    cur += c;
+  }
+}
+
+int
+dns_getdn(dnscc_t *pkt, dnscc_t **cur, dnscc_t *end,
+          register dnsc_t *dn, unsigned dnsiz) {
+  unsigned c;
+  dnscc_t *pp = *cur;		/* current packet pointer */
+  dnsc_t *dp = dn;		/* current dn pointer */
+  dnsc_t *const de		/* end of the DN dest */
+       = dn + (dnsiz < DNS_MAXDN ? dnsiz : DNS_MAXDN);
+  dnscc_t *jump = NULL;		/* ptr after first jump if any */
+  unsigned loop = 100;		/* jump loop counter */
+
+  for(;;) {		/* loop by labels */
+    if (pp >= end)		/* reached end of packet? */
+      return -1;
+    c = *pp++;			/* length of the label */
+    if (!c) {			/* empty label: terminate */
+      if (dn >= de)		/* can't fit terminator */
+        goto noroom;
+      *dp++ = 0;
+      /* return next pos: either after the first jump or current */
+      *cur = jump ? jump : pp;
+      return dp - dn;
+    }
+    if (c & 192) {		/* jump */
+      if (pp >= end)		/* eop instead of jump pos */
+        return -1;
+      if (!jump) jump = pp + 1;	/* remember first jump */
+      else if (!--loop) return -1; /* too many jumps */
+      c = ((c & ~192) << 8) | *pp; /* new pos */
+      if (c < DNS_HSIZE)	/* don't allow jump into the header */
+        return -1;
+      pp = pkt + c;
+      continue;
+    }
+    if (c > DNS_MAXLABEL)	/* too long label? */
+      return -1;
+    if (pp + c > end)		/* label does not fit in packet? */
+      return -1;
+    if (dp + c + 1 > de)	/* if enouth room for the label */
+      goto noroom;
+    *dp++ = c;			/* label length */
+    memcpy(dp, pp, c);		/* and the label itself */
+    dp += c;
+    pp += c;			/* advance to the next label */
+  }
+noroom:
+  return dnsiz < DNS_MAXDN ? 0 : -1;
+}
+
+void dns_rewind(struct dns_parse *p, dnscc_t *qdn) {
+  p->dnsp_qdn = qdn;
+  p->dnsp_cur = p->dnsp_ans;
+  p->dnsp_rrl = dns_numan(p->dnsp_pkt);
+  p->dnsp_ttl = 0xffffffffu;
+  p->dnsp_nrr = 0;
+}
+
+void
+dns_initparse(struct dns_parse *p, dnscc_t *qdn,
+              dnscc_t *pkt, dnscc_t *cur, dnscc_t *end) {
+  p->dnsp_pkt = pkt;
+  p->dnsp_end = end;
+  p->dnsp_rrl = dns_numan(pkt);
+  p->dnsp_qdn = qdn;
+  assert(cur + 4 <= end);
+  if ((p->dnsp_qtyp = dns_get16(cur+0)) == DNS_T_ANY) p->dnsp_qtyp = 0;
+  if ((p->dnsp_qcls = dns_get16(cur+2)) == DNS_C_ANY) p->dnsp_qcls = 0;
+  p->dnsp_cur = p->dnsp_ans = cur + 4;
+  p->dnsp_ttl = 0xffffffffu;
+  p->dnsp_nrr = 0;
+}
+
+int dns_nextrr(struct dns_parse *p, struct dns_rr *rr) {
+  dnscc_t *cur = p->dnsp_cur;
+  while(p->dnsp_rrl > 0) {
+    --p->dnsp_rrl;
+    if (dns_getdn(p->dnsp_pkt, &cur, p->dnsp_end,
+                  rr->dnsrr_dn, sizeof(rr->dnsrr_dn)) <= 0)
+      return -1;
+    if (cur + 10 > p->dnsp_end)
+      return -1;
+    rr->dnsrr_typ = dns_get16(cur);
+    rr->dnsrr_cls = dns_get16(cur+2);
+    rr->dnsrr_ttl = dns_get32(cur+4);
+    rr->dnsrr_dsz = dns_get16(cur+8);
+    rr->dnsrr_dptr = cur = cur + 10;
+    rr->dnsrr_dend = cur = cur + rr->dnsrr_dsz;
+    if (cur > p->dnsp_end)
+      return -1;
+    if (p->dnsp_qdn && !dns_dnequal(p->dnsp_qdn, rr->dnsrr_dn))
+      continue;
+    if ((!p->dnsp_qcls || p->dnsp_qcls == rr->dnsrr_cls) &&
+        (!p->dnsp_qtyp || p->dnsp_qtyp == rr->dnsrr_typ)) {
+      p->dnsp_cur = cur;
+      ++p->dnsp_nrr;
+      if (p->dnsp_ttl > rr->dnsrr_ttl) p->dnsp_ttl = rr->dnsrr_ttl;
+      return 1;
+    }
+    if (p->dnsp_qdn && rr->dnsrr_typ == DNS_T_CNAME && !p->dnsp_nrr) {
+      if (dns_getdn(p->dnsp_pkt, &rr->dnsrr_dptr, p->dnsp_end,
+                    p->dnsp_dnbuf, sizeof(p->dnsp_dnbuf)) <= 0 ||
+          rr->dnsrr_dptr != rr->dnsrr_dend)
+        return -1;
+      p->dnsp_qdn = p->dnsp_dnbuf;
+      if (p->dnsp_ttl > rr->dnsrr_ttl) p->dnsp_ttl = rr->dnsrr_ttl;
+    }
+  }
+  p->dnsp_cur = cur;
+  return 0;
+}
+
+int dns_stdrr_size(const struct dns_parse *p) {
+  return
+    dns_dntop_size(p->dnsp_qdn) +
+    (p->dnsp_qdn == dns_payload(p->dnsp_pkt) ? 0 :
+     dns_dntop_size(dns_payload(p->dnsp_pkt)));
+}
+
+void *dns_stdrr_finish(struct dns_rr_null *ret, char *cp,
+                       const struct dns_parse *p) {
+  cp += dns_dntop(p->dnsp_qdn, (ret->dnsn_cname = cp), DNS_MAXNAME);
+  if (p->dnsp_qdn == dns_payload(p->dnsp_pkt))
+    ret->dnsn_qname = ret->dnsn_cname;
+  else
+    dns_dntop(dns_payload(p->dnsp_pkt), (ret->dnsn_qname = cp), DNS_MAXNAME);
+  ret->dnsn_ttl = p->dnsp_ttl;
+  return ret;
+}

3rdparty/udns/udns_rr_a.c

@@ -0,0 +1,126 @@
+/* udns_rr_a.c
+   parse/query A/AAAA IN records
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#ifndef WINDOWS
+# include <sys/types.h>
+# include <netinet/in.h>
+#endif
+#include "udns.h"
+
+/* here, we use common routine to parse both IPv4 and IPv6 addresses.
+ */
+
+/* this structure should match dns_rr_a[46] */
+struct dns_rr_a {
+  dns_rr_common(dnsa);
+  unsigned char *dnsa_addr;
+};
+
+static int
+dns_parse_a(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+            void **result, unsigned dsize) {
+  struct dns_rr_a *ret;
+  struct dns_parse p;
+  struct dns_rr rr;
+  int r;
+
+  /* first, validate and count number of addresses */
+  dns_initparse(&p, qdn, pkt, cur, end);
+  while((r = dns_nextrr(&p, &rr)) > 0)
+    if (rr.dnsrr_dsz != dsize)
+      return DNS_E_PROTOCOL;
+  if (r < 0)
+    return DNS_E_PROTOCOL;
+  else if (!p.dnsp_nrr)
+    return DNS_E_NODATA;
+
+  ret = malloc(sizeof(*ret) + dsize * p.dnsp_nrr + dns_stdrr_size(&p));
+  if (!ret)
+    return DNS_E_NOMEM;
+
+  ret->dnsa_nrr = p.dnsp_nrr;
+  ret->dnsa_addr = (unsigned char*)(ret+1);
+
+  /* copy the RRs */
+  for (dns_rewind(&p, qdn), r = 0; dns_nextrr(&p, &rr); ++r)
+    memcpy(ret->dnsa_addr + dsize * r, rr.dnsrr_dptr, dsize);
+
+  dns_stdrr_finish((struct dns_rr_null *)ret,
+                   (char *)(ret->dnsa_addr + dsize * p.dnsp_nrr), &p);
+  *result = ret;
+  return 0;
+}
+
+int
+dns_parse_a4(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+             void **result) {
+#ifdef AF_INET
+  assert(sizeof(struct in_addr) == 4);
+#endif
+  assert(dns_get16(cur+2) == DNS_C_IN && dns_get16(cur+0) == DNS_T_A);
+  return dns_parse_a(qdn, pkt, cur, end, result, 4);
+}
+
+struct dns_query *
+dns_submit_a4(struct dns_ctx *ctx, const char *name, int flags,
+              dns_query_a4_fn *cbck, void *data) {
+  return
+    dns_submit_p(ctx, name, DNS_C_IN, DNS_T_A, flags,
+                 dns_parse_a4, (dns_query_fn*)cbck, data);
+}
+
+struct dns_rr_a4 *
+dns_resolve_a4(struct dns_ctx *ctx, const char *name, int flags) {
+  return (struct dns_rr_a4 *)
+    dns_resolve_p(ctx, name, DNS_C_IN, DNS_T_A, flags, dns_parse_a4);
+}
+
+int
+dns_parse_a6(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+             void **result) {
+#ifdef AF_INET6
+  assert(sizeof(struct in6_addr) == 16);
+#endif
+  assert(dns_get16(cur+2) == DNS_C_IN && dns_get16(cur+0) == DNS_T_AAAA);
+  return dns_parse_a(qdn, pkt, cur, end, result, 16);
+}
+
+struct dns_query *
+dns_submit_a6(struct dns_ctx *ctx, const char *name, int flags,
+              dns_query_a6_fn *cbck, void *data) {
+  return
+    dns_submit_p(ctx, name, DNS_C_IN, DNS_T_AAAA, flags,
+                 dns_parse_a6, (dns_query_fn*)cbck, data);
+}
+
+struct dns_rr_a6 *
+dns_resolve_a6(struct dns_ctx *ctx, const char *name, int flags) {
+  return (struct dns_rr_a6 *)
+    dns_resolve_p(ctx, name, DNS_C_IN, DNS_T_AAAA, flags, dns_parse_a6);
+}

3rdparty/udns/udns_rr_mx.c

@@ -0,0 +1,91 @@
+/* udns_rr_mx.c
+   parse/query MX IN records
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include "udns.h"
+
+int
+dns_parse_mx(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+             void **result) {
+  struct dns_rr_mx *ret;
+  struct dns_parse p;
+  struct dns_rr rr;
+  int r, l;
+  char *sp;
+  dnsc_t mx[DNS_MAXDN];
+
+  assert(dns_get16(cur+2) == DNS_C_IN && dns_get16(cur+0) == DNS_T_MX);
+
+  /* first, validate the answer and count size of the result */
+  l = 0;
+  dns_initparse(&p, qdn, pkt, cur, end);
+  while((r = dns_nextrr(&p, &rr)) > 0) {
+    cur = rr.dnsrr_dptr + 2;
+    r = dns_getdn(pkt, &cur, end, mx, sizeof(mx));
+    if (r <= 0 || cur != rr.dnsrr_dend)
+      return DNS_E_PROTOCOL;
+    l += dns_dntop_size(mx);
+  }
+  if (r < 0)
+    return DNS_E_PROTOCOL;
+  if (!p.dnsp_nrr)
+    return DNS_E_NODATA;
+
+  /* next, allocate and set up result */
+  l += dns_stdrr_size(&p);
+  ret = malloc(sizeof(*ret) + sizeof(struct dns_mx) * p.dnsp_nrr + l);
+  if (!ret)
+    return DNS_E_NOMEM;
+  ret->dnsmx_nrr = p.dnsp_nrr;
+  ret->dnsmx_mx = (struct dns_mx *)(ret+1);
+
+  /* and 3rd, fill in result, finally */
+  sp = (char*)(ret->dnsmx_mx + p.dnsp_nrr);
+  for (dns_rewind(&p, qdn), r = 0; dns_nextrr(&p, &rr); ++r) {
+    ret->dnsmx_mx[r].name = sp;
+    cur = rr.dnsrr_dptr;
+    ret->dnsmx_mx[r].priority = dns_get16(cur);
+    cur += 2;
+    dns_getdn(pkt, &cur, end, mx, sizeof(mx));
+    sp += dns_dntop(mx, sp, DNS_MAXNAME);
+  }
+  dns_stdrr_finish((struct dns_rr_null *)ret, sp, &p);
+  *result = ret;
+  return 0;
+}
+
+struct dns_query *
+dns_submit_mx(struct dns_ctx *ctx, const char *name, int flags,
+              dns_query_mx_fn *cbck, void *data) {
+  return
+    dns_submit_p(ctx, name, DNS_C_IN, DNS_T_MX, flags,
+                 dns_parse_mx, (dns_query_fn *)cbck, data);
+}
+
+struct dns_rr_mx *
+dns_resolve_mx(struct dns_ctx *ctx, const char *name, int flags) {
+  return (struct dns_rr_mx *)
+    dns_resolve_p(ctx, name, DNS_C_IN, DNS_T_MX, flags, dns_parse_mx);
+}

3rdparty/udns/udns_rr_naptr.c

@@ -0,0 +1,128 @@
+/* udns_rr_naptr.c
+   parse/query NAPTR IN records
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   Copyright (C) 2006  Mikael Magnusson <mikma@users.sourceforge.net>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include "udns.h"
+
+/* Get a single string for NAPTR record, pretty much like a DN label.
+ * String length is in first byte in *cur, so it can't be >255.
+ */
+static int dns_getstr(dnscc_t **cur, dnscc_t *ep, char *buf)
+{
+  unsigned l;
+  dnscc_t *cp = *cur;
+
+  l = *cp++;
+  if (cp + l > ep)
+    return DNS_E_PROTOCOL;
+  if (buf) {
+    memcpy(buf, cp, l);
+    buf[l] = '\0';
+  }
+  cp += l;
+
+  *cur = cp;
+  return l + 1;
+}
+
+int
+dns_parse_naptr(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+                void **result) {
+  struct dns_rr_naptr *ret;
+  struct dns_parse p;
+  struct dns_rr rr;
+  int r, l;
+  char *sp;
+  dnsc_t dn[DNS_MAXDN];
+
+  assert(dns_get16(cur+2) == DNS_C_IN && dns_get16(cur+0) == DNS_T_NAPTR);
+
+  /* first, validate the answer and count size of the result */
+  l = 0;
+  dns_initparse(&p, qdn, pkt, cur, end);
+  while((r = dns_nextrr(&p, &rr)) > 0) {
+    int i;
+    dnscc_t *ep = rr.dnsrr_dend;
+
+    /* first 4 bytes: order & preference */
+    cur = rr.dnsrr_dptr + 4;
+
+    /* flags, services and regexp */
+    for (i = 0; i < 3; i++) {
+      r = dns_getstr(&cur, ep, NULL);
+      if (r < 0)
+        return r;
+      l += r;
+    }
+    /* replacement */
+    r = dns_getdn(pkt, &cur, end, dn, sizeof(dn));
+    if (r <= 0 || cur != rr.dnsrr_dend)
+      return DNS_E_PROTOCOL;
+    l += dns_dntop_size(dn);
+  }
+  if (r < 0)
+    return DNS_E_PROTOCOL;
+  if (!p.dnsp_nrr)
+    return DNS_E_NODATA;
+
+  /* next, allocate and set up result */
+  l += dns_stdrr_size(&p);
+  ret = malloc(sizeof(*ret) + sizeof(struct dns_naptr) * p.dnsp_nrr + l);
+  if (!ret)
+    return DNS_E_NOMEM;
+  ret->dnsnaptr_nrr = p.dnsp_nrr;
+  ret->dnsnaptr_naptr = (struct dns_naptr *)(ret+1);
+
+  /* and 3rd, fill in result, finally */
+  sp = (char*)(&ret->dnsnaptr_naptr[p.dnsp_nrr]);
+  for (dns_rewind(&p, qdn), r = 0; dns_nextrr(&p, &rr); ++r) {
+    cur = rr.dnsrr_dptr;
+    ret->dnsnaptr_naptr[r].order = dns_get16(cur); cur += 2;
+    ret->dnsnaptr_naptr[r].preference = dns_get16(cur); cur += 2;
+    sp += dns_getstr(&cur, end, (ret->dnsnaptr_naptr[r].flags = sp));
+    sp += dns_getstr(&cur, end, (ret->dnsnaptr_naptr[r].service = sp));
+    sp += dns_getstr(&cur, end, (ret->dnsnaptr_naptr[r].regexp = sp));
+    dns_getdn(pkt, &cur, end, dn, sizeof(dn));
+    sp += dns_dntop(dn, (ret->dnsnaptr_naptr[r].replacement = sp), DNS_MAXNAME);
+  }
+  dns_stdrr_finish((struct dns_rr_null *)ret, sp, &p);
+  *result = ret;
+  return 0;
+}
+
+struct dns_query *
+dns_submit_naptr(struct dns_ctx *ctx, const char *name, int flags,
+                 dns_query_naptr_fn *cbck, void *data) {
+  return
+    dns_submit_p(ctx, name, DNS_C_IN, DNS_T_NAPTR, flags,
+                 dns_parse_naptr, (dns_query_fn *)cbck, data);
+}
+
+struct dns_rr_naptr *
+dns_resolve_naptr(struct dns_ctx *ctx, const char *name, int flags) {
+  return (struct dns_rr_naptr *)
+    dns_resolve_p(ctx, name, DNS_C_IN, DNS_T_NAPTR, flags, dns_parse_naptr);
+}

3rdparty/udns/udns_rr_ptr.c

@@ -0,0 +1,109 @@
+/* udns_rr_ptr.c
+   parse/query PTR records
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include <stdlib.h>
+#include <assert.h>
+#include "udns.h"
+
+int
+dns_parse_ptr(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+              void **result) {
+  struct dns_rr_ptr *ret;
+  struct dns_parse p;
+  struct dns_rr rr;
+  int r, l, c;
+  char *sp;
+  dnsc_t ptr[DNS_MAXDN];
+
+  assert(dns_get16(cur+2) == DNS_C_IN && dns_get16(cur+0) == DNS_T_PTR);
+
+  /* first, validate the answer and count size of the result */
+  l = c = 0;
+  dns_initparse(&p, qdn, pkt, cur, end);
+  while((r = dns_nextrr(&p, &rr)) > 0) {
+    cur = rr.dnsrr_dptr;
+    r = dns_getdn(pkt, &cur, end, ptr, sizeof(ptr));
+    if (r <= 0 || cur != rr.dnsrr_dend)
+      return DNS_E_PROTOCOL;
+    l += dns_dntop_size(ptr);
+    ++c;
+  }
+  if (r < 0)
+    return DNS_E_PROTOCOL;
+  if (!c)
+    return DNS_E_NODATA;
+
+  /* next, allocate and set up result */
+  ret = malloc(sizeof(*ret) + sizeof(char **) * c + l + dns_stdrr_size(&p));
+  if (!ret)
+    return DNS_E_NOMEM;
+  ret->dnsptr_nrr = c;
+  ret->dnsptr_ptr = (char **)(ret+1);
+
+  /* and 3rd, fill in result, finally */
+  sp = (char*)(ret->dnsptr_ptr + c);
+  c = 0;
+  dns_rewind(&p, qdn);
+  while((r = dns_nextrr(&p, &rr)) > 0) {
+    ret->dnsptr_ptr[c] = sp;
+    cur = rr.dnsrr_dptr;
+    dns_getdn(pkt, &cur, end, ptr, sizeof(ptr));
+    sp += dns_dntop(ptr, sp, DNS_MAXNAME);
+    ++c;
+  }
+  dns_stdrr_finish((struct dns_rr_null *)ret, sp, &p);
+  *result = ret;
+  return 0;
+}
+
+struct dns_query *
+dns_submit_a4ptr(struct dns_ctx *ctx, const struct in_addr *addr,
+                 dns_query_ptr_fn *cbck, void *data) {
+  dnsc_t dn[DNS_A4RSIZE];
+  dns_a4todn(addr, 0, dn, sizeof(dn));
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_PTR, DNS_NOSRCH,
+                  dns_parse_ptr, (dns_query_fn *)cbck, data);
+}
+
+struct dns_rr_ptr *
+dns_resolve_a4ptr(struct dns_ctx *ctx, const struct in_addr *addr) {
+  return (struct dns_rr_ptr *)
+    dns_resolve(ctx, dns_submit_a4ptr(ctx, addr, NULL, NULL));
+}
+
+struct dns_query *
+dns_submit_a6ptr(struct dns_ctx *ctx, const struct in6_addr *addr,
+                 dns_query_ptr_fn *cbck, void *data) {
+  dnsc_t dn[DNS_A6RSIZE];
+  dns_a6todn(addr, 0, dn, sizeof(dn));
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_PTR, DNS_NOSRCH,
+                  dns_parse_ptr, (dns_query_fn *)cbck, data);
+}
+
+struct dns_rr_ptr *
+dns_resolve_a6ptr(struct dns_ctx *ctx, const struct in6_addr *addr) {
+  return (struct dns_rr_ptr *)
+    dns_resolve(ctx, dns_submit_a6ptr(ctx, addr, NULL, NULL));
+}

3rdparty/udns/udns_rr_srv.c

@@ -0,0 +1,155 @@
+/* udns_rr_srv.c
+   parse/query SRV IN (rfc2782) records
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+   Copyright 2005 Thadeu Lima de Souza Cascardo <cascardo@minaslivre.org>
+
+   2005-09-11:
+   Changed MX parser file into a SRV parser file
+
+ */
+
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include "udns.h"
+
+int
+dns_parse_srv(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+              void **result) {
+  struct dns_rr_srv *ret;
+  struct dns_parse p;
+  struct dns_rr rr;
+  int r, l;
+  char *sp;
+  dnsc_t srv[DNS_MAXDN];
+
+  assert(dns_get16(cur+2) == DNS_C_IN && dns_get16(cur+0) == DNS_T_SRV);
+
+  /* first, validate the answer and count size of the result */
+  l = 0;
+  dns_initparse(&p, qdn, pkt, cur, end);
+  while((r = dns_nextrr(&p, &rr)) > 0) {
+    cur = rr.dnsrr_dptr + 6;
+    r = dns_getdn(pkt, &cur, end, srv, sizeof(srv));
+    if (r <= 0 || cur != rr.dnsrr_dend)
+      return DNS_E_PROTOCOL;
+    l += dns_dntop_size(srv);
+  }
+  if (r < 0)
+    return DNS_E_PROTOCOL;
+  if (!p.dnsp_nrr)
+    return DNS_E_NODATA;
+
+  /* next, allocate and set up result */
+  l += dns_stdrr_size(&p);
+  ret = malloc(sizeof(*ret) + sizeof(struct dns_srv) * p.dnsp_nrr + l);
+  if (!ret)
+    return DNS_E_NOMEM;
+  ret->dnssrv_nrr = p.dnsp_nrr;
+  ret->dnssrv_srv = (struct dns_srv *)(ret+1);
+
+  /* and 3rd, fill in result, finally */
+  sp = (char*)(ret->dnssrv_srv + p.dnsp_nrr);
+  for (dns_rewind(&p, qdn), r = 0; dns_nextrr(&p, &rr); ++r) {
+    ret->dnssrv_srv[r].name = sp;
+    cur = rr.dnsrr_dptr;
+    ret->dnssrv_srv[r].priority = dns_get16(cur);
+    ret->dnssrv_srv[r].weight = dns_get16(cur+2);
+    ret->dnssrv_srv[r].port = dns_get16(cur+4);
+    cur += 6;
+    dns_getdn(pkt, &cur, end, srv, sizeof(srv));
+    sp += dns_dntop(srv, sp, DNS_MAXNAME);
+  }
+  dns_stdrr_finish((struct dns_rr_null *)ret, sp, &p);
+  *result = ret;
+  return 0;
+}
+
+/* Add a single service or proto name prepending an undescore (_),
+ * according to rfc2782 rules.
+ * Return 0 or the label length.
+ * Routing assumes dn holds enouth space for a single DN label. */
+static int add_sname(dnsc_t *dn, const char *sn) {
+  int l = dns_ptodn(sn, 0, dn + 1, DNS_MAXLABEL-1, NULL);
+  if (l <= 1 || l - 2 != dn[1])
+    /* Should we really check if sn is exactly one label?  Do we care? */
+    return 0;
+  dn[0] = l - 1;
+  dn[1] = '_';
+  return l;
+}
+
+/* Construct a domain name for SRV query from the given name, service and proto.
+ * The code allows any combinations of srv and proto (both are non-NULL,
+ * both NULL, or either one is non-NULL).  Whenever it makes any sense or not
+ * is left as an exercise to programmer.
+ * Return negative value on error (malformed query) or addition query flag(s).
+ */
+static int
+build_srv_dn(dnsc_t *dn, const char *name, const char *srv, const char *proto)
+{
+  int p = 0, l, isabs;
+  if (srv) {
+    l = add_sname(dn + p, srv);
+    if (!l)
+      return -1;
+    p += l;
+  }
+  if (proto) {
+    l = add_sname(dn + p, proto);
+    if (!l)
+      return -1;
+    p += l;
+  }
+  l = dns_ptodn(name, 0, dn + p, DNS_MAXDN - p, &isabs);
+  if (l < 0)
+    return -1;
+  return isabs ? DNS_NOSRCH : 0;
+}
+
+struct dns_query *
+dns_submit_srv(struct dns_ctx *ctx,
+               const char *name, const char *srv, const char *proto,
+               int flags, dns_query_srv_fn *cbck, void *data) {
+  dnsc_t dn[DNS_MAXDN];
+  int r = build_srv_dn(dn, name, srv, proto);
+  if (r < 0) {
+    dns_setstatus (ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return
+    dns_submit_dn(ctx, dn, DNS_C_IN, DNS_T_SRV, flags | r,
+                  dns_parse_srv, (dns_query_fn *)cbck, data);
+}
+
+struct dns_rr_srv *
+dns_resolve_srv(struct dns_ctx *ctx,
+                const char *name, const char *srv, const char *proto, int flags)
+{
+  dnsc_t dn[DNS_MAXDN];
+  int r = build_srv_dn(dn, name, srv, proto);
+  if (r < 0) {
+    dns_setstatus(ctx, DNS_E_BADQUERY);
+    return NULL;
+  }
+  return (struct dns_rr_srv *)
+    dns_resolve_dn(ctx, dn, DNS_C_IN, DNS_T_SRV, flags | r, dns_parse_srv);
+}

3rdparty/udns/udns_rr_txt.c

@@ -0,0 +1,98 @@
+/* udns_rr_txt.c
+   parse/query TXT records
+
+   Copyright (C) 2005  Michael Tokarev <mjt@corpit.ru>
+   This file is part of UDNS library, an async DNS stub resolver.
+
+   This library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   This library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with this library, in file named COPYING.LGPL; if not,
+   write to the Free Software Foundation, Inc., 59 Temple Place,
+   Suite 330, Boston, MA  02111-1307  USA
+
+ */
+
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include "udns.h"
+
+int
+dns_parse_txt(dnscc_t *qdn, dnscc_t *pkt, dnscc_t *cur, dnscc_t *end,
+              void **result) {
+  struct dns_rr_txt *ret;
+  struct dns_parse p;
+  struct dns_rr rr;
+  int r, l;
+  dnsc_t *sp;
+  dnscc_t *cp, *ep;
+
+  assert(dns_get16(cur+0) == DNS_T_TXT);
+
+  /* first, validate the answer and count size of the result */
+  l = 0;
+  dns_initparse(&p, qdn, pkt, cur, end);
+  while((r = dns_nextrr(&p, &rr)) > 0) {
+    cp = rr.dnsrr_dptr; ep = rr.dnsrr_dend;
+    while(cp < ep) {
+      r = *cp++;
+      if (cp + r > ep)
+        return DNS_E_PROTOCOL;
+      l += r;
+      cp += r;
+    }
+  }
+  if (r < 0)
+    return DNS_E_PROTOCOL;
+  if (!p.dnsp_nrr)
+    return DNS_E_NODATA;
+
+  /* next, allocate and set up result */
+  l +=  (sizeof(struct dns_txt) + 1) * p.dnsp_nrr + dns_stdrr_size(&p);
+  ret = malloc(sizeof(*ret) + l);
+  if (!ret)
+    return DNS_E_NOMEM;
+  ret->dnstxt_nrr = p.dnsp_nrr;
+  ret->dnstxt_txt = (struct dns_txt *)(ret+1);
+
+  /* and 3rd, fill in result, finally */
+  sp = (dnsc_t*)(ret->dnstxt_txt + p.dnsp_nrr);
+  for(dns_rewind(&p, qdn), r = 0; dns_nextrr(&p, &rr) > 0; ++r) {
+    ret->dnstxt_txt[r].txt = sp;
+    cp = rr.dnsrr_dptr; ep = rr.dnsrr_dend;
+    while(cp < ep) {
+      l = *cp++;
+      memcpy(sp, cp, l);
+      sp += l;
+      cp += l;
+    }
+    ret->dnstxt_txt[r].len = sp - ret->dnstxt_txt[r].txt;
+    *sp++ = '\0';
+  }
+  dns_stdrr_finish((struct dns_rr_null *)ret, (char*)sp, &p);
+  *result = ret;
+  return 0;
+}
+
+struct dns_query *
+dns_submit_txt(struct dns_ctx *ctx, const char *name, int qcls, int flags,
+               dns_query_txt_fn *cbck, void *data) {
+  return
+    dns_submit_p(ctx, name, qcls, DNS_T_TXT, flags,
+                 dns_parse_txt, (dns_query_fn *)cbck, data);
+}
+
+struct dns_rr_txt *
+dns_resolve_txt(struct dns_ctx *ctx, const char *name, int qcls, int flags) {
+  return (struct dns_rr_txt *)
+    dns_resolve_p(ctx, name, qcls, DNS_T_TXT, flags, dns_parse_txt);
+}

CMakeLists.txt

@@ -117,7 +117,7 @@ option(ENABLE_RTI       "Set to ON to build SimGear with RTI support" OFF)
 option(ENABLE_TESTS     "Set to OFF to disable building SimGear's test applications" ON)
 option(ENABLE_SOUND     "Set to OFF to disable building SimGear's sound support" ON)
 option(ENABLE_PKGUTIL   "Set to ON to build the sg_pkgutil application (default)" ON)
-option(ENABLE_CURL      "Set to ON to use libCurl as the HTTP client backend" OFF)
+option(ENABLE_DNS       "Set to ON to use udns library and DNS service resolver" ON)
 
 if (MSVC)
   GET_FILENAME_COMPONENT(PARENT_DIR ${PROJECT_BINARY_DIR} PATH)
@@ -209,11 +209,7 @@ else()
 endif(SIMGEAR_HEADLESS)
 
 find_package(ZLIB REQUIRED)
-
-if (ENABLE_CURL)
-  find_package(CURL REQUIRED)
-  message(STATUS "Curl HTTP client: ENABLED")
-endif()
+find_package(CURL REQUIRED)
 
 if (SYSTEM_EXPAT)
     message(STATUS "Requested to use system Expat library, forcing SIMGEAR_SHARED to true")
@@ -420,6 +416,13 @@ endif()
 install (FILES ${PROJECT_BINARY_DIR}/simgear/simgear_config.h  DESTINATION include/simgear/)
 
 include_directories(3rdparty/utf8/source)
+if (ENABLE_DNS)
+    message(STATUS "DNS resolver: ENABLED")
+    include_directories(3rdparty/udns)
+else()
+    message(STATUS "DNS resolver: DISABLED")
+endif()
+
 
 add_subdirectory(3rdparty)
 add_subdirectory(simgear)

simgear/bucket/newbucket.cxx

@@ -286,6 +286,54 @@ double SGBucket::get_height_m() const {
     return SG_BUCKET_SPAN * degree_height;
 }
 
+unsigned int SGBucket::siblings( int dx, int dy, std::vector<SGBucket>& buckets ) const 
+{
+    if (!isValid()) {
+        SG_LOG(SG_TERRAIN, SG_WARN, "SGBucket::sibling: requesting sibling of invalid bucket");
+        return 0;
+    }
+    
+    double src_span = sg_bucket_span( get_center_lat() );
+   
+    double clat = get_center_lat() + dy * SG_BUCKET_SPAN;
+    // return invalid here instead of clipping, so callers can discard
+    // invalid buckets without having to check if it's an existing one
+    if ((clat < -90.0) || (clat > 90.0)) {
+        return 0;
+    }
+    
+    // find the lon span for the new latitude
+    double trg_span = sg_bucket_span( clat );
+    
+    // if target span < src_span, return multiple buckets...
+    if ( trg_span < src_span ) {
+        // calc center longitude of westernmost sibling
+        double start_lon = get_center_lat() - src_span/2 + trg_span/2;
+        
+        unsigned int num_buckets = src_span/trg_span;
+        for ( unsigned int x = 0; x < num_buckets; x++ ) {
+            double tmp = start_lon + x * trg_span;
+            tmp = SGMiscd::normalizePeriodic(-180.0, 180.0, tmp);
+            
+            SGBucket b;
+            b.innerSet(tmp, clat);
+            
+            buckets.push_back( b );
+        }
+    } else {
+        // just return the single sibling
+        double tmp = get_center_lon() + dx * trg_span;
+        tmp = SGMiscd::normalizePeriodic(-180.0, 180.0, tmp);
+    
+        SGBucket b;
+        b.innerSet(tmp, clat);
+        
+        buckets.push_back( b );
+    }
+    
+    return buckets.size();
+}
+
 SGBucket SGBucket::sibling(int dx, int dy) const
 {
     if (!isValid()) {

simgear/bucket/newbucket.hxx

@@ -288,6 +288,11 @@ public:
      */
     SGBucket sibling(int dx, int dy) const;
     
+    /**
+     * @return multiple buckets offset from this by dx,dy
+     */
+    unsigned int siblings(int dz, int dy, std::vector<SGBucket>& buckets) const;
+    
     // friends
 
     friend std::ostream& operator<< ( std::ostream&, const SGBucket& );

simgear/canvas/CanvasSystemAdapter.hxx

@@ -37,7 +37,7 @@ namespace canvas
       virtual FontPtr getFont(const std::string& name) const = 0;
       virtual void addCamera(osg::Camera* camera) const = 0;
       virtual void removeCamera(osg::Camera* camera) const = 0;
-      virtual osg::Image* getImage(const std::string& path) const = 0;
+      virtual osg::ref_ptr<osg::Image> getImage(const std::string& path) const = 0;
       virtual SGSubsystem* getSubsystem(const std::string& name) const = 0;
       virtual HTTP::Client* getHTTPClient() const = 0;
   };

simgear/canvas/elements/CanvasImage.cxx

@@ -155,8 +155,9 @@ namespace canvas
   //----------------------------------------------------------------------------
   Image::~Image()
   {
-    if( _http_request )
-      _http_request->abort("image destroyed");
+      if( _http_request ) {
+          Canvas::getSystemAdapter()->getHTTPClient()->cancelRequest(_http_request, "image destroyed");
+      }
   }
 
   //----------------------------------------------------------------------------
@@ -463,7 +464,7 @@ namespace canvas
   }
 
   //----------------------------------------------------------------------------
-  void Image::setImage(osg::Image *img)
+  void Image::setImage(osg::ref_ptr<osg::Image> img)
   {
     // remove canvas...
     setSrcCanvas( CanvasPtr() );
@@ -618,7 +619,7 @@ namespace canvas
       // Abort pending request
       if( _http_request )
       {
-        _http_request->abort("setting new image");
+          Canvas::getSystemAdapter()->getHTTPClient()->cancelRequest(_http_request, "setting new image");
         _http_request.reset();
       }
 

simgear/canvas/elements/CanvasImage.hxx

@@ -59,7 +59,7 @@ namespace canvas
       void setSrcCanvas(CanvasPtr canvas);
       CanvasWeakPtr getSrcCanvas() const;
 
-      void setImage(osg::Image *img);
+      void setImage(osg::ref_ptr<osg::Image> img);
       void setFill(const std::string& fill);
 
       /**

simgear/constants.h

@@ -211,5 +211,10 @@ const float SG_RADIANS_TO_DEGREES = 180.0f / SG_PI;
 /** for backwards compatibility */
 #define SG_SCENERY_FILE_FORMAT "0.4"
 
+/** Default range in m at which all objects are displayed.  Overridden by /sim/rendering/static-lod/rough **/
+#define SG_OBJECT_RANGE     9000.0
+
+/** Radius of scenery tiles in m **/
+#define SG_TILE_RADIUS      14000.0
 
 #endif // _SG_CONSTANTS_H

simgear/debug/debug_types.h

@@ -33,7 +33,8 @@ typedef enum {
     SG_NAVAID      = 0x00400000,
     SG_GUI         = 0x00800000,
     SG_TERRASYNC   = 0x01000000,
-    SG_UNDEFD      = 0x02000000, // For range checking
+    SG_PARTICLES   = 0x02000000,
+    SG_UNDEFD      = 0x04000000, // For range checking
 
     SG_ALL         = 0xFFFFFFFF
 } sgDebugClass;

simgear/debug/logstream.cxx

@@ -72,6 +72,7 @@ const char* debugClassToString(sgDebugClass c)
     case SG_NAVAID:     return "navaid";
     case SG_GUI:        return "gui";
     case SG_TERRASYNC:  return "terrasync";
+    case SG_PARTICLES:  return "particles";
     default:            return "unknown";
     }
 }

simgear/environment/metar.cxx

@@ -23,6 +23,16 @@
 /**
  * @file metar.cxx
  * Interface for encoded Meteorological Aerodrome Reports (METAR).
+ *
+ * @see WMO-49
+ * Technical Regulations, Basic Documents No. 2 (WMO No. 49)
+ * Volume II - Meteorological Service for International Air Navigation
+ * http://library.wmo.int/pmb_ged/wmo_49-v2_2013_en.pdf
+ *
+ * Refer to Table A3-2 (Template for METAR and SPECI) following page 78.
+ *
+ * For general information:
+ * World Meteorological Organization http://library.wmo.int
  */
 #ifdef HAVE_CONFIG_H
 #  include <simgear_config.h>
@@ -465,6 +475,7 @@ bool SGMetar::scanRwyVisRange()
 	char *m = _m;
 	int i;
 	SGMetarRunway r;
+
 	if (*m++ != 'R')
 		return false;
 	if (!scanNumber(&m, &i, 2))
@@ -582,6 +593,14 @@ bool SGMetar::scanWeather()
 	string weather;
 	const struct Token *a;
 
+	// @see WMO-49 Section 4.4.2.9
+	// Denotes a temporary failure of the sensor
+	if (!strncmp(m, "// ", 3)) {
+		_m += 3;
+		_grpcount++;
+		return false;
+	}
+
 	if ((a = scanToken(&m, special))) {
 		if (!scanBoundary(&m))
 			return false;
@@ -591,7 +610,7 @@ bool SGMetar::scanWeather()
 	}
 
 	string pre, post;
-    struct Weather w;
+	struct Weather w;
 	if (*m == '-')
 		m++, pre = "light ", w.intensity = LIGHT;
 	else if (*m == '+')
@@ -685,7 +704,7 @@ bool SGMetar::scanSkyCondition()
 			return false;
 
 		if (i == 3) {
-            cl._coverage = SGMetarCloud::COVERAGE_CLEAR;
+			cl._coverage = SGMetarCloud::COVERAGE_CLEAR;
 			_clouds.push_back(cl);
 		} else {
 			_cavok = true;
@@ -716,7 +735,7 @@ bool SGMetar::scanSkyCondition()
 	} else if (!scanNumber(&m, &i, 3))
 		i = -1;
 
-    if (cl._coverage == SGMetarCloud::COVERAGE_NIL) {
+	if (cl._coverage == SGMetarCloud::COVERAGE_NIL) {
 		if (!scanBoundary(&m))
 			return false;
 		if (i == -1)			// 'VV///'
@@ -735,9 +754,15 @@ bool SGMetar::scanSkyCondition()
 		cl._type = a->id;
 		cl._type_long = a->text;
 	}
+
+	// @see WMO-49 Section 4.5.4.5
+	// Denotes temporary failure of sensor and covers cases like FEW045///
+	if (!strncmp(m, "///", 3))
+		m += 3;
 	if (!scanBoundary(&m))
 		return false;
 	_clouds.push_back(cl);
+
 	_m = m;
 	_grpcount++;
 	return true;

simgear/environment/test_metar.cxx

@@ -58,6 +58,37 @@ void test_basic()
     COMPARE(m1.getWindDir(), 270);
     FUZZY_COMPARE(m1.getWindSpeed_kt(), 12, TEST_EPSILON);
     
+    COMPARE(m1.getWeather().size(), 1);
+    COMPARE(m1.getClouds().size(), 2);
+
+    FUZZY_COMPARE(m1.getTemperature_C(), 10, TEST_EPSILON);
+    FUZZY_COMPARE(m1.getDewpoint_C(), 5, TEST_EPSILON);
+    FUZZY_COMPARE(m1.getPressure_hPa(), 1025, TEST_EPSILON);
+}
+
+void test_sensor_failure_weather()
+{
+    SGMetar m1("2011/10/20 11:25 EHAM 201125Z 27012KT 240V300 9999 // FEW025CB SCT048 10/05 Q1025");
+    COMPARE(m1.getWindDir(), 270);
+    FUZZY_COMPARE(m1.getWindSpeed_kt(), 12, TEST_EPSILON);
+
+    COMPARE(m1.getWeather().size(), 0);
+    COMPARE(m1.getClouds().size(), 2);
+
+    FUZZY_COMPARE(m1.getTemperature_C(), 10, TEST_EPSILON);
+    FUZZY_COMPARE(m1.getDewpoint_C(), 5, TEST_EPSILON);
+    FUZZY_COMPARE(m1.getPressure_hPa(), 1025, TEST_EPSILON);
+}
+
+void test_sensor_failure_cloud()
+{
+    SGMetar m1("2011/10/20 11:25 EHAM 201125Z 27012KT 240V300 9999 FEW025CB/// SCT048/// 10/05 Q1025");
+    COMPARE(m1.getWindDir(), 270);
+    FUZZY_COMPARE(m1.getWindSpeed_kt(), 12, TEST_EPSILON);
+
+    COMPARE(m1.getWeather().size(), 0);
+    COMPARE(m1.getClouds().size(), 2);
+
     FUZZY_COMPARE(m1.getTemperature_C(), 10, TEST_EPSILON);
     FUZZY_COMPARE(m1.getDewpoint_C(), 5, TEST_EPSILON);
     FUZZY_COMPARE(m1.getPressure_hPa(), 1025, TEST_EPSILON);
@@ -66,7 +97,9 @@ void test_basic()
 int main(int argc, char* argv[])
 {
     try {
-    test_basic();
+        test_basic();
+        test_sensor_failure_weather();
+        test_sensor_failure_cloud();
     } catch (sg_exception& e) {
         cerr << "got exception:" << e.getMessage() << endl;
         return -1;

simgear/io/AbstractRepository.cxx

@@ -0,0 +1,34 @@
+// AbstractRepository.cxx -- abstract API for TerraSync remote
+//
+// Copyright (C) 2016  James Turner <zakalawe@mac.com>
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License as
+// published by the Free Software Foundation; either version 2 of the
+// License, or (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+
+#include "AbstractRepository.hxx"
+
+namespace simgear
+{
+
+AbstractRepository::~AbstractRepository()
+{
+
+}
+
+size_t AbstractRepository::bytesStillToDownload() const
+{
+    return 0;
+}
+
+} // of namespace simgear

simgear/io/AbstractRepository.hxx

@@ -0,0 +1,72 @@
+// AbstractRepository.hxx - API for terrasyc to access remote server
+//
+// Copyright (C) 2016  James Turner <zakalawe@mac.com>
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License as
+// published by the Free Software Foundation; either version 2 of the
+// License, or (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+
+
+#ifndef SG_IO_ABSTRACT_REPOSITORY_HXX
+#define SG_IO_ABSTRACT_REPOSITORY_HXX
+
+#include <string>
+
+#include <simgear/misc/sg_path.hxx>
+
+namespace simgear  {
+
+  namespace HTTP {
+    class Client;
+  }
+
+class AbstractRepository
+{
+public:
+
+    virtual ~AbstractRepository();
+
+    virtual SGPath fsBase() const = 0;
+
+    virtual void setBaseUrl(const std::string& url) =0;
+    virtual std::string baseUrl() const = 0;;
+
+    virtual HTTP::Client* http() const = 0;
+
+    virtual void update() = 0;
+
+    virtual bool isDoingSync() const = 0;
+
+    virtual size_t bytesStillToDownload() const;
+
+    enum ResultCode {
+        REPO_NO_ERROR = 0,
+        REPO_ERROR_NOT_FOUND,
+        REPO_ERROR_SOCKET,
+        SVN_ERROR_XML,
+        SVN_ERROR_TXDELTA,
+        REPO_ERROR_IO,
+        REPO_ERROR_CHECKSUM,
+        REPO_ERROR_FILE_NOT_FOUND,
+        REPO_ERROR_HTTP,
+        REPO_PARTIAL_UPDATE
+    };
+
+    virtual ResultCode failure() const = 0;
+  protected:
+
+};
+
+} // of namespace simgear
+
+#endif // of SG_IO_ABSTRACT_REPOSITORY_HXX

simgear/io/CMakeLists.txt

@@ -18,10 +18,12 @@ set(HEADERS
     HTTPFileRequest.hxx
     HTTPMemoryRequest.hxx
     HTTPRequest.hxx
+    AbstractRepository.hxx
     DAVMultiStatus.hxx
     SVNRepository.hxx
     SVNDirectory.hxx
     SVNReportParser.hxx
+    HTTPRepository.hxx
     )
 
 set(SOURCES
@@ -40,15 +42,17 @@ set(SOURCES
     HTTPFileRequest.cxx
     HTTPMemoryRequest.cxx
     HTTPRequest.cxx
+    AbstractRepository.cxx
     DAVMultiStatus.cxx
     SVNRepository.cxx
     SVNDirectory.cxx
     SVNReportParser.cxx
+    HTTPRepository.cxx
     )
 
-if (NOT ENABLE_CURL)
-  list(APPEND SOURCES HTTPContentDecode.cxx)
-  list(APPEND HEADERS HTTPContentDecode.hxx)
+if(ENABLE_DNS)
+  list(APPEND SOURCES DNSClient.cxx)
+  list(APPEND HEADERS DNSClient.hxx)
 endif()
 
 simgear_component(io io "${SOURCES}" "${HEADERS}")
@@ -66,9 +70,18 @@ target_link_libraries(test_http ${TEST_LIBS})
 
 add_test(http ${EXECUTABLE_OUTPUT_PATH}/test_http)
 
+if(ENABLE_DNS)
+	add_executable(test_dns test_DNS.cxx)
+	target_link_libraries(test_dns ${TEST_LIBS})
+	add_test(dns ${EXECUTABLE_OUTPUT_PATH}/test_dns)
+endif()
+
 add_executable(httpget httpget.cxx)
 target_link_libraries(httpget ${TEST_LIBS})
 
+add_executable(http_repo_sync http_repo_sync.cxx)
+target_link_libraries(http_repo_sync ${TEST_LIBS})
+
 add_executable(decode_binobj decode_binobj.cxx)
 target_link_libraries(decode_binobj ${TEST_LIBS})
 
@@ -77,4 +90,8 @@ target_link_libraries(test_binobj ${TEST_LIBS})
 
 add_test(binobj ${EXECUTABLE_OUTPUT_PATH}/test_binobj)
 
+add_executable(test_repository test_repository.cxx)
+target_link_libraries(test_repository ${TEST_LIBS})
+add_test(http_repository ${EXECUTABLE_OUTPUT_PATH}/test_repository)
+
 endif(ENABLE_TESTS)

simgear/io/DNSClient.cxx

@@ -0,0 +1,144 @@
+/**
+ * \file DNSClient.cxx - simple DNS resolver client engine for SimGear
+ */
+
+// Written by James Turner
+//
+// Copyright (C) 2016 Torsten Dreyer - torsten (at) t3r (dot) de
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+//
+
+#include "DNSClient.hxx"
+#include "udns.h"
+#include <time.h>
+#include <simgear/debug/logstream.hxx>
+
+namespace simgear {
+
+namespace DNS {
+
+class Client::ClientPrivate {
+public:
+    ClientPrivate() {
+        if (dns_init(NULL, 0) < 0)
+            SG_LOG(SG_IO, SG_ALERT, "Can't init udns library" );
+
+        if( dns_open(NULL) < 0 )
+            SG_LOG(SG_IO, SG_ALERT, "Can't open udns context" );
+    }
+
+    ~ClientPrivate() {
+        dns_close(NULL);
+    }
+};
+
+Request::Request( const std::string & dn ) :
+        _dn(dn),
+        _type(DNS_T_ANY),
+        _complete(false),
+        _timeout_secs(5),
+        _start(0)
+{
+}
+
+Request::~Request()
+{
+}
+
+bool Request::isTimeout() const
+{
+    return (time(NULL) - _start) > _timeout_secs;
+}
+
+NAPTRRequest::NAPTRRequest( const std::string & dn ) :
+        Request(dn)
+{
+    _type = DNS_T_NAPTR;
+}
+
+static bool sortNAPTR( const NAPTRRequest::NAPTR_ptr a, const NAPTRRequest::NAPTR_ptr b )
+{
+    if( a->order > b->order ) return false;
+    if( a->order < b->order ) return true;
+    return a->preference < b->preference;
+}
+
+static void dnscbNAPTR(struct dns_ctx *ctx, struct dns_rr_naptr *result, void *data)
+{
+    NAPTRRequest * r = static_cast<NAPTRRequest*>(data);
+    if (result) {
+        r->cname = result->dnsnaptr_cname;
+        r->qname = result->dnsnaptr_qname;
+        r->ttl = result->dnsnaptr_ttl;
+        for (int i = 0; i < result->dnsnaptr_nrr; i++) {
+            if( !r->qservice.empty() && r->qservice != result->dnsnaptr_naptr[i].service )
+                return;
+
+            //TODO: case ignore and result flags may have more than one flag
+            if( !r->qflags.empty() && r->qflags != result->dnsnaptr_naptr[i].flags )
+                return;
+
+            NAPTRRequest::NAPTR_ptr naptr(new NAPTRRequest::NAPTR);
+            r->entries.push_back(naptr);
+            naptr->order = result->dnsnaptr_naptr[i].order;
+            naptr->preference = result->dnsnaptr_naptr[i].preference;
+            naptr->flags = result->dnsnaptr_naptr[i].flags;
+            naptr->service = result->dnsnaptr_naptr[i].service;
+            naptr->regexp = result->dnsnaptr_naptr[i].regexp;
+            naptr->replacement = result->dnsnaptr_naptr[i].replacement;
+        }
+        std::sort( r->entries.begin(), r->entries.end(), sortNAPTR );
+        free(result);
+    }
+    r->setComplete();
+}
+
+void NAPTRRequest::submit()
+{
+    if (!dns_submit_naptr(NULL, getDn().c_str(), 0, dnscbNAPTR, this )) {
+        SG_LOG(SG_IO, SG_ALERT, "Can't submit dns request for " << getDn());
+        return;
+    }
+    _start = time(NULL);
+}
+
+
+Client::~Client()
+{
+}
+
+Client::Client() :
+    d(new ClientPrivate)
+{
+}
+
+void Client::makeRequest(const Request_ptr& r)
+{
+    r->submit();
+}
+
+
+void Client::update(int waitTimeout)
+{
+    time_t now = time(NULL);
+    if( dns_timeouts( NULL, waitTimeout, now ) < 0 )
+        return;
+    dns_ioevent(NULL, now);
+}
+
+} // of namespace DNS
+
+} // of namespace simgear

simgear/io/DNSClient.hxx

@@ -0,0 +1,109 @@
+/**
+ * \file DNSClient.hxx - simple DNS resolver client for SimGear
+ */
+
+// Written by Torsten Dreyer
+//
+// Copyright (C) 2016 Torsten Dreyer - torsten (at) t3r (dot) de
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+//
+
+#ifndef SG_DNS_CLIENT_HXX
+#define SG_DNS_CLIENT_HXX
+
+#include <memory> // for std::auto_ptr
+#include <string>
+#include <vector>
+#include <simgear/structure/SGReferenced.hxx>
+#include <simgear/structure/SGSharedPtr.hxx>
+
+namespace simgear
+{
+
+namespace DNS
+{
+
+class Request : public SGReferenced
+{
+public:
+    Request( const std::string & dn );
+    virtual ~Request();
+    std::string getDn() const { return _dn; }
+    int getType() const { return _type; }
+    bool isComplete() const { return _complete; }
+    bool isTimeout() const;
+    void setComplete( bool b = true ) { _complete = b; }
+
+    virtual void submit() = 0;
+
+    std::string cname;
+    std::string qname;
+    unsigned ttl;
+protected:
+    std::string _dn;
+    int _type;
+    bool _complete;
+    time_t _timeout_secs;
+    time_t _start;
+};
+
+class NAPTRRequest : public Request
+{
+public:
+    NAPTRRequest( const std::string & dn );
+    virtual void submit();
+
+    struct NAPTR : SGReferenced {
+        int order;
+        int preference;
+        std::string flags;
+        std::string service;
+        std::string regexp;
+        std::string replacement;
+    };
+    typedef SGSharedPtr<NAPTR> NAPTR_ptr;
+    typedef std::vector<NAPTR_ptr> NAPTR_list;
+    NAPTR_list entries;
+
+    std::string qflags;
+    std::string qservice;
+};
+
+typedef SGSharedPtr<Request> Request_ptr;
+
+class Client
+{
+public:
+    Client();
+    ~Client();
+
+    void update(int waitTimeout = 0);
+
+    void makeRequest(const Request_ptr& r);
+
+//    void cancelRequest(const Request_ptr& r, std::string reason = std::string());
+
+private:
+
+    class ClientPrivate;
+    std::auto_ptr<ClientPrivate> d;
+};
+
+} // of namespace DNS
+
+} // of namespace simgear
+
+#endif // of SG_DNS_CLIENT_HXX

simgear/io/HTTPClient.cxx

@@ -38,11 +38,7 @@
 
 #include <simgear/simgear_config.h>
 
-#if defined(ENABLE_CURL)
-  #include <curl/multi.h>
-#else
-    #include <simgear/io/HTTPContentDecode.hxx>
-#endif
+#include <curl/multi.h>
 
 #include <simgear/io/sg_netChat.hxx>
 
@@ -68,7 +64,6 @@ namespace HTTP
 
 extern const int DEFAULT_HTTP_PORT = 80;
 const char* CONTENT_TYPE_URL_ENCODED = "application/x-www-form-urlencoded";
-const unsigned int MAX_INFLIGHT_REQUESTS = 32;
 
 class Connection;
 typedef std::multimap<std::string, Connection*> ConnectionDict;
@@ -77,719 +72,131 @@ typedef std::list<Request_ptr> RequestList;
 class Client::ClientPrivate
 {
 public:
-#if defined(ENABLE_CURL)
     CURLM* curlMulti;
-    bool haveActiveRequests;
-#else
-    NetChannelPoller poller;
-// connections by host (potentially more than one)
-    ConnectionDict connections;
-#endif
+
+    void createCurlMulti()
+    {
+        curlMulti = curl_multi_init();
+        // see https://curl.haxx.se/libcurl/c/CURLMOPT_PIPELINING.html
+        // we request HTTP 1.1 pipelining
+        curl_multi_setopt(curlMulti, CURLMOPT_PIPELINING, 1 /* aka CURLPIPE_HTTP1 */);
+        curl_multi_setopt(curlMulti, CURLMOPT_MAX_TOTAL_CONNECTIONS, (long) maxConnections);
+        curl_multi_setopt(curlMulti, CURLMOPT_MAX_PIPELINE_LENGTH,
+                          (long) maxPipelineDepth);
+        curl_multi_setopt(curlMulti, CURLMOPT_MAX_HOST_CONNECTIONS,
+                          (long) maxHostConnections);
+
+
+    }
+
+    typedef std::map<Request_ptr, CURL*> RequestCurlMap;
+    RequestCurlMap requests;
 
     std::string userAgent;
     std::string proxy;
     int proxyPort;
     std::string proxyAuth;
     unsigned int maxConnections;
+    unsigned int maxHostConnections;
+    unsigned int maxPipelineDepth;
 
     RequestList pendingRequests;
 
-
-
     SGTimeStamp timeTransferSample;
     unsigned int bytesTransferred;
     unsigned int lastTransferRate;
     uint64_t totalBytesDownloaded;
 };
 
-#if !defined(ENABLE_CURL)
-class Connection : public NetChat
-{
-public:
-    Connection(Client* pr, const std::string& conId) :
-        client(pr),
-        state(STATE_CLOSED),
-        port(DEFAULT_HTTP_PORT),
-        connectionId(conId)
-    {
-    }
-
-    virtual ~Connection()
-    {
-    }
-
-    virtual void handleBufferRead (NetBuffer& buffer)
-    {
-      if( !activeRequest || !activeRequest->isComplete() )
-        return NetChat::handleBufferRead(buffer);
-
-      // Request should be aborted (signaled by setting its state to complete).
-
-      // force the state to GETTING_BODY, to simplify logic in
-      // responseComplete and handleClose
-      state = STATE_GETTING_BODY;
-      responseComplete();
-    }
-
-    void setServer(const std::string& h, short p)
-    {
-        host = h;
-        port = p;
-    }
-
-    // socket-level errors
-    virtual void handleError(int error)
-    {
-        const char* errStr = strerror(error);
-        SG_LOG(SG_IO, SG_WARN, "HTTP Connection handleError:" << error << " ("
-               << errStr << ")");
-
-        debugDumpRequests();
-
-        if (!activeRequest)
-        {
-        // connection level failure, eg name lookup or routing
-        // we won't have an active request yet, so let's fail all of the
-        // requests since we presume it's a systematic failure for
-        // the host in question
-            BOOST_FOREACH(Request_ptr req, sentRequests) {
-                req->setFailure(error, errStr);
-            }
-
-            BOOST_FOREACH(Request_ptr req, queuedRequests) {
-                req->setFailure(error, errStr);
-            }
-
-            sentRequests.clear();
-            queuedRequests.clear();
-        }
-
-        NetChat::handleError(error);
-        if (activeRequest) {
-            activeRequest->setFailure(error, errStr);
-            activeRequest = NULL;
-            _contentDecoder.reset();
-        }
-
-        state = STATE_SOCKET_ERROR;
-    }
-
-    void handleTimeout()
-    {
-        handleError(ETIMEDOUT);
-    }
-
-    virtual void handleClose()
-    {
-        NetChat::handleClose();
-
-    // closing of the connection from the server side when getting the body,
-        bool canCloseState = (state == STATE_GETTING_BODY);
-        if (canCloseState && activeRequest) {
-        // force state here, so responseComplete can avoid closing the
-        // socket again
-            state =  STATE_CLOSED;
-            responseComplete();
-        } else {
-            if (state == STATE_WAITING_FOR_RESPONSE) {
-                assert(!sentRequests.empty());
-                sentRequests.front()->setFailure(500, "server closed connection unexpectedly");
-                // no active request, but don't restore the front sent one
-                sentRequests.erase(sentRequests.begin());
-            }
-
-            if (activeRequest) {
-                activeRequest->setFailure(500, "server closed connection");
-                // remove the failed request from sentRequests, so it does
-                // not get restored
-                RequestList::iterator it = std::find(sentRequests.begin(),
-                    sentRequests.end(), activeRequest);
-                if (it != sentRequests.end()) {
-                    sentRequests.erase(it);
-                }
-                activeRequest = NULL;
-                _contentDecoder.reset();
-            }
-
-            state = STATE_CLOSED;
-        }
-
-      if (sentRequests.empty()) {
-        return;
-      }
-
-    // restore sent requests to the queue, so they will be re-sent
-    // when the connection opens again
-      queuedRequests.insert(queuedRequests.begin(),
-                              sentRequests.begin(), sentRequests.end());
-      sentRequests.clear();
-    }
-
-    void queueRequest(const Request_ptr& r)
-    {
-        queuedRequests.push_back(r);
-        tryStartNextRequest();
-    }
-
-    void beginResponse()
-    {
-        assert(!sentRequests.empty());
-        assert(state == STATE_WAITING_FOR_RESPONSE);
-
-        activeRequest = sentRequests.front();
-        try {
-            activeRequest->responseStart(buffer);
-        } catch (sg_exception& e) {
-            handleError(EIO);
-            return;
-        }
-
-      state = STATE_GETTING_HEADERS;
-      buffer.clear();
-      if (activeRequest->responseCode() == 204) {
-        noMessageBody = true;
-      } else if (activeRequest->method() == "HEAD") {
-        noMessageBody = true;
-      } else {
-        noMessageBody = false;
-      }
-
-      bodyTransferSize = -1;
-      chunkedTransfer = false;
-      _contentDecoder.reset();
-    }
-
-    void tryStartNextRequest()
-    {
-      while( !queuedRequests.empty()
-          && queuedRequests.front()->isComplete() )
-        queuedRequests.pop_front();
-
-      if (queuedRequests.empty()) {
-        idleTime.stamp();
-        return;
-      }
-
-      if (sentRequests.size() > MAX_INFLIGHT_REQUESTS) {
-        return;
-      }
-
-      if (state == STATE_CLOSED) {
-          if (!connectToHost()) {
-
-              return;
-          }
-
-          setTerminator("\r\n");
-          state = STATE_IDLE;
-      }
-
-      Request_ptr r = queuedRequests.front();
-      r->requestStart();
-
-      std::stringstream headerData;
-      std::string path = r->path();
-      assert(!path.empty());
-      std::string query = r->query();
-      std::string bodyData;
-
-      if (!client->proxyHost().empty()) {
-          path = r->scheme() + "://" + r->host() + r->path();
-      }
-
-      if (r->bodyType() == CONTENT_TYPE_URL_ENCODED) {
-          headerData << r->method() << " " << path << " HTTP/1.1\r\n";
-          bodyData = query.substr(1); // URL-encode, drop the leading '?'
-          headerData << "Content-Type:" << CONTENT_TYPE_URL_ENCODED << "\r\n";
-          headerData << "Content-Length:" << bodyData.size() << "\r\n";
-      } else {
-          headerData << r->method() << " " << path << query << " HTTP/1.1\r\n";
-          if( r->hasBodyData() )
-          {
-            headerData << "Content-Length:" << r->bodyLength() << "\r\n";
-            headerData << "Content-Type:" << r->bodyType() << "\r\n";
-          }
-      }
-
-      headerData << "Host: " << r->hostAndPort() << "\r\n";
-      headerData << "User-Agent:" << client->userAgent() << "\r\n";
-      headerData << "Accept-Encoding: deflate, gzip\r\n";
-      if (!client->proxyAuth().empty()) {
-          headerData << "Proxy-Authorization: " << client->proxyAuth() << "\r\n";
-      }
-
-      BOOST_FOREACH(const StringMap::value_type& h, r->requestHeaders()) {
-          headerData << h.first << ": " << h.second << "\r\n";
-      }
-
-      headerData << "\r\n"; // final CRLF to terminate the headers
-      if (!bodyData.empty()) {
-          headerData << bodyData;
-      }
-
-      bool ok = push(headerData.str().c_str());
-      if (!ok) {
-          SG_LOG(SG_IO, SG_WARN, "HTTPClient: over-stuffed the socket");
-          // we've over-stuffed the socket, give up for now, let things
-          // drain down before trying to start any more requests.
-          return;
-      }
-
-      if( r->hasBodyData() )
-        for(size_t body_bytes_sent = 0; body_bytes_sent < r->bodyLength();)
-        {
-          char buf[4096];
-          size_t len = r->getBodyData(buf, body_bytes_sent, 4096);
-          if( len )
-          {
-            if( !bufferSend(buf, len) )
-            {
-              SG_LOG(SG_IO,
-                     SG_WARN,
-                     "overflow the HTTP::Connection output buffer");
-              state = STATE_SOCKET_ERROR;
-              return;
-            }
-            body_bytes_sent += len;
-          }
-          else
-          {
-            SG_LOG(SG_IO,
-                   SG_WARN,
-                   "HTTP asynchronous request body generation is unsupported");
-            break;
-          }
-        }
-
-        SG_LOG(SG_IO, SG_DEBUG, "con:" << connectionId << " did start request:" << r->url());
-      // successfully sent, remove from queue, and maybe send the next
-      queuedRequests.pop_front();
-      sentRequests.push_back(r);
-      state = STATE_WAITING_FOR_RESPONSE;
-
-      // pipelining, let's maybe send the next request right away
-      tryStartNextRequest();
-    }
-
-    virtual void collectIncomingData(const char* s, int n)
-    {
-        idleTime.stamp();
-        client->receivedBytes(static_cast<unsigned int>(n));
-
-        if(   (state == STATE_GETTING_BODY)
-           || (state == STATE_GETTING_CHUNKED_BYTES) )
-          _contentDecoder.receivedBytes(s, n);
-        else
-          buffer.append(s, n);
-    }
-
-    virtual void foundTerminator(void)
-    {
-        idleTime.stamp();
-        switch (state) {
-        case STATE_WAITING_FOR_RESPONSE:
-            beginResponse();
-            break;
-
-        case STATE_GETTING_HEADERS:
-            processHeader();
-            buffer.clear();
-            break;
-
-        case STATE_GETTING_BODY:
-            responseComplete();
-            break;
-
-        case STATE_GETTING_CHUNKED:
-            processChunkHeader();
-            break;
-
-        case STATE_GETTING_CHUNKED_BYTES:
-            setTerminator("\r\n");
-            state = STATE_GETTING_CHUNKED;
-            buffer.clear();
-            break;
-
-
-        case STATE_GETTING_TRAILER:
-            processTrailer();
-            buffer.clear();
-            break;
-
-        case STATE_IDLE:
-            SG_LOG(SG_IO, SG_WARN, "HTTP got data in IDLE state, bad server?");
-
-        default:
-            break;
-        }
-    }
-
-    bool hasIdleTimeout() const
-    {
-        if ((state != STATE_IDLE) && (state != STATE_CLOSED)) {
-            return false;
-        }
-
-        assert(sentRequests.empty());
-        bool isTimedOut = (idleTime.elapsedMSec() > (1000 * 10)); // 10 seconds
-        return isTimedOut;
-    }
-
-    bool hasErrorTimeout() const
-    {
-      if ((state == STATE_IDLE) || (state == STATE_CLOSED)) {
-        return false;
-      }
-
-        bool isTimedOut = (idleTime.elapsedMSec() > (1000 * 30)); // 30 seconds
-        return isTimedOut;
-    }
-
-    bool hasError() const
-    {
-        return (state == STATE_SOCKET_ERROR);
-    }
-
-    bool shouldStartNext() const
-    {
-      return !queuedRequests.empty() && (sentRequests.size() < MAX_INFLIGHT_REQUESTS);
-    }
-
-    bool isActive() const
-    {
-        return !queuedRequests.empty() || !sentRequests.empty();
-    }
-
-    void debugDumpRequests() const
-    {
-        SG_LOG(SG_IO, SG_DEBUG, "requests for:" << host << ":" << port << " (conId=" << connectionId
-               << "; state=" << state << ")");
-        if (activeRequest) {
-            SG_LOG(SG_IO, SG_DEBUG, "\tactive:" << activeRequest->url());
-        } else {
-            SG_LOG(SG_IO, SG_DEBUG, "\tNo active request");
-        }
-
-        BOOST_FOREACH(Request_ptr req, sentRequests) {
-            SG_LOG(SG_IO, SG_DEBUG, "\tsent:" << req->url());
-        }
-
-        BOOST_FOREACH(Request_ptr req, queuedRequests) {
-            SG_LOG(SG_IO, SG_DEBUG, "\tqueued:" << req->url());
-        }
-    }
-private:
-    bool connectToHost()
-    {
-        SG_LOG(SG_IO, SG_DEBUG, "HTTP connecting to " << host << ":" << port);
-
-        if (!open()) {
-            SG_LOG(SG_IO, SG_WARN, "HTTP::Connection: connectToHost: open() failed");
-            return false;
-        }
-
-        if (connect(host.c_str(), port) != 0) {
-            SG_LOG(SG_IO, SG_WARN, "HTTP::Connection: connectToHost: connect() failed");
-            return false;
-        }
-
-        return true;
-    }
-
-
-    void processHeader()
-    {
-        std::string h = strutils::simplify(buffer);
-        if (h.empty()) { // blank line terminates headers
-            headersComplete();
-            return;
-        }
-
-        int colonPos = buffer.find(':');
-        if (colonPos < 0) {
-            SG_LOG(SG_IO, SG_WARN, "malformed HTTP response header:" << h);
-            return;
-        }
-
-        std::string key = strutils::simplify(buffer.substr(0, colonPos));
-        std::string lkey = boost::to_lower_copy(key);
-        std::string value = strutils::strip(buffer.substr(colonPos + 1));
-
-        // only consider these if getting headers (as opposed to trailers
-        // of a chunked transfer)
-        if (state == STATE_GETTING_HEADERS) {
-            if (lkey == "content-length") {
-
-                int sz = strutils::to_int(value);
-                if (bodyTransferSize <= 0) {
-                    bodyTransferSize = sz;
-                }
-                activeRequest->setResponseLength(sz);
-            } else if (lkey == "transfer-length") {
-                bodyTransferSize = strutils::to_int(value);
-            } else if (lkey == "transfer-encoding") {
-                processTransferEncoding(value);
-            } else if (lkey == "content-encoding") {
-                _contentDecoder.setEncoding(value);
-            }
-        }
-
-        activeRequest->responseHeader(lkey, value);
-    }
-
-    void processTransferEncoding(const std::string& te)
-    {
-        if (te == "chunked") {
-            chunkedTransfer = true;
-        } else {
-            SG_LOG(SG_IO, SG_WARN, "unsupported transfer encoding:" << te);
-            // failure
-        }
-    }
-
-    void processChunkHeader()
-    {
-        if (buffer.empty()) {
-            // blank line after chunk data
-            return;
-        }
-
-        int chunkSize = 0;
-        int semiPos = buffer.find(';');
-        if (semiPos >= 0) {
-            // extensions ignored for the moment
-            chunkSize = strutils::to_int(buffer.substr(0, semiPos), 16);
-        } else {
-            chunkSize = strutils::to_int(buffer, 16);
-        }
-
-        buffer.clear();
-        if (chunkSize == 0) {  //  trailer start
-            state = STATE_GETTING_TRAILER;
-            return;
-        }
-
-        state = STATE_GETTING_CHUNKED_BYTES;
-        setByteCount(chunkSize);
-    }
-
-    void processTrailer()
-    {
-        if (buffer.empty()) {
-            // end of trailers
-            responseComplete();
-            return;
-        }
-
-    // process as a normal header
-        processHeader();
-    }
-
-    void headersComplete()
-    {
-        activeRequest->responseHeadersComplete();
-        _contentDecoder.initWithRequest(activeRequest);
-
-        if (chunkedTransfer) {
-            state = STATE_GETTING_CHUNKED;
-        } else if (noMessageBody || (bodyTransferSize == 0)) {
-            // force the state to GETTING_BODY, to simplify logic in
-            // responseComplete and handleClose
-            state = STATE_GETTING_BODY;
-            responseComplete();
-        } else {
-            setByteCount(bodyTransferSize); // may be -1, that's fine
-            state = STATE_GETTING_BODY;
-        }
-    }
-
-    void responseComplete()
-    {
-        Request_ptr completedRequest = activeRequest;
-        _contentDecoder.finish();
-
-        assert(sentRequests.front() == activeRequest);
-        sentRequests.pop_front();
-        bool doClose = activeRequest->closeAfterComplete();
-        activeRequest = NULL;
-
-        if ((state == STATE_GETTING_BODY) || (state == STATE_GETTING_TRAILER)) {
-            if (doClose) {
-          // this will bring us into handleClose() above, which updates
-          // state to STATE_CLOSED
-              close();
-
-          // if we have additional requests waiting, try to start them now
-              tryStartNextRequest();
-            }
-        }
-
-        if (state != STATE_CLOSED)  {
-            state = sentRequests.empty() ? STATE_IDLE : STATE_WAITING_FOR_RESPONSE;
-        }
-
-    // notify request after we change state, so this connection is idle
-    // if completion triggers other requests (which is likely)
-        completedRequest->responseComplete();
-        client->requestFinished(this);
-
-        setTerminator("\r\n");
-    }
-
-    enum ConnectionState {
-        STATE_IDLE = 0,
-        STATE_WAITING_FOR_RESPONSE,
-        STATE_GETTING_HEADERS,
-        STATE_GETTING_BODY,
-        STATE_GETTING_CHUNKED,
-        STATE_GETTING_CHUNKED_BYTES,
-        STATE_GETTING_TRAILER,
-        STATE_SOCKET_ERROR,
-        STATE_CLOSED             ///< connection should be closed now
-    };
-
-    Client* client;
-    Request_ptr activeRequest;
-    ConnectionState state;
-    std::string host;
-    short port;
-    std::string buffer;
-    int bodyTransferSize;
-    SGTimeStamp idleTime;
-    bool chunkedTransfer;
-    bool noMessageBody;
-
-    RequestList queuedRequests;
-    RequestList sentRequests;
-
-    ContentDecoder _contentDecoder;
-    std::string connectionId;
-};
-#endif // of !ENABLE_CURL
-
 Client::Client() :
     d(new ClientPrivate)
 {
     d->proxyPort = 0;
     d->maxConnections = 4;
+    d->maxHostConnections = 4;
     d->bytesTransferred = 0;
     d->lastTransferRate = 0;
     d->timeTransferSample.stamp();
     d->totalBytesDownloaded = 0;
-
+    d->maxPipelineDepth = 5;
     setUserAgent("SimGear-" SG_STRINGIZE(SIMGEAR_VERSION));
-#if defined(ENABLE_CURL)
+
     static bool didInitCurlGlobal = false;
     if (!didInitCurlGlobal) {
       curl_global_init(CURL_GLOBAL_ALL);
       didInitCurlGlobal = true;
     }
 
-    d->curlMulti = curl_multi_init();
-#endif
+    d->createCurlMulti();
 }
 
 Client::~Client()
 {
-#if defined(ENABLE_CURL)
   curl_multi_cleanup(d->curlMulti);
-#endif
 }
 
 void Client::setMaxConnections(unsigned int maxCon)
 {
-    if (maxCon < 1) {
-        throw sg_range_exception("illegal HTTP::Client::setMaxConnections value");
-    }
-
     d->maxConnections = maxCon;
-#if defined(ENABLE_CURL)
-    curl_multi_setopt(d->curlMulti, CURLMOPT_MAXCONNECTS, (long) maxCon);
-#endif
+    curl_multi_setopt(d->curlMulti, CURLMOPT_MAX_TOTAL_CONNECTIONS, (long) maxCon);
+}
+
+void Client::setMaxHostConnections(unsigned int maxHostCon)
+{
+    d->maxHostConnections = maxHostCon;
+    curl_multi_setopt(d->curlMulti, CURLMOPT_MAX_HOST_CONNECTIONS, (long) maxHostCon);
+}
+
+void Client::setMaxPipelineDepth(unsigned int depth)
+{
+    d->maxPipelineDepth = depth;
+    curl_multi_setopt(d->curlMulti, CURLMOPT_MAX_PIPELINE_LENGTH, (long) depth);
 }
 
 void Client::update(int waitTimeout)
 {
-#if defined(ENABLE_CURL)
     int remainingActive, messagesInQueue;
     curl_multi_perform(d->curlMulti, &remainingActive);
-    d->haveActiveRequests = (remainingActive > 0);
 
     CURLMsg* msg;
     while ((msg = curl_multi_info_read(d->curlMulti, &messagesInQueue))) {
       if (msg->msg == CURLMSG_DONE) {
-        Request* req;
+        Request* rawReq = 0;
         CURL *e = msg->easy_handle;
-        curl_easy_getinfo(e, CURLINFO_PRIVATE, &req);
+        curl_easy_getinfo(e, CURLINFO_PRIVATE, &rawReq);
+
+        // ensure request stays valid for the moment
+        // eg if responseComplete cancels us
+        Request_ptr req(rawReq);
 
         long responseCode;
         curl_easy_getinfo(e, CURLINFO_RESPONSE_CODE, &responseCode);
 
+          // remove from the requests map now,
+          // in case the callbacks perform a cancel. We'll use
+          // the absence from the request dict in cancel to avoid
+          // a double remove
+          ClientPrivate::RequestCurlMap::iterator it = d->requests.find(req);
+          assert(it != d->requests.end());
+          assert(it->second == e);
+          d->requests.erase(it);
+
         if (msg->data.result == 0) {
           req->responseComplete();
         } else {
-          fprintf(stderr, "Result: %d - %s\n",
-                msg->data.result, curl_easy_strerror(msg->data.result));
+          SG_LOG(SG_IO, SG_WARN, "CURL Result:" << msg->data.result << " " << curl_easy_strerror(msg->data.result));
           req->setFailure(msg->data.result, curl_easy_strerror(msg->data.result));
         }
 
         curl_multi_remove_handle(d->curlMulti, e);
-
-        // balance the reference we take in makeRequest
-        SGReferenced::put(req);
         curl_easy_cleanup(e);
-      }
-      else {
-        SG_LOG(SG_IO, SG_ALERT, "CurlMSG:" << msg->msg);
+      } else {
+          // should never happen since CURLMSG_DONE is the only code
+          // defined!
+          SG_LOG(SG_IO, SG_ALERT, "unknown CurlMSG:" << msg->msg);
       }
     } // of curl message processing loop
-#else
-    if (!d->poller.hasChannels() && (waitTimeout > 0)) {
-        SGTimeStamp::sleepForMSec(waitTimeout);
-    } else {
-        d->poller.poll(waitTimeout);
-    }
-
-    bool waitingRequests = !d->pendingRequests.empty();
-    ConnectionDict::iterator it = d->connections.begin();
-    for (; it != d->connections.end(); ) {
-        Connection* con = it->second;
-        if (con->hasIdleTimeout() ||
-            con->hasError() ||
-            con->hasErrorTimeout() ||
-            (!con->isActive() && waitingRequests))
-        {
-            if (con->hasErrorTimeout()) {
-                // tell the connection we're timing it out
-                con->handleTimeout();
-            }
-
-        // connection has been idle for a while, clean it up
-        // (or if we have requests waiting for a different host,
-        // or an error condition
-            ConnectionDict::iterator del = it++;
-            delete del->second;
-            d->connections.erase(del);
-        } else {
-            if (it->second->shouldStartNext()) {
-                it->second->tryStartNextRequest();
-            }
-            ++it;
-        }
-    } // of connection iteration
-
-    if (waitingRequests && (d->connections.size() < d->maxConnections)) {
-        RequestList waiting(d->pendingRequests);
-        d->pendingRequests.clear();
-
-        // re-submit all waiting requests in order; this takes care of
-        // finding multiple pending items targetted to the same (new)
-        // connection
-        BOOST_FOREACH(Request_ptr req, waiting) {
-            makeRequest(req);
-        }
-    }
-#endif
+    SGTimeStamp::sleepForMSec(waitTimeout);
 }
 
 void Client::makeRequest(const Request_ptr& r)
@@ -802,12 +209,16 @@ void Client::makeRequest(const Request_ptr& r)
         return;
     }
 
-#if defined(ENABLE_CURL)
+    r->_client = this;
+
+    ClientPrivate::RequestCurlMap::iterator rit = d->requests.find(r);
+    assert(rit == d->requests.end());
+
     CURL* curlRequest = curl_easy_init();
     curl_easy_setopt(curlRequest, CURLOPT_URL, r->url().c_str());
 
-    // manually increase the ref count of the request
-    SGReferenced::get(r.get());
+    d->requests[r] = curlRequest;
+
     curl_easy_setopt(curlRequest, CURLOPT_PRIVATE, r.get());
     // disable built-in libCurl progress feedback
     curl_easy_setopt(curlRequest, CURLOPT_NOPROGRESS, 1);
@@ -818,6 +229,7 @@ void Client::makeRequest(const Request_ptr& r)
     curl_easy_setopt(curlRequest, CURLOPT_HEADERDATA, r.get());
 
     curl_easy_setopt(curlRequest, CURLOPT_USERAGENT, d->userAgent.c_str());
+    curl_easy_setopt(curlRequest, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
 
     if (!d->proxy.empty()) {
       curl_easy_setopt(curlRequest, CURLOPT_PROXY, d->proxy.c_str());
@@ -872,78 +284,31 @@ void Client::makeRequest(const Request_ptr& r)
     }
 
     curl_multi_add_handle(d->curlMulti, curlRequest);
-    d->haveActiveRequests = true;
 
-// FIXME - premature?
+// this seems premature, but we don't have a callback from Curl we could
+// use to trigger when the requst is actually sent.
     r->requestStart();
+}
 
-#else
-    if( r->url().find("http://") != 0 ) {
-        r->setFailure(EINVAL, "only HTTP protocol is supported");
+void Client::cancelRequest(const Request_ptr &r, std::string reason)
+{
+    ClientPrivate::RequestCurlMap::iterator it = d->requests.find(r);
+    if(it == d->requests.end()) {
+        // already being removed, presumably inside ::update()
+        // nothing more to do
         return;
     }
 
-    std::string host = r->host();
-    int port = r->port();
-    if (!d->proxy.empty()) {
-        host = d->proxy;
-        port = d->proxyPort;
-    }
+    CURLMcode err = curl_multi_remove_handle(d->curlMulti, it->second);
+    assert(err == CURLM_OK);
 
-    Connection* con = NULL;
-    std::stringstream ss;
-    ss << host << "-" << port;
-    std::string connectionId = ss.str();
-    bool havePending = !d->pendingRequests.empty();
-    bool atConnectionsLimit = d->connections.size() >= d->maxConnections;
-    ConnectionDict::iterator consEnd = d->connections.end();
+    // clear the request pointer form the curl-easy object
+    curl_easy_setopt(it->second, CURLOPT_PRIVATE, 0);
 
-    // assign request to an existing Connection.
-    // various options exist here, examined in order
-    ConnectionDict::iterator it = d->connections.find(connectionId);
-    if (atConnectionsLimit && (it == consEnd)) {
-        // maximum number of connections active, queue this request
-        // when a connection goes inactive, we'll start this one
-        d->pendingRequests.push_back(r);
-        return;
-    }
+    curl_easy_cleanup(it->second);
+    d->requests.erase(it);
 
-    // scan for an idle Connection to the same host (likely if we're
-    // retrieving multiple resources from the same host in quick succession)
-    // if we have pending requests (waiting for a free Connection), then
-    // force new requests on this id to always use the first Connection
-    // (instead of the random selection below). This ensures that when
-    // there's pressure on the number of connections to keep alive, one
-    // host can't DoS every other.
-    int count = 0;
-    for (; (it != consEnd) && (it->first == connectionId); ++it, ++count) {
-        if (havePending || !it->second->isActive()) {
-            con = it->second;
-            break;
-        }
-    }
-
-    if (!con && atConnectionsLimit) {
-        // all current connections are busy (active), and we don't
-        // have free connections to allocate, so let's assign to
-        // an existing one randomly. Ideally we'd used whichever one will
-        // complete first but we don't have that info.
-        int index = rand() % count;
-        for (it = d->connections.find(connectionId); index > 0; --index) { ; }
-        con = it->second;
-    }
-
-    // allocate a new connection object
-    if (!con) {
-        con = new Connection(this, connectionId);
-        con->setServer(host, port);
-        d->poller.addChannel(con);
-        d->connections.insert(d->connections.end(),
-            ConnectionDict::value_type(connectionId, con));
-    }
-
-    con->queueRequest(r);
-#endif
+    r->setFailure(-1, reason);
 }
 
 //------------------------------------------------------------------------------
@@ -999,16 +364,7 @@ void Client::setProxy( const std::string& proxy,
 
 bool Client::hasActiveRequests() const
 {
-  #if defined(ENABLE_CURL)
-    return d->haveActiveRequests;
-  #else
-    ConnectionDict::const_iterator it = d->connections.begin();
-    for (; it != d->connections.end(); ++it) {
-        if (it->second->isActive()) return true;
-    }
-
-    return false;
-#endif
+    return !d->requests.empty();
 }
 
 void Client::receivedBytes(unsigned int count)
@@ -1051,9 +407,14 @@ uint64_t Client::totalBytesDownloaded() const
 size_t Client::requestWriteCallback(char *ptr, size_t size, size_t nmemb, void *userdata)
 {
   size_t byteSize = size * nmemb;
-
   Request* req = static_cast<Request*>(userdata);
   req->processBodyBytes(ptr, byteSize);
+
+  Client* cl = req->http();
+  if (cl) {
+    cl->receivedBytes(byteSize);
+  }
+
   return byteSize;
 }
 
@@ -1107,16 +468,18 @@ size_t Client::requestHeaderCallback(char *rawBuffer, size_t size, size_t nitems
 
 void Client::debugDumpRequests()
 {
-#if defined(ENABLE_CURL)
-
-#else
-    SG_LOG(SG_IO, SG_INFO, "== HTTP connection dump");
-    ConnectionDict::iterator it = d->connections.begin();
-    for (; it != d->connections.end(); ++it) {
-        it->second->debugDumpRequests();
+    SG_LOG(SG_IO, SG_INFO, "== HTTP request dump");
+    ClientPrivate::RequestCurlMap::iterator it = d->requests.begin();
+    for (; it != d->requests.end(); ++it) {
+        SG_LOG(SG_IO, SG_INFO, "\t" << it->first->url());
     }
     SG_LOG(SG_IO, SG_INFO, "==");
-#endif
+}
+
+void Client::clearAllConnections()
+{
+    curl_multi_cleanup(d->curlMulti);
+    d->createCurlMulti();
 }
 
 } // of namespace HTTP

simgear/io/HTTPClient.hxx

@@ -49,6 +49,8 @@ public:
 
     void makeRequest(const Request_ptr& r);
 
+    void cancelRequest(const Request_ptr& r, std::string reason = std::string());
+
     /**
      * Download a resource and save it to a file.
      *
@@ -75,6 +77,13 @@ public:
      */
     void setMaxConnections(unsigned int maxCons);
 
+    void setMaxHostConnections(unsigned int maxHostConns);
+
+    /**
+     * maximum depth to pipeline requests - set to 0 to disable pipelining
+     */
+    void setMaxPipelineDepth(unsigned int depth);
+
     const std::string& userAgent() const;
 
     const std::string& proxyHost() const;
@@ -100,6 +109,8 @@ public:
     uint64_t totalBytesDownloaded() const;
 
     void debugDumpRequests();
+
+    void clearAllConnections();
 private:
     // libCurl callbacks
     static size_t requestWriteCallback(char *ptr, size_t size, size_t nmemb, void *userdata);

simgear/io/HTTPContentDecode.cxx

@@ -1,269 +0,0 @@
-// Written by James Turner
-//
-// Copyright (C) 2013  James Turner  <zakalawe@mac.com>
-//
-// This library is free software; you can redistribute it and/or
-// modify it under the terms of the GNU Library General Public
-// License as published by the Free Software Foundation; either
-// version 2 of the License, or (at your option) any later version.
-//
-// This library is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-// Library General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License
-// along with this program; if not, write to the Free Software
-// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
-//
-
-#include "HTTPContentDecode.hxx"
-
-#include <cassert>
-#include <cstdlib> // rand()
-#include <cstring> // for memset, memcpy
-
-#include <simgear/debug/logstream.hxx>
-#include <simgear/structure/exception.hxx>
-#include <simgear/io/lowlevel.hxx> // for sgEndian stuff
-
-namespace simgear
-{
-
-namespace HTTP
-{
-
-    const int ZLIB_DECOMPRESS_BUFFER_SIZE = 32 * 1024;
-    const int ZLIB_INFLATE_WINDOW_BITS = -MAX_WBITS;
-  
-    // see http://www.ietf.org/rfc/rfc1952.txt for these values and
-    // detailed description of the logic 
-    const int GZIP_HEADER_ID1 = 31;
-    const int GZIP_HEADER_ID2 = 139;
-    const int GZIP_HEADER_METHOD_DEFLATE = 8;
-    const unsigned int GZIP_HEADER_SIZE = 10;
-    const int GZIP_HEADER_FEXTRA = 1 << 2;
-    const int GZIP_HEADER_FNAME = 1 << 3;
-    const int GZIP_HEADER_COMMENT = 1 << 4;
-    const int GZIP_HEADER_CRC = 1 << 1;
-    
-ContentDecoder::ContentDecoder() :
-    _output(NULL),
-    _zlib(NULL),
-    _input(NULL),
-    _inputAllocated(0),
-    _inputSize(0)
-{
-}
-
-ContentDecoder::~ContentDecoder()
-{
-    free(_output);
-    free(_input);
-    free(_zlib);
-}
-
-void ContentDecoder::setEncoding(const std::string& encoding)
-{
-    if (encoding == "gzip") {
-      _contentDeflate = true;
-      _needGZipHeader = true;
-    } else if (encoding == "deflate") {
-      _contentDeflate = true;
-      _needGZipHeader = false;
-    } else if (encoding != "identity") {
-      SG_LOG(SG_IO, SG_WARN, "unsupported content encoding:" << encoding);
-    }
-}
-
-void ContentDecoder::reset()
-{
-    _request = NULL;
-    _contentDeflate = false;
-    _needGZipHeader = false;
-    _inputSize = 0;
-}
-
-void ContentDecoder::initWithRequest(Request_ptr req)
-{
-    _request = req;
-    if (!_contentDeflate) {
-        return;
-    }
-
-    if (!_zlib) {
-        _zlib = (z_stream*) malloc(sizeof(z_stream));
-    }
-    
-    memset(_zlib, 0, sizeof(z_stream));
-    if (!_output) {
-         _output = (unsigned char*) malloc(ZLIB_DECOMPRESS_BUFFER_SIZE);
-    }
-    
-    _inputSize = 0;
-    // NULLs means we'll get default alloc+free methods
-    // which is absolutely fine
-    _zlib->avail_out = ZLIB_DECOMPRESS_BUFFER_SIZE;
-    _zlib->next_out = _output;
-    if (inflateInit2(_zlib, ZLIB_INFLATE_WINDOW_BITS) != Z_OK) {
-      SG_LOG(SG_IO, SG_WARN, "inflateInit2 failed");
-    }
-}
-
-void ContentDecoder::finish()
-{    
-    if (_contentDeflate) {
-        runDecoder();
-      inflateEnd(_zlib);
-    }
-}
-
-void ContentDecoder::receivedBytes(const char* n, size_t s)
-{
-    if (!_contentDeflate) {
-        _request->processBodyBytes(n, s);
-        return;
-    }    
-    
-// allocate more space if needed (this will only happen rarely once the
-// buffer has hit something proportionate to the server's compression
-// window size)
-    size_t requiredSize = _inputSize + s;
-    if (requiredSize > _inputAllocated) {
-        reallocateInputBuffer(requiredSize);
-    }
-    
-// copy newly recieved bytes into the buffer
-    memcpy(_input + _inputSize, n, s);
-    _inputSize += s;
-    
-    if (_needGZipHeader && !consumeGZipHeader()) {
-        // still waiting on the full GZIP header, so done
-        return;
-    }
-    
-    runDecoder();
-}
-
-void ContentDecoder::consumeBytes(size_t consumed)
-{    
-    assert(_inputSize >= consumed);
-// move existing (consumed) bytes down
-    if (consumed > 0) {
-        size_t newSize = _inputSize - consumed;
-        memmove(_input, _input + consumed, newSize);
-        _inputSize = newSize;
-    }
-}
-
-void ContentDecoder::reallocateInputBuffer(size_t newSize)
-{    
-    _input = (unsigned char*) realloc(_input, newSize);
-    _inputAllocated = newSize;
-}
-
-void ContentDecoder::runDecoder()
-{
-    _zlib->next_in = (unsigned char*) _input;
-    _zlib->avail_in = _inputSize;
-    int writtenSize;
-    
-    // loop, running zlib() inflate and sending output bytes to
-    // our request body handler. Keep calling inflate until no bytes are
-    // written, and ZLIB has consumed all available input
-    do {
-        _zlib->next_out = _output;
-        _zlib->avail_out = ZLIB_DECOMPRESS_BUFFER_SIZE;
-      int result = inflate(_zlib, Z_NO_FLUSH);
-      if (result == Z_OK || result == Z_STREAM_END) {
-          // nothing to do
-      } else if (result == Z_BUF_ERROR) {
-          // transient error, fall through
-      } else {
-        //  _error = result;          
-        return;
-      }
-          
-      writtenSize = ZLIB_DECOMPRESS_BUFFER_SIZE - _zlib->avail_out;      
-      if (writtenSize > 0) {
-          _request->processBodyBytes((char*) _output, writtenSize);
-      }
-      
-      if (result == Z_STREAM_END) {
-          break;
-      }
-    } while ((_zlib->avail_in > 0) || (writtenSize > 0));
-    
-    // update input buffers based on what we consumed
-    consumeBytes(_inputSize - _zlib->avail_in);
-}
-
-bool ContentDecoder::consumeGZipHeader()
-{    
-    size_t avail = _inputSize;
-    if (avail < GZIP_HEADER_SIZE) {
-      return false; // need more header bytes
-    }
-    
-    if ((_input[0] != GZIP_HEADER_ID1) ||
-        (_input[1] != GZIP_HEADER_ID2) ||
-        (_input[2] != GZIP_HEADER_METHOD_DEFLATE))
-    {
-      return false; // invalid GZip header
-    }
-    
-    char flags = _input[3];
-    unsigned int gzipHeaderSize =  GZIP_HEADER_SIZE;
-    if (flags & GZIP_HEADER_FEXTRA) {
-      gzipHeaderSize += 2;
-      if (avail < gzipHeaderSize) {
-        return false; // need more header bytes
-      }
-      
-      unsigned short extraHeaderBytes = *(reinterpret_cast<unsigned short*>(_input + GZIP_HEADER_FEXTRA));
-      if ( sgIsBigEndian() ) {
-          sgEndianSwap( &extraHeaderBytes );
-      }
-      
-      gzipHeaderSize += extraHeaderBytes;
-      if (avail < gzipHeaderSize) {
-        return false; // need more header bytes
-      }
-    }
-
-#if 0
-    if (flags & GZIP_HEADER_FNAME) {
-      gzipHeaderSize++;
-      while (gzipHeaderSize <= avail) {
-        if (_input[gzipHeaderSize-1] == 0) {
-          break; // found terminating NULL character
-        }
-      }
-    }
-    
-    if (flags & GZIP_HEADER_COMMENT) {
-      gzipHeaderSize++;
-      while (gzipHeaderSize <= avail) {
-        if (_input[gzipHeaderSize-1] == 0) {
-          break; // found terminating NULL character
-        }
-      }
-    }
-#endif
-        
-    if (flags & GZIP_HEADER_CRC) {
-      gzipHeaderSize += 2;
-    }
-    
-    if (avail < gzipHeaderSize) {
-      return false; // need more header bytes
-    }
-    
-    consumeBytes(gzipHeaderSize);
-    _needGZipHeader = false;
-    return true;
-}
-
-} // of namespace HTTP
-
-} // of namespace simgear

simgear/io/HTTPContentDecode.hxx

@@ -1,72 +0,0 @@
-// Written by James Turner
-//
-// Copyright (C) 2013  James Turner  <zakalawe@mac.com>
-//
-// This library is free software; you can redistribute it and/or
-// modify it under the terms of the GNU Library General Public
-// License as published by the Free Software Foundation; either
-// version 2 of the License, or (at your option) any later version.
-//
-// This library is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-// Library General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License
-// along with this program; if not, write to the Free Software
-// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
-//
-
-#ifndef SG_HTTP_CONTENT_DECODER_HXX
-#define SG_HTTP_CONTENT_DECODER_HXX
-
-#include <string>
-
-#include <zlib.h>
-
-#include <simgear/io/HTTPRequest.hxx>
-
-
-namespace simgear
-{
-
-namespace HTTP
-{
-
-class ContentDecoder
-{
-public:
-    ContentDecoder();
-    ~ContentDecoder();
-    
-    void reset();
-    
-    void initWithRequest(Request_ptr req);
-    
-    void finish();
-    
-    void setEncoding(const std::string& encoding);
-    
-    void receivedBytes(const char* n, size_t s);
-
-private:
-    bool consumeGZipHeader();
-    void runDecoder();
-    
-    void consumeBytes(size_t consumed);
-    void reallocateInputBuffer(size_t newSize);
-    
-    Request_ptr _request;
-    unsigned char* _output;
-    
-    z_stream* _zlib;
-    unsigned char* _input;
-    size_t _inputAllocated, _inputSize;
-    bool _contentDeflate, _needGZipHeader;
-};
-
-} // of namespace HTTP
-
-} // of namespace simgear
-
-#endif // of SG_HTTP_CONTENT_DECODER_HXX

simgear/io/HTTPFileRequest.cxx

@@ -59,8 +59,6 @@ namespace HTTP
         SG_WARN,
         "HTTP::FileRequest: failed to open file '" << _filename << "'"
       );
-
-      abort("Failed to open file.");
     }
   }
 

simgear/io/HTTPRepository.hxx

@@ -0,0 +1,61 @@
+// HTTPRepository.hxx - plain HTTP TerraSync remote server client
+//
+// Copyright (C) 2016  James Turner <zakalawe@mac.com>
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License as
+// published by the Free Software Foundation; either version 2 of the
+// License, or (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful, but
+// WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+
+
+#ifndef SG_IO_HTTP_REPOSITORY_HXX
+#define SG_IO_HTTP_REPOSITORY_HXX
+
+#include <simgear/io/AbstractRepository.hxx>
+#include <memory>
+
+namespace simgear  {
+
+class HTTPRepoPrivate;
+
+class HTTPRepository : public AbstractRepository
+{
+public:
+
+    HTTPRepository(const SGPath& root, HTTP::Client* cl);
+    virtual ~HTTPRepository();
+
+    virtual SGPath fsBase() const;
+
+    virtual void setBaseUrl(const std::string& url);
+    virtual std::string baseUrl() const;
+
+    virtual HTTP::Client* http() const;
+
+    virtual void update();
+
+    virtual bool isDoingSync() const;
+
+    virtual ResultCode failure() const;
+
+    virtual size_t bytesToDownload() const;
+
+    virtual size_t bytesDownloaded() const;
+private:
+    bool isBare() const;
+
+    std::auto_ptr<HTTPRepoPrivate> _d;
+};
+
+} // of namespace simgear
+
+#endif // of HTTPRepository

simgear/io/HTTPRequest.cxx

@@ -32,6 +32,7 @@ extern const int DEFAULT_HTTP_PORT;
 
 //------------------------------------------------------------------------------
 Request::Request(const std::string& url, const std::string method):
+  _client(0),
   _method(method),
   _url(url),
   _responseVersion(HTTP_VERSION_UNKNOWN),
@@ -39,7 +40,8 @@ Request::Request(const std::string& url, const std::string method):
   _responseLength(0),
   _receivedBodyBytes(0),
   _ready_state(UNSENT),
-  _willClose(false)
+  _willClose(false),
+  _connectionCloseHeader(false)
 {
 
 }
@@ -147,7 +149,10 @@ void Request::responseStart(const std::string& r)
 void Request::responseHeader(const std::string& key, const std::string& value)
 {
   if( key == "connection" ) {
-    _willClose = (value.find("close") != std::string::npos);
+    _connectionCloseHeader = (value.find("close") != std::string::npos);
+      // track willClose seperately because other conditions (abort, for
+      // example) can also set it
+    _willClose = _connectionCloseHeader;
   } else if (key == "content-length") {
     int sz = strutils::to_int(value);
     setResponseLength(sz);
@@ -360,19 +365,6 @@ void Request::setReadyState(ReadyState state)
   _cb_always(this);
 }
 
-//------------------------------------------------------------------------------
-void Request::abort()
-{
-  abort("Request aborted.");
-}
-
-//----------------------------------------------------------------------------
-void Request::abort(const std::string& reason)
-{
-  setFailure(-1, reason);
-  _willClose = true;
-}
-
 //------------------------------------------------------------------------------
 bool Request::closeAfterComplete() const
 {
@@ -380,6 +372,19 @@ bool Request::closeAfterComplete() const
   return _willClose || (_responseVersion != HTTP_1_1);
 }
 
+//------------------------------------------------------------------------------
+
+void Request::setCloseAfterComplete()
+{
+    _willClose = true;
+}
+
+//------------------------------------------------------------------------------
+bool Request::serverSupportsPipelining() const
+{
+    return (_responseVersion == HTTP_1_1) && !_connectionCloseHeader;
+}
+
 //------------------------------------------------------------------------------
 bool Request::isComplete() const
 {

simgear/io/HTTPRequest.hxx

@@ -37,6 +37,8 @@ namespace simgear
 namespace HTTP
 {
 
+class Client;
+
 /**
  * Base class for HTTP request (and answer).
  */
@@ -131,6 +133,9 @@ public:
     virtual std::string url() const
         { return _url; }
 
+    Client* http() const
+    { return _client; }
+
     virtual std::string scheme() const;
     virtual std::string path() const;
     virtual std::string host() const;
@@ -194,19 +199,16 @@ public:
 
     ReadyState readyState() const { return _ready_state; }
 
-    /**
-     * Request aborting this request.
-     */
-    void abort();
-
-    /**
-     * Request aborting this request and specify the reported reaseon for it.
-     */
-    void abort(const std::string& reason);
-
     bool closeAfterComplete() const;
     bool isComplete() const;
 
+    /**
+     * Check if the server response indicates pipelining should be continued.
+     * Currently tests that HTTP_1_1 is explicitly supported, and that the
+     * server/proxy did not request Connection: close
+     */
+    bool serverSupportsPipelining() const;
+
 protected:
     Request(const std::string& url, const std::string method = "GET");
 
@@ -234,6 +236,10 @@ private:
     void processBodyBytes(const char* s, int n);
     void setReadyState(ReadyState state);
 
+    void setCloseAfterComplete();
+
+    Client*       _client; // HTTP client we're active on
+
     std::string   _method;
     std::string   _url;
     StringMap     _request_headers;
@@ -253,6 +259,7 @@ private:
 
     ReadyState    _ready_state;
     bool          _willClose;
+    bool          _connectionCloseHeader;
 };
 
 typedef SGSharedPtr<Request> Request_ptr;

simgear/io/SVNDirectory.cxx

@@ -136,7 +136,7 @@ void SVNDirectory::parseCache()
         _cachedRevision = versionName;
         doneSelf = true;
       } else {
-        DAVResource* child = addChildDirectory(hrefPtr)->collection();
+        DAVResource* child = parseChildDirectory(hrefPtr)->collection();
           string s = strutils::strip(versionName);
           if (!s.empty()) {
               child->setVersionName(versionName);
@@ -235,6 +235,17 @@ SVNDirectory::addChildDirectory(const std::string& dirName)
     return child;
 }
 
+SVNDirectory*
+SVNDirectory::parseChildDirectory(const std::string& dirName)
+{
+    assert(!dav->childWithName(dirName));
+    DAVCollection* childCol = dav->createChildCollection(dirName);
+    SVNDirectory* child = new SVNDirectory(this, childCol);
+    childCol->setVersionName(child->cachedRevision());
+    _children.push_back(child);
+    return child;
+}
+
 void SVNDirectory::deleteChildByName(const std::string& nm)
 {
     DAVResource* child = dav->childWithName(nm);

simgear/io/SVNDirectory.hxx

@@ -91,7 +91,8 @@ private:
   void writeCache();
   
   DirectoryList::iterator findChildDir(const std::string& dirName);
-      
+  SVNDirectory* parseChildDirectory(const std::string& dirName);
+
   SGPath localPath;
   DAVCollection* dav;
   SVNRepository* repo;

simgear/io/SVNReportParser.cxx

@@ -244,7 +244,7 @@ class SVNReportParser::SVNReportParserPrivate
 public:
   SVNReportParserPrivate(SVNRepository* repo) :
     tree(repo),
-    status(SVNRepository::SVN_NO_ERROR),
+    status(AbstractRepository::REPO_NO_ERROR),
     parserInited(false),
     currentPath(repo->fsBase())
   {
@@ -258,7 +258,7 @@ public:
   
   void startElement (const char * name, const char** attributes)
   {    
-      if (status != SVNRepository::SVN_NO_ERROR) {
+      if (status != AbstractRepository::REPO_NO_ERROR) {
           return;
       }
       
@@ -277,7 +277,7 @@ public:
       currentPath = filePath;
        
       if (!filePath.exists()) {
-          fail(SVNRepository::SVN_ERROR_FILE_NOT_FOUND);
+          fail(AbstractRepository::REPO_ERROR_FILE_NOT_FOUND);
           return;
       }
 
@@ -386,7 +386,7 @@ public:
   
   void endElement (const char * name)
   {
-      if (status != SVNRepository::SVN_NO_ERROR) {
+      if (status != SVNRepository::REPO_NO_ERROR) {
           return;
       }
       
@@ -420,7 +420,7 @@ public:
     } else if (!strcmp(name, SVN_DAV_MD5_CHECKSUM)) {
       // validate against (presumably) just written file
       if (decodedFileMd5 != md5Sum) {
-        fail(SVNRepository::SVN_ERROR_CHECKSUM);
+        fail(SVNRepository::REPO_ERROR_CHECKSUM);
       }
     } else if (!strcmp(name, SVN_OPEN_DIRECTORY_TAG)) {
         currentDir->updateReportComplete();
@@ -443,7 +443,7 @@ public:
   
   void data (const char * s, int length)
   {
-      if (status != SVNRepository::SVN_NO_ERROR) {
+      if (status != SVNRepository::REPO_NO_ERROR) {
           return;
       }
       
@@ -544,7 +544,7 @@ SVNReportParser::~SVNReportParser()
 SVNRepository::ResultCode
 SVNReportParser::innerParseXML(const char* data, int size)
 {
-    if (_d->status != SVNRepository::SVN_NO_ERROR) {
+    if (_d->status != SVNRepository::REPO_NO_ERROR) {
         return _d->status;
     }
     
@@ -568,7 +568,7 @@ SVNReportParser::innerParseXML(const char* data, int size)
 SVNRepository::ResultCode
 SVNReportParser::parseXML(const char* data, int size)
 {
-    if (_d->status != SVNRepository::SVN_NO_ERROR) {
+    if (_d->status != SVNRepository::REPO_NO_ERROR) {
         return _d->status;
     }
     
@@ -586,7 +586,7 @@ SVNReportParser::parseXML(const char* data, int size)
 
 SVNRepository::ResultCode SVNReportParser::finishParse()
 {
-    if (_d->status != SVNRepository::SVN_NO_ERROR) {
+    if (_d->status != SVNRepository::REPO_NO_ERROR) {
         return _d->status;
     }
     

simgear/io/SVNRepository.cxx

@@ -26,6 +26,7 @@
 #include <map>
 #include <set>
 #include <fstream>
+#include <memory>
 
 #include <boost/foreach.hpp>
 
@@ -51,12 +52,12 @@ typedef std::vector<HTTP::Request_ptr> RequestVector;
 class SVNRepoPrivate
 {
 public:
-    SVNRepoPrivate(SVNRepository* parent) : 
-        p(parent), 
+    SVNRepoPrivate(SVNRepository* parent) :
+        p(parent),
         isUpdating(false),
-        status(SVNRepository::SVN_NO_ERROR)
+        status(SVNRepository::REPO_NO_ERROR)
     { ; }
-    
+
     SVNRepository* p; // link back to outer
     SVNDirectory* rootCollection;
     HTTP::Client* http;
@@ -65,12 +66,12 @@ public:
     std::string targetRevision;
     bool isUpdating;
     SVNRepository::ResultCode status;
-    
+
     void svnUpdateDone()
     {
         isUpdating = false;
     }
-    
+
     void updateFailed(HTTP::Request* req, SVNRepository::ResultCode err)
     {
         SG_LOG(SG_TERRASYNC, SG_WARN, "SVN: failed to update from:" << req->url()
@@ -78,29 +79,29 @@ public:
         isUpdating = false;
         status = err;
     }
-      
+
     void propFindComplete(HTTP::Request* req, DAVCollection* col);
     void propFindFailed(HTTP::Request* req, SVNRepository::ResultCode err);
 };
 
 
 namespace { // anonmouse
-    
+
     string makeAbsoluteUrl(const string& url, const string& base)
     {
       if (strutils::starts_with(url, "http://"))
         return url; // already absolute
-  
+
       assert(strutils::starts_with(base, "http://"));
       int schemeEnd = base.find("://");
       int hostEnd = base.find('/', schemeEnd + 3);
       if (hostEnd < 0) {
         return url;
       }
-  
+
       return base.substr(0, hostEnd) + url;
     }
-    
+
     // keep the responses small by only requesting the properties we actually
     // care about; the ETag, length and MD5-sum
     const char* PROPFIND_REQUEST_BODY =
@@ -132,17 +133,17 @@ namespace { // anonmouse
         if (responseCode() == 207) {
             // fine
         } else if (responseCode() == 404) {
-            _repo->propFindFailed(this, SVNRepository::SVN_ERROR_NOT_FOUND);
+            _repo->propFindFailed(this, SVNRepository::REPO_ERROR_NOT_FOUND);
         } else {
             SG_LOG(SG_TERRASYNC, SG_WARN, "request for:" << url() <<
                 " return code " << responseCode());
-            _repo->propFindFailed(this, SVNRepository::SVN_ERROR_SOCKET);
+            _repo->propFindFailed(this, SVNRepository::REPO_ERROR_SOCKET);
             _repo = NULL;
         }
 
         Request::responseHeadersComplete();
       }
-  
+
       virtual void onDone()
       {
         if (responseCode() == 207) {
@@ -150,11 +151,11 @@ namespace { // anonmouse
           if (_davStatus.isValid()) {
               _repo->propFindComplete(this, (DAVCollection*) _davStatus.resource());
           } else {
-              _repo->propFindFailed(this, SVNRepository::SVN_ERROR_SOCKET);
+              _repo->propFindFailed(this, SVNRepository::REPO_ERROR_SOCKET);
           }
         }
       }
-  
+
       virtual void gotBodyData(const char* s, int n)
       {
         if (responseCode() != 207) {
@@ -162,16 +163,16 @@ namespace { // anonmouse
         }
         _davStatus.parseXML(s, n);
       }
-        
+
         virtual void onFail()
         {
             HTTP::Request::onFail();
 			if (_repo) {
-				_repo->propFindFailed(this, SVNRepository::SVN_ERROR_SOCKET);
+				_repo->propFindFailed(this, SVNRepository::REPO_ERROR_SOCKET);
 				_repo = NULL;
 			}
         }
-        
+
     private:
       SVNRepoPrivate* _repo;
       DAVMultiStatus _davStatus;
@@ -181,14 +182,14 @@ class UpdateReportRequest:
   public HTTP::Request
 {
 public:
-  UpdateReportRequest(SVNRepoPrivate* repo, 
+  UpdateReportRequest(SVNRepoPrivate* repo,
       const std::string& aVersionName,
       bool startEmpty) :
     HTTP::Request("", "REPORT"),
     _parser(repo->p),
     _repo(repo),
     _failed(false)
-  {       
+  {
     setUrl(repo->vccUrl);
     std::string request =
     "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n"
@@ -218,7 +219,7 @@ protected:
       if (_failed) {
           return;
       }
-      
+
     if (responseCode() == 200) {
           SVNRepository::ResultCode err = _parser.finishParse();
           if (err) {
@@ -228,26 +229,26 @@ protected:
               _repo->svnUpdateDone();
           }
     } else if (responseCode() == 404) {
-        _repo->updateFailed(this, SVNRepository::SVN_ERROR_NOT_FOUND);
+        _repo->updateFailed(this, SVNRepository::REPO_ERROR_NOT_FOUND);
         _failed = true;
     } else {
         SG_LOG(SG_TERRASYNC, SG_WARN, "SVN: request for:" << url() <<
         " got HTTP status " << responseCode());
-        _repo->updateFailed(this, SVNRepository::SVN_ERROR_HTTP);
+        _repo->updateFailed(this, SVNRepository::REPO_ERROR_HTTP);
         _failed = true;
     }
   }
 
   virtual void gotBodyData(const char* s, int n)
-  {    
+  {
       if (_failed) {
           return;
       }
-      
+
     if (responseCode() != 200) {
         return;
     }
-    
+
     SVNRepository::ResultCode err = _parser.parseXML(s, n);
     if (err) {
         _failed = true;
@@ -261,7 +262,7 @@ protected:
     {
         HTTP::Request::onFail();
         if (_repo) {
-            _repo->updateFailed(this, SVNRepository::SVN_ERROR_SOCKET);
+            _repo->updateFailed(this, SVNRepository::REPO_ERROR_SOCKET);
             _repo = NULL;
         }
     }
@@ -270,15 +271,15 @@ private:
   SVNRepoPrivate* _repo;
   bool _failed;
 };
-        
-} // anonymous 
+
+} // anonymous
 
 SVNRepository::SVNRepository(const SGPath& base, HTTP::Client *cl) :
-  _d(new SVNRepoPrivate(this))
+    _d(new SVNRepoPrivate(this))
 {
   _d->http = cl;
   _d->rootCollection = new SVNDirectory(this, base);
-  _d->baseUrl = _d->rootCollection->url();  
+  _d->baseUrl = _d->rootCollection->url();
 }
 
 SVNRepository::~SVNRepository()
@@ -312,42 +313,42 @@ bool SVNRepository::isBare() const
     if (!fsBase().exists() || Dir(fsBase()).isEmpty()) {
         return true;
     }
-    
+
     if (_d->vccUrl.empty()) {
         return true;
     }
-    
+
     return false;
 }
 
 void SVNRepository::update()
-{  
-    _d->status = SVN_NO_ERROR;
-    if (_d->targetRevision.empty() || _d->vccUrl.empty()) {        
-        _d->isUpdating = true;        
+{
+    _d->status = REPO_NO_ERROR;
+    if (_d->targetRevision.empty() || _d->vccUrl.empty()) {
+        _d->isUpdating = true;
         PropFindRequest* pfr = new PropFindRequest(_d.get());
         http()->makeRequest(pfr);
         return;
     }
-        
+
     if (_d->targetRevision == rootDir()->cachedRevision()) {
         SG_LOG(SG_TERRASYNC, SG_DEBUG, baseUrl() << " in sync at version " << _d->targetRevision);
         _d->isUpdating = false;
         return;
     }
-    
+
     _d->isUpdating = true;
-    UpdateReportRequest* urr = new UpdateReportRequest(_d.get(), 
+    UpdateReportRequest* urr = new UpdateReportRequest(_d.get(),
         _d->targetRevision, isBare());
     http()->makeRequest(urr);
 }
-  
+
 bool SVNRepository::isDoingSync() const
 {
-    if (_d->status != SVN_NO_ERROR) {
+    if (_d->status != REPO_NO_ERROR) {
         return false;
     }
-    
+
     return _d->isUpdating || _d->rootCollection->isDoingSync();
 }
 
@@ -368,16 +369,16 @@ void SVNRepoPrivate::propFindComplete(HTTP::Request* req, DAVCollection* c)
 {
     targetRevision = c->versionName();
     vccUrl = makeAbsoluteUrl(c->versionControlledConfiguration(), baseUrl);
-    rootCollection->collection()->setVersionControlledConfiguration(vccUrl);    
+    rootCollection->collection()->setVersionControlledConfiguration(vccUrl);
     p->update();
 }
-  
+
 void SVNRepoPrivate::propFindFailed(HTTP::Request *req, SVNRepository::ResultCode err)
 {
-    if (err != SVNRepository::SVN_ERROR_NOT_FOUND) {
+    if (err != SVNRepository::REPO_ERROR_NOT_FOUND) {
         SG_LOG(SG_TERRASYNC, SG_WARN, "PropFind failed for:" << req->url());
     }
-    
+
     isUpdating = false;
     status = err;
 }

simgear/io/SVNRepository.hxx

@@ -17,62 +17,43 @@
 // Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
 
-#ifndef SG_IO_DAVMIRRORTREE_HXX
-#define SG_IO_DAVMIRRORTREE_HXX
+#ifndef SG_IO_SVN_REPOSITORY_HXX
+#define SG_IO_SVN_REPOSITORY_HXX
 
-#include <string>
-#include <vector>
+#include <simgear/io/AbstractRepository.hxx>
 #include <memory>
 
-#include <simgear/misc/sg_path.hxx>
-
 namespace simgear  {
-  
-  namespace HTTP {
-    class Client;
-  }
-  
-class SVNDirectory;  
+
+class SVNDirectory;
 class SVNRepoPrivate;
 
-class SVNRepository
+class SVNRepository : public AbstractRepository
 {
 public:
-  
+
     SVNRepository(const SGPath& root, HTTP::Client* cl);
-    ~SVNRepository();
+    virtual ~SVNRepository();
 
     SVNDirectory* rootDir() const;
-    SGPath fsBase() const;
+    virtual SGPath fsBase() const;
 
-    void setBaseUrl(const std::string& url);
-    std::string baseUrl() const;
+    virtual void setBaseUrl(const std::string& url);
+    virtual std::string baseUrl() const;
 
-    HTTP::Client* http() const;
+    virtual HTTP::Client* http() const;
 
-    void update();
+    virtual void update();
 
-    bool isDoingSync() const;
-    
-    enum ResultCode {
-        SVN_NO_ERROR = 0,
-        SVN_ERROR_NOT_FOUND,
-        SVN_ERROR_SOCKET,
-        SVN_ERROR_XML,
-        SVN_ERROR_TXDELTA,
-        SVN_ERROR_IO,
-        SVN_ERROR_CHECKSUM,
-        SVN_ERROR_FILE_NOT_FOUND,
-        SVN_ERROR_HTTP
-    };
-    
-    ResultCode failure() const;
+    virtual bool isDoingSync() const;
+
+    virtual ResultCode failure() const;
 private:
     bool isBare() const;
-    
+
     std::auto_ptr<SVNRepoPrivate> _d;
 };
 
 } // of namespace simgear
 
-#endif // of SG_IO_DAVMIRRORTREE_HXX
+#endif // of SG_IO_SVN_REPOSITORY_HXX

simgear/io/http_repo_sync.cxx

@@ -0,0 +1,82 @@
+
+#include <cstdio>
+#include <cstring>
+#include <signal.h>
+
+#include <iostream>
+#include <boost/foreach.hpp>
+
+
+#include <simgear/io/sg_file.hxx>
+#include <simgear/io/HTTPClient.hxx>
+#include <simgear/io/HTTPRepository.hxx>
+#include <simgear/misc/strutils.hxx>
+#include <simgear/timing/timestamp.hxx>
+#include <simgear/misc/sg_dir.hxx>
+#include <simgear/debug/logstream.hxx>
+
+using namespace simgear;
+using std::cout;
+using std::endl;
+using std::cerr;
+using std::string;
+
+int main(int argc, char* argv[])
+{
+    HTTP::Client cl;
+    string proxy, proxyAuth;
+    string_list headers;
+    string url;
+
+    sglog().setLogLevels( SG_ALL, SG_INFO );
+
+    for (int a=0; a<argc;++a) {
+        if (argv[a][0] == '-') {
+            if (!strcmp(argv[a], "--proxy")) {
+                proxy = argv[++a];
+            } else if (!strcmp(argv[a], "--auth")) {
+                proxyAuth = argv[++a];
+            }
+        } else { // of argument starts with a hyphen
+            url = argv[a];
+        }
+    } // of arguments iteration
+
+    if (!proxy.empty()) {
+        int colonPos = proxy.find(':');
+        string proxyHost = proxy;
+        int proxyPort = 8800;
+        if (colonPos >= 0) {
+            proxyHost = proxy.substr(0, colonPos);
+            proxyPort = strutils::to_int(proxy.substr(colonPos + 1));
+        }
+
+        cl.setProxy(proxyHost, proxyPort, proxyAuth);
+    }
+
+#ifndef WIN32
+    signal(SIGPIPE, SIG_IGN);
+#endif
+
+    if (url.empty()) {
+        cerr << "no URL argument specificed" << endl;
+        return EXIT_FAILURE;
+    }
+
+    SGPath rootPath = simgear::Dir::current().path();
+    HTTPRepository* repo = new HTTPRepository(rootPath, &cl);
+    repo->setBaseUrl(url);
+    repo->update();
+
+    while (repo->isDoingSync()) {
+        cl.update();
+        SGTimeStamp::sleepForMSec(100);
+    }
+
+    if (repo->failure() != AbstractRepository::REPO_NO_ERROR) {
+        cerr << "got response:" << repo->failure() << endl;
+        return EXIT_FAILURE;
+    }
+
+    return EXIT_SUCCESS;
+}

simgear/io/raw_socket.cxx

@@ -684,6 +684,15 @@ bool Socket::isNonBlockingError ()
 #endif
 }
 
+int Socket::errorNumber()
+{
+#if defined(WINSOCK)
+    return WSAGetLastError();
+#else
+    return errno;
+#endif
+}
+
 
 //////////////////////////////////////////////////////////////////////
 //

simgear/io/raw_socket.hxx

@@ -101,7 +101,8 @@ public:
   void setBroadcast ( bool broadcast ) ;
 
   static bool isNonBlockingError () ;
-  
+  static int errorNumber();
+
   static int select ( Socket** reads, Socket** writes, int timeout ) ;
 } ;
 

simgear/io/sg_file.cxx

@@ -45,8 +45,9 @@
 #include "sg_file.hxx"
 
 
-SGFile::SGFile(const std::string &file, int repeat_)
-    : file_name(file), fp(-1), eof_flag(true), repeat(repeat_), iteration(0)
+SGFile::SGFile(const std::string &file, int repeat_, int extraoflags_ )
+    : file_name(file), fp(-1), eof_flag(true), repeat(repeat_), iteration(0),
+      extraoflags(extraoflags_)
 {
     set_type( sgFileType );
 }
@@ -70,13 +71,13 @@ bool SGFile::open( const SGProtocolDir d ) {
 
     if ( get_dir() == SG_IO_OUT ) {
 #ifdef _WIN32
-        int mode = 00666;
+        int mode = _S_IREAD | _S_IWRITE;
 #else
         mode_t mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH;
 #endif
-	fp = ::open( file_name.c_str(), O_WRONLY | O_CREAT | O_TRUNC, mode );
+	fp = ::open( file_name.c_str(), O_WRONLY | O_CREAT | O_TRUNC | extraoflags, mode );
     } else if ( get_dir() == SG_IO_IN ) {
-	fp = ::open( file_name.c_str(), O_RDONLY );
+	fp = ::open( file_name.c_str(), O_RDONLY | extraoflags );
     } else {
 	SG_LOG( SG_IO, SG_ALERT, 
 		"Error:  bidirection mode not available for files." );
@@ -178,3 +179,12 @@ bool SGFile::close() {
     eof_flag = true;
     return true;
 }
+
+SGBinaryFile::SGBinaryFile( const std::string& file, int repeat_ ) :
+#ifdef _WIN32
+    SGFile(file,repeat_, _O_BINARY)
+#else
+    SGFile(file,repeat_, 0)
+#endif
+{
+}

simgear/io/sg_file.hxx

@@ -39,6 +39,7 @@ class SGFile : public SGIOChannel {
     const int repeat;
     int iteration;              // number of current repetition,
                                 // starting at 0
+    int extraoflags;
 
 public:
 
@@ -50,7 +51,7 @@ public:
      * @param file name of file to open
      * @param repeat On eof restart at the beginning of the file
      */
-    SGFile( const std::string& file, int repeat_ = 1 );
+    SGFile( const std::string& file, int repeat_ = 1, int extraoflags = 0);
 
     /**
      * Create an SGFile from an existing, open file-descriptor
@@ -85,4 +86,9 @@ public:
     virtual bool eof() const { return eof_flag; };
 };
 
+class SGBinaryFile : public SGFile {
+public:
+    SGBinaryFile( const std::string& file, int repeat_ = 1 );
+};
+
 #endif // _SG_FILE_HXX

simgear/io/sg_netChannel.cxx

@@ -114,7 +114,7 @@ NetChannel::send (const void * buffer, int size, int flags)
     write_blocked = true ;
     return 0;
   } else {
-    this->handleError (result);
+    this->handleError (errorNumber());
     close();
     return -1;
   }
@@ -134,7 +134,7 @@ NetChannel::recv (void * buffer, int size, int flags)
   } else if (isNonBlockingError ()) {
     return 0;
   } else {
-    this->handleError (result);
+    this->handleError (errorNumber());
     close();
     return -1;
   }
@@ -207,7 +207,7 @@ NetChannel::handleResolve()
         return 0;
     } else {
         // some other error condition
-        handleError (result);
+        handleError (errorNumber());
         close();
         return -1;
     }

simgear/io/test_DNS.cxx

@@ -0,0 +1,117 @@
+#include <cstdlib>
+
+#include <iostream>
+#include <map>
+#include <sstream>
+#include <errno.h>
+
+#include <boost/algorithm/string/case_conv.hpp>
+
+#include <simgear/simgear_config.h>
+
+#include "DNSClient.hxx"
+
+#include "test_DNS.hxx"
+
+#include <simgear/debug/logstream.hxx>
+#include <simgear/misc/strutils.hxx>
+#include <simgear/timing/timestamp.hxx>
+
+using std::cout;
+using std::cerr;
+using std::endl;
+
+using namespace simgear;
+
+#define COMPARE(a, b) \
+    if ((a) != (b))  { \
+        cerr << "failed:" << #a << " != " << #b << endl; \
+        cerr << "\tgot:'" << a << "'" << endl; \
+        exit(1); \
+    }
+
+
+int main(int argc, char* argv[])
+{
+    sglog().setLogLevels( SG_ALL, SG_DEBUG );
+
+    DNS::Client cl;
+#define EXISTING_RECORD "terrasync.flightgear.org"
+
+    // test existing NAPTR
+    // fgtest.t3r.de.                600        IN        NAPTR        999 99 "U" "test" "!^.*$!http://dnstest.flightgear.org/!" .
+    {
+        DNS::NAPTRRequest * naptrRequest = new DNS::NAPTRRequest(EXISTING_RECORD);
+        DNS::Request_ptr r(naptrRequest);
+        cl.makeRequest(r);
+        while( !r->isComplete() && !r->isTimeout()) {
+            SGTimeStamp::sleepForMSec(200);
+            cl.update(0);
+        }
+
+        if( r->isTimeout() ) {
+            cerr << "timeout testing existing record " EXISTING_RECORD << endl;
+            return EXIT_FAILURE;
+        }
+        if(naptrRequest->entries.empty()) {
+            cerr << "no results for " EXISTING_RECORD << endl;
+            return EXIT_FAILURE;
+        }
+
+        // test for ascending preference/order
+        int order = -1, preference = -1;
+        for( DNS::NAPTRRequest::NAPTR_list::const_iterator it = naptrRequest->entries.begin(); it != naptrRequest->entries.end(); ++it ) {
+            // currently only support "U" which implies empty replacement
+            COMPARE((*it)->flags, "U" );
+            COMPARE(naptrRequest->entries[0]->replacement, "" );
+
+            // currently only support ws20
+            COMPARE((*it)->service, "ws20" );
+
+            if( (*it)->order < order ) {
+                cerr << "NAPTR entries not ascending for field 'order'" << endl;
+                return EXIT_FAILURE;
+            } else if( (*it)->order > order ) {
+                order = (*it)->order;
+                preference = (*it)->preference;
+            } else {
+                if( (*it)->preference < preference ) {
+                    cerr << "NAPTR entries not ascending for field 'preference', order=" << order << endl;
+                    return EXIT_FAILURE;
+                }
+                preference = (*it)->preference;
+            }
+
+            if( false == simgear::strutils::starts_with( (*it)->regexp, "!^.*$!" ) ) {
+                cerr << "NAPTR entry with unsupported regexp: " << (*it)->regexp << endl;
+                return EXIT_FAILURE;
+            }
+
+            if( false == simgear::strutils::ends_with( (*it)->regexp, "!" ) ) {
+                cerr << "NAPTR entry with unsupported regexp: " << (*it)->regexp << endl;
+                return EXIT_FAILURE;
+            }
+
+        }
+    }
+
+    // test non-existing NAPTR
+    {
+        DNS::NAPTRRequest * naptrRequest = new DNS::NAPTRRequest("jurkxkqdiufqzpfvzqok.prozhqrlcaavbxifkkhf");
+        DNS::Request_ptr r(naptrRequest);
+        cl.makeRequest(r);
+        while( !r->isComplete() && !r->isTimeout()) {
+            SGTimeStamp::sleepForMSec(200);
+            cl.update(0);
+        }
+
+        if( r->isTimeout() ) {
+            cerr << "timeout testing non-existing record." << endl;
+            return EXIT_FAILURE;
+        }
+        COMPARE(naptrRequest->entries.size(), 0 );
+    }
+
+    cout << "all tests passed ok" << endl;
+    return EXIT_SUCCESS;
+}

simgear/io/test_DNS.hxx

@@ -0,0 +1,13 @@
+#ifndef SIMGEAR_IO_TEST_DNS_HXX
+#define SIMGEAR_IO_TEST_DNS_HXX
+
+#include <sstream>
+
+#include <simgear/io/sg_netChat.hxx>
+#include <simgear/misc/strutils.hxx>
+
+namespace simgear
+{
+} // of namespace simgear
+
+#endif // of SIMGEAR_IO_TEST_DNS_HXX

simgear/io/test_HTTP.cxx

@@ -12,14 +12,13 @@
 #include "HTTPClient.hxx"
 #include "HTTPRequest.hxx"
 
-#include <simgear/io/sg_netChat.hxx>
+#include "test_HTTP.hxx"
+
 #include <simgear/misc/strutils.hxx>
 #include <simgear/timing/timestamp.hxx>
 #include <simgear/debug/logstream.hxx>
 
-#if defined(ENABLE_CURL)
 #include <curl/multi.h>
-#endif
 
 using std::cout;
 using std::cerr;
@@ -85,7 +84,7 @@ protected:
 
     virtual void gotBodyData(const char* s, int n)
     {
-      //std::cout << "got body data:'" << string(s, n) << "'" <<std::endl;
+        //std::cout << "got body data:'" << string(s, n) << "'" <<std::endl;
         bodyData += string(s, n);
     }
 
@@ -96,97 +95,12 @@ protected:
     }
 };
 
-class TestServerChannel : public NetChat
+class HTTPTestChannel : public TestServerChannel
 {
 public:
-    enum State
+
+    virtual void processRequestHeaders()
     {
-        STATE_IDLE = 0,
-        STATE_HEADERS,
-        STATE_CLOSING,
-        STATE_REQUEST_BODY
-    };
-
-    TestServerChannel()
-    {
-        state = STATE_IDLE;
-        setTerminator("\r\n");
-
-    }
-
-    virtual void collectIncomingData(const char* s, int n)
-    {
-        buffer += string(s, n);
-    }
-
-    virtual void foundTerminator(void)
-    {
-        if (state == STATE_IDLE) {
-            state = STATE_HEADERS;
-            string_list line = strutils::split(buffer, NULL, 3);
-            if (line.size() < 3) {
-                cerr << "malformed request:" << buffer << endl;
-                exit(-1);
-            }
-
-            method = line[0];
-            path = line[1];
-
-            string::size_type queryPos = path.find('?');
-            if (queryPos != string::npos) {
-                parseArgs(path.substr(queryPos + 1));
-                path = path.substr(0, queryPos);
-            }
-
-            httpVersion = line[2];
-            requestHeaders.clear();
-            buffer.clear();
-        } else if (state == STATE_HEADERS) {
-            string s = strutils::simplify(buffer);
-            if (s.empty()) {
-                buffer.clear();
-                receivedRequestHeaders();
-                return;
-            }
-
-            string::size_type colonPos = buffer.find(':');
-            if (colonPos == string::npos) {
-                cerr << "test malformed HTTP response header:" << buffer << endl;
-                buffer.clear();
-                return;
-            }
-
-            string key = strutils::simplify(buffer.substr(0, colonPos));
-            string value = strutils::strip(buffer.substr(colonPos + 1));
-            requestHeaders[key] = value;
-            buffer.clear();
-        } else if (state == STATE_REQUEST_BODY) {
-            receivedBody();
-            setTerminator("\r\n");
-        } else if (state == STATE_CLOSING) {
-          // ignore!
-        }
-    }
-
-    void parseArgs(const string& argData)
-    {
-        string_list argv = strutils::split(argData, "&");
-        for (unsigned int a=0; a<argv.size(); ++a) {
-            string::size_type eqPos = argv[a].find('=');
-            if (eqPos == string::npos) {
-                cerr << "malformed HTTP argument:" << argv[a] << endl;
-                continue;
-            }
-
-            string key = argv[a].substr(0, eqPos);
-            string value = argv[a].substr(eqPos + 1);
-            args[key] = value;
-        }
-    }
-
-    void receivedRequestHeaders()
-    {
-        state = STATE_IDLE;
         if (path == "/test1") {
             string contentStr(BODY1);
             stringstream d;
@@ -340,25 +254,44 @@ public:
               requestContentLength = strutils::to_int(requestHeaders["Content-Length"]);
               setByteCount(requestContentLength);
               state = STATE_REQUEST_BODY;
+        } else if (path == "/test_get_during_send") {
+            // indicate we will send some number of bytes, but only send
+            // some of them now.
+            waitingOnNextRequestToContinue = true;
+
+            string contentStr(BODY3, 100); // only send first 100 chars
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << strlen(BODY3) << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+        } else if (path == "/test_get_during_send_2") {
+            stringstream d;
+
+            if (waitingOnNextRequestToContinue) {
+                waitingOnNextRequestToContinue = false;
+                // push the rest of the first request
+                d << string(BODY3).substr(100);
+            }
+
+
+            // push the response to this request
+            string contentStr(BODY1);
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << contentStr.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+
         } else {
-            sendErrorResponse(404, false, "");
+          TestServerChannel::processRequestHeaders();
         }
     }
 
-    void closeAfterSending()
+    void processRequestBody()
     {
-      state = STATE_CLOSING;
-      closeWhenDone();
-    }
-
-    void receivedBody()
-    {
-        state = STATE_IDLE;
-        if (method == "POST") {
-            parseArgs(buffer);
-        }
-
-        if (path == "/test_post") {
+      if (path == "/test_post") {
             if ((args["foo"] != "abc") || (args["bar"] != "1234") || (args["username"] != "johndoe")) {
                 sendErrorResponse(400, true, "bad arguments");
                 return;
@@ -391,11 +324,8 @@ public:
 
           push(d.str().c_str());
         } else {
-          std::cerr << "weird URL " << path << std::endl;
-          sendErrorResponse(400, true, "bad URL:" + path);
+          TestServerChannel::processRequestBody();
         }
-
-        buffer.clear();
     }
 
     void sendBody2()
@@ -408,84 +338,10 @@ public:
         bufferSend(body2, body2Size);
     }
 
-    void sendErrorResponse(int code, bool close, string content)
-    {
-        cerr << "sending error " << code << " for " << path << endl;
-        cerr << "\tcontent:" << content << endl;
-
-        stringstream headerData;
-        headerData << "HTTP/1.1 " << code << " " << reasonForCode(code) << "\r\n";
-        headerData << "Content-Length:" << content.size() << "\r\n";
-        headerData << "\r\n"; // final CRLF to terminate the headers
-        push(headerData.str().c_str());
-        push(content.c_str());
-
-        if (close) {
-            closeWhenDone();
-        }
-    }
-
-    string reasonForCode(int code)
-    {
-        switch (code) {
-            case 200: return "OK";
-            case 201: return "Created";
-            case 204: return "no content";
-            case 404: return "not found";
-            case 407: return "proxy authentication required";
-            default: return "unknown code";
-        }
-    }
-
-    State state;
-    string buffer;
-    string method;
-    string path;
-    string httpVersion;
-    std::map<string, string> requestHeaders;
-    std::map<string, string> args;
-    int requestContentLength;
+    bool waitingOnNextRequestToContinue;
 };
 
-class TestServer : public NetChannel
-{
-    simgear::NetChannelPoller _poller;
-public:
-    TestServer()
-    {
-        Socket::initSockets();
-
-        open();
-        bind(NULL, 2000); // localhost, any port
-        listen(5);
-
-        _poller.addChannel(this);
-    }
-
-    virtual ~TestServer()
-    {
-    }
-
-    virtual bool writable (void) { return false ; }
-
-    virtual void handleAccept (void)
-    {
-        simgear::IPAddress addr ;
-        int handle = accept ( &addr ) ;
-        //cout << "did accept from " << addr.getHost() << ":" << addr.getPort() << endl;
-        TestServerChannel* chan = new TestServerChannel();
-        chan->setHandle(handle);
-
-        _poller.addChannel(chan);
-    }
-
-    void poll()
-    {
-        _poller.poll();
-    }
-};
-
-TestServer testServer;
+TestServer<HTTPTestChannel> testServer;
 
 void waitForComplete(HTTP::Client* cl, TestRequest* tr)
 {
@@ -677,11 +533,7 @@ int main(int argc, char* argv[])
 
 
 
-#if defined(ENABLE_CURL)
-      const int HOST_NOT_FOUND_CODE = CURLE_COULDNT_RESOLVE_HOST;
-#else
-      const int HOST_NOT_FOUND_CODE = ENOENT;
-#endif
+        const int HOST_NOT_FOUND_CODE = CURLE_COULDNT_RESOLVE_HOST;
         COMPARE(tr->responseCode(), HOST_NOT_FOUND_CODE);
     }
 
@@ -693,11 +545,7 @@ int main(int argc, char* argv[])
         cl.makeRequest(tr);
         waitForFailed(&cl, tr);
 
-  #if defined(ENABLE_CURL)
         const int SERVER_NO_DATA_CODE = CURLE_GOT_NOTHING;
-  #else
-        const int SERVER_NO_DATA_CODE = 500;
-  #endif
         COMPARE(tr->responseCode(), SERVER_NO_DATA_CODE);
     }
 
@@ -714,7 +562,6 @@ cout << "testing proxy close" << endl;
         COMPARE(tr->bodyData, string(body2, body2Size));
     }
 
-#if defined(ENABLE_CURL)
     {
         cl.setProxy("localhost", 2000, "johndoe:swordfish");
         TestRequest* tr = new TestRequest("http://www.google.com/test3");
@@ -725,12 +572,13 @@ cout << "testing proxy close" << endl;
         COMPARE(tr->responseBytesReceived(), body2Size);
         COMPARE(tr->bodyData, string(body2, body2Size));
     }
-#endif
 
 // pipelining
     cout << "testing HTTP 1.1 pipelining" << endl;
 
     {
+        testServer.resetConnectCount();
+        cl.clearAllConnections();
 
         cl.setProxy("", 80);
         TestRequest* tr = new TestRequest("http://localhost:2000/test1");
@@ -756,6 +604,8 @@ cout << "testing proxy close" << endl;
         COMPARE(tr2->bodyData, string(BODY3));
 
         COMPARE(tr3->bodyData, string(BODY1));
+
+        COMPARE(testServer.connectCount(), 1);
     }
 
 // multiple requests with an HTTP 1.0 server
@@ -836,6 +686,103 @@ cout << "testing proxy close" << endl;
         COMPARE(tr->responseBytesReceived(), 0);
     }
 
+    // test cancel
+    {
+        cout <<  "cancel  request" << endl;
+        testServer.resetConnectCount();
+        cl.clearAllConnections();
+
+        cl.setProxy("", 80);
+        TestRequest* tr = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/testLorem");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        TestRequest* tr3 = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own3(tr3);
+        cl.makeRequest(tr3);
+
+        cl.cancelRequest(tr, "my reason 1");
+
+        cl.cancelRequest(tr2, "my reason 2");
+
+        waitForComplete(&cl, tr3);
+
+        COMPARE(tr->responseCode(), -1);
+        COMPARE(tr2->responseReason(), "my reason 2");
+
+        COMPARE(tr3->responseLength(), strlen(BODY1));
+        COMPARE(tr3->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr3->bodyData, string(BODY1));
+    }
+
+    // test cancel
+    {
+        cout <<  "cancel middle request" << endl;
+        testServer.resetConnectCount();
+        cl.clearAllConnections();
+
+        cl.setProxy("", 80);
+        TestRequest* tr = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/testLorem");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        TestRequest* tr3 = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own3(tr3);
+        cl.makeRequest(tr3);
+
+        cl.cancelRequest(tr2, "middle request");
+
+        waitForComplete(&cl, tr3);
+
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseLength(), strlen(BODY1));
+        COMPARE(tr->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr->bodyData, string(BODY1));
+
+        COMPARE(tr2->responseCode(), -1);
+
+        COMPARE(tr3->responseLength(), strlen(BODY1));
+        COMPARE(tr3->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr3->bodyData, string(BODY1));
+    }
+
+    {
+        cout << "get-during-response-send" << endl;
+        cl.clearAllConnections();
+        //test_get_during_send
+
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_get_during_send");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        // kick things along
+        for (int i=0; i<10; ++i) {
+            SGTimeStamp::sleepForMSec(1);
+            cl.update();
+            testServer.poll();
+
+        }
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/test_get_during_send_2");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        waitForComplete(&cl, tr2);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->bodyData, string(BODY3));
+        COMPARE(tr->responseBytesReceived(), strlen(BODY3));
+        COMPARE(tr2->responseCode(), 200);
+        COMPARE(tr2->bodyData, string(BODY1));
+        COMPARE(tr2->responseBytesReceived(), strlen(BODY1));
+    }
 
     cout << "all tests passed ok" << endl;
     return EXIT_SUCCESS;

simgear/io/test_HTTP.hxx

@@ -0,0 +1,228 @@
+#ifndef SIMGEAR_IO_TEST_HTTP_HXX
+#define SIMGEAR_IO_TEST_HTTP_HXX
+
+#include <sstream>
+
+#include <simgear/io/sg_netChat.hxx>
+#include <simgear/misc/strutils.hxx>
+
+namespace simgear
+{
+
+class TestServerChannel : public NetChat
+{
+public:
+    enum State
+    {
+        STATE_IDLE = 0,
+        STATE_HEADERS,
+        STATE_CLOSING,
+        STATE_REQUEST_BODY
+    };
+
+    TestServerChannel()
+    {
+        state = STATE_IDLE;
+        setTerminator("\r\n");
+
+    }
+
+    virtual void collectIncomingData(const char* s, int n)
+    {
+        buffer += std::string(s, n);
+    }
+
+    virtual void foundTerminator(void)
+    {
+        if (state == STATE_IDLE) {
+            state = STATE_HEADERS;
+            string_list line = strutils::split(buffer, NULL, 3);
+            if (line.size() < 3) {
+                std::cerr << "malformed request:" << buffer << std::endl;
+                exit(-1);
+            }
+
+            method = line[0];
+            path = line[1];
+
+            std::string::size_type queryPos = path.find('?');
+            if (queryPos != std::string::npos) {
+                parseArgs(path.substr(queryPos + 1));
+                path = path.substr(0, queryPos);
+            }
+
+            httpVersion = line[2];
+            requestHeaders.clear();
+            buffer.clear();
+        } else if (state == STATE_HEADERS) {
+            std::string s = strutils::simplify(buffer);
+            if (s.empty()) {
+                buffer.clear();
+                receivedRequestHeaders();
+                return;
+            }
+
+            std::string::size_type colonPos = buffer.find(':');
+            if (colonPos == std::string::npos) {
+                std::cerr << "test malformed HTTP response header:" << buffer << std::endl;
+                buffer.clear();
+                return;
+            }
+
+            std::string key = strutils::simplify(buffer.substr(0, colonPos));
+            std::string value = strutils::strip(buffer.substr(colonPos + 1));
+            requestHeaders[key] = value;
+            buffer.clear();
+        } else if (state == STATE_REQUEST_BODY) {
+            receivedBody();
+            setTerminator("\r\n");
+        } else if (state == STATE_CLOSING) {
+          // ignore!
+        }
+    }
+
+    void parseArgs(const std::string& argData)
+    {
+        string_list argv = strutils::split(argData, "&");
+        for (unsigned int a=0; a<argv.size(); ++a) {
+            std::string::size_type eqPos = argv[a].find('=');
+            if (eqPos == std::string::npos) {
+                std::cerr << "malformed HTTP argument:" << argv[a] << std::endl;
+                continue;
+            }
+
+            std::string key = argv[a].substr(0, eqPos);
+            std::string value = argv[a].substr(eqPos + 1);
+            args[key] = value;
+        }
+    }
+
+    void receivedRequestHeaders()
+    {
+        state = STATE_IDLE;
+        processRequestHeaders();
+    }
+
+    virtual void processRequestHeaders()
+    {
+      sendErrorResponse(404, false, "");
+    }
+
+    void closeAfterSending()
+    {
+      state = STATE_CLOSING;
+      closeWhenDone();
+    }
+
+    void receivedBody()
+    {
+        state = STATE_IDLE;
+        if (method == "POST") {
+            parseArgs(buffer);
+        }
+
+        processRequestBody();
+
+        buffer.clear();
+    }
+
+    virtual void processRequestBody()
+    {
+      sendErrorResponse(404, false, "");
+    }
+
+    void sendErrorResponse(int code, bool close, std::string content)
+    {
+        std::cerr << "sending error " << code << " for " << path << std::endl;
+        std::cerr << "\tcontent:" << content << std::endl;
+
+        std::stringstream headerData;
+        headerData << "HTTP/1.1 " << code << " " << reasonForCode(code) << "\r\n";
+        headerData << "Content-Length:" << content.size() << "\r\n";
+        headerData << "\r\n"; // final CRLF to terminate the headers
+        push(headerData.str().c_str());
+        push(content.c_str());
+
+        if (close) {
+            closeWhenDone();
+        }
+    }
+
+    std::string reasonForCode(int code)
+    {
+        switch (code) {
+            case 200: return "OK";
+            case 201: return "Created";
+            case 204: return "no content";
+            case 404: return "not found";
+            case 407: return "proxy authentication required";
+            default: return "unknown code";
+        }
+    }
+
+    State state;
+    std::string buffer;
+    std::string method;
+    std::string path;
+    std::string httpVersion;
+    std::map<std::string, std::string> requestHeaders;
+    std::map<std::string, std::string> args;
+    int requestContentLength;
+};
+
+template <class T>
+class TestServer : public NetChannel
+{
+    simgear::NetChannelPoller _poller;
+    int _connectCount;
+public:
+    TestServer()
+    {
+        Socket::initSockets();
+
+        _connectCount = 0;
+
+        open();
+        bind(NULL, 2000); // localhost, any port
+        listen(16);
+
+        _poller.addChannel(this);
+    }
+
+    virtual ~TestServer()
+    {
+    }
+
+    virtual bool writable (void) { return false ; }
+
+    virtual void handleAccept (void)
+    {
+        simgear::IPAddress addr ;
+        int handle = accept ( &addr ) ;
+        TestServerChannel* chan = new T();
+        chan->setHandle(handle);
+
+        _poller.addChannel(chan);
+
+        _connectCount++;
+    }
+
+    void poll()
+    {
+        _poller.poll();
+    }
+
+    void resetConnectCount()
+    {
+        _connectCount = 0;
+    }
+
+    int connectCount()
+    {
+        return _connectCount;
+    }
+};
+
+} // of namespace simgear
+
+#endif // of SIMGEAR_IO_TEST_HTTP_HXX

simgear/io/test_repository.cxx

@@ -0,0 +1,599 @@
+#include <cstdlib>
+#include <iostream>
+#include <map>
+#include <sstream>
+#include <errno.h>
+#include <fcntl.h>
+
+#include <boost/algorithm/string/case_conv.hpp>
+
+#include <simgear/simgear_config.h>
+
+#include "test_HTTP.hxx"
+#include "HTTPRepository.hxx"
+#include "HTTPClient.hxx"
+
+#include <simgear/misc/strutils.hxx>
+#include <simgear/misc/sg_hash.hxx>
+#include <simgear/timing/timestamp.hxx>
+#include <simgear/debug/logstream.hxx>
+#include <simgear/misc/sg_dir.hxx>
+#include <simgear/structure/exception.hxx>
+#include <simgear/io/sg_file.hxx>
+
+using namespace simgear;
+
+std::string dataForFile(const std::string& parentName, const std::string& name, int revision)
+{
+    std::ostringstream os;
+    // random content but which definitely depends on our tree location
+    // and revision.
+    for (int i=0; i<100; ++i) {
+        os << i << parentName << "_" << name << "_" << revision;
+    }
+
+    return os.str();
+}
+
+std::string hashForData(const std::string& d)
+{
+    simgear::sha1nfo info;
+    sha1_init(&info);
+    sha1_write(&info, d.data(), d.size());
+    return strutils::encodeHex(sha1_result(&info), HASH_LENGTH);
+}
+
+class TestRepoEntry
+{
+public:
+    TestRepoEntry(TestRepoEntry* parent, const std::string& name, bool isDir);
+    ~TestRepoEntry();
+
+    TestRepoEntry* parent;
+    std::string name;
+
+    std::string indexLine() const;
+
+    std::string hash() const;
+
+    std::vector<TestRepoEntry*> children;
+
+    size_t sizeInBytes() const
+    {
+        return data().size();
+    }
+
+    bool isDir;
+    int revision; // for files
+    int requestCount;
+    bool getWillFail;
+    bool returnCorruptData;
+
+    void clearRequestCounts();
+
+    void setGetWillFail(bool b)
+    {
+        getWillFail = b;
+    }
+
+    void setReturnCorruptData(bool d)
+    {
+        returnCorruptData = d;
+    }
+
+    std::string pathInRepo() const
+    {
+        return parent ? (parent->pathInRepo() + "/" + name) : name;
+    }
+
+    std::string data() const;
+
+    void defineFile(const std::string& path, int rev = 1)
+    {
+        string_list pathParts = strutils::split(path, "/");
+        if (pathParts.size() == 1) {
+            children.push_back(new TestRepoEntry(this, pathParts.front(), false));
+            children.back()->revision = rev;
+        } else {
+            // recurse
+            TestRepoEntry* c = childEntry(pathParts.front());
+            if (!c) {
+                // define a new directory child
+                c = new TestRepoEntry(this, pathParts.front(), true);
+                children.push_back(c);
+            }
+
+            size_t frontPartLength = pathParts.front().size();
+            c->defineFile(path.substr(frontPartLength + 1), rev);
+        }
+    }
+
+    TestRepoEntry* findEntry(const std::string& path)
+    {
+        if (path.empty()) {
+            return this;
+        }
+        
+        string_list pathParts = strutils::split(path, "/");
+        TestRepoEntry* entry = childEntry(pathParts.front());
+        if (pathParts.size() == 1) {
+            return entry; // might be NULL
+        }
+
+        if (!entry) {
+            std::cerr << "bad path: " << path << std::endl;
+            return NULL;
+        }
+
+        size_t part0Length = pathParts.front().size() + 1;
+        return entry->findEntry(path.substr(part0Length));
+    }
+
+    TestRepoEntry* childEntry(const std::string& name) const
+    {
+        assert(isDir);
+        for (size_t i=0; i<children.size(); ++i) {
+            if (children[i]->name == name) {
+                return children[i];
+            }
+        }
+
+        return NULL;
+    }
+
+    void removeChild(const std::string& name)
+    {
+        std::vector<TestRepoEntry*>::iterator it;
+        for (it = children.begin(); it != children.end(); ++it) {
+            if ((*it)->name == name) {
+                delete *it;
+                children.erase(it);
+                return;
+            }
+        }
+        std::cerr << "child not found:" << name << std::endl;
+    }
+};
+
+TestRepoEntry::TestRepoEntry(TestRepoEntry* pr, const std::string& nm, bool d) :
+    parent(pr), name(nm), isDir(d)
+{
+    revision = 2;
+    requestCount = 0;
+    getWillFail = false;
+    returnCorruptData = false;
+}
+
+TestRepoEntry::~TestRepoEntry()
+{
+    for (size_t i=0; i<children.size(); ++i) {
+        delete children[i];
+    }
+}
+
+std::string TestRepoEntry::data() const
+{
+    if (isDir) {
+        std::ostringstream os;
+        os << "version:1\n";
+        os << "path:" << pathInRepo() << "\n";
+        for (size_t i=0; i<children.size(); ++i) {
+            os << children[i]->indexLine() << "\n";
+        }
+        return os.str();
+    } else {
+        return dataForFile(parent->name, name, revision);
+    }
+}
+
+std::string TestRepoEntry::indexLine() const
+{
+    std::ostringstream os;
+    os << (isDir ? "d:" : "f:") << name << ":" << hash()
+        << ":" << sizeInBytes();
+    return os.str();
+}
+
+std::string TestRepoEntry::hash() const
+{
+    simgear::sha1nfo info;
+    sha1_init(&info);
+    std::string d(data());
+    sha1_write(&info, d.data(), d.size());
+    return strutils::encodeHex(sha1_result(&info), HASH_LENGTH);
+}
+
+void TestRepoEntry::clearRequestCounts()
+{
+    requestCount = 0;
+    if (isDir) {
+        for (size_t i=0; i<children.size(); ++i) {
+            children[i]->clearRequestCounts();
+        }
+    }
+}
+
+TestRepoEntry* global_repo = NULL;
+
+class TestRepositoryChannel : public TestServerChannel
+{
+public:
+
+    virtual void processRequestHeaders()
+    {
+        state = STATE_IDLE;
+        if (path.find("/repo/") == 0) {
+//            std::cerr << "get for:" << path << std::endl;
+
+            std::string repoPath = path.substr(6);
+            bool lookingForDir = false;
+            std::string::size_type suffix = repoPath.find(".dirindex");
+            if (suffix != std::string::npos) {
+                lookingForDir = true;
+                if (suffix > 0) {
+                    // trim the preceeding '/' as well, for non-root dirs
+                    suffix--;
+                }
+
+                repoPath = repoPath.substr(0, suffix);
+            }
+
+            TestRepoEntry* entry = global_repo->findEntry(repoPath);
+            if (!entry) {
+                sendErrorResponse(404, false, "unknown repo path:" + repoPath);
+                return;
+            }
+
+            if (entry->isDir != lookingForDir) {
+                sendErrorResponse(404, false, "mismatched path type:" + repoPath);
+                return;
+            }
+
+            if (entry->getWillFail) {
+                sendErrorResponse(404, false, "entry marked to fail explicitly:" + repoPath);
+                return;
+            }
+
+            entry->requestCount++;
+
+            std::string content;
+            if (entry->returnCorruptData) {
+                content = dataForFile("!$£$!" + entry->parent->name,
+                                      "corrupt_" + entry->name,
+                                      entry->revision);
+            } else {
+                content = entry->data();
+            }
+
+            std::stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << content.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << content;
+            push(d.str().c_str());
+        } else {
+            sendErrorResponse(404, false, "");
+        }
+    }
+};
+
+std::string test_computeHashForPath(const SGPath& p)
+{
+    if (!p.exists())
+        return std::string();
+    sha1nfo info;
+    sha1_init(&info);
+    char* buf = static_cast<char*>(alloca(1024 * 1024));
+    size_t readLen;
+
+    SGBinaryFile f(p.str());
+    f.open(SG_IO_IN);
+
+    while ((readLen = f.read(buf, 1024 * 1024)) > 0) {
+        sha1_write(&info, buf, readLen);
+    }
+
+    std::string hashBytes((char*) sha1_result(&info), HASH_LENGTH);
+    return strutils::encodeHex(hashBytes);
+}
+
+void verifyFileState(const SGPath& fsRoot, const std::string& relPath)
+{
+    TestRepoEntry* entry = global_repo->findEntry(relPath);
+    if (!entry) {
+        throw sg_error("Missing test repo entry", relPath);
+    }
+
+    SGPath p(fsRoot);
+    p.append(relPath);
+    if (!p.exists()) {
+        throw sg_error("Missing file system entry", relPath);
+    }
+
+    std::string hashOnDisk = test_computeHashForPath(p);
+    if (hashOnDisk != entry->hash()) {
+        throw sg_error("Checksum mismatch", relPath);
+    }
+}
+
+void verifyRequestCount(const std::string& relPath, int count)
+{
+    TestRepoEntry* entry = global_repo->findEntry(relPath);
+    if (!entry) {
+        throw sg_error("Missing test repo entry", relPath);
+    }
+
+    if (entry->requestCount != count) {
+        throw sg_exception("Bad request count", relPath);
+    }
+}
+
+void createFile(const SGPath& basePath, const std::string& relPath, int revision)
+{
+    string_list comps = strutils::split(relPath, "/");
+
+    SGPath p(basePath);
+    p.append(relPath);
+
+    simgear::Dir d(p.dir());
+    d.create(0700);
+
+    std::string prName = comps.at(comps.size() - 2);
+    {
+        std::ofstream f(p.c_str(), std::ios::trunc | std::ios::out);
+        f << dataForFile(prName, comps.back(), revision);
+    }
+}
+
+TestServer<TestRepositoryChannel> testServer;
+
+void waitForUpdateComplete(HTTP::Client* cl, HTTPRepository* repo)
+{
+    SGTimeStamp start(SGTimeStamp::now());
+    while (start.elapsedMSec() <  10000) {
+        cl->update();
+        testServer.poll();
+
+        if (!repo->isDoingSync()) {
+            return;
+        }
+        SGTimeStamp::sleepForMSec(15);
+    }
+
+    std::cerr << "timed out" << std::endl;
+}
+
+void testBasicClone(HTTP::Client* cl)
+{
+    std::auto_ptr<HTTPRepository> repo;
+    SGPath p(simgear::Dir::current().path());
+    p.append("http_repo_basic");
+    simgear::Dir pd(p);
+    pd.removeChildren();
+    
+    repo.reset(new HTTPRepository(p, cl));
+    repo->setBaseUrl("http://localhost:2000/repo");
+    repo->update();
+
+    waitForUpdateComplete(cl, repo.get());
+
+    verifyFileState(p, "fileA");
+    verifyFileState(p, "dirA/subdirA/fileAAA");
+    verifyFileState(p, "dirC/subdirA/subsubA/fileCAAA");
+
+    global_repo->findEntry("fileA")->revision++;
+    global_repo->findEntry("dirB/subdirA/fileBAA")->revision++;
+    global_repo->defineFile("dirC/fileCA"); // new file
+    global_repo->findEntry("dirB/subdirA")->removeChild("fileBAB");
+    global_repo->findEntry("dirA")->removeChild("subdirA"); // remove a dir
+
+    repo->update();
+
+    // verify deltas
+    waitForUpdateComplete(cl, repo.get());
+
+    verifyFileState(p, "fileA");
+    verifyFileState(p, "dirC/fileCA");
+
+    std::cout << "Passed test: basic clone and update" << std::endl;
+}
+
+void testModifyLocalFiles(HTTP::Client* cl)
+{
+    std::auto_ptr<HTTPRepository> repo;
+    SGPath p(simgear::Dir::current().path());
+    p.append("http_repo_modify_local_2");
+    simgear::Dir pd(p);
+    if (pd.exists()) {
+        pd.removeChildren();
+    }
+
+    repo.reset(new HTTPRepository(p, cl));
+    repo->setBaseUrl("http://localhost:2000/repo");
+    repo->update();
+
+    waitForUpdateComplete(cl, repo.get());
+    verifyFileState(p, "dirB/subdirA/fileBAA");
+
+    SGPath modFile(p);
+    modFile.append("dirB/subdirA/fileBAA");
+    {
+        std::ofstream of(modFile.c_str(), std::ios::out | std::ios::trunc);
+        of << "complete nonsense";
+        of.close();
+    }
+
+    global_repo->clearRequestCounts();
+    repo->update();
+    waitForUpdateComplete(cl, repo.get());
+    verifyFileState(p, "dirB/subdirA/fileBAA");
+    verifyRequestCount("dirB", 0);
+    verifyRequestCount("dirB/subdirA", 0);
+    verifyRequestCount("dirB/subdirA/fileBAA", 1);
+
+    std::cout << "Passed test: identify and fix locally modified files" << std::endl;
+}
+
+void testNoChangesUpdate()
+{
+
+}
+
+void testMergeExistingFileWithoutDownload(HTTP::Client* cl)
+{
+    std::auto_ptr<HTTPRepository> repo;
+    SGPath p(simgear::Dir::current().path());
+    p.append("http_repo_merge_existing");
+    simgear::Dir pd(p);
+    if (pd.exists()) {
+        pd.removeChildren();
+    }
+
+    repo.reset(new HTTPRepository(p, cl));
+    repo->setBaseUrl("http://localhost:2000/repo");
+
+    createFile(p, "dirC/fileCB", 4); // should match
+    createFile(p, "dirC/fileCC", 3); // mismatch
+
+    global_repo->defineFile("dirC/fileCB", 4);
+    global_repo->defineFile("dirC/fileCC", 10);
+
+    // new sub-tree
+    createFile(p, "dirD/fileDA", 4);
+    createFile(p, "dirD/subdirDA/fileDAA", 6);
+    createFile(p, "dirD/subdirDB/fileDBA", 6);
+
+    global_repo->defineFile("dirD/fileDA", 4);
+    global_repo->defineFile("dirD/subdirDA/fileDAA", 6);
+    global_repo->defineFile("dirD/subdirDB/fileDBA", 6);
+
+    repo->update();
+    waitForUpdateComplete(cl, repo.get());
+    verifyFileState(p, "dirC/fileCB");
+    verifyFileState(p, "dirC/fileCC");
+    verifyRequestCount("dirC/fileCB", 0);
+    verifyRequestCount("dirC/fileCC", 1);
+
+    verifyRequestCount("dirD/fileDA", 0);
+    verifyRequestCount("dirD/subdirDA/fileDAA", 0);
+    verifyRequestCount("dirD/subdirDB/fileDBA", 0);
+
+    std::cout << "Passed test: merge existing files with matching hash" << std::endl;
+}
+
+void testLossOfLocalFiles(HTTP::Client* cl)
+{
+    std::auto_ptr<HTTPRepository> repo;
+    SGPath p(simgear::Dir::current().path());
+    p.append("http_repo_lose_local");
+    simgear::Dir pd(p);
+    if (pd.exists()) {
+        pd.removeChildren();
+    }
+
+    repo.reset(new HTTPRepository(p, cl));
+    repo->setBaseUrl("http://localhost:2000/repo");
+    repo->update();
+    waitForUpdateComplete(cl, repo.get());
+    verifyFileState(p, "dirB/subdirA/fileBAA");
+
+    SGPath lostPath(p);
+    lostPath.append("dirB/subdirA");
+    simgear::Dir lpd(lostPath);
+    lpd.remove(true);
+
+    global_repo->clearRequestCounts();
+
+    repo->update();
+    waitForUpdateComplete(cl, repo.get());
+    verifyFileState(p, "dirB/subdirA/fileBAA");
+
+    verifyRequestCount("dirB", 0);
+    verifyRequestCount("dirB/subdirA", 1);
+    verifyRequestCount("dirB/subdirA/fileBAC", 1);
+
+    std::cout << "Passed test: lose and replace local files" << std::endl;
+}
+
+void testAbandonMissingFiles(HTTP::Client* cl)
+{
+    std::auto_ptr<HTTPRepository> repo;
+    SGPath p(simgear::Dir::current().path());
+    p.append("http_repo_missing_files");
+    simgear::Dir pd(p);
+    if (pd.exists()) {
+        pd.removeChildren();
+    }
+
+    global_repo->defineFile("dirA/subdirE/fileAEA");
+    global_repo->findEntry("dirA/subdirE/fileAEA")->setGetWillFail(true);
+
+    repo.reset(new HTTPRepository(p, cl));
+    repo->setBaseUrl("http://localhost:2000/repo");
+    repo->update();
+    waitForUpdateComplete(cl, repo.get());
+    if (repo->failure() != AbstractRepository::REPO_PARTIAL_UPDATE) {
+        throw sg_exception("Bad result from missing files test");
+    }
+
+    global_repo->findEntry("dirA/subdirE/fileAEA")->setGetWillFail(false);
+}
+
+void testAbandonCorruptFiles(HTTP::Client* cl)
+{
+    std::auto_ptr<HTTPRepository> repo;
+    SGPath p(simgear::Dir::current().path());
+    p.append("http_repo_corrupt_files");
+    simgear::Dir pd(p);
+    if (pd.exists()) {
+        pd.removeChildren();
+    }
+
+    global_repo->defineFile("dirB/subdirG/fileBGA");
+    global_repo->findEntry("dirB/subdirG/fileBGA")->setReturnCorruptData(true);
+
+    repo.reset(new HTTPRepository(p, cl));
+    repo->setBaseUrl("http://localhost:2000/repo");
+    repo->update();
+    waitForUpdateComplete(cl, repo.get());
+    if (repo->failure() != AbstractRepository::REPO_PARTIAL_UPDATE) {
+        throw sg_exception("Bad result from corrupt files test");
+    }
+
+    std::cout << "Passed test: detect corrupted download" << std::endl;
+}
+
+int main(int argc, char* argv[])
+{
+  sglog().setLogLevels( SG_ALL, SG_INFO );
+
+  HTTP::Client cl;
+  cl.setMaxConnections(1);
+
+    global_repo = new TestRepoEntry(NULL, "root", true);
+    global_repo->defineFile("fileA");
+    global_repo->defineFile("fileB");
+    global_repo->defineFile("dirA/fileAA");
+    global_repo->defineFile("dirA/fileAB");
+    global_repo->defineFile("dirA/fileAC");
+    global_repo->defineFile("dirA/subdirA/fileAAA");
+    global_repo->defineFile("dirA/subdirA/fileAAB");
+    global_repo->defineFile("dirB/subdirA/fileBAA");
+    global_repo->defineFile("dirB/subdirA/fileBAB");
+    global_repo->defineFile("dirB/subdirA/fileBAC");
+    global_repo->defineFile("dirC/subdirA/subsubA/fileCAAA");
+
+    testBasicClone(&cl);
+
+    testModifyLocalFiles(&cl);
+
+    testLossOfLocalFiles(&cl);
+
+    testMergeExistingFileWithoutDownload(&cl);
+
+    testAbandonMissingFiles(&cl);
+
+    testAbandonCorruptFiles(&cl);
+
+    return 0;
+}

simgear/io/text_DNS.cxx

@@ -0,0 +1,787 @@
+#include <cstdlib>
+
+#include <iostream>
+#include <map>
+#include <sstream>
+#include <errno.h>
+
+#include <boost/algorithm/string/case_conv.hpp>
+
+#include <simgear/simgear_config.h>
+
+#include "HTTPClient.hxx"
+#include "HTTPRequest.hxx"
+
+#include "test_HTTP.hxx"
+
+#include <simgear/misc/strutils.hxx>
+#include <simgear/timing/timestamp.hxx>
+#include <simgear/debug/logstream.hxx>
+
+#include <curl/multi.h>
+
+using std::cout;
+using std::cerr;
+using std::endl;
+using std::string;
+using std::stringstream;
+
+using namespace simgear;
+
+const char* BODY1 = "The quick brown fox jumps over a lazy dog.";
+const char* BODY3 = "Cras ut neque nulla. Duis ut velit neque, sit amet "
+"pharetra risus. In est ligula, lacinia vitae congue in, sollicitudin at "
+"libero. Mauris pharetra pretium elit, nec placerat dui semper et. Maecenas "
+"magna magna, placerat sed luctus ac, commodo et ligula. Mauris at purus et "
+"nisl molestie auctor placerat at quam. Donec sapien magna, venenatis sed "
+"iaculis id, fringilla vel arcu. Duis sed neque nisi. Cras a arcu sit amet "
+"risus ultrices varius. Integer sagittis euismod dui id varius. Cras vel "
+"justo gravida metus.";
+
+const unsigned int body2Size = 8 * 1024;
+char body2[body2Size];
+
+#define COMPARE(a, b) \
+    if ((a) != (b))  { \
+        cerr << "failed:" << #a << " != " << #b << endl; \
+        cerr << "\tgot:'" << a << "'" << endl; \
+        exit(1); \
+    }
+
+#define VERIFY(a) \
+    if (!(a))  { \
+        cerr << "failed:" << #a << endl; \
+        exit(1); \
+    }
+
+class TestRequest : public HTTP::Request
+{
+public:
+    bool complete;
+    bool failed;
+    string bodyData;
+
+    TestRequest(const std::string& url, const std::string method = "GET") :
+        HTTP::Request(url, method),
+        complete(false),
+        failed(false)
+    {
+
+    }
+
+    std::map<string, string> headers;
+protected:
+
+    virtual void onDone()
+    {
+        complete = true;
+    }
+
+    virtual void onFail()
+    {
+        failed = true;
+    }
+
+    virtual void gotBodyData(const char* s, int n)
+    {
+        //std::cout << "got body data:'" << string(s, n) << "'" <<std::endl;
+        bodyData += string(s, n);
+    }
+
+    virtual void responseHeader(const string& header, const string& value)
+    {
+        Request::responseHeader(header, value);
+        headers[header] =  value;
+    }
+};
+
+class HTTPTestChannel : public TestServerChannel
+{
+public:
+
+    virtual void processRequestHeaders()
+    {
+        if (path == "/test1") {
+            string contentStr(BODY1);
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << contentStr.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+        } else if (path == "/testLorem") {
+            string contentStr(BODY3);
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << contentStr.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+        } else if (path == "/test_zero_length_content") {
+            string contentStr;
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << contentStr.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+        } else if (path == "/test_headers") {
+            COMPARE(requestHeaders["X-Foo"], string("Bar"));
+            COMPARE(requestHeaders["X-AnotherHeader"], string("A longer value"));
+
+            string contentStr(BODY1);
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << contentStr.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+        } else if (path == "/test2") {
+            sendBody2();
+        } else if (path == "/testchunked") {
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Transfer-Encoding:chunked\r\n";
+            d << "\r\n";
+            d << "8\r\n"; // first chunk
+            d << "ABCDEFGH\r\n";
+            d << "6\r\n"; // second chunk
+            d << "ABCDEF\r\n";
+            d << "10\r\n"; // third chunk
+            d << "ABCDSTUVABCDSTUV\r\n";
+            d << "0\r\n"; // start of trailer
+            d << "X-Foobar: wibble\r\n"; // trailer data
+            d << "\r\n";
+            push(d.str().c_str());
+        } else if (path == "http://www.google.com/test2") {
+            // proxy test
+            if (requestHeaders["Host"] != "www.google.com") {
+                sendErrorResponse(400, true, "bad destination");
+            }
+
+            if (requestHeaders["Proxy-Authorization"] != string()) {
+                sendErrorResponse(401, false, "bad auth, not empty"); // shouldn't supply auth
+            }
+
+            sendBody2();
+        } else if (path == "http://www.google.com/test3") {
+            // proxy test
+            if (requestHeaders["Host"] != "www.google.com") {
+                sendErrorResponse(400, true, "bad destination");
+            }
+
+            string credentials = requestHeaders["Proxy-Authorization"];
+            if (credentials.substr(0, 5) != "Basic") {
+              // request basic auth
+              stringstream d;
+              d << "HTTP/1.1 " << 407 << " " << reasonForCode(407) << "\r\n";
+              d << "WWW-Authenticate: Basic real=\"simgear\"\r\n";
+              d << "\r\n"; // final CRLF to terminate the headers
+              push(d.str().c_str());
+              return;
+            }
+
+            std::vector<unsigned char> userAndPass;
+            strutils::decodeBase64(credentials.substr(6), userAndPass);
+            std::string decodedUserPass((char*) userAndPass.data(), userAndPass.size());
+
+            if (decodedUserPass != "johndoe:swordfish") {
+                std::map<string, string>::const_iterator it;
+                for (it = requestHeaders.begin(); it != requestHeaders.end(); ++it) {
+                  cerr << "header:" << it->first << " = " << it->second << endl;
+                }
+
+                sendErrorResponse(401, false, "bad auth, not as set"); // forbidden
+            }
+
+            sendBody2();
+        } else if (strutils::starts_with(path, "/test_1_0")) {
+            string contentStr(BODY1);
+            if (strutils::ends_with(path, "/B")) {
+                contentStr = BODY3;
+            }
+            stringstream d;
+            d << "HTTP/1.0 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+            closeAfterSending();
+        } else if (path == "/test_close") {
+            string contentStr(BODY1);
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Connection: close\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+            closeAfterSending();
+        } else if (path == "/test_abrupt_close") {
+            // simulate server doing socket close before sending any
+            // response - this used to cause a TerraSync failure since we
+            // would get stuck restarting the request
+            closeAfterSending();
+
+        } else if (path == "/test_args") {
+            if ((args["foo"] != "abc") || (args["bar"] != "1234") || (args["username"] != "johndoe")) {
+                sendErrorResponse(400, true, "bad arguments");
+                return;
+            }
+
+            string contentStr(BODY1);
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << contentStr.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+        } else if (path == "/test_post") {
+            if (requestHeaders["Content-Type"] != "application/x-www-form-urlencoded") {
+                cerr << "bad content type: '" << requestHeaders["Content-Type"] << "'" << endl;
+                 sendErrorResponse(400, true, "bad content type");
+                 return;
+            }
+
+            requestContentLength = strutils::to_int(requestHeaders["Content-Length"]);
+            setByteCount(requestContentLength);
+            state = STATE_REQUEST_BODY;
+        } else if ((path == "/test_put") || (path == "/test_create")) {
+              if (requestHeaders["Content-Type"] != "x-application/foobar") {
+                  cerr << "bad content type: '" << requestHeaders["Content-Type"] << "'" << endl;
+                   sendErrorResponse(400, true, "bad content type");
+                   return;
+              }
+
+              requestContentLength = strutils::to_int(requestHeaders["Content-Length"]);
+              setByteCount(requestContentLength);
+              state = STATE_REQUEST_BODY;
+        } else if (path == "/test_get_during_send") {
+            // indicate we will send some number of bytes, but only send
+            // some of them now.
+            waitingOnNextRequestToContinue = true;
+
+            string contentStr(BODY3, 100); // only send first 100 chars
+            stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << strlen(BODY3) << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+        } else if (path == "/test_get_during_send_2") {
+            stringstream d;
+
+            if (waitingOnNextRequestToContinue) {
+                waitingOnNextRequestToContinue = false;
+                // push the rest of the first request
+                d << string(BODY3).substr(100);
+            }
+
+
+            // push the response to this request
+            string contentStr(BODY1);
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << contentStr.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << contentStr;
+            push(d.str().c_str());
+
+        } else {
+          TestServerChannel::processRequestHeaders();
+        }
+    }
+
+    void processRequestBody()
+    {
+      if (path == "/test_post") {
+            if ((args["foo"] != "abc") || (args["bar"] != "1234") || (args["username"] != "johndoe")) {
+                sendErrorResponse(400, true, "bad arguments");
+                return;
+            }
+
+            stringstream d;
+            d << "HTTP/1.1 " << 204 << " " << reasonForCode(204) << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            push(d.str().c_str());
+        } else if (path == "/test_put") {
+          std::cerr << "sending PUT response" << std::endl;
+
+          COMPARE(buffer, BODY3);
+          stringstream d;
+          d << "HTTP/1.1 " << 204 << " " << reasonForCode(204) << "\r\n";
+          d << "\r\n"; // final CRLF to terminate the headers
+          push(d.str().c_str());
+        } else if (path == "/test_create") {
+          std::cerr << "sending create response" << std::endl;
+
+          std::string entityStr = "http://localhost:2000/something.txt";
+
+          COMPARE(buffer, BODY3);
+          stringstream d;
+          d << "HTTP/1.1 " << 201 << " " << reasonForCode(201) << "\r\n";
+          d << "Location:" << entityStr << "\r\n";
+          d << "Content-Length:" << entityStr.size() << "\r\n";
+          d << "\r\n"; // final CRLF to terminate the headers
+          d << entityStr;
+
+          push(d.str().c_str());
+        } else {
+          TestServerChannel::processRequestBody();
+        }
+    }
+
+    void sendBody2()
+    {
+        stringstream d;
+        d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+        d << "Content-Length:" << body2Size << "\r\n";
+        d << "\r\n"; // final CRLF to terminate the headers
+        push(d.str().c_str());
+        bufferSend(body2, body2Size);
+    }
+
+    bool waitingOnNextRequestToContinue;
+};
+
+TestServer<HTTPTestChannel> testServer;
+
+void waitForComplete(HTTP::Client* cl, TestRequest* tr)
+{
+    SGTimeStamp start(SGTimeStamp::now());
+    while (start.elapsedMSec() <  10000) {
+        cl->update();
+        testServer.poll();
+
+        if (tr->complete) {
+            return;
+        }
+        SGTimeStamp::sleepForMSec(15);
+    }
+
+    cerr << "timed out" << endl;
+}
+
+void waitForFailed(HTTP::Client* cl, TestRequest* tr)
+{
+    SGTimeStamp start(SGTimeStamp::now());
+    while (start.elapsedMSec() <  10000) {
+        cl->update();
+        testServer.poll();
+
+        if (tr->failed) {
+            return;
+        }
+        SGTimeStamp::sleepForMSec(15);
+    }
+
+    cerr << "timed out waiting for failure" << endl;
+}
+
+int main(int argc, char* argv[])
+{
+    sglog().setLogLevels( SG_ALL, SG_INFO );
+
+    HTTP::Client cl;
+    // force all requests to use the same connection for this test
+    cl.setMaxConnections(1);
+
+// test URL parsing
+    TestRequest* tr1 = new TestRequest("http://localhost.woo.zar:2000/test1?foo=bar");
+    COMPARE(tr1->scheme(), "http");
+    COMPARE(tr1->hostAndPort(), "localhost.woo.zar:2000");
+    COMPARE(tr1->host(), "localhost.woo.zar");
+    COMPARE(tr1->port(), 2000);
+    COMPARE(tr1->path(), "/test1");
+
+    TestRequest* tr2 = new TestRequest("http://192.168.1.1/test1/dir/thing/file.png");
+    COMPARE(tr2->scheme(), "http");
+    COMPARE(tr2->hostAndPort(), "192.168.1.1");
+    COMPARE(tr2->host(), "192.168.1.1");
+    COMPARE(tr2->port(), 80);
+    COMPARE(tr2->path(), "/test1/dir/thing/file.png");
+
+// basic get request
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseReason(), string("OK"));
+        COMPARE(tr->responseLength(), strlen(BODY1));
+        COMPARE(tr->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr->bodyData, string(BODY1));
+    }
+
+    {
+      TestRequest* tr = new TestRequest("http://localhost:2000/testLorem");
+      HTTP::Request_ptr own(tr);
+      cl.makeRequest(tr);
+
+      waitForComplete(&cl, tr);
+      COMPARE(tr->responseCode(), 200);
+      COMPARE(tr->responseReason(), string("OK"));
+      COMPARE(tr->responseLength(), strlen(BODY3));
+      COMPARE(tr->responseBytesReceived(), strlen(BODY3));
+      COMPARE(tr->bodyData, string(BODY3));
+    }
+
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_args?foo=abc&bar=1234&username=johndoe");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+    }
+
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_headers");
+        HTTP::Request_ptr own(tr);
+        tr->requestHeader("X-Foo") = "Bar";
+        tr->requestHeader("X-AnotherHeader") = "A longer value";
+        cl.makeRequest(tr);
+
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseReason(), string("OK"));
+        COMPARE(tr->responseLength(), strlen(BODY1));
+        COMPARE(tr->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr->bodyData, string(BODY1));
+    }
+
+// larger get request
+    for (unsigned int i=0; i<body2Size; ++i) {
+        body2[i] = (i << 4) | (i >> 2);
+    }
+
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test2");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseBytesReceived(), body2Size);
+        COMPARE(tr->bodyData, string(body2, body2Size));
+    }
+
+    cerr << "testing chunked transfer encoding" << endl;
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/testchunked");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseReason(), string("OK"));
+        COMPARE(tr->responseBytesReceived(), 30);
+        COMPARE(tr->bodyData, "ABCDEFGHABCDEFABCDSTUVABCDSTUV");
+    // check trailers made it too
+        COMPARE(tr->headers["x-foobar"], string("wibble"));
+    }
+
+// test 404
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/not-found");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 404);
+        COMPARE(tr->responseReason(), string("not found"));
+        COMPARE(tr->responseLength(), 0);
+    }
+
+    cout << "done 404 test" << endl;
+
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_args?foo=abc&bar=1234&username=johndoe");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+    }
+
+    cout << "done1" << endl;
+// test HTTP/1.0
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_1_0");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseLength(), strlen(BODY1));
+        COMPARE(tr->bodyData, string(BODY1));
+    }
+
+    cout << "done2" << endl;
+// test HTTP/1.1 Connection::close
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_close");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseLength(), strlen(BODY1));
+        COMPARE(tr->bodyData, string(BODY1));
+    }
+    cout << "done3" << endl;
+// test connectToHost failure
+
+    {
+        TestRequest* tr = new TestRequest("http://not.found/something");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForFailed(&cl, tr);
+
+        const int HOST_NOT_FOUND_CODE = CURLE_COULDNT_RESOLVE_HOST;
+        COMPARE(tr->responseCode(), HOST_NOT_FOUND_CODE);
+    }
+
+  cout << "testing abrupt close" << endl;
+    // test server-side abrupt close
+    {
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_abrupt_close");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForFailed(&cl, tr);
+
+        const int SERVER_NO_DATA_CODE = CURLE_GOT_NOTHING;
+        COMPARE(tr->responseCode(), SERVER_NO_DATA_CODE);
+    }
+
+cout << "testing proxy close" << endl;
+// test proxy
+    {
+        cl.setProxy("localhost", 2000);
+        TestRequest* tr = new TestRequest("http://www.google.com/test2");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseLength(), body2Size);
+        COMPARE(tr->bodyData, string(body2, body2Size));
+    }
+
+    {
+        cl.setProxy("localhost", 2000, "johndoe:swordfish");
+        TestRequest* tr = new TestRequest("http://www.google.com/test3");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseBytesReceived(), body2Size);
+        COMPARE(tr->bodyData, string(body2, body2Size));
+    }
+
+// pipelining
+    cout << "testing HTTP 1.1 pipelining" << endl;
+
+    {
+        testServer.resetConnectCount();
+        cl.clearAllConnections();
+
+        cl.setProxy("", 80);
+        TestRequest* tr = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/testLorem");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        TestRequest* tr3 = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own3(tr3);
+        cl.makeRequest(tr3);
+
+        waitForComplete(&cl, tr3);
+        VERIFY(tr->complete);
+        VERIFY(tr2->complete);
+        COMPARE(tr->bodyData, string(BODY1));
+
+        COMPARE(tr2->responseLength(), strlen(BODY3));
+        COMPARE(tr2->responseBytesReceived(), strlen(BODY3));
+        COMPARE(tr2->bodyData, string(BODY3));
+
+        COMPARE(tr3->bodyData, string(BODY1));
+
+        COMPARE(testServer.connectCount(), 1);
+    }
+
+// multiple requests with an HTTP 1.0 server
+    {
+        cout << "http 1.0 multiple requests" << endl;
+
+        cl.setProxy("", 80);
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_1_0/A");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/test_1_0/B");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        TestRequest* tr3 = new TestRequest("http://localhost:2000/test_1_0/C");
+        HTTP::Request_ptr own3(tr3);
+        cl.makeRequest(tr3);
+
+        waitForComplete(&cl, tr3);
+        VERIFY(tr->complete);
+        VERIFY(tr2->complete);
+
+        COMPARE(tr->responseLength(), strlen(BODY1));
+        COMPARE(tr->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr->bodyData, string(BODY1));
+
+        COMPARE(tr2->responseLength(), strlen(BODY3));
+        COMPARE(tr2->responseBytesReceived(), strlen(BODY3));
+        COMPARE(tr2->bodyData, string(BODY3));
+        COMPARE(tr3->bodyData, string(BODY1));
+    }
+
+// POST
+    {
+        cout << "testing POST" << endl;
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_post?foo=abc&bar=1234&username=johndoe", "POST");
+        tr->setBodyData("", "application/x-www-form-urlencoded");
+
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 204);
+    }
+
+    // PUT
+        {
+            cout << "testing PUT" << endl;
+            TestRequest* tr = new TestRequest("http://localhost:2000/test_put", "PUT");
+            tr->setBodyData(BODY3, "x-application/foobar");
+
+            HTTP::Request_ptr own(tr);
+            cl.makeRequest(tr);
+            waitForComplete(&cl, tr);
+            COMPARE(tr->responseCode(), 204);
+        }
+
+        {
+            cout << "testing PUT create" << endl;
+            TestRequest* tr = new TestRequest("http://localhost:2000/test_create", "PUT");
+            tr->setBodyData(BODY3, "x-application/foobar");
+
+            HTTP::Request_ptr own(tr);
+            cl.makeRequest(tr);
+            waitForComplete(&cl, tr);
+            COMPARE(tr->responseCode(), 201);
+        }
+
+    // test_zero_length_content
+    {
+        cout << "zero-length-content-response" << endl;
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_zero_length_content");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+        waitForComplete(&cl, tr);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->bodyData, string());
+        COMPARE(tr->responseBytesReceived(), 0);
+    }
+
+    // test cancel
+    {
+        cout <<  "cancel  request" << endl;
+        testServer.resetConnectCount();
+        cl.clearAllConnections();
+
+        cl.setProxy("", 80);
+        TestRequest* tr = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/testLorem");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        TestRequest* tr3 = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own3(tr3);
+        cl.makeRequest(tr3);
+
+        cl.cancelRequest(tr, "my reason 1");
+
+        cl.cancelRequest(tr2, "my reason 2");
+
+        waitForComplete(&cl, tr3);
+
+        COMPARE(tr->responseCode(), -1);
+        COMPARE(tr2->responseReason(), "my reason 2");
+
+        COMPARE(tr3->responseLength(), strlen(BODY1));
+        COMPARE(tr3->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr3->bodyData, string(BODY1));
+    }
+
+    // test cancel
+    {
+        cout <<  "cancel middle request" << endl;
+        testServer.resetConnectCount();
+        cl.clearAllConnections();
+
+        cl.setProxy("", 80);
+        TestRequest* tr = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/testLorem");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        TestRequest* tr3 = new TestRequest("http://localhost:2000/test1");
+        HTTP::Request_ptr own3(tr3);
+        cl.makeRequest(tr3);
+
+        cl.cancelRequest(tr2, "middle request");
+
+        waitForComplete(&cl, tr3);
+
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->responseLength(), strlen(BODY1));
+        COMPARE(tr->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr->bodyData, string(BODY1));
+
+        COMPARE(tr2->responseCode(), -1);
+
+        COMPARE(tr3->responseLength(), strlen(BODY1));
+        COMPARE(tr3->responseBytesReceived(), strlen(BODY1));
+        COMPARE(tr3->bodyData, string(BODY1));
+    }
+
+    {
+        cout << "get-during-response-send" << endl;
+        cl.clearAllConnections();
+        //test_get_during_send
+
+        TestRequest* tr = new TestRequest("http://localhost:2000/test_get_during_send");
+        HTTP::Request_ptr own(tr);
+        cl.makeRequest(tr);
+
+        // kick things along
+        for (int i=0; i<10; ++i) {
+            SGTimeStamp::sleepForMSec(1);
+            cl.update();
+            testServer.poll();
+
+        }
+
+        TestRequest* tr2 = new TestRequest("http://localhost:2000/test_get_during_send_2");
+        HTTP::Request_ptr own2(tr2);
+        cl.makeRequest(tr2);
+
+        waitForComplete(&cl, tr2);
+        COMPARE(tr->responseCode(), 200);
+        COMPARE(tr->bodyData, string(BODY3));
+        COMPARE(tr->responseBytesReceived(), strlen(BODY3));
+        COMPARE(tr2->responseCode(), 200);
+        COMPARE(tr2->bodyData, string(BODY1));
+        COMPARE(tr2->responseBytesReceived(), strlen(BODY1));
+    }
+
+    cout << "all tests passed ok" << endl;
+    return EXIT_SUCCESS;
+}

simgear/misc/CMakeLists.txt

@@ -10,6 +10,7 @@ set(HEADERS
     interpolator.hxx
     make_new.hxx
     sg_dir.hxx
+    sg_hash.hxx
     sg_path.hxx
     sgstream.hxx
     stdint.hxx
@@ -29,6 +30,7 @@ set(SOURCES
     interpolator.cxx
     sg_dir.cxx
     sg_path.cxx
+    sg_hash.cxx
     sgstream.cxx
     strutils.cxx
     tabbed_values.cxx

simgear/misc/sg_hash.cxx

@@ -0,0 +1,28 @@
+// Hash functions with simgear API
+//
+// Copyright (C) 2016  James Turner <zakalawe@mac.com>
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+#include "sg_hash.hxx"
+
+#include <cstring>
+
+namespace simgear
+{
+
+#include "sha1.c"
+
+}

simgear/misc/sg_hash.hxx

@@ -0,0 +1,61 @@
+// Hash functions with simgear API
+//
+// Copyright (C) 2016  James Turner <zakalawe@mac.com>
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Library General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Library General Public License for more details.
+//
+// You should have received a copy of the GNU Library General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA
+
+#include <stdint.h>
+#include <sys/types.h>
+
+namespace simgear
+{
+
+  /* header */
+
+  #define HASH_LENGTH 20
+  #define BLOCK_LENGTH 64
+
+  typedef struct sha1nfo {
+  	uint32_t buffer[BLOCK_LENGTH/4];
+  	uint32_t state[HASH_LENGTH/4];
+  	uint32_t byteCount;
+  	uint8_t bufferOffset;
+  	uint8_t keyBuffer[BLOCK_LENGTH];
+  	uint8_t innerHash[HASH_LENGTH];
+  } sha1nfo;
+
+  /* public API - prototypes - TODO: doxygen*/
+
+  /**
+   */
+  void sha1_init(sha1nfo *s);
+  /**
+   */
+  void sha1_writebyte(sha1nfo *s, uint8_t data);
+  /**
+   */
+  void sha1_write(sha1nfo *s, const char *data, size_t len);
+  /**
+   */
+  uint8_t* sha1_result(sha1nfo *s);
+  /**
+   */
+  void sha1_initHmac(sha1nfo *s, const uint8_t* key, int keyLength);
+  /**
+   */
+  uint8_t* sha1_resultHmac(sha1nfo *s);
+
+
+}

simgear/misc/sg_path.cxx

@@ -233,7 +233,8 @@ SGPath::SGPath(const SGPath& p) :
   _exists(p._exists),
   _isDir(p._isDir),
   _isFile(p._isFile),
-  _modTime(p._modTime)
+  _modTime(p._modTime),
+  _size(p._size)
 {
 }
 
@@ -250,6 +251,7 @@ SGPath& SGPath::operator=(const SGPath& p)
   _isDir = p._isDir;
   _isFile = p._isFile;
   _modTime = p._modTime;
+  _size = p._size;
   return *this;
 }
 
@@ -283,6 +285,7 @@ SGPath::PermissionChecker SGPath::getPermissionChecker() const
 void SGPath::set_cached(bool cached)
 {
   _cacheEnabled = cached;
+  _cached = false;
 }
 
 // append another piece to the existing path
@@ -445,6 +448,7 @@ void SGPath::validate() const
     _isFile = ((S_IFREG & buf.st_mode ) !=0);
     _isDir = ((S_IFDIR & buf.st_mode ) !=0);
     _modTime = buf.st_mtime;
+    _size = buf.st_size;
   }
 
 #else
@@ -457,6 +461,7 @@ void SGPath::validate() const
     _isFile = ((S_ISREG(buf.st_mode )) != 0);
     _isDir = ((S_ISDIR(buf.st_mode )) != 0);
     _modTime = buf.st_mtime;
+    _size = buf.st_size;
   }
   
 #endif
@@ -686,6 +691,12 @@ time_t SGPath::modTime() const
     return _modTime;
 }
 
+size_t SGPath::sizeInBytes() const
+{
+    validate();
+    return _size;
+}
+
 bool SGPath::operator==(const SGPath& other) const
 {
     return (path == other.path);

simgear/misc/sg_path.hxx

@@ -252,7 +252,12 @@ public:
      * modification time of the file
      */
     time_t modTime() const;
-    
+
+    /**
+     *
+     */
+    size_t sizeInBytes() const;
+
     /**
      * rename the file / directory we point at, to a new name
      * this may fail if the new location is on a different volume / share,
@@ -315,6 +320,7 @@ private:
     mutable bool _isDir : 1;
     mutable bool _isFile : 1;
     mutable time_t _modTime;
+    mutable size_t _size;
 };
 
 /// Output to an ostream

simgear/misc/sha1.c

@@ -0,0 +1,291 @@
+/* This code is public-domain - it is based on libcrypt
+ * placed in the public domain by Wei Dai and other contributors.
+ */
+// gcc -Wall -DSHA1TEST -o sha1test sha1.c && ./sha1test
+
+//#include <stdint.h>
+//#include <string.h>
+
+
+#ifdef __BIG_ENDIAN__
+# define SHA_BIG_ENDIAN
+#elif defined __LITTLE_ENDIAN__
+/* override */
+#elif defined __BYTE_ORDER
+# if __BYTE_ORDER__ ==  __ORDER_BIG_ENDIAN__
+# define SHA_BIG_ENDIAN
+# endif
+#elif defined _WIN32
+/* assume little-endian, there is no endian.h on MSVC */
+#else // ! defined __LITTLE_ENDIAN__
+# include <endian.h> // machine/endian.h
+# if __BYTE_ORDER__ ==  __ORDER_BIG_ENDIAN__
+#  define SHA_BIG_ENDIAN
+# endif
+#endif
+
+
+/* code */
+#define SHA1_K0  0x5a827999
+#define SHA1_K20 0x6ed9eba1
+#define SHA1_K40 0x8f1bbcdc
+#define SHA1_K60 0xca62c1d6
+
+void sha1_init(sha1nfo *s) {
+	s->state[0] = 0x67452301;
+	s->state[1] = 0xefcdab89;
+	s->state[2] = 0x98badcfe;
+	s->state[3] = 0x10325476;
+	s->state[4] = 0xc3d2e1f0;
+	s->byteCount = 0;
+	s->bufferOffset = 0;
+}
+
+uint32_t sha1_rol32(uint32_t number, uint8_t bits) {
+	return ((number << bits) | (number >> (32-bits)));
+}
+
+void sha1_hashBlock(sha1nfo *s) {
+	uint8_t i;
+	uint32_t a,b,c,d,e,t;
+
+	a=s->state[0];
+	b=s->state[1];
+	c=s->state[2];
+	d=s->state[3];
+	e=s->state[4];
+	for (i=0; i<80; i++) {
+		if (i>=16) {
+			t = s->buffer[(i+13)&15] ^ s->buffer[(i+8)&15] ^ s->buffer[(i+2)&15] ^ s->buffer[i&15];
+			s->buffer[i&15] = sha1_rol32(t,1);
+		}
+		if (i<20) {
+			t = (d ^ (b & (c ^ d))) + SHA1_K0;
+		} else if (i<40) {
+			t = (b ^ c ^ d) + SHA1_K20;
+		} else if (i<60) {
+			t = ((b & c) | (d & (b | c))) + SHA1_K40;
+		} else {
+			t = (b ^ c ^ d) + SHA1_K60;
+		}
+		t+=sha1_rol32(a,5) + e + s->buffer[i&15];
+		e=d;
+		d=c;
+		c=sha1_rol32(b,30);
+		b=a;
+		a=t;
+	}
+	s->state[0] += a;
+	s->state[1] += b;
+	s->state[2] += c;
+	s->state[3] += d;
+	s->state[4] += e;
+}
+
+void sha1_addUncounted(sha1nfo *s, uint8_t data) {
+	uint8_t * const b = (uint8_t*) s->buffer;
+#ifdef SHA_BIG_ENDIAN
+	b[s->bufferOffset] = data;
+#else
+	b[s->bufferOffset ^ 3] = data;
+#endif
+	s->bufferOffset++;
+	if (s->bufferOffset == BLOCK_LENGTH) {
+		sha1_hashBlock(s);
+		s->bufferOffset = 0;
+	}
+}
+
+void sha1_writebyte(sha1nfo *s, uint8_t data) {
+	++s->byteCount;
+	sha1_addUncounted(s, data);
+}
+
+void sha1_write(sha1nfo *s, const char *data, size_t len) {
+	for (;len--;) sha1_writebyte(s, (uint8_t) *data++);
+}
+
+void sha1_pad(sha1nfo *s) {
+	// Implement SHA-1 padding (fips180-2 §5.1.1)
+
+	// Pad with 0x80 followed by 0x00 until the end of the block
+	sha1_addUncounted(s, 0x80);
+	while (s->bufferOffset != 56) sha1_addUncounted(s, 0x00);
+
+	// Append length in the last 8 bytes
+	sha1_addUncounted(s, 0); // We're only using 32 bit lengths
+	sha1_addUncounted(s, 0); // But SHA-1 supports 64 bit lengths
+	sha1_addUncounted(s, 0); // So zero pad the top bits
+	sha1_addUncounted(s, s->byteCount >> 29); // Shifting to multiply by 8
+	sha1_addUncounted(s, s->byteCount >> 21); // as SHA-1 supports bitstreams as well as
+	sha1_addUncounted(s, s->byteCount >> 13); // byte.
+	sha1_addUncounted(s, s->byteCount >> 5);
+	sha1_addUncounted(s, s->byteCount << 3);
+}
+
+uint8_t* sha1_result(sha1nfo *s) {
+	// Pad to complete the last block
+	sha1_pad(s);
+
+#ifndef SHA_BIG_ENDIAN
+	// Swap byte order back
+	int i;
+	for (i=0; i<5; i++) {
+		s->state[i]=
+			  (((s->state[i])<<24)& 0xff000000)
+			| (((s->state[i])<<8) & 0x00ff0000)
+			| (((s->state[i])>>8) & 0x0000ff00)
+			| (((s->state[i])>>24)& 0x000000ff);
+	}
+#endif
+
+	// Return pointer to hash (20 characters)
+	return (uint8_t*) s->state;
+}
+
+#define HMAC_IPAD 0x36
+#define HMAC_OPAD 0x5c
+
+void sha1_initHmac(sha1nfo *s, const uint8_t* key, int keyLength) {
+	uint8_t i;
+	memset(s->keyBuffer, 0, BLOCK_LENGTH);
+	if (keyLength > BLOCK_LENGTH) {
+		// Hash long keys
+		sha1_init(s);
+		for (;keyLength--;) sha1_writebyte(s, *key++);
+		memcpy(s->keyBuffer, sha1_result(s), HASH_LENGTH);
+	} else {
+		// Block length keys are used as is
+		memcpy(s->keyBuffer, key, keyLength);
+	}
+	// Start inner hash
+	sha1_init(s);
+	for (i=0; i<BLOCK_LENGTH; i++) {
+		sha1_writebyte(s, s->keyBuffer[i] ^ HMAC_IPAD);
+	}
+}
+
+uint8_t* sha1_resultHmac(sha1nfo *s) {
+	uint8_t i;
+	// Complete inner hash
+	memcpy(s->innerHash,sha1_result(s),HASH_LENGTH);
+	// Calculate outer hash
+	sha1_init(s);
+	for (i=0; i<BLOCK_LENGTH; i++) sha1_writebyte(s, s->keyBuffer[i] ^ HMAC_OPAD);
+	for (i=0; i<HASH_LENGTH; i++) sha1_writebyte(s, s->innerHash[i]);
+	return sha1_result(s);
+}
+
+/* self-test */
+
+#if SHA1TEST
+#include <stdio.h>
+
+uint8_t hmacKey1[]={
+	0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x0a,0x0b,0x0c,0x0d,0x0e,0x0f,
+	0x10,0x11,0x12,0x13,0x14,0x15,0x16,0x17,0x18,0x19,0x1a,0x1b,0x1c,0x1d,0x1e,0x1f,
+	0x20,0x21,0x22,0x23,0x24,0x25,0x26,0x27,0x28,0x29,0x2a,0x2b,0x2c,0x2d,0x2e,0x2f,
+	0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3a,0x3b,0x3c,0x3d,0x3e,0x3f
+};
+uint8_t hmacKey2[]={
+	0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3a,0x3b,0x3c,0x3d,0x3e,0x3f,
+	0x40,0x41,0x42,0x43
+};
+uint8_t hmacKey3[]={
+	0x50,0x51,0x52,0x53,0x54,0x55,0x56,0x57,0x58,0x59,0x5a,0x5b,0x5c,0x5d,0x5e,0x5f,
+	0x60,0x61,0x62,0x63,0x64,0x65,0x66,0x67,0x68,0x69,0x6a,0x6b,0x6c,0x6d,0x6e,0x6f,
+	0x70,0x71,0x72,0x73,0x74,0x75,0x76,0x77,0x78,0x79,0x7a,0x7b,0x7c,0x7d,0x7e,0x7f,
+	0x80,0x81,0x82,0x83,0x84,0x85,0x86,0x87,0x88,0x89,0x8a,0x8b,0x8c,0x8d,0x8e,0x8f,
+	0x90,0x91,0x92,0x93,0x94,0x95,0x96,0x97,0x98,0x99,0x9a,0x9b,0x9c,0x9d,0x9e,0x9f,
+	0xa0,0xa1,0xa2,0xa3,0xa4,0xa5,0xa6,0xa7,0xa8,0xa9,0xaa,0xab,0xac,0xad,0xae,0xaf,
+	0xb0,0xb1,0xb2,0xb3
+};
+uint8_t hmacKey4[]={
+	0x70,0x71,0x72,0x73,0x74,0x75,0x76,0x77,0x78,0x79,0x7a,0x7b,0x7c,0x7d,0x7e,0x7f,
+	0x80,0x81,0x82,0x83,0x84,0x85,0x86,0x87,0x88,0x89,0x8a,0x8b,0x8c,0x8d,0x8e,0x8f,
+	0x90,0x91,0x92,0x93,0x94,0x95,0x96,0x97,0x98,0x99,0x9a,0x9b,0x9c,0x9d,0x9e,0x9f,
+	0xa0
+};
+
+void printHash(uint8_t* hash) {
+	int i;
+	for (i=0; i<20; i++) {
+		printf("%02x", hash[i]);
+	}
+	printf("\n");
+}
+
+
+int main (int argc, char **argv) {
+	uint32_t a;
+	sha1nfo s;
+
+	// SHA tests
+	printf("Test: FIPS 180-2 C.1 and RFC3174 7.3 TEST1\n");
+	printf("Expect:a9993e364706816aba3e25717850c26c9cd0d89d\n");
+	printf("Result:");
+	sha1_init(&s);
+	sha1_write(&s, "abc", 3);
+	printHash(sha1_result(&s));
+	printf("\n\n");
+
+	printf("Test: FIPS 180-2 C.2 and RFC3174 7.3 TEST2\n");
+	printf("Expect:84983e441c3bd26ebaae4aa1f95129e5e54670f1\n");
+	printf("Result:");
+	sha1_init(&s);
+	sha1_write(&s, "abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq", 56);
+	printHash(sha1_result(&s));
+	printf("\n\n");
+
+	printf("Test: RFC3174 7.3 TEST4\n");
+	printf("Expect:dea356a2cddd90c7a7ecedc5ebb563934f460452\n");
+	printf("Result:");
+	sha1_init(&s);
+	for (a=0; a<80; a++) sha1_write(&s, "01234567", 8);
+	printHash(sha1_result(&s));
+	printf("\n\n");
+
+	// HMAC tests
+	printf("Test: FIPS 198a A.1\n");
+	printf("Expect:4f4ca3d5d68ba7cc0a1208c9c61e9c5da0403c0a\n");
+	printf("Result:");
+	sha1_initHmac(&s, hmacKey1, 64);
+	sha1_write(&s, "Sample #1",9);
+	printHash(sha1_resultHmac(&s));
+	printf("\n\n");
+
+	printf("Test: FIPS 198a A.2\n");
+	printf("Expect:0922d3405faa3d194f82a45830737d5cc6c75d24\n");
+	printf("Result:");
+	sha1_initHmac(&s, hmacKey2, 20);
+	sha1_write(&s, "Sample #2", 9);
+	printHash(sha1_resultHmac(&s));
+	printf("\n\n");
+
+	printf("Test: FIPS 198a A.3\n");
+	printf("Expect:bcf41eab8bb2d802f3d05caf7cb092ecf8d1a3aa\n");
+	printf("Result:");
+	sha1_initHmac(&s, hmacKey3,100);
+	sha1_write(&s, "Sample #3", 9);
+	printHash(sha1_resultHmac(&s));
+	printf("\n\n");
+
+	printf("Test: FIPS 198a A.4\n");
+	printf("Expect:9ea886efe268dbecce420c7524df32e0751a2a26\n");
+	printf("Result:");
+	sha1_initHmac(&s, hmacKey4,49);
+	sha1_write(&s, "Sample #4", 9);
+	printHash(sha1_resultHmac(&s));
+	printf("\n\n");
+
+	// Long tests
+	printf("Test: FIPS 180-2 C.3 and RFC3174 7.3 TEST3\n");
+	printf("Expect:34aa973cd4c4daa4f61eeb2bdbad27316534016f\n");
+	printf("Result:");
+	sha1_init(&s);
+	for (a=0; a<1000000; a++) sha1_writebyte(&s, 'a');
+	printHash(sha1_result(&s));
+
+	return 0;
+}
+#endif /* self-test */

simgear/misc/strutils.cxx

@@ -24,11 +24,15 @@
 #include <cstring>
 #include <sstream>
 #include <algorithm>
+#include <string.h>             // strerror_r() and strerror_s()
+#include <errno.h>
 
 #include "strutils.hxx"
 
 #include <simgear/debug/logstream.hxx>
 #include <simgear/package/md5.h>
+#include <simgear/compiler.h>   // SG_WINDOWS
+#include <simgear/structure/exception.hxx>
 
 using std::string;
 using std::vector;
@@ -597,6 +601,49 @@ string sanitizePrintfFormat(const string& input)
     return input;
 }
 
+std::string error_string(int errnum)
+{
+  char buf[512];                // somewhat arbitrary...
+  // This could be simplified with C11 (annex K, optional...), which offers:
+  //
+  //   errno_t strerror_s( char *buf, rsize_t bufsz, errno_t errnum );
+  //   size_t strerrorlen_s( errno_t errnum );
+
+#if defined(_WIN32)
+  errno_t retcode;
+  // Always makes the string in 'buf' null-terminated
+  retcode = strerror_s(buf, sizeof(buf), errnum);
+#elif defined(_GNU_SOURCE)
+  return std::string(strerror_r(errnum, buf, sizeof(buf)));
+#elif (_POSIX_C_SOURCE >= 200112L) || defined(SG_MAC)
+  int retcode;
+  // POSIX.1-2001 and POSIX.1-2008
+  retcode = strerror_r(errnum, buf, sizeof(buf));
+#else
+#error "Could not find a thread-safe alternative to strerror()."
+#endif
+
+#if !defined(_GNU_SOURCE)
+  if (retcode) {
+    std::string msg = "unable to get error message for a given error number";
+    // C++11 would make this shorter with std::to_string()
+    std::ostringstream ostr;
+    ostr << errnum;
+
+#if !defined(_WIN32)
+    if (retcode == ERANGE) {    // more specific error message in this case
+      msg = std::string("buffer too small to hold the error message for "
+                        "the specified error number");
+    }
+#endif
+
+    throw sg_error(msg, ostr.str());
+  }
+
+  return std::string(buf);
+#endif  // !defined(_GNU_SOURCE)
+}
+
 } // end namespace strutils
     
 } // end namespace simgear

simgear/misc/strutils.hxx

@@ -216,6 +216,14 @@ namespace simgear {
        */
       std::string sanitizePrintfFormat(const std::string& input);
 
+    /**
+     * Get the message corresponding to a given value of errno.
+     *
+     * Similar to strerror(), except it should be thread-safe and returns an
+     * std::string.
+     */
+    std::string error_string(int errnum);
+
   } // end namespace strutils
 } // end namespace simgear
 

simgear/misc/strutils_test.cxx

@@ -2,6 +2,10 @@
 #define BOOST_TEST_MODULE misc
 #include <BoostTestTargetConfig.h>
 
+#include <errno.h>
+#include <stdlib.h>             // _set_errno() on Windows
+#include <string>
+#include <fstream>              // std::ifstream
 #include <simgear/compiler.h>
 #include "strutils.hxx"
 
@@ -78,3 +82,24 @@ BOOST_AUTO_TEST_CASE( md5_hex )
   // md5
   BOOST_CHECK_EQUAL(strutils::md5("test"), "098f6bcd4621d373cade4e832627b4f6");
 }
+
+BOOST_AUTO_TEST_CASE( error_string )
+{
+#if defined(_WIN32)
+  _set_errno(0);
+#else
+  errno = 0;
+#endif
+
+  std::ifstream f("/\\/non-existent/file/a8f7bz97-3ffe-4f5b-b8db-38ccurJL-");
+
+#if defined(_WIN32)
+  errno_t saved_errno = errno;
+#else
+  int saved_errno = errno;
+#endif
+
+  BOOST_CHECK(!f.is_open());
+  BOOST_CHECK_NE(saved_errno, 0);
+  BOOST_CHECK_GT(strutils::error_string(saved_errno).size(), 0);
+}

simgear/nasal/mathlib.c

@@ -117,15 +117,14 @@ static naRef f_fmod(naContext c, naRef me, int argc, naRef* args)
 static naRef f_clamp(naContext c, naRef me, int argc, naRef* args)
 {
     naRef a = naNumValue(argc > 0 ? args[0] : naNil());
-    naRef b = naNumValue(argc > 1 ? args[1] : naNil());
-    naRef x = naNumValue(argc > 2 ? args[2] : naNil());
+    naRef min = naNumValue(argc > 1 ? args[1] : naNil());
+    naRef max = naNumValue(argc > 2 ? args[2] : naNil());
     
-    if(naIsNil(a) || naIsNil(b) || naIsNil(x))
+    if(naIsNil(a) || naIsNil(min) || naIsNil(max))
         naRuntimeError(c, "non numeric arguments to clamp()");
-    
-    if (a.num < b.num) b.num = a.num;
-    if (b.num > x.num) b.num = x.num;
-    return VALIDATE(b);
+
+    a.num = a.num < min.num ? min.num : ( a.num > max.num ? max.num : a.num );
+    return VALIDATE(a);
 }
 
 static naRef f_periodic(naContext c, naRef me, int argc, naRef* args)

simgear/package/Catalog.cxx

@@ -31,6 +31,7 @@
 #include <simgear/package/Package.hxx>
 #include <simgear/package/Root.hxx>
 #include <simgear/package/Install.hxx>
+#include <simgear/misc/strutils.hxx>
 
 namespace simgear {
 
@@ -40,18 +41,32 @@ bool checkVersion(const std::string& aVersion, SGPropertyNode_ptr props)
 {
     BOOST_FOREACH(SGPropertyNode* v, props->getChildren("version")) {
         std::string s(v->getStringValue());
-        if (s== aVersion) {
+        if (s == aVersion) {
             return true;
         }
 
-        // allow 3.5.* to match any of 3.5.0, 3.5.1rc1, 3.5.11 or so on
-        if (strutils::ends_with(s, ".*")) {
-            size_t lastDot = aVersion.rfind('.');
-            std::string ver = aVersion.substr(0, lastDot);
-            if (ver == s.substr(0, s.length() - 2)) {
-                return true;
+        // examine each dot-seperated component in turn, supporting a wildcard
+        // in the versions from the catalog.
+        string_list appVersionParts = simgear::strutils::split(aVersion, ".");
+        string_list catVersionParts = simgear::strutils::split(s, ".");
+
+        size_t partCount = appVersionParts.size();
+        if (partCount != catVersionParts.size()) {
+            continue;
+        }
+
+        bool ok = true;
+        for (unsigned int p=0; p < partCount; ++p) {
+            if (catVersionParts[p] == "*") {
+                // always passes
+            } else if (appVersionParts[p] != catVersionParts[p]) {
+                ok = false;
             }
         }
+
+        if (ok) {
+            return true;
+        }
     }
     return false;
 }
@@ -104,7 +119,7 @@ protected:
         try {
             readProperties(m_buffer.data(), m_buffer.size(), props);
             m_owner->parseProps(props);
-        } catch (sg_exception& e) {
+        } catch (sg_exception& ) {
             SG_LOG(SG_GENERAL, SG_ALERT, "catalog parse failure:" << m_owner->url());
             m_owner->refreshComplete(Delegate::FAIL_EXTRACT);
             return;
@@ -192,26 +207,17 @@ CatalogRef Catalog::createFromPath(Root* aRoot, const SGPath& aPath)
     try {
         props = new SGPropertyNode;
         readProperties(xml.str(), props);
-    } catch (sg_exception& e) {
+    } catch (sg_exception& ) {
         return NULL;
     }
 
-    if (!checkVersion(aRoot->applicationVersion(), props)) {
-        std::string redirect = redirectUrlForVersion(aRoot->applicationVersion(), props);
-        if (!redirect.empty()) {
-            SG_LOG(SG_GENERAL, SG_WARN, "catalog at " << aPath << ", version mismatch:\n\t"
-                   << "redirecting to alternate URL:" << redirect);
-            CatalogRef c = Catalog::createFromUrl(aRoot, redirect);
-            c->m_installRoot = aPath;
-            return c;
-        } else {
-            SG_LOG(SG_GENERAL, SG_WARN, "skipping catalog at " << aPath << ", version mismatch:\n\t"
-               << props->getStringValue("version") << " vs required " << aRoot->catalogVersion());
-            return NULL;
-        }
+    bool versionCheckOk = checkVersion(aRoot->applicationVersion(), props);
 
+    if (!versionCheckOk) {
+        SG_LOG(SG_GENERAL, SG_INFO, "catalog at:" << aPath << " failed version check: need" << aRoot->applicationVersion());
+        // keep the catalog but mark it as needing an update
     } else {
-        SG_LOG(SG_GENERAL, SG_INFO, "creating catalog from:" << aPath);
+        SG_LOG(SG_GENERAL, SG_DEBUG, "creating catalog from:" << aPath);
     }
 
     CatalogRef c = new Catalog(aRoot);
@@ -219,8 +225,12 @@ CatalogRef Catalog::createFromPath(Root* aRoot, const SGPath& aPath)
     c->parseProps(props);
     c->parseTimestamp();
 
-    // parsed XML ok, mark status as valid
-    c->changeStatus(Delegate::STATUS_SUCCESS);
+    if (versionCheckOk) {
+        // parsed XML ok, mark status as valid
+        c->changeStatus(Delegate::STATUS_SUCCESS);
+    } else {
+        c->changeStatus(Delegate::FAIL_VERSION);
+    }
 
     return c;
 }
@@ -469,6 +479,11 @@ unsigned int Catalog::ageInSeconds() const
 
 bool Catalog::needsRefresh() const
 {
+    // always refresh in these cases
+    if ((m_status == Delegate::FAIL_VERSION) || (m_status == Delegate::FAIL_DOWNLOAD)) {
+        return true;
+    }
+
     unsigned int maxAge = m_props->getIntValue("max-age-sec", m_root->maxAgeSeconds());
     return (ageInSeconds() > maxAge);
 }

simgear/package/CatalogTest.cxx

@@ -23,15 +23,102 @@
 
 #include <cstdlib>
 #include <iostream>
+#include <fstream>
 
 #include <simgear/package/Catalog.hxx>
 #include <simgear/package/Root.hxx>
 #include <simgear/package/Package.hxx>
+#include <simgear/package/Install.hxx>
 
+#include <simgear/misc/test_macros.hxx>
 #include <simgear/misc/sg_dir.hxx>
+#include <simgear/timing/timestamp.hxx>
+
+#include <simgear/io/test_HTTP.hxx>
+#include <simgear/io/HTTPClient.hxx>
+#include <simgear/io/sg_file.hxx>
+#include <simgear/structure/exception.hxx>
 
 using namespace simgear;
 
+std::string readFileIntoString(const SGPath& path)
+{
+    std::string contents;
+
+    size_t readLen;
+    SGBinaryFile f(path.str());
+    if (!f.open(SG_IO_IN)) {
+        throw sg_io_exception("Couldn't open file", path);
+    }
+
+    char buf[8192];
+    while ((readLen = f.read(buf, 8192)) > 0) {
+        contents += std::string(buf, readLen);
+    }
+
+    return contents;
+}
+
+SGPath global_serverFilesRoot;
+unsigned int global_catalogVersion = 0;
+
+class TestPackageChannel : public TestServerChannel
+{
+public:
+
+    virtual void processRequestHeaders()
+    {
+        state = STATE_IDLE;
+        SGPath localPath(global_serverFilesRoot);
+
+
+        if (path == "/catalogTest1/catalog.xml") {
+            if (global_catalogVersion > 0) {
+                std::stringstream ss;
+                ss << "/catalogTest1/catalog-v" << global_catalogVersion << ".xml";
+                path = ss.str();
+            }
+        }
+
+        localPath.append(path);
+
+      //  SG_LOG(SG_IO, SG_INFO, "local path is:" << localPath.str());
+
+        if (localPath.exists()) {
+            std::string content = readFileIntoString(localPath);
+            std::stringstream d;
+            d << "HTTP/1.1 " << 200 << " " << reasonForCode(200) << "\r\n";
+            d << "Content-Length:" << content.size() << "\r\n";
+            d << "\r\n"; // final CRLF to terminate the headers
+            d << content;
+
+            std::string ds(d.str());
+            bufferSend(ds.data(), ds.size());
+        } else {
+            sendErrorResponse(404, false, "");
+        }
+    }
+};
+
+TestServer<TestPackageChannel> testServer;
+
+void waitForUpdateComplete(HTTP::Client* cl, pkg::Root* root)
+{
+    SGTimeStamp start(SGTimeStamp::now());
+    while (start.elapsedMSec() <  10000) {
+        cl->update();
+        testServer.poll();
+
+        if (!cl->hasActiveRequests()) {
+            return;
+        }
+
+        SGTimeStamp::sleepForMSec(15);
+    }
+
+    std::cerr << "timed out" << std::endl;
+}
+
 int parseTest()
 {
     SGPath rootPath = simgear::Dir::current().path();
@@ -42,7 +129,7 @@ int parseTest()
     VERIFY(cat.valid());
 
     COMPARE(cat->id(), "org.flightgear.test.catalog1");
-    COMPARE(cat->url(), "http://download.flightgear.org/catalog1/catalog.xml");
+    COMPARE(cat->url(), "http://localhost:2000/catalogTest1/catalog.xml");
     COMPARE(cat->description(), "First test catalog");
 
 // check the packages too
@@ -55,7 +142,7 @@ int parseTest()
     COMPARE(p1->qualifiedId(), "org.flightgear.test.catalog1.alpha");
     COMPARE(p1->name(), "Alpha package");
     COMPARE(p1->revision(), 8);
-    COMPARE(p1->fileSizeBytes(), 1234567);
+    COMPARE(p1->fileSizeBytes(), 593);
 
 
     pkg::PackageRef p2 = cat->getPackageById("c172p");
@@ -99,9 +186,245 @@ int parseTest()
     return EXIT_SUCCESS;
 }
 
+void testAddCatalog(HTTP::Client* cl)
+{
+// erase dir
+    SGPath rootPath(simgear::Dir::current().path());
+    rootPath.append("pkg_add_catalog");
+    simgear::Dir pd(rootPath);
+    pd.removeChildren();
+
+
+    pkg::RootRef root(new pkg::Root(rootPath, "8.1.2"));
+    // specify a test dir
+    root->setHTTPClient(cl);
+
+    pkg::CatalogRef c = pkg::Catalog::createFromUrl(root.ptr(), "http://localhost:2000/catalogTest1/catalog.xml");
+
+    waitForUpdateComplete(cl, root);
+
+// verify on disk state
+    SGPath p(rootPath);
+    p.append("org.flightgear.test.catalog1");
+    p.append("catalog.xml");
+    VERIFY(p.exists());
+    COMPARE(root->allPackages().size(), 3);
+    COMPARE(root->catalogs().size(), 1);
+
+    pkg::PackageRef p1 = root->getPackageById("alpha");
+    COMPARE(p1->id(), "alpha");
+
+    pkg::PackageRef p2 = root->getPackageById("org.flightgear.test.catalog1.c172p");
+    COMPARE(p2->id(), "c172p");
+    COMPARE(p2->qualifiedId(), "org.flightgear.test.catalog1.c172p");
+
+}
+
+void testInstallPackage(HTTP::Client* cl)
+{
+    SGPath rootPath(simgear::Dir::current().path());
+    rootPath.append("pkg_install_with_dep");
+    simgear::Dir pd(rootPath);
+    pd.removeChildren();
+
+    pkg::RootRef root(new pkg::Root(rootPath, "8.1.2"));
+    // specify a test dir
+    root->setHTTPClient(cl);
+
+    pkg::CatalogRef c = pkg::Catalog::createFromUrl(root.ptr(), "http://localhost:2000/catalogTest1/catalog.xml");
+    waitForUpdateComplete(cl, root);
+
+    pkg::PackageRef p1 = root->getPackageById("org.flightgear.test.catalog1.c172p");
+    pkg::InstallRef ins = p1->install();
+
+    VERIFY(ins->isQueued());
+
+    waitForUpdateComplete(cl, root);
+    VERIFY(p1->isInstalled());
+    VERIFY(p1->existingInstall() == ins);
+
+    pkg::PackageRef commonDeps = root->getPackageById("common-sounds");
+    VERIFY(commonDeps->existingInstall());
+
+    // verify on disk state
+    SGPath p(rootPath);
+    p.append("org.flightgear.test.catalog1");
+    p.append("Aircraft");
+    p.append("c172p");
+
+    COMPARE(p, ins->path());
+
+    p.append("c172p-floats-set.xml");
+    VERIFY(p.exists());
+
+    SGPath p2(rootPath);
+    p2.append("org.flightgear.test.catalog1");
+    p2.append("Aircraft");
+    p2.append("sounds");
+    p2.append("sharedfile.txt");
+    VERIFY(p2.exists());
+}
+
+void testUninstall(HTTP::Client* cl)
+{
+    SGPath rootPath(simgear::Dir::current().path());
+    rootPath.append("pkg_uninstall");
+    simgear::Dir pd(rootPath);
+    pd.removeChildren();
+
+    pkg::RootRef root(new pkg::Root(rootPath, "8.1.2"));
+    root->setHTTPClient(cl);
+
+    pkg::CatalogRef c = pkg::Catalog::createFromUrl(root.ptr(), "http://localhost:2000/catalogTest1/catalog.xml");
+    waitForUpdateComplete(cl, root);
+
+    pkg::PackageRef p1 = root->getPackageById("org.flightgear.test.catalog1.c172p");
+    pkg::InstallRef ins = p1->install();
+
+    waitForUpdateComplete(cl, root);
+
+    VERIFY(p1->isInstalled());
+
+    ins->uninstall();
+
+    VERIFY(!ins->path().exists());
+}
+
+void testRemoveCatalog(HTTP::Client* cl)
+{
+    SGPath rootPath(simgear::Dir::current().path());
+    rootPath.append("pkg_catalog_remove");
+    simgear::Dir pd(rootPath);
+    pd.removeChildren();
+
+    pkg::RootRef root(new pkg::Root(rootPath, "8.1.2"));
+    root->setHTTPClient(cl);
+
+    {
+        pkg::CatalogRef c = pkg::Catalog::createFromUrl(root.ptr(), "http://localhost:2000/catalogTest1/catalog.xml");
+        waitForUpdateComplete(cl, root);
+    }
+
+    {
+        pkg::PackageRef p1 = root->getPackageById("org.flightgear.test.catalog1.c172p");
+        pkg::InstallRef ins = p1->install();
+
+        waitForUpdateComplete(cl, root);
+
+        VERIFY(p1->isInstalled());
+    }
+
+    root->removeCatalogById("org.flightgear.test.catalog1");
+
+
+    SGPath p2(rootPath);
+    p2.append("org.flightgear.test.catalog1");
+    VERIFY(!p2.exists());
+
+    VERIFY(root->allPackages().empty());
+    VERIFY(root->catalogs().empty());
+
+    pkg::CatalogRef c = root->getCatalogById("org.flightgear.test.catalog1");
+    COMPARE(c, pkg::CatalogRef());
+}
+
+template <class T>
+bool contains(const std::vector<T>& vec, const T& item)
+{
+    typename std::vector<T>::const_iterator it = std::find(vec.begin(), vec.end(), item);
+    return (it != vec.end());
+}
+
+void testRefreshCatalog(HTTP::Client* cl)
+{
+    SGPath rootPath(simgear::Dir::current().path());
+    rootPath.append("pkg_catalog_refresh_update");
+    simgear::Dir pd(rootPath);
+    pd.removeChildren();
+
+    pkg::RootRef root(new pkg::Root(rootPath, "8.1.2"));
+    root->setHTTPClient(cl);
+
+    {
+        pkg::CatalogRef c = pkg::Catalog::createFromUrl(root.ptr(), "http://localhost:2000/catalogTest1/catalog.xml");
+        waitForUpdateComplete(cl, root);
+    }
+
+    {
+        pkg::PackageRef p1 = root->getPackageById("org.flightgear.test.catalog1.c172p");
+        pkg::InstallRef ins = p1->install();
+
+        pkg::PackageRef p2 = root->getPackageById("org.flightgear.test.catalog1.alpha");
+        pkg::InstallRef ins2 = p2->install();
+
+        waitForUpdateComplete(cl, root);
+
+        VERIFY(p1->isInstalled());
+        VERIFY(p2->isInstalled());
+    }
+
+    VERIFY(root->packagesNeedingUpdate().empty());
+
+    global_catalogVersion = 2;
+
+    VERIFY(!cl->hasActiveRequests());
+    root->refresh();
+
+    // should be a no-op due to catalog age testing
+    VERIFY(!cl->hasActiveRequests());
+
+    // force it this time
+    root->refresh(true);
+    VERIFY(cl->hasActiveRequests());
+    waitForUpdateComplete(cl, root);
+
+    pkg::CatalogRef c = root->getCatalogById("org.flightgear.test.catalog1");
+    COMPARE(c->ageInSeconds(), 0);
+
+    VERIFY(root->getPackageById("dc3") != pkg::PackageRef());
+    COMPARE(root->packagesNeedingUpdate().size(), 2);
+
+    pkg::PackageList needingUpdate = root->packagesNeedingUpdate();
+    VERIFY(contains(needingUpdate, root->getPackageById("common-sounds")));
+    VERIFY(contains(needingUpdate, root->getPackageById("org.flightgear.test.catalog1.alpha")));
+
+    pkg::InstallRef ins = root->getPackageById("alpha")->existingInstall();
+    VERIFY(ins->hasUpdate());
+
+    for (pkg::PackageList::const_iterator it = needingUpdate.begin();
+         it != needingUpdate.end(); ++it)
+    {
+        root->scheduleToUpdate((*it)->existingInstall());
+    }
+
+    waitForUpdateComplete(cl, root);
+
+    VERIFY(root->packagesNeedingUpdate().empty());
+    COMPARE(root->getPackageById("common-sounds")->revision(), 11);
+}
+
+
 int main(int argc, char* argv[])
 {
+    sglog().setLogLevels( SG_ALL, SG_DEBUG );
+
+    HTTP::Client cl;
+    cl.setMaxConnections(1);
+
+    global_serverFilesRoot = SGPath(SRC_DIR);
+
+    testAddCatalog(&cl);
+
     parseTest();
+
+    testInstallPackage(&cl);
+    
+    testUninstall(&cl);
+
+    testRemoveCatalog(&cl);
+
+    testRefreshCatalog(&cl);
+
     std::cout << "Successfully passed all tests!" << std::endl;
     return EXIT_SUCCESS;
 }

simgear/package/Install.cxx

@@ -166,6 +166,9 @@ protected:
             return;
         }
 
+        // extract_xxxx directory is now empty, so remove it
+        m_extractPath.remove();
+
         m_owner->m_revision = m_owner->package()->revision();
         m_owner->writeRevisionFile();
         m_owner->m_download.reset(); // so isDownloading reports false
@@ -273,7 +276,7 @@ private:
                     throw sg_io_exception("failed to go to next file in the archive");
                 }
             }
-        } catch (sg_exception& e) {
+        } catch (sg_exception& ) {
             result = false;
         }
 
@@ -409,7 +412,7 @@ size_t Install::downloadedBytes() const
 void Install::cancelDownload()
 {
     if (m_download.valid()) {
-        m_download->abort("User cancelled download");
+        m_package->catalog()->root()->cancelHTTPRequest(m_download, "User cancelled download");
     }
 
     if (m_revision == 0) {

simgear/package/Package.cxx

@@ -142,7 +142,7 @@ InstallRef Package::existingInstall(const InstallCallback& cb) const
     InstallRef install;
     try {
         install = m_catalog->root()->existingInstallForPackage(const_cast<Package*>(this));
-    } catch (std::exception& e) {
+    } catch (std::exception& ) {
         return InstallRef();
     }
 
@@ -281,7 +281,7 @@ PackageList Package::dependencies() const
     PackageList result;
     
     BOOST_FOREACH(SGPropertyNode* dep, m_props->getChildren("depends")) {
-        std::string depName = dep->getStringValue("package");
+        std::string depName = dep->getStringValue("id");
         unsigned int rev = dep->getIntValue("revision", 0);
         
     // prefer local hangar package if possible, in case someone does something

simgear/package/Root.cxx

@@ -159,6 +159,7 @@ public:
     std::string locale;
     HTTP::Client* http;
     CatalogDict catalogs;
+    CatalogList disabledCatalogs;
     unsigned int maxAgeSeconds;
     std::string version;
 
@@ -225,6 +226,20 @@ void Root::makeHTTPRequest(HTTP::Request *req)
     d->httpPendingRequests.push_back(req);
 }
 
+void Root::cancelHTTPRequest(HTTP::Request *req, const std::string &reason)
+{
+    if (d->http) {
+        d->http->cancelRequest(req, reason);
+    }
+
+    std::deque<HTTP::Request_ptr>::iterator it = std::find(d->httpPendingRequests.begin(),
+                                                           d->httpPendingRequests.end(),
+                                                           req);
+    if (it != d->httpPendingRequests.end()) {
+        d->httpPendingRequests.erase(it);
+    }
+}
+
 Root::Root(const SGPath& aPath, const std::string& aVersion) :
     d(new RootPrivate)
 {
@@ -240,10 +255,16 @@ Root::Root(const SGPath& aPath, const std::string& aVersion) :
         return;
     }
 
-    BOOST_FOREACH(SGPath c, dir.children(Dir::TYPE_DIR)) {
+    BOOST_FOREACH(SGPath c, dir.children(Dir::TYPE_DIR | Dir::NO_DOT_OR_DOTDOT)) {
         CatalogRef cat = Catalog::createFromPath(this, c);
         if (cat) {
-           d->catalogs[cat->id()] = cat;
+            if (cat->status() == Delegate::STATUS_SUCCESS) {
+                d->catalogs[cat->id()] = cat;
+            } else {
+                // catalog has problems, such as needing an update
+                // keep it out of the main collection for now
+                d->disabledCatalogs.push_back(cat);
+            }
         }
     } // of child directories iteration
 }
@@ -357,14 +378,28 @@ Root::packagesNeedingUpdate() const
 void Root::refresh(bool aForce)
 {
     bool didStartAny = false;
+
+    // copy all candidate catalogs to a seperate list, since refreshing
+    // can modify both the main collection and/or the disabled list
+    CatalogList toRefresh;
     CatalogDict::iterator it = d->catalogs.begin();
     for (; it != d->catalogs.end(); ++it) {
-        if (aForce || it->second->needsRefresh()) {
-            it->second->refresh();
-            didStartAny = true;
+        int age = it->second->ageInSeconds();
+        if (aForce || (age > maxAgeSeconds())) {
+            toRefresh.push_back(it->second);
         }
     }
 
+    toRefresh.insert(toRefresh.end(), d->disabledCatalogs.begin(),
+                     d->disabledCatalogs.end());
+
+
+    CatalogList::iterator j = toRefresh.begin();
+    for (; j != toRefresh.end(); ++j) {
+        (*j)->refresh();
+        didStartAny =  true;
+    }
+
     if (!didStartAny) {
         // signal refresh complete to the delegate already
         d->fireRefreshStatus(CatalogRef(), Delegate::STATUS_REFRESHED);
@@ -486,23 +521,39 @@ void Root::catalogRefreshStatus(CatalogRef aCat, Delegate::StatusCode aReason)
         d->refreshing.erase(aCat);
     }
 
-    if ((aReason != Delegate::STATUS_REFRESHED) && (aReason != Delegate::STATUS_IN_PROGRESS)) {
-        // if the failure is permanent, delete the catalog from our
-        // list (don't touch it on disk)
-        bool isPermanentFailure = (aReason == Delegate::FAIL_VERSION);
-        if (isPermanentFailure) {
-            SG_LOG(SG_GENERAL, SG_WARN, "permanent failure for catalog:" << aCat->id());
-            if (catIt != d->catalogs.end()) {
-                d->catalogs.erase(catIt);
-            }
-        }
-    }
-
     if ((aReason == Delegate::STATUS_REFRESHED) && (catIt == d->catalogs.end())) {
         assert(!aCat->id().empty());
         d->catalogs.insert(catIt, CatalogDict::value_type(aCat->id(), aCat));
+
+        // catalog might have been previously disabled, let's remove in that case
+        CatalogList::iterator j = std::find(d->disabledCatalogs.begin(),
+                                            d->disabledCatalogs.end(),
+                                            aCat);
+        if (j != d->disabledCatalogs.end()) {
+            SG_LOG(SG_GENERAL, SG_INFO, "re-enabling disabled catalog:" << aCat->id());
+            d->disabledCatalogs.erase(j);
+        }
     }
 
+    if ((aReason != Delegate::STATUS_REFRESHED) &&
+        (aReason != Delegate::STATUS_IN_PROGRESS) &&
+        (aReason != Delegate::STATUS_SUCCESS))
+    {
+        // catalog has errors, disable it
+        CatalogList::iterator j = std::find(d->disabledCatalogs.begin(),
+                                            d->disabledCatalogs.end(),
+                                            aCat);
+        if (j == d->disabledCatalogs.end()) {
+            SG_LOG(SG_GENERAL, SG_INFO, "disabling catalog:" << aCat->id());
+            d->disabledCatalogs.push_back(aCat);
+        }
+
+        // and remove it from the active collection
+        if (catIt != d->catalogs.end()) {
+            d->catalogs.erase(catIt);
+        }
+    } // of catalog has errors case
+
     if (d->refreshing.empty()) {
         d->fireRefreshStatus(CatalogRef(), Delegate::STATUS_REFRESHED);
         d->firePackagesChanged();
@@ -511,17 +562,31 @@ void Root::catalogRefreshStatus(CatalogRef aCat, Delegate::StatusCode aReason)
 
 bool Root::removeCatalogById(const std::string& aId)
 {
+    CatalogRef cat;
+
     CatalogDict::iterator catIt = d->catalogs.find(aId);
     if (catIt == d->catalogs.end()) {
-        SG_LOG(SG_GENERAL, SG_WARN, "removeCatalogById: unknown ID:" << aId);
-        return false;
+        // check the disabled list
+        CatalogList::iterator j = d->disabledCatalogs.begin();
+        for (; j != d->disabledCatalogs.end(); ++j) {
+            if ((*j)->id() == aId) {
+                break;
+            }
+        }
+
+        if (j == d->disabledCatalogs.end()) {
+            SG_LOG(SG_GENERAL, SG_WARN, "removeCatalogById: no catalog with id:" << aId);
+            return false;
+        }
+
+        cat = *j;
+        d->disabledCatalogs.erase(j);
+    } else {
+        cat = catIt->second;
+        // drop the reference
+        d->catalogs.erase(catIt);
     }
 
-    CatalogRef cat = catIt->second;
-
-    // drop the reference
-    d->catalogs.erase(catIt);
-
     bool ok = cat->uninstall();
     if (!ok) {
         SG_LOG(SG_GENERAL, SG_WARN, "removeCatalogById: catalog :" << aId

simgear/package/Root.hxx

@@ -81,6 +81,11 @@ public:
      * set yet.
      */
     void makeHTTPRequest(HTTP::Request* req);
+
+    /**
+     * Cancel an HTTP request.
+     */
+    void cancelHTTPRequest(HTTP::Request* req, const std::string& reason);
     
     /**
      * The catalog XML/property version in use. This is used to make incomaptible

simgear/package/catalogTest1/alpha/alpha-set.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+
+<PropertyList>
+  <sim>
+    
+  </sim>
+</PropertyList>

simgear/package/catalogTest1/alpha/somefile.txt

@@ -0,0 +1 @@
+The quick brown fox jumps over a lazy dog.

simgear/package/catalogTest1/c172p/c172p-2d-panel-set.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+
+<PropertyList>
+  <sim>
+
+  </sim>
+</PropertyList>

simgear/package/catalogTest1/c172p/c172p-floats-set.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+
+<PropertyList>
+  <sim>
+
+  </sim>
+</PropertyList>

simgear/package/catalogTest1/c172p/c172p-set.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+
+<PropertyList>
+  <sim>
+
+  </sim>
+</PropertyList>