From Peter Hrenka, (note from Robert Osfield, renamed GenericPrimitiveFunctor mention below to TemplatePrimitiveFunctor).

"Since we desperately needed a means for picking Lines
and Points I implemented (hopefully!) proper geometrical tests
for the PolytopeIntersector.

First of all I implemented a new "GenericPrimiteFunctor"
which is basically an extended copy TriangleFunctor which also
handles Points, Lines and Quads through suitable overloads of
operator(). I would have liked to call it "PrimitiveFunctor"
but that name was already used...
I used a template method to remove redundancy in the
drawElements method overloads. If you know of platforms where
this will not work I can change it to the style used
in TriangleFunctor.

In PolytopeIntersector.cpp I implemented a
"PolytopePrimitiveIntersector" which provides the needed
overloads for Points, Lines, Triangles and Quads to
the GenericPrimitiveFunctor. This is then used in the
intersect method of PolytopeIntersector.

Implementation summary:
- Points: Check distance to all planes
- Lines: Check distance of both ends against each plane.
  If both are outside -> line is out
  If both are in -> continue checking
  One is in, one is out -> compute intersection point (candidate)
  Then check all candidates against all other polytope
  planes. The remaining candidates are the proper
  intersection points of the line with the polytope.
- Triangles: Perform Line-Checks for all edges of the
  triangle as above. If there is an proper intersection
  -> done.
  In the case where there are more than 2 polytope
  plane to check against we have to check for the case
  where the triangle encloses the polytope.
  In that case the intersection lines of the polytope
  planes are computed and checked against the triangle.
- Quads: handled as two triangles.

This is implementation is certainly not the fastest.
There are certainly ways and strategies to improve it.


I also enabled the code for PolytopeIntersector
in osgkeyboardmouse and added keybindings to
switch the type of intersector ('p') and the picking
coordinate system ('c') on the fly. Since the
PolytopeIntersector does not have a canonical
ordering for its intersections (as opposed to
the LineSegementIntersector) I chaged the
implementation to toggle all hit geometries.


I tested the functionality with osgkeyboardmouse
and several models and it seems to work for
polygonal models. Special nodes such as billboards
do not work.


The next thing on my todo-list is to implement
a an improved Intersection-Structure for the
PolytopeIntersector. We need to know
which primitives where hit (and where).

"

examples/osgkeyboardmouse/osgkeyboardmouse.cpp

@@ -143,7 +143,9 @@ class PickHandler : public osgGA::GUIEventHandler
 public: 
 
     PickHandler():
-        _mx(0.0),_my(0.0) {}
+        _mx(0.0),_my(0.0),
+        _usePolytopeIntersector(false),
+        _useWindowCoordinates(false) {}
 
     ~PickHandler() {}
 
@@ -165,6 +167,26 @@ public:
                     osg::notify(osg::NOTICE)<<"Saved model to file 'saved_model.osg'"<<std::endl;
                     osgDB::writeNodeFile(*(viewer->getSceneData()), "saved_model.osg");
                 }
+                else if (ea.getKey()=='p')
+                {
+                    _usePolytopeIntersector = !_usePolytopeIntersector;
+                    if (_usePolytopeIntersector)
+                    {
+                        osg::notify(osg::NOTICE)<<"Using PolytopeIntersector"<<std::endl;
+                    } else {
+                        osg::notify(osg::NOTICE)<<"Using LineSegmentIntersector"<<std::endl;
+                    }
+                }
+                else if (ea.getKey()=='c')
+                {
+                    _useWindowCoordinates = !_useWindowCoordinates;
+                    if (_useWindowCoordinates)
+                    {
+                        osg::notify(osg::NOTICE)<<"Using window coordinates for picking"<<std::endl;
+                    } else {
+                        osg::notify(osg::NOTICE)<<"Using projection coordiates for picking"<<std::endl;
+                    }
+                }
                 else if (ea.getKey()==osgGA::GUIEventAdapter::KEY_Delete || ea.getKey()==osgGA::GUIEventAdapter::KEY_BackSpace)
                 {
                     osg::notify(osg::NOTICE)<<"Delete"<<std::endl;
@@ -206,60 +228,66 @@ public:
         osg::Node* node = 0;
         osg::Group* parent = 0;
 
-        bool usePolytopePicking = false;
-        if (usePolytopePicking)
+        if (_usePolytopeIntersector)
         {
+            osgUtil::PolytopeIntersector* picker;
+            if (_useWindowCoordinates)
+            {
+                // use window coordinates
+                // remap the mouse x,y into viewport coordinates.
+                osg::Viewport* viewport = viewer->getCamera()->getViewport();
+                double mx = viewport->x() + (int)((double )viewport->width()*(ea.getXnormalized()*0.5+0.5));
+                double my = viewport->y() + (int)((double )viewport->height()*(ea.getYnormalized()*0.5+0.5));
 
-#if 0
-            // use window coordinates
-            // remap the mouse x,y into viewport coordinates.
-            osg::Viewport* viewport = viewer->getCamera()->getViewport();
-            double mx = viewport->x() + (int)((double )viewport->width()*(ea.getXnormalized()*0.5+0.5));
-            double my = viewport->y() + (int)((double )viewport->height()*(ea.getYnormalized()*0.5+0.5));
-
-            // half width, height.
-            double w = 5.0f;
-            double h = 5.0f;
-            osgUtil::PolytopeIntersector* picker = new osgUtil::PolytopeIntersector( osgUtil::Intersector::WINDOW, mx-w, my-h, mx+w, my+h );
-#else
-            double mx = ea.getXnormalized();
-            double my = ea.getYnormalized();
-            double w = 0.05;
-            double h = 0.05;
-            osgUtil::PolytopeIntersector* picker = new osgUtil::PolytopeIntersector( osgUtil::Intersector::PROJECTION, mx-w, my-h, mx+w, my+h );
-#endif
+                // half width, height.
+                double w = 5.0f;
+                double h = 5.0f;
+                picker = new osgUtil::PolytopeIntersector( osgUtil::Intersector::WINDOW, mx-w, my-h, mx+w, my+h );
+            } else {
+                double mx = ea.getXnormalized();
+                double my = ea.getYnormalized();
+                double w = 0.05;
+                double h = 0.05;
+                picker = new osgUtil::PolytopeIntersector( osgUtil::Intersector::PROJECTION, mx-w, my-h, mx+w, my+h );
+            }
             osgUtil::IntersectionVisitor iv(picker);
 
             viewer->getCamera()->accept(iv);
 
             if (picker->containsIntersections())
             {
-                osgUtil::PolytopeIntersector::Intersection intersection = picker->getFirstIntersection();
+                osgUtil::PolytopeIntersector::Intersections& intersections = picker->getIntersections();
 
-                osg::NodePath& nodePath = intersection.nodePath;
-                node = (nodePath.size()>=1)?nodePath[nodePath.size()-1]:0;
-                parent = (nodePath.size()>=2)?dynamic_cast<osg::Group*>(nodePath[nodePath.size()-2]):0;
+                for (osgUtil::PolytopeIntersector::Intersections::iterator it=intersections.begin();
+                     it!=intersections.end(); ++it) {
+                    osgUtil::PolytopeIntersector::Intersection intersection=*it;
 
-                if (node) std::cout<<"  Hits "<<node->className()<<" nodePath size"<<nodePath.size()<<std::endl;
+                    osg::NodePath& nodePath = intersection.nodePath;
+                    node = (nodePath.size()>=1)?nodePath[nodePath.size()-1]:0;
+                    parent = (nodePath.size()>=2)?dynamic_cast<osg::Group*>(nodePath[nodePath.size()-2]):0;
+
+                    if (node) std::cout<<"  Hits "<<node->className()<<" nodePath size"<<nodePath.size()<<std::endl;
+                    toggleScribe(parent, node);
+                }
 
             }
 
         }
         else
         {
-
-            #if 0
-            // use non dimensional coordinates - in projection/clip space
-            osgUtil::LineSegmentIntersector* picker = new osgUtil::LineSegmentIntersector( osgUtil::Intersector::PROJECTION, ea.getXnormalized(),ea.getYnormalized() );
-            #else
-            // use window coordinates
-            // remap the mouse x,y into viewport coordinates.
-            osg::Viewport* viewport = viewer->getCamera()->getViewport();
-            float mx = viewport->x() + (int)((float)viewport->width()*(ea.getXnormalized()*0.5f+0.5f));
-            float my = viewport->y() + (int)((float)viewport->height()*(ea.getYnormalized()*0.5f+0.5f));
-            osgUtil::LineSegmentIntersector* picker = new osgUtil::LineSegmentIntersector( osgUtil::Intersector::WINDOW, mx, my );
-            #endif
-
+            osgUtil::LineSegmentIntersector* picker;
+            if (!_useWindowCoordinates)
+            {
+                // use non dimensional coordinates - in projection/clip space
+                picker = new osgUtil::LineSegmentIntersector( osgUtil::Intersector::PROJECTION, ea.getXnormalized(),ea.getYnormalized() );
+            } else {
+                // use window coordinates
+                // remap the mouse x,y into viewport coordinates.
+                osg::Viewport* viewport = viewer->getCamera()->getViewport();
+                float mx = viewport->x() + (int)((float)viewport->width()*(ea.getXnormalized()*0.5f+0.5f));
+                float my = viewport->y() + (int)((float)viewport->height()*(ea.getYnormalized()*0.5f+0.5f));
+                picker = new osgUtil::LineSegmentIntersector( osgUtil::Intersector::WINDOW, mx, my );
+            }
             osgUtil::IntersectionVisitor iv(picker);
 
             viewer->getCamera()->accept(iv);
@@ -274,36 +302,38 @@ public:
                 parent = (nodePath.size()>=2)?dynamic_cast<osg::Group*>(nodePath[nodePath.size()-2]):0;
 
                 if (node) std::cout<<"  Hits "<<node->className()<<" nodePath size"<<nodePath.size()<<std::endl;
-
+                toggleScribe(parent, node);
             }
         }        
 
         // now we try to decorate the hit node by the osgFX::Scribe to show that its been "picked"
-        if (parent && node)
+    }
+
+    void toggleScribe(osg::Group* parent, osg::Node* node) {
+        if (!parent || !node) return;
+
+        std::cout<<"  parent "<<parent->className()<<std::endl;
+
+        osgFX::Scribe* parentAsScribe = dynamic_cast<osgFX::Scribe*>(parent);
+        if (!parentAsScribe)
         {
-
-            std::cout<<"  parent "<<parent->className()<<std::endl;
-
-            osgFX::Scribe* parentAsScribe = dynamic_cast<osgFX::Scribe*>(parent);
-            if (!parentAsScribe)
+            // node not already picked, so highlight it with an osgFX::Scribe
+            osgFX::Scribe* scribe = new osgFX::Scribe();
+            scribe->addChild(node);
+            parent->replaceChild(node,scribe);
+        }
+        else
+        {
+            // node already picked so we want to remove scribe to unpick it.
+            osg::Node::ParentList parentList = parentAsScribe->getParents();
+            for(osg::Node::ParentList::iterator itr=parentList.begin();
+                itr!=parentList.end();
+                ++itr)
             {
-                // node not already picked, so highlight it with an osgFX::Scribe
-                osgFX::Scribe* scribe = new osgFX::Scribe();
-                scribe->addChild(node);
-                parent->replaceChild(node,scribe);
-            }
-            else
-            {
-                // node already picked so we want to remove scribe to unpick it.
-                osg::Node::ParentList parentList = parentAsScribe->getParents();
-                for(osg::Node::ParentList::iterator itr=parentList.begin();
-                    itr!=parentList.end();
-                    ++itr)
-                {
-                    (*itr)->replaceChild(parentAsScribe,node);
-                }
+                (*itr)->replaceChild(parentAsScribe,node);
             }
         }
+
     }
 
     void saveSelectedModel(osg::Node* scene)
@@ -323,6 +353,8 @@ public:
 protected:
 
     float _mx,_my;
+    bool _usePolytopeIntersector;
+    bool _useWindowCoordinates;
 };
 
 int main( int argc, char **argv )

include/osg/TemplatePrimitiveFunctor

@@ -0,0 +1,302 @@
+/* -*-c++-*- OpenSceneGraph - Copyright (C) 1998-2006 Robert Osfield
+ *
+ * This library is open source and may be redistributed and/or modified under
+ * the terms of the OpenSceneGraph Public License (OSGPL) version 0.0 or
+ * (at your option) any later version.  The full license is in LICENSE file
+ * included with this distribution, and on the openscenegraph.org website.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * OpenSceneGraph Public License for more details.
+*/
+
+#ifndef OSG_TERMPLATEPRIMITIVEFUNCTOR
+#define OSG_TERMPLATEPRIMITIVEFUNCTOR 1
+
+#include <osg/PrimitiveSet>
+#include <osg/Notify>
+
+namespace osg {
+
+
+    /** Provides access to the primitives that compose an \c osg::Drawable.
+     *  <p>Notice that \c TemplatePrimitiveFunctor is a class template, and that it inherits
+     *  from its template parameter \c T. This template parameter must implement
+     *  <tt>operator()(const osg::Vec3 v1, const osg::Vec3 v2, const osg::Vec3
+     *  v3, bool treatVertexDataAsTemporary)</tt>,
+     *  <tt>operator()(const osg::Vec3 v1, const osg::Vec3 v2, bool
+     *  treatVertexDataAsTemporary)</tt>, <tt>operator()(const osg::Vec3 v1,
+     *  const osg::Vec3 v2, const osg::Vec3 v3, bool treatVertexDataAsTemporary)</tt>,
+     *  and <tt>operator()(const osg::Vec3 v1, const osg::Vec3 v2, const osg::Vec3 v3,
+     *  const osg::Vec3 v4, bool treatVertexDataAsTemporary)</tt> which will be called
+     *  for the matching primitive when the functor is applied to a \c Drawable.
+     *  Parameters \c v1, \c v2, \c v3, and \c v4 are the vertices of the primitive.
+     *  The last parameter, \c treatVertexDataAsTemporary, indicates whether these
+     *  vertices are coming from a "real" vertex array, or from a temporary vertex array,
+     *  created by the \c TemplatePrimitiveFunctor from some other geometry representation.
+     *  @see \c PrimitiveFunctor for general usage hints.
+     */
+    template<class T>
+    class TemplatePrimitiveFunctor : public PrimitiveFunctor, public T
+    {
+    public:
+
+        TemplatePrimitiveFunctor()
+        {
+            _vertexArraySize=0;
+            _vertexArrayPtr=0;
+            _modeCache=0;
+            _treatVertexDataAsTemporary=false;
+        }
+
+        virtual ~TemplatePrimitiveFunctor() {}
+
+        void setTreatVertexDataAsTemporary(bool treatVertexDataAsTemporary) { _treatVertexDataAsTemporary=treatVertexDataAsTemporary; }
+        bool getTreatVertexDataAsTemporary() const { return _treatVertexDataAsTemporary; }
+
+        virtual void setVertexArray(unsigned int,const Vec2*)
+        {
+            notify(WARN)<<"Triangle Functor does not support Vec2* vertex arrays"<<std::endl;
+        }
+
+        virtual void setVertexArray(unsigned int count,const Vec3* vertices)
+        {
+            _vertexArraySize = count;
+            _vertexArrayPtr = vertices;
+        }
+
+        virtual void setVertexArray(unsigned int,const Vec4* )
+        {
+            notify(WARN)<<"Triangle Functor does not support Vec4* vertex arrays"<<std::endl;
+        }
+
+
+        virtual void drawArrays(GLenum mode,GLint first,GLsizei count)
+        {
+            if (_vertexArrayPtr==0 || count==0) return;
+
+            switch(mode)
+                {
+                case(GL_TRIANGLES): {
+                    const Vec3* vlast = &_vertexArrayPtr[first+count];
+                    for(const Vec3* vptr=&_vertexArrayPtr[first];vptr<vlast;vptr+=3)
+                        this->operator()(*(vptr),*(vptr+1),*(vptr+2),_treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_TRIANGLE_STRIP): {
+                    const Vec3* vptr = &_vertexArrayPtr[first];
+                    for(GLsizei i=2;i<count;++i,++vptr)
+                    {
+                        if ((i%2)) this->operator()(*(vptr),*(vptr+2),*(vptr+1),_treatVertexDataAsTemporary);
+                        else       this->operator()(*(vptr),*(vptr+1),*(vptr+2),_treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_QUADS): {
+                    const Vec3* vptr = &_vertexArrayPtr[first];
+                    for(GLsizei i=3;i<count;i+=4,vptr+=4)
+                    {
+                        this->operator()(*(vptr),*(vptr+1),*(vptr+2),*(vptr+3),_treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_QUAD_STRIP): {
+                    const Vec3* vptr = &_vertexArrayPtr[first];
+                    for(GLsizei i=3;i<count;i+=2,vptr+=2)
+                    {
+                        this->operator()(*(vptr),*(vptr+1),*(vptr+3),*(vptr+2),_treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_POLYGON): // treat polygons as GL_TRIANGLE_FAN
+                case(GL_TRIANGLE_FAN): {
+                    const Vec3* vfirst = &_vertexArrayPtr[first];
+                    const Vec3* vptr = vfirst+1;
+                    for(GLsizei i=2;i<count;++i,++vptr)
+                    {
+                        this->operator()(*(vfirst),*(vptr),*(vptr+1),_treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_POINTS): {
+                    const Vec3* vlast = &_vertexArrayPtr[first+count];
+                    for(const Vec3* vptr=&_vertexArrayPtr[first];vptr<vlast;vptr+=1)
+                        this->operator()(*(vptr),_treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_LINES): {
+                    const Vec3* vlast = &_vertexArrayPtr[first+count];
+                    for(const Vec3* vptr=&_vertexArrayPtr[first];vptr<vlast;vptr+=2)
+                        this->operator()(*(vptr),*(vptr+1),_treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_LINE_STRIP): {
+                    const Vec3* vlast = &_vertexArrayPtr[first+count];
+                    for(const Vec3* vptr=&_vertexArrayPtr[first];vptr<vlast;vptr+=1)
+                        this->operator()(*(vptr),*(vptr+1),_treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_LINE_LOOP): {
+                    const Vec3* vlast = &_vertexArrayPtr[first+count];
+                    for(const Vec3* vptr=&_vertexArrayPtr[first];vptr<vlast;vptr+=1)
+                        this->operator()(*(vptr),*(vptr+1),_treatVertexDataAsTemporary);
+                    this->operator()(*(vlast-1),_vertexArrayPtr[first],_treatVertexDataAsTemporary);
+                    break;
+                }
+                default:
+                    break;
+                }
+        }
+
+        template<class IndexType>
+        void drawElementsTemplate(GLenum mode,GLsizei count,const IndexType* indices)
+        {
+            if (indices==0 || count==0) return;
+
+            typedef const IndexType* IndexPointer;
+
+            switch(mode)
+                {
+                case(GL_TRIANGLES): {
+                    IndexPointer ilast = &indices[count];
+                    for(IndexPointer  iptr=indices;iptr<ilast;iptr+=3)
+                        this->operator()(_vertexArrayPtr[*iptr],_vertexArrayPtr[*(iptr+1)],_vertexArrayPtr[*(iptr+2)],_treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_TRIANGLE_STRIP): {
+                    IndexPointer iptr = indices;
+                    for(GLsizei i=2;i<count;++i,++iptr)
+                    {
+                        if ((i%2)) this->operator()(_vertexArrayPtr[*(iptr)],_vertexArrayPtr[*(iptr+2)],
+                                                    _vertexArrayPtr[*(iptr+1)],_treatVertexDataAsTemporary);
+                        else       this->operator()(_vertexArrayPtr[*(iptr)],_vertexArrayPtr[*(iptr+1)],
+                                                    _vertexArrayPtr[*(iptr+2)],_treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_QUADS): {
+                    IndexPointer iptr = indices;
+                    for(GLsizei i=3;i<count;i+=4,iptr+=4)
+                    {
+                        this->operator()(_vertexArrayPtr[*(iptr)],_vertexArrayPtr[*(iptr+1)],
+                                         _vertexArrayPtr[*(iptr+2)],_vertexArrayPtr[*(iptr+3)],
+                                         _treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_QUAD_STRIP): {
+                    IndexPointer iptr = indices;
+                    for(GLsizei i=3;i<count;i+=2,iptr+=2)
+                    {
+                        this->operator()(_vertexArrayPtr[*(iptr)],_vertexArrayPtr[*(iptr+1)],
+                                         _vertexArrayPtr[*(iptr+3)],_vertexArrayPtr[*(iptr+2)],
+                                         _treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_POLYGON): // treat polygons as GL_TRIANGLE_FAN
+                case(GL_TRIANGLE_FAN): {
+                    IndexPointer iptr = indices;
+                    const Vec3& vfirst = _vertexArrayPtr[*iptr];
+                    ++iptr;
+                    for(GLsizei i=2;i<count;++i,++iptr)
+                    {
+                        this->operator()(vfirst,_vertexArrayPtr[*(iptr)],_vertexArrayPtr[*(iptr+1)],
+                                         _treatVertexDataAsTemporary);
+                    }
+                    break;
+                }
+                case(GL_POINTS): {
+                    IndexPointer ilast = &indices[count];
+                    for(IndexPointer  iptr=indices;iptr<ilast;iptr+=1)
+                        this->operator()(_vertexArrayPtr[*iptr],_treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_LINES): {
+                    IndexPointer ilast = &indices[count];
+                    for(IndexPointer  iptr=indices;iptr<ilast;iptr+=2)
+                        this->operator()(_vertexArrayPtr[*iptr],_vertexArrayPtr[*(iptr+1)],
+                                         _treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_LINE_STRIP): {
+                    IndexPointer ilast = &indices[count];
+                    for(IndexPointer  iptr=indices;iptr<ilast;iptr+=1)
+                        this->operator()(_vertexArrayPtr[*iptr],_vertexArrayPtr[*(iptr+1)],
+                                         _treatVertexDataAsTemporary);
+                    break;
+                }
+                case(GL_LINE_LOOP): {
+                    IndexPointer ilast = &indices[count];
+                    for(IndexPointer  iptr=indices;iptr<ilast;iptr+=1)
+                        this->operator()(_vertexArrayPtr[*iptr],_vertexArrayPtr[*(iptr+1)],
+                                         _treatVertexDataAsTemporary);
+                    break;
+                    this->operator()(_vertexArrayPtr[*(ilast-1)],_vertexArrayPtr[indices[0]],
+                                     _treatVertexDataAsTemporary);
+                }
+                default:
+                    break;
+                }
+        }
+
+
+        virtual void drawElements(GLenum mode,GLsizei count,const GLubyte* indices)
+        {
+            drawElementsTemplate(mode, count, indices);
+        }
+
+        virtual void drawElements(GLenum mode,GLsizei count,const GLushort* indices)
+        {
+            drawElementsTemplate(mode, count, indices);
+        }
+
+        virtual void drawElements(GLenum mode,GLsizei count,const GLuint* indices)
+        {
+            drawElementsTemplate(mode, count, indices);
+        }
+
+        /** Note:
+         * begin(..),vertex(..) & end() are convenience methods for adapting
+         * non vertex array primitives to vertex array based primitives.
+         * This is done to simplify the implementation of primtive functor
+         * subclasses - users only need override drawArray and drawElements.
+         */
+        virtual void begin(GLenum mode)
+        {
+            _modeCache = mode;
+            _vertexCache.clear();
+        }
+
+        virtual void vertex(const Vec2& vert) { _vertexCache.push_back(osg::Vec3(vert[0],vert[1],0.0f)); }
+        virtual void vertex(const Vec3& vert) { _vertexCache.push_back(vert); }
+        virtual void vertex(const Vec4& vert) { _vertexCache.push_back(osg::Vec3(vert[0],vert[1],vert[2])/vert[3]); }
+        virtual void vertex(float x,float y) { _vertexCache.push_back(osg::Vec3(x,y,0.0f)); }
+        virtual void vertex(float x,float y,float z) { _vertexCache.push_back(osg::Vec3(x,y,z)); }
+        virtual void vertex(float x,float y,float z,float w) { _vertexCache.push_back(osg::Vec3(x,y,z)/w); }
+        virtual void end()
+        {
+            if (!_vertexCache.empty())
+            {
+                setVertexArray(_vertexCache.size(),&_vertexCache.front());
+                _treatVertexDataAsTemporary = true;
+                drawArrays(_modeCache,0,_vertexCache.size());
+            }
+        }
+
+    protected:
+
+
+        unsigned int        _vertexArraySize;
+        const Vec3*         _vertexArrayPtr;
+
+        GLenum              _modeCache;
+        std::vector<Vec3>   _vertexCache;
+        bool                _treatVertexDataAsTemporary;
+    };
+
+
+}
+
+#endif

src/osgUtil/PolytopeIntersector.cpp

@@ -1,13 +1,13 @@
-/* -*-c++-*- OpenSceneGraph - Copyright (C) 1998-2006 Robert Osfield 
+/* -*-c++-*- OpenSceneGraph - Copyright (C) 1998-2006 Robert Osfield
  *
- * This library is open source and may be redistributed and/or modified under  
- * the terms of the OpenSceneGraph Public License (OSGPL) version 0.0 or 
+ * This library is open source and may be redistributed and/or modified under
+ * the terms of the OpenSceneGraph Public License (OSGPL) version 0.0 or
  * (at your option) any later version.  The full license is in LICENSE file
  * included with this distribution, and on the openscenegraph.org website.
- * 
+ *
  * This library is distributed in the hope that it will be useful,
  * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the 
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  * OpenSceneGraph Public License for more details.
 */
 
@@ -17,11 +17,308 @@
 #include <osg/Geometry>
 #include <osg/Notify>
 #include <osg/io_utils>
-#include <osg/TriangleFunctor>
+#include <osg/TemplatePrimitiveFunctor>
 
 using namespace osgUtil;
 
 
+
+class PolytopePrimitiveIntersector {
+public:
+#ifdef OSG_USE_DOUBLE_PLANE
+    typedef double value_type;
+    typedef osg::Vec3d Vec3_type;
+#else
+    typedef float value_type;
+    typedef osg::Vec3f Vec3_type;
+#endif
+
+    typedef osg::Polytope::ClippingMask PlaneMask;
+    typedef osg::Polytope::PlaneList PlaneList;
+    typedef std::vector<std::pair<PlaneMask,Vec3_type> > CandList_t;
+
+    /// a line defined by the intersection of two planes
+    struct PlanesLine {
+        PlanesLine(PlaneMask m, Vec3_type p, Vec3_type d) :
+            mask(m), pos(p), dir(d) {}
+        PlaneMask mask;
+        Vec3_type pos;
+        Vec3_type dir;
+    };
+    typedef std::vector<PlanesLine> LinesList;
+
+    PolytopePrimitiveIntersector() :
+        numPoints(0),
+        numLines(0),
+        numTriangles(0),
+        numQuads(0),
+        _numIntersections(0) {}
+
+    value_type eps() { return 1e-6; }
+
+    /// check which candidate points lie within the polytope volume
+    /// mark outliers with mask == 0, return number of remaining candidates
+    unsigned int checkCandidatePoints(PlaneMask inside_mask, CandList_t& cand) {
+        PlaneMask selector_mask = 0x1;
+        unsigned int numCands=cand.size();
+        for (PlaneList::const_iterator it=_planes.begin(); it!=_planes.end() && numCands>0;
+             ++it, selector_mask <<= 1) {
+            const osg::Plane& plane=*it;
+            if (selector_mask & inside_mask) continue;
+            for (CandList_t::iterator pointIt=cand.begin(); pointIt!=cand.end(); ++pointIt) {
+                PlaneMask mask=pointIt->first;
+                if (mask==0) continue;
+                if (selector_mask & mask) continue;
+                if (plane.distance(pointIt->second)<0.0f) {
+                    mask=0;  // mark as outside
+                    --numCands;
+                    if (numCands==0) return 0;
+                }
+            }
+        }
+        return numCands;
+    }
+
+    // handle points
+    void operator()(const Vec3_type v1, bool treatVertexDataAsTemporary) {
+        ++numPoints;
+        for (PlaneList::const_iterator it=_planes.begin(); it!=_planes.end(); ++it) {
+            const osg::Plane& plane=*it;
+            const value_type d1=plane.distance(v1);
+            if (d1<0.0f) return;   // point outside
+        }
+        ++_numIntersections;
+    }
+
+    // handle lines
+    void operator()(const Vec3_type v1, const Vec3_type v2, bool treatVertexDataAsTemporary)
+    {
+        ++numLines;
+        PlaneMask selector_mask = 0x1;
+        PlaneMask inside_mask = 0x0;
+        CandList_t cand;
+
+        for (PlaneList::const_iterator it=_planes.begin(); it!=_planes.end(); ++it, selector_mask<<=1) {
+            const osg::Plane& plane=*it;
+            const value_type d1=plane.distance(v1);
+            const value_type d2=plane.distance(v2);
+            const bool d1IsNegative = (d1<0.0f);
+            const bool d2IsNegative = (d2<0.0f);
+            if (d1IsNegative && d2IsNegative) return;      // line outside
+            if (!d1IsNegative && !d2IsNegative) {
+                inside_mask |= selector_mask;
+                continue;   // completly inside
+            }
+            if (d1==0.0f) {
+                cand.push_back( CandList_t::value_type(selector_mask, v1) );
+            } else if (d2==0.0f) {
+                cand.push_back( CandList_t::value_type(selector_mask, v2) );
+            } else if (d1IsNegative && !d2IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v1-(v2-v1)*(d1/(-d1+d2))) ) );
+            } else if (!d1IsNegative && d2IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v1+(v2-v1)*(d1/(d1-d2))) ) );
+            }
+
+        }
+        if (inside_mask==_plane_mask) {
+            ++_numIntersections;
+            return;
+        }
+
+        unsigned int numCands=checkCandidatePoints(inside_mask, cand);
+        if (numCands>0) {
+            ++_numIntersections;
+        }
+
+    }
+
+    // handle triangles
+    void operator()(const Vec3_type v1, const Vec3_type v2, const Vec3_type v3,
+                    bool treatVertexDataAsTemporary)
+    {
+        ++numTriangles;
+        PlaneMask selector_mask = 0x1;
+        PlaneMask inside_mask = 0x0;
+        CandList_t cand;
+        for (PlaneList::const_iterator it=_planes.begin(); it!=_planes.end();
+             ++it, selector_mask <<= 1) {
+            const osg::Plane& plane=*it;
+            const value_type d1=plane.distance(v1);
+            const value_type d2=plane.distance(v2);
+            const value_type d3=plane.distance(v3);
+            const bool d1IsNegative = (d1<0.0f);
+            const bool d2IsNegative = (d2<0.0f);
+            const bool d3IsNegative = (d3<0.0f);
+            if (d1IsNegative && d2IsNegative && d3IsNegative) return;      // triangle outside
+            if (!d1IsNegative && !d2IsNegative && !d3IsNegative) {
+                inside_mask |= selector_mask;
+                continue;   // completly inside
+            }
+
+            // edge v1-v2 intersects
+            if (d1==0.0f) {
+                cand.push_back( CandList_t::value_type(selector_mask, v1) );
+            } else if (d2==0.0f) {
+                cand.push_back( CandList_t::value_type(selector_mask, v2) );
+            } else if (d1IsNegative && !d2IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v1-(v2-v1)*(d1/(-d1+d2))) ) );
+            } else if (!d1IsNegative && d2IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v1+(v2-v1)*(d1/(d1-d2))) ) );
+            }
+
+            // edge v1-v3 intersects
+            if (d3==0.0f) {
+                cand.push_back( CandList_t::value_type(selector_mask, v3) );
+            } else if (d1IsNegative && !d3IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v1-(v3-v1)*(d1/(-d1+d3))) ) );
+            } else if (!d1IsNegative && d3IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v1+(v3-v1)*(d1/(d1-d3))) ) );
+            }
+
+            // edge v2-v3 intersects
+            if (d2IsNegative && !d3IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v2-(v3-v2)*(d2/(-d2+d3))) ) );
+            } else if (!d2IsNegative && d3IsNegative) {
+                cand.push_back( CandList_t::value_type(selector_mask, (v2+(v3-v2)*(d2/(d2-d3))) ) );
+            }
+        }
+
+        if (_plane_mask==inside_mask) { // triangle lies inside of all planes
+            ++_numIntersections;
+            return;
+        }
+
+        if (cand.empty() && _planes.size()<3) {
+            return;
+        }
+
+        unsigned int numCands=checkCandidatePoints(inside_mask, cand);
+
+        if (numCands>0) {
+            ++_numIntersections;
+            return;
+        }
+
+        // handle case where the polytope goes through the triangle
+        // without containing any point of it
+
+        LinesList& lines=getPolytopeLines();
+        cand.clear();
+
+        // check all polytope lines against the triangle
+        // use algorithm from "Real-time rendering" (second edition) pp.580
+        const Vec3_type e1=v2-v1;
+        const Vec3_type e2=v3-v1;
+
+        for (LinesList::const_iterator it=lines.begin(); it!=lines.end(); ++it) {
+            const PlanesLine& line=*it;
+
+            Vec3_type p=line.dir^e2;
+            const value_type a=e1*p;
+            if (osg::absolute(a)<eps()) continue;
+            const value_type f=1.0f/a;
+            const Vec3_type s=(line.pos-v1);
+            const value_type u=f*(s*p);
+            if (u<0.0f || u>1.0f) continue;
+            const Vec3_type q=s^e1;
+            const value_type v=f*(line.dir*q);
+            if (v<0.0f || u+v>1.0f) continue;
+            const value_type t=f*(e2*q);
+
+            cand.push_back(CandList_t::value_type(line.mask, line.pos+line.dir*t));
+        }
+
+        numCands=checkCandidatePoints(inside_mask, cand);
+
+        if (numCands>0) {
+            ++_numIntersections;
+            return;
+        }
+
+    }
+
+    /// handle quads
+    void operator()(const Vec3_type v1, const Vec3_type v2, const Vec3_type v3, const Vec3_type v4,
+                    bool treatVertexDataAsTemporary) {
+        ++numQuads;
+        this->operator()(v1,v2,v3,treatVertexDataAsTemporary);
+        this->operator()(v1,v3,v4,treatVertexDataAsTemporary);
+        numTriangles-=2;
+    }
+
+    void setPolytope(osg::Polytope& polytope) {
+
+        const PlaneMask currentMask = polytope.getCurrentMask();
+        PlaneMask selector_mask = 0x1;
+
+        const PlaneList& planeList = polytope.getPlaneList();
+        unsigned int numActivePlanes = 0;
+
+        PlaneList::const_iterator itr;
+        for(itr=planeList.begin(); itr!=planeList.end(); ++itr) {
+            if (currentMask&selector_mask) ++numActivePlanes;
+            selector_mask <<= 1;
+        }
+
+        _plane_mask = 0x0;
+        _planes.clear();
+        _planes.reserve(numActivePlanes);
+        _lines.clear();
+
+        selector_mask=0x1;
+        for(itr=planeList.begin(); itr!=planeList.end(); ++itr) {
+            if (currentMask&selector_mask) {
+                _planes.push_back(*itr);
+                _plane_mask <<= 1;
+                _plane_mask |= 0x1;
+            }
+            selector_mask <<= 1;
+        }
+    }
+
+    /// get boundary lines of polytope
+    LinesList& getPolytopeLines() {
+        if (!_lines.empty()) return _lines;
+
+        PlaneMask selector_mask = 0x1;
+        for (PlaneList::const_iterator it=_planes.begin(); it!=_planes.end(); ++it, selector_mask <<= 1 ) {
+            const osg::Plane& plane1=*it;
+            const Vec3_type normal1=plane1.getNormal();
+            const Vec3_type point1=normal1*(-plane1[3]);   /// canonical point on plane1
+            PlaneMask sub_selector_mask = (selector_mask<<1);
+            for (PlaneList::const_iterator jt=it+1; jt!=_planes.end(); ++jt, sub_selector_mask <<= 1 ) {
+                const osg::Plane& plane2=*jt;
+                const Vec3_type normal2=plane2.getNormal();
+                if (osg::absolute(normal1*normal2) > (1.0-eps())) continue;
+                const Vec3_type lineDirection = normal1^normal2;
+
+                const Vec3_type searchDirection = lineDirection^normal1; /// search dir in plane1
+                const value_type seachDist = -plane2.distance(point1)/(searchDirection*normal2);
+                if (osg::isNaN(seachDist)) continue;
+                const Vec3_type linePoint=point1+searchDirection*seachDist;
+                _lines.push_back(PlanesLine(selector_mask|sub_selector_mask, linePoint, lineDirection));
+            }
+        }
+        return _lines;
+    }
+
+    unsigned int getNumIntersections() const { return _numIntersections; }
+    unsigned int getNumPrimitives() const { return numPoints+numLines+numTriangles; }
+    unsigned int getNumPlanes() const { return _planes.size(); }
+
+    unsigned int numPoints;
+    unsigned int numLines;
+    unsigned int numTriangles;
+    unsigned int numQuads;
+
+private:
+    PlaneList _planes;                   ///< active planes extracted from polytope
+    LinesList _lines;                    ///< all intersection lines of two polytope planes
+    PlaneMask _plane_mask;               ///< mask for all planes of the polytope
+    unsigned int _numIntersections;
+};
+
+
 ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
 //
 //  PolytopeIntersector
@@ -73,18 +370,18 @@ Intersector* PolytopeIntersector::clone(osgUtil::IntersectionVisitor& iv)
     osg::Matrix matrix;
     switch (_coordinateFrame)
     {
-        case(WINDOW): 
+        case(WINDOW):
             if (iv.getWindowMatrix()) matrix.preMult( *iv.getWindowMatrix() );
             if (iv.getProjectionMatrix()) matrix.preMult( *iv.getProjectionMatrix() );
             if (iv.getViewMatrix()) matrix.preMult( *iv.getViewMatrix() );
             if (iv.getModelMatrix()) matrix.preMult( *iv.getModelMatrix() );
             break;
-        case(PROJECTION): 
+        case(PROJECTION):
             if (iv.getProjectionMatrix()) matrix.preMult( *iv.getProjectionMatrix() );
             if (iv.getViewMatrix()) matrix.preMult( *iv.getViewMatrix() );
             if (iv.getModelMatrix()) matrix.preMult( *iv.getModelMatrix() );
             break;
-        case(VIEW): 
+        case(VIEW):
             if (iv.getViewMatrix()) matrix.preMult( *iv.getViewMatrix() );
             if (iv.getModelMatrix()) matrix.preMult( *iv.getModelMatrix() );
             break;
@@ -117,17 +414,24 @@ void PolytopeIntersector::intersect(osgUtil::IntersectionVisitor& iv, osg::Drawa
 {
     if ( !_polytope.contains( drawable->getBound() ) ) return;
 
-    osg::Geometry* geometry = drawable->asGeometry();
-    osg::Vec3Array* vertices = geometry ? dynamic_cast<osg::Vec3Array*>(geometry->getVertexArray()) : 0;
-    if (vertices)
-    {
-        if (!_polytope.contains(*vertices)) return;
+    osg::TemplatePrimitiveFunctor<PolytopePrimitiveIntersector> func;
+    func.setPolytope(_polytope);
+
+    drawable->accept(func);
+
+    if (func.getNumIntersections()==0) {
+       return;
     }
 
+    osg::notify(osg::INFO) << func.getNumIntersections() << " intersections with "
+               << func.numPoints<<" points, "<< func.numLines
+               <<" lines, "<< func.numTriangles
+               <<" triangles, "<<func.numQuads<<" quads"<<std::endl;
+
     Intersection hit;
     hit.nodePath = iv.getNodePath();
     hit.drawable = drawable;
-    
+
     insertIntersection(hit);
 }
 
@@ -135,6 +439,6 @@ void PolytopeIntersector::intersect(osgUtil::IntersectionVisitor& iv, osg::Drawa
 void PolytopeIntersector::reset()
 {
     Intersector::reset();
-    
+
     _intersections.clear();
 }