forcing users to use osgDB::readRef*File() methods. The later is preferable as it closes a potential threading bug when using paging databases in conjunction
with the osgDB::Registry Object Cache. This threading bug occurs when one thread gets an object from the Cache via an osgDB::read*File() call where only
a pointer to the object is passed back, so taking a reference to the object is delayed till it gets reassigned to a ref_ptr<>, but at the same time another
thread calls a flush of the Object Cache deleting this object as it's referenceCount is now zero. Using osgDB::readREf*File() makes sure the a ref_ptr<> is
passed back and the referenceCount never goes to zero.
To ensure the OSG builds when OSG_PROVIDE_READFILE is to OFF the many cases of osgDB::read*File() usage had to be replaced with a ref_ptr<> osgDB::readRef*File()
usage. The avoid this change causing lots of other client code to be rewritten to handle the use of ref_ptr<> in place of C pointer I introduced a serious of
templte methods in various class to adapt ref_ptr<> to the underly C pointer to be passed to old OSG API's, example of this is found in include/osg/Group:
bool addChild(Node* child); // old method which can only be used with a Node*
tempalte<class T> bool addChild(const osg::ref_ptr<T>& child) { return addChild(child.get()); } // adapter template method
These changes together cover 149 modified files, so it's a large submission. This extent of changes are warrent to make use of the Object Cache
and multi-threaded loaded more robust.
git-svn-id: http://svn.openscenegraph.org/osg/OpenSceneGraph/trunk@15164 16af8721-9629-0410-8352-f15c8da7e697
The problem is that the readObjectFields method will add the object to the _identifierMap. So all the other instances of that image in the same file will be replaced by the created dummy object. In my fix this was an dummy image and I didn't notice it in our scene's, probably because it covered a small part of an object. In your fix the dummy object was not an image and that leads to a crash when something tries to use it as an image. I have attached a small fix for this bug.
"
git-svn-id: http://svn.openscenegraph.org/osg/OpenSceneGraph/trunk@14471 16af8721-9629-0410-8352-f15c8da7e697
I'm not sure if the IVE was simply generated incorrectly, or if the Image::getTotalSizeInBytesIncludingMipmaps() was modified since the file was generated. Either way, I added a simple check in the IVE loader so that it clears the mipmap offsets if the actual data size does not match the computed data size. This seems like a safe fallback since the mipmap data can be automatically generated, and it fixes the problem in my case.
Also, while looking into this issue, I noticed that the osgDB::InputStream class applies the serialized image allocation mode. However, since the serializer is allocating the image data itself, it seems like it should force the allocation mode to USE_NEW_DELETE.
"
the wrappers it contains. The registration creates a prototype
object for all of the wrapped classes. For some of the higher-level
classes this can be a bit heavy.
I noticed a problem with a model which required a single class from
osgSim. When osgdb_serializers_osgsim.so was loaded it registered
wrappers and created prototype objects for all of the osgSim classes,
including osgSim::ScalarBar. The constructor for that class creates
several drawables, and loads arial.ttf using the freetype plugin. I
don't need that, and don't even ship the font or plugin with my
application, resulting in an unexplained warning message loading
the model.
I've modified the ObjectWrapper class to defer the prototype object
creation until if & when actually required."
// A custom class
namespace CustomDomain {
class MyGroup : public osg::Group
{
public:
META_Node( CustomDomain, MyGroup );
void setMyName( const std::string& n );
const std::string& getMyName() const;
void setMyID( int id );
int getMyID() const;
...
};
}
// The serialization wrapper using a custom domain name
REGISTER_CUSTOM_OBJECT_WRAPPER( MyDomain,
CustomDomain_MyGroup,
new CustomDomain::MyGroup,
CustomDomain::MyGroup,
"osg::Object osg::Node osg::Group CustomDomain::MyGroup" )
{
ADD_STRING_SERIALIZER( MyName, std::string() );
{
UPDATE_TO_VERSION_SCOPED( 1 ); // Updated for a new domain version
ADD_INT_SERIALIZER( MyID, 0 );
}
}
Save the class instance as follows:
osgDB::writeNodeFile( *myGroup, "serializer_test.osgt", new osgDB::Options("CustomDomains=MyDomain:1") );
The output file will include the domain version definition and all the class data, and can be read back. We can also force setting the domain version by the CustomDomains option while reading the saved files. If we save the class instance without any options, MyID will be ignored because the default domain version is 0.
This may help third-party libraries like osgEarth to maintain their own serializers without regarding to the OSG soversion changes.
Another feature added is a more robust binary format, which in fact adds a size-offset at each block's beginning. When there are problems or unsupported data types while reading, we can now directly jump to the block end indicated by the offset value. So a .osgb file will automatically ignore bad data and read remains as normal (at present it will fail at all). This feature will not break the backward compatibility, and can be disabled by setting "RobustBinaryFormat=false" while writing out.
Hope these changes can work smoothly with present and future community projects. Maybe we should also consider have an osgserializer example to test and demonstrate all things we can do now."
Bug description:
Let's say we have class A
namespace Bug
{
class A : public osg::Object
{
public:
//...
typedef std::vector<osg::ref_ptr<A> > AList;
protected:
AList _alist;
//...
}
}
REGISTER_OBJECT_WRAPPER( A,
new Bug::A,
Bug::A,
"osg::Object Bug::A" )
{
ADD_LIST_SERIALIZER(A,Bug::A::AList);
}
Bug:
We create say 3 instances of class A: A1,A2,A3 and then we add A2 and A3 and A1 as child instances of A1 so we get next structure:
A1
|- A2,A3,A1
we call osgDB::writeObjectFile(A1,"/data/a.osgt") -> saved correctly( third element in list is saved as unique id that references parentClass
now we call
A1 = osgDB::readObjectFile("/data/a.osgt");
Everything is deserialized correctely except last element in list which should be same instance as parent A1.
The attached code resolves this issue by passing UniqueID in readObjectFields method and saving object in _identifierMap as soon as we have valid object instance so we can make reference to parent object from any child instance.
"
[examples/osgphotoalbum/PhotoArchive.cpp:56]: (error) Memory leak: fileIndentifier
[examples/osgphotoalbum/PhotoArchive.cpp:257]: (error) Deallocating a deallocated pointer: newData
[examples/osgphotoalbum/PhotoArchive.cpp:318]: (error) Deallocating a deallocated pointer: newData
[src/osg/ImageUtils.cpp:116]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:307]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:312]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:367]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:399]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:400]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:482]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:483]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:484]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:519]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/ImageUtils.cpp:536]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:71]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:74]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:77]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:82]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:102]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:107]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:599]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:600]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:601]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:602]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:603]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:604]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:605]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osg/OcclusionQueryNode.cpp:606]: (portability) Extra qualification 'osg::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:134]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:135]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:136]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:137]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:139]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:177]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:178]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:195]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:198]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:203]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:205]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/ExternalFileWriter.cpp:253]: (portability) Extra qualification 'osgDB::' unnecessary and considered an error by many compilers.
[src/osgDB/InputStream.cpp:553]: (error) Memory leak: data
[src/osgDB/OutputStream.cpp:393]: (error) Memory leak: data
[src/osgPlugins/Inventor/ConvertToInventor.cpp:656]: (error) Mismatching allocation and deallocation: tmpArray
[src/osgPlugins/Inventor/ReaderWriterIV.cpp:237]: (error) Common realloc mistake: 'buf' nulled but not freed upon failure
[src/osgPlugins/OpenFlight/expGeometryRecords.cpp:167]: (portability) Extra qualification 'flt::' unnecessary and considered an error by many compilers.
[src/osgPlugins/OpenFlight/expGeometryRecords.cpp:373]: (portability) Extra qualification 'flt::' unnecessary and considered an error by many compilers.
[src/osgPlugins/cfg/CameraConfig.cpp:635]: (error) Unusual pointer arithmetic
[src/osgPlugins/freetype/FreeTypeLibrary.cpp:122]: (error) Memory leak: buffer
[src/osgPlugins/geo/ReaderWriterGEO.cpp:210]: (error) Possible null pointer dereference: gfd - otherwise it is redundant to check if gfd is null at line 211
[src/osgPlugins/geo/ReaderWriterGEO.cpp:227]: (error) Possible null pointer dereference: gfd - otherwise it is redundant to check if gfd is null at line 228
[src/osgPlugins/geo/ReaderWriterGEO.cpp:903]: (error) Possible null pointer dereference: gfd - otherwise it is redundant to check if gfd is null at line 904
[src/osgPlugins/geo/osgGeoNodes.h:180]: (error) Memory leak: geoHeaderGeo::intVars
[src/osgPlugins/geo/osgGeoNodes.h:181]: (error) Memory leak: geoHeaderGeo::useVars
[src/osgPlugins/geo/osgGeoNodes.h:182]: (error) Memory leak: geoHeaderGeo::extVars
[src/osgPlugins/md2/ReaderWriterMD2.cpp:180]: (error) Memory leak: mapbase
[src/osgPlugins/md2/ReaderWriterMD2.cpp:166]: (error) Resource leak: file_fd
[src/osgPlugins/pic/ReaderWriterPIC.cpp:152]: (error) Mismatching allocation and deallocation: tmpbuf
[src/osgPlugins/pic/ReaderWriterPIC.cpp:153]: (error) Mismatching allocation and deallocation: buffer
[src/osgPlugins/ply/plyfile.cpp:843]: (error) Memory leak: plyfile
[src/osgPlugins/pvr/ReaderWriterPVR.cpp:179]: (error) Memory leak: imageData
[src/osgPlugins/shp/ESRIShapeParser.cpp:29]: (error) Resource leak: fd
[src/osgPlugins/shp/XBaseParser.cpp:96]: (error) Resource leak: fd
[src/osgPlugins/zip/unzip.cpp:3158]: (error) Possible null pointer dereference: s - otherwise it is redundant to check if s is null at line 3159
[src/osgPlugins/zip/unzip.cpp:4155]: (error) Dangerous usage of 'rd' (strncpy doesn't always 0-terminate it)
[src/osgShadow/MinimalCullBoundsShadowMap.cpp:334]: (error) Possible null pointer dereference: rl - otherwise it is redundant to check if rl is null at line 331
[src/osgViewer/ScreenCaptureHandler.cpp:617]: (error) Possible null pointer dereference: camera - otherwise it is redundant to check if camera is null at line 611
[src/osgViewer/ScreenCaptureHandler.cpp:632]: (error) Possible null pointer dereference: camera - otherwise it is redundant to check if camera is null at line 626
[src/osgVolume/Locator.cpp:209]: (error) Dangerous iterator usage after erase()-method.
[src/osgVolume/RayTracedTechnique.cpp:274]: (error) Possible null pointer dereference: imageLayer - otherwise it is redundant to check if imageLayer is null at line 259
[src/osgVolume/RayTracedTechnique.cpp:275]: (error) Possible null pointer dereference: imageLayer - otherwise it is redundant to check if imageLayer is null at line 259
[src/osgWrappers/serializers/osg/ShaderBinary.cpp:28]: (error) Mismatching allocation and deallocation: data
functionalities. It includes two main parts: a version checking macro
for handling backward-compatiblity since 3.0, and enhencement of
current schema mechanism. I also change the option handling process to
use getPluginStringData(), and add new USE_SERIALIZER_WRAPPER macro in
the Registry header to allow for static-link usage as well.
The enhencement of schema machanism just tells the type of each
serializer while outputting them, such as:
osg::Group = Children:1
The meaning of the number can be found in the osgDB/Serializer header,
BaseSerializer::Type enum. It may help 3rdparty utilities understand
the structure of the wrapper and do some reflection work in the
future.
The new macro UPDATE_TO_VERSION can help indicate the InputStream (no
affect on the writer) that a serializer is added/removed since certain
OSG version. An example wrapper file is also attached. The
Geode_modified.cpp is based on the serializers/osg/Geode.cpp file
(hey, don't merge it :-), but assumes that a new user serializer
'Test' is added since version 65 (that is, the OSG_SOVERSION):
REGISTER_OBJECT_WRAPPER( Geode, ... )
{
ADD_USER_SERIALIZER( Drawables ); // origin ones
UPDATE_TO_VERSION( 65 )
{
ADD_USER_SERIALIZER( Test ); // a serializer added from version 65
}
}
All kinds of ADD_... macros following UPDATE_TO_VERSION will
automatically apply the updated version. The braces here are only for
typesetting!
While reading an osgt/osgb/osgx file, OSG will now check if the file
version (recorded as the writer's soversion, instead of previous
meaningless "#Version 2") is equal or greater than Test's version, and
try reading it, or just ignore it if file version is lesser.
And we also have the REMOVE_SERIALIZER macro will mark a named
serializer as removed in some version, with which all files generated
by further versions will just ignore it:
UPDATE_TO_VERSION( 70 )
{
REMOVE_SERIALIZER( Test );
}
This means that from version 70, the serializer Test is removed (but
not actually erased from the list) and should not be read anymore. If
the read file version is less than 70 (and equal or greater than 65),
Test will still be handled when reading; otherwise it will be ignored
to keep compatiblity on different OSG versions.
"
change was to use doubles for reading and writing matrices regardless of type of Matrix
being serialized.
Change does break backwards compatibility though, so code
path supporting original format has been left in for the
time being. However, this code is not reliable enough and
is over complicated compared to the simplified handling. Once
the new code has been bedded down for a while I'll remove this code block.
I changed
_in->setStream( new std::stringstream(data) );
to
_dataDecompress = new std::stringstream(data);
_in->setStream( _dataDecompress );
Then when the destructor is of InputStream is called I delete the
dataDecompress stringstream.
"
the OutputStream/InputStream implementations, which was just finished
last weekend with a few tests on Windows and Ubuntu. Hope it could
work and get more feedbacks soon.
I've added a new option "SchemaData" to the osg2 plugin. Developers
may test the new feature with the command line:
# osgconv cow.osg cow.osgb -O SchemaData
It will record all serializer properties used in the scene graph, at
the beginning of the generated file. And when osgviewer and user
applications is going to read the osgb file, the inbuilt data will be
automatically read and applied first, to keep backwards compatibility
partly. This will not affect osgb files generated with older versions.
"
using osgDB::XmlParser. The extension for XML-formatted scenes is
.osgx, corresponding to .osgb for binary and .osgt for ascii. It could
either be rendered in osgviewer or edited by common web browsers and
xml editors because of a range of changes to fit the XML syntax. For
example, the recorded class names are slight modified, from
'osg::Geode' to 'osg--Geode'.
To quickly get an XML file:
# ./osgconv cow.osg cow.osgx
The StreamOperator header, InputStreram and OutputStream classes are
modified to be more portable for triple ascii/binary/XML formats. I
also fixed a bug in readImage()/writeImage() to share image objects if
needed.
The ReaderWriterOSG2 class now supports all three formats and
reading/writing scene objects (not nodes or images), thanks to
Torben's advice before.
"
The options where not passed on to the image reader plugins when reading the new osg2 format files, so I added the options to the osgDB.:readImageFile function call.
"
of OSG. I modified the osgDB::InputStream and OutputStream and the
PagedLOD wrapper as well. Now all seems to work fine with paged
scenes. I've tested with the puget terrain data and the osgdem
application from VPB:
# osgdem --xx 10 --yy 10 -t ps_texture_4k.tif --xx 10 --yy 10 -d
ps_height_4k.tif -l 8 -v 0.1 -o puget.osgb
As the ive plugin does, The PagedLOD wrapper now automatically add the
latest file path to PagedLODs' databasePath member, to help them find
correct child positions. I also changed the image storage strategy of
the OutputStream class, to store them inline by default. The osgt
extension should also work, in case the image files are also written
to the disk.
"
1. Rewrite the reading/writing exception handlers to work like the ive
plugin exceptions.
2. Write a header writing/checking function in ReaderWriterOSG2.cpp,
which may help decide if the stream is ascii or binary. The
readInputIterator() function will return null pointer if the input
file is nither osgb nor osgt format, which indicates that the old .osg
format could be used here, in case we've merged the two plugins
together.
3. Add a new ForceReadingImage option in the InputStream, which will
allocate an empty image object with the filename if specifed external
image file is missed. It may be useful for format converting in some
cases.
4. Add new osgParticle wrappers, as well as some modification to the
osgParticle headers, for instance, change isEnabled() to getEnabled().
5. Some fixes to the osg serialization wrappers."
to the osgDB::Registry. Added a osgDB::Registry::getObjectWrapperManager() for access of this object wrapper manager. This
change centralises the singleton management in osgDB.
Merged the osgDB::GlobalLookUpTable functionality into ObjectWrapperManger to keep down the number of singletons in use.
From Robert Osfield, refactor of Wang Rui's original osg2 into 3 parts - parts placed into osgDB, the ReaderWriter placed into src/osg/Plugin/osg and wrappers into src/osgWrappers/serializers/osg
