"I have found a "bug" in the new audio decoding code (actually I think the bug is in ffmpeg, but anyway it should be wise to protect the OSG plug-in about it). I am attaching a security check in FFmpegDecoderAudio.cpp.
If anybody is curious about the problem, it happens sometimes when decoding an AAC audio stream. It eventually includes a PCE block inside the AAC audio frame and then ffmpeg audio decoding function signals a "new_frame" with 1024 samples, but a null pointer instead of the audio data. It can be easily detected because in these cases number of channels is 0. Maybe this is the intended behaviour for ffmpeg, but I find it quite weird.
"
" It seems that libav does not have a channels attribute in AVFrame structure. This new version should do."
git-svn-id: http://svn.openscenegraph.org/osg/OpenSceneGraph/trunk@14676 16af8721-9629-0410-8352-f15c8da7e697
$ osgmovie --audio movie.avi.ffmpeg
FFmpegImageStream::open audio failed, audio stream will be disabled: unknown audio format
With the attached FFmpegDecoderAudio.cpp, audio sounds correctly.
I am also attaching a modified version of FindFFmpeg.cmake that allows to set as FFMPEG_DIR the ffmpeg compiled in the source directory structure. It should not break anything as it only adds some additional search paths.
"
git-svn-id: http://svn.openscenegraph.org/osg/OpenSceneGraph/trunk@14654 16af8721-9629-0410-8352-f15c8da7e697
$ osgmovie --audio movie.avi.ffmpeg
FFmpegImageStream::open audio failed, audio stream will be disabled: unknown audio format
With the attached FFmpegDecoderAudio.cpp, audio sounds correctly.
I am also attaching a modified version of FindFFmpeg.cmake that allows to set as FFMPEG_DIR the ffmpeg compiled in the source directory structure. It should not break anything as it only adds some additional search paths.
"
Note from Robert Osfield, I have found in testing that audio quality is not good for planar floating point formats, even with adding support for SDL2 to the osgmovie example. I haven't yet tracked down the cause of these audio problems or a solution.
git-svn-id: http://svn.openscenegraph.org/osg/OpenSceneGraph/trunk@14609 16af8721-9629-0410-8352-f15c8da7e697
been removed in the most recent versions of libavcodec/ffmpeg. You're
already using avcodec_open2() elsewhere, but one appears to have been
missed. The change is trivial:
[skynet](0) $ svn diff
Index: src/osgPlugins/ffmpeg/FFmpegDecoderAudio.cpp
===================================================================
--- src/osgPlugins/ffmpeg/FFmpegDecoderAudio.cpp (revision 13355)
+++ src/osgPlugins/ffmpeg/FFmpegDecoderAudio.cpp (working copy)
@@ -109,7 +109,7 @@
// m_context->flags |= CODEC_FLAG_TRUNCATED;
// Open codec
- if (avcodec_open(m_context, p_codec) < 0)
+ if (avcodec_open2(m_context, p_codec, NULL) < 0)
throw std::runtime_error("avcodec_open() failed");
}
[skynet](0) $
I've applied similar changes this year to many other packages, including
cheese, blender, linphone, ad nauseam. It's been tested by verifying that
with my patch, OSG builds against the newest libavcodec and ffmpeg, whereas
otherwise it does not.
The modified src/osgPlugins/ffmpeg/FFmpegDecoderAudio.cpp is attached. It
originates in svn trunk revision 13355. Thanks!"
Header update
FindFFmpeg.cmake has been changed in order to support the new header include format for FFmpeg. In the 1.0 release, a new file had been added with the name “time.h” in the avutil library. The previous method of adding includes caused conflicts with the ANSI C “time.h” file. Now the include directive will only use the main include folder. All files using the old include format have been updated to reflect the change.
Added __STDC_CONSTANT_MACROS define to CMakeLists.txt
Since there is no guarantee that FFmpegHeaders.hpp will be included before stdint.h is included, the define has been moved from FFmpegHeaders.hpp to be part of the CMakeLists.txt for the FFmpeg plugin. This will allow the define to work on all compilers regardless of include order.
Replaced AVFormatParameters with AVDictionary
AVFormatParameters is no longer supported in FFmpeg and has been replaced with a key/value map of strings for each setting. FFmpegParameters and FFmpegDecoder has been updated to reflect this.
Replaced av_open_input_file with avformat_open_input
FFmpeg now opens files using avformat_open_input. Since the av_open_input_file method is deprecated, the FFmpegDecoder class has been updated to reflect this change.
Added custom AVIOContext field to options
Since some formats and inputs may not be supported by FFmpeg, I have added a new parameter that allows a user to allocate their own AVIOContext. This class will allow for creating a read, seek, and write callback if they desire.
Checking for start_time validity
It is possible for some file formats to not provide a start_time to FFmpeg. This would cause stuttering in the video since the clocks class would be incorrect.
Removed findVideoStream and findAudioStream
The new FFmpeg release already has a function that will find the best audio and video stream. The code has been replaced with this function.
Updated error reporting
Some functions would not log an error when opening a file or modifying a file failed. New logs have been added as well as a function to convert error numbers to their string descriptions.
decode_video has been replaced
The old decode_video function would remove extra data that some decoders use in order to properly decode a packet. Now av_codec_decode_video2 has replaced that function.
Picture format changed from RGBA32 to RGB24
Since most video will not contain an alpha channel, using a 24 bit texture will use less memory."
the required header mathematics.h is not being included explicitly.
I have just included it, and verified that this also works with current
version 0.7, since the header is also available there."
I don’t have access to an OSX or Linux dev machine to make the changes required to the quick time plugin. This plugin will just default to returning 0."
OpenSceneGraph\src\osgPlugins\ffmpeg\FFmpegDecoderAudio.cpp
It solves a wrong sample format being set for audiostreams type S32 and F32, for the
enum AVSampleFormat (in include/avutil/samplefmt.h) doesn't match
enum SampleFormat (in include/osg/AudioStream) for anything but
SAMPLE_FORMAT_U8 and SAMPLE_FORMAT_S16
"
and follow up that address versioning issue
"I can find the change, avutil version 50.38.0 introduced a new header file:
libavutil/samplefmt.h
older versions use
libavcodec/avcodec.h
with the sampleformats without AV_
an intermideate fix with the FF_API_OLD_SAMPLE_FMT is removed from ffmpeg 0.8.3 so
I think this is the correct fix:
#if LIBAVUTIL_VERSION_INT < AV_VERSION_INT(50,38,0)
#define AV_SAMPLE_FMT_NONE SAMPLE_FMT_NONE
#define AV_SAMPLE_FMT_U8 SAMPLE_FMT_U8
#define AV_SAMPLE_FMT_S16 SAMPLE_FMT_S16
#define AV_SAMPLE_FMT_S32 SAMPLE_FMT_S32
#define AV_SAMPLE_FMT_FLT SAMPLE_FMT_FLT
#define AV_SAMPLE_FMT_DBL SAMPLE_FMT_DBL
#define AV_SAMPLE_FMT_NB SAMPLE_FMT_NB
#endif
"
deprecated, which led to compilation errors in the OSG plugin. I tried
fixing them and tested with the version ffmpeg-git-5d4fd1d (ffmpeg
version > 0.8, libavcodec = 53.7.0) under Windows.
"
Now we can use ffmpeg to render webcam video (using the vfwcap device) under Windows:
osgmovie 0 -e ffmpeg -O "format=vfwcap frame_rate=25"
The number 0 just indicates a default device number in ffmpeg.
I think this can work under Linux, too, and should be a bit better than comparing the filename with a '/dev/' string. Just type:
./osgmovie /dev/yourcam -e ffmpeg -O "format=video4linux2 frame_rate=30 size=320x240""
- missing copy attribute _referenceFrame in ClipNode in copy constructor
- checked iterators against the end in osgText
- close codec context in ffmpeg plugin to avoid memory leak
"
"enable thread locking in libavcodec
This is required for a multithreaded application using ffmpeg from
another thread."
"Prevent the audio from videos from hanging on exit if they are paused.
The video decoder already has similar logic."
"Add a way to retrieve the creation time for MPEG-4 files."
"fmpeg, improve wait for close logic
Both audio and video destructors have been succesfully using the logic,
if(isRunning())
{
m_exit = true;
join();
}
since it was introduced,
but the close routines are using,
m_exit = true;
if(isRunning() && waitForThreadToExit)
{
while(isRunning()) { OpenThreads::Thread::YieldCurrentThread(); }
}
which not only is it doing an unnecessary busy wait, but it doesn't
guaranteed that the other thread has terminated, just that it has
progressed far enough that OpenThreads has set the thread status as
not running. Like the destructor set the m_exit after checking
isRunning() to avoid the race condition of not getting to join()
because the thread was running, but isRunning() returns false.
Now that FFmpeg*close is fixed, call it from the destructor as well
to have that code in only one location."
The file FFmpegHeaders.hpp sets this definition. However, if stdint.h is
already included through other files, it won't take any effect.
Include FFmpeg headers as early as possible in order to avoid stdint.h being
included on other paths.
"
functionality and I also modified osgmovie example to support "seek"."
Note from Robert Osfield, changes osgmovie to use '>' for the seek as '+' was already used in a separate submission that had been merged.
