§ ¶DirectSound Renderer latency woes
I'm tweaking my laptop-as-a-TV setup again, and am running into problems with latency in the DirectShow filter graph.
The video path I cobbled together a while ago, and works as follows: I have preview acceleration mode activated VirtualDub and set to interlaced fields / even field first, which then pushes the video through a Direct3D pixel shader that does adaptive deinterlacing and upsampling to field rate with a little bit of bicubic filtering and gamma adjustment thrown in. That works surprisingly well, with good latency and a decent image. The problem is the audio, which initially starts out fine but slowly accumulates a lot of latency over time. I've spent a while fiddling with the filter graph clock and perusing the various (many) interfaces, and have determined the following:
- Rate matching by data rate is active on the audio renderer.
- The capture filter is sending 40ms audio buffers, and doesn't seem to respond to IAMBufferNegotiation requests to change it. Therefore, that's 40ms of audio latency I can't get rid of.
- The DirectSound buffer size is about one second.
- The buffer level reported through IAMAudioRendererStats very slowly rises over time, from about 4% to 90%.
- Neither SyncUsingStreamOffset() + SetMaxGraphLatency() nor SetErrorTolerance() appear to help.
I had this same problem with my Adaptec GameBridge, and now it's happening with my ATI capture device, so I don't think it's capture driver specific.
It looks like the current problem is that the DirectSound Renderer is intent upon slowly filling up its one second buffer over time, even though that is a huge amount of latency to add to a live stream. For a live TV stream, that usually isn't a problem, as the audio and video just need to have the same amount of delay to be in sync. However, since I'm trying to play Ar Tonelico, any delay on either the audio or video stream results in a delay relative to the controller inputs, and I really don't want that amount of latency anywhere in the system. It's a bit frustrating that the audio system runs well at first but gradually slews to unusability until I reset the filter graph again.
Does anyone have any experience with this problem or tips as to how to solve it? I really don't want to have to write my own audio renderer, but I'm beginning to think that might be the best solution.
I had a similar problem playing avi files
with ntsc framerates where the audio would fall out of sync.
I found the easiest way to fix it was to use
reclock which adjusts the framerate to the refresh rate of your monitor.
Maybe a similar approach would work?
whitetiger - 11 07 10 - 20:10
I had something similar with the WinTV software for a Hauppauge HVR-1100 when feeding the card a composite signal - the sound would become delayed over time. There were no glitches in the sound, so it must have been resampling it slightly to stretch it. I didn't notice any significant delay on the video, so the problem is audio-only.
I eventually "fixed" it by using DScaler, which I believe uses something other than DirectShow to capture and display the audio/video. I don't know if using DScaler's method would be any good for you.
Torkell (link) - 11 07 10 - 20:32
What is the true clock rate for your audio output device, looks like it might be a tad slow and the stupid renderer is not correcting it. Can you quickly fudge the audio stream rate to 48001 or more, this will do a quick test of the hypothesis.
IanB - 13 07 10 - 14:02
...That's the reason for ASIO existence...
Jam_One - 13 07 10 - 20:10
Um, no. We're talking about up to a SECOND of latency here, not 10-100ms. DirectSound itself is perfectly capable of acceptable latency for this task.
Phaeron - 14 07 10 - 16:22
You said "... which initially starts out fine but slowly accumulates a lot of latency over time
I took this to mean the input data is arriving at a rate very slightly faster than the renderer is dispensing with that data, hence the ever increasing latency.
And you said "* Rate matching by data rate is active on the audio renderer.
" But for the above to happen, the rate matching cannot be working correctly.
IanB - 16 07 10 - 22:35
Well, it sort of is and isn't.
Initially, there is nearly zero delay, but the delay builds up over time as the buffer slowly fills up, until it converges around 90%. This means that the renderer is not accurately rate matching initially, but eventually does so. This is what I would expect for a rate matching algorithm configured to "soft start" with a slight bias until it achieves the desired level of buffering. The other possibility would be that the rate matching isn't working correctly and it's instead being bumped by a periodic buffer-full condition, but I would think that would lead to stuttering.
IAMAudioRendererStats does report the current value of the rate matching algorithm, and I can see it applying corrections. I haven't checked whether this matches the incoming rate.
Phaeron - 17 07 10 - 13:58
"... instead being bumped by a periodic buffer-full condition ...
" Sound's more like the dopey renderer has "achieves the desired level of buffering.
" at a level that does not suit your use.
Thus my initial stab of "fudge the audio stream rate to 48001 or more.
" to force the renderer into starvation.
IanB - 17 07 10 - 20:28
MPC HC has a directshow based standalone audio renderer "MPC Audio Renderer" perhaps it can be of some use to you
zee - 21 07 10 - 06:15
Avery, few questions:
1. You didn't mention which operating system you are having this problem with? Vista? Win7? WinXP? 32-bit? 64-bit? All of them?
2. Which audio card? Onboard or addon (such as PCI-e X-Fi Titanium)? Or all have the same problem?
3. Generic Microsoft audio drivers? Latest manufacturer's drivers?
4. Have you checked your DPC latency? Maybe it is a video driver issue causing a delay. Have you tried another video card and drivers?
Finally, perhaps your video frame rate is off by a bit and it accumulates.
Igor Levicki (link) - 24 07 10 - 11:01
Um, did you read my post or are you just guessing? Why would DPC latency matter with regard to user space buffering on the order of one second, and what would a video driver delay matter when the audio renderer is in data rate matching mode?
Phaeron - 24 07 10 - 22:26
I had the same problem but was able to fix it (at least in WinXP) by using DVDIdle Pro. I haven't run into the problem in Windows 7, so maybe they fixed it? Anyway they have a free trial of DVDIdle pro so it won't cost you anything to try it, and it boosted overall performance as well as battery life in my laptop. What it does is slice off an amount you choose of your RAM or HDD to use as a buffer. Since RAM is cheap I used 512Mb of RAM as a cache which seemed to fix the problem for me.
kb - 25 07 10 - 00:30
No, I wrote a comment without reading your post, I have psychic powers :p
Of course I read your post.
I maybe guessing but I also asked for additional information, pretty much standard practice in troubleshooting while you sound as if you already know what might be the problem, and you are sticking to it thus making yourself blind to other options.
AFAIK, DirectSound Renderer acts as a wrapper for an audio device. If the rate matching is on and you get the accumulating delay, perhaps the audio driver is broken that is why I asked if you tried different audio drivers and devices.
It may also be a problem with DirectSound that is OS specific (or even worse, OS and driver specific), that is why I asked you which operating systems you've tried this on.
Regarding DPC latency, if video driver (or any other driver for that matter) takes a lot of time to do the processing in DPC, then all sorts of nasty things can happen in both kernel and user mode. For example, I've seen laptop WiFi card's drivers making the whole computer sluggish when they can't connect. CPU usage in task manager is 0%, yet you can't work with the laptop and if you check DPCs you see that the driver is taking a good 40-60% of the CPU time. It might have nothing to do with your problem, but IMO it is worth checking out just in case -- it wouldn't be the first time that DPC latency causes audio problems either:
Igor Levicki (link) - 25 07 10 - 00:42
I've experienced a similar sounding issue with MPC-HC and the HDHomerun TV tuner. In both cases, the programs were using ffdshow filters for mpeg2 files or streams and the audio would trail the video over time or when I skip. My solution was to go into the ffdshow video decoder configuration and switch from the default libavcodec decoder to the libmpeg2 decoder for mpeg2 video. Don't know why or how, but it solved the problem for me.
Pong - 27 07 10 - 20:19
Ran into a thread that discussed buffers the other day:
May have some clue or not.
Also do you know of any open source projects that are close in scope to a "directshow audio source filter for tracking wave out in vista"? (anything that might be good to fork it and proceed from there)?
roger (link) - 30 07 10 - 06:07
OK this is under the right topic now... I apologize if I'm being Captain Obvious by suggesting this, but sometimes the obvious gets overlooked. If the only real goal is playing a console game and using the laptop as a TV, there may be a simple workaround. Does the laptop have a microphone port? Try using the microphone port for audio and the capture card plus Virtualdub for video and see if the latency problem goes away.
I don't have game system, a laptop or a ATI theater 750 USB capture device to test with. I tested using two DTA converter boxes tuned to the same channel. One was connected to a TV ,and the other to the ATI TV Wonder 650 PCIE capture device installed in my desktop PC. The PC was 3-4 sylables behind the TV when using the capture card for both audio and video. That's about a 1 second delay.
When I set up using the microphone port for audio, and the capture card for video, it eliminated the delay as far as I can tell. The audio and video on the PC side still seemed pretty well synchronized even without going through the capture card. I ran the test for about 20 minutes, and everything still seemed fine at the end.
I have repeated the same things several times and had the same results consistently. I even ran the same tests with DScaler and it did not behave any differently than Virtualdub. I haven't gotten the card working yet with VLC, so I don't know what will happen there. Using the microphone port for audio input may not work for every combination of software and hardware, but if the proper cables are at hand, it is easy to run the experiment.
Ms. R - 21 08 10 - 17:07
Please have a look at EAX-extensions (version 2 tops, originally v1). Furher versions are trademarks of Creative soundcards but the issue gets worsening. There are a few older cards that use hardware driven clock-sampling rate (the exact term is hardware acceleretad extensions). The problem in discussion (audio vs video syncronization) lays not in Virtualdub. Beggining with Vista, Microsoft dropped out the Direct Hardware acces to soundcard-devices and by doing so it leaves us only DirectSound interface to work around.
The fact is that most cards don't have hardware accelerated extended functions and O/S prior to Vista had offered these "raw" functions using "hardware wavetable". It was the hardware that processed the output, and Windows had added more options through the DirectX package.
Many soundcards lack these hardware overlay interface EAX-functions and the buffer used to desyncronize it seems it got overlooked. Vista offers "support" for EAX but there is a small difference between the Original-Hardware-Related EAX-extensions and the implementation done in Microsoft Vista. It allows acces only to software interfaces since Vista dropped the direct-to-hardware acces. Please someone notify Microsoft what they did is bad.
The limits are obvious in winamp if I may say so. In windows 98 you had the output device for soundcards chose from "wave mapper" and let's say "maestro2". The first option is using DirectSound interface to hardware and the 2nd is using direct hardare acces. When using the second Windows98 could not acces the hardware by overlaying its sound-notify events. If you were using the 1st (virtual) device the playing of a fiel was done using the "wave mapper" something to be looked at as a HUB = any audio inputs => single speaker.
Windows XP drivers lacks part of these functions while the speaker plays sound no matter what output device is chosen. Virtualdub is capturing OK since XP has hardware acces layers to the soundcard. A capturing session done in VirtualDub when using EAX-extensions -able as Maestro2 soundcard (windows XP tops) is running perfect. A few problems appear when using no-signal-live antenna signal.
1.I don't put my many on newer soundcards like ASUS XONAR which offers EAX just to find out Microsoft does not.
2.Somebody be kind and put this matter on the table with Microsoft.
Bogdan (link) - 24 08 10 - 22:21