DirectShow Encoding

[superrudi]

Hi,

i have some problems with the DirectShow encoding. I build the following graph with graphedit:

SourceFilter(800x600, RGB32, 20fps) -> Intel Media SDK H.264 Encoder -> Intel Media SDK MP4 Muxer -> FileWriter

When i start the graph, the number of frames encoded is rising, but the file-size remains very small and i can't play the file with VLC or Windows Media Player. What's wrong?

The second problem is that i need the InfinitePinTee or SmartTee to double the stream before encoding, but i can't connect the Tee Filters with the Intel Encoder Filter.


My system:

Asus P8H67-V
i7-2600
NVidia GTX 550 Ti
3 displays, 2xNVidia, 1x onboard
Win7 x86
Media SDK 3.0 Beta 4
newest Intel HD graphics driver
Visual Studio 2010 with Win SDK 7.1

Thanks,
Superrudi

[Eric_S_Intel]

Hello Superrudi,

I went back and checked the MSDK 3.0 Beta 4 Dshow filters, and found that I could connect the smart and infinite T filters to the h264 encoder filters without problem. Can you make sure the filters that are being loaded are the ones you are expecting?

Regarding whats happening with your encode, I am not sure I have enough information to make a guess. Can you explain, or share your source filter and the clip its outputting? Is it a push or pull filter? This usage model has been used over and over, so Id like focus in on the source filter first. A bit more information will help me get you up and running.

Thanks

-Eric

[superrudi]

Hi,

the filters are the right ones, which are located in the Media-SDK/bin/win32 directory. My filter for testing is a Push-Source derived from CBaseFilter and the pin is derived from CBaseOutputPin. In short the deliver-function does the following (moving white rectangle on black background):

IMediaSample *pSample=0;
HRESULT hr = pin->GetDeliveryBuffer(&pSample, 0, 0, 0);
pSample->GetPointer(&pData);

//---------------------------------
memset(pData, 0, width*height*4);
unsigned char* pos = pData + (x + width*y)*4;
for(int i=0;i{
memset(pos, 255, 4*size);
pos += width*4;
}
x++;
//----------------------------------------

pSample->SetActualDataLength(width*height*4);
start = sampleTime;
sampleTime += m_rtFrameLength;
pSample->SetTime(&start, &sampleTime);
pSample->SetSyncPoint(TRUE);
pin->Deliver(pSample);
pSample->Release();


The MediaType is set as follows:

CMediaType *pmt;
VIDEOINFOHEADER *pvi = (VIDEOINFOHEADER *) pmt->AllocFormatBuffer(sizeof(VIDEOINFOHEADER));
ZeroMemory(pvi, sizeof(VIDEOINFOHEADER));

pvi->bmiHeader.biCompression = BI_RGB;
pvi->bmiHeader.biBitCount = 32;
pvi->bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
pvi->bmiHeader.biWidth = width;
pvi->bmiHeader.biHeight = height;
pvi->bmiHeader.biPlanes = 1;
pvi->bmiHeader.biSizeImage = GetBitmapSize(&pvi->bmiHeader);
// pvi->bmiHeader.biSizeImage = 0;
pvi->bmiHeader.biClrImportant = 0;
pvi->AvgTimePerFrame = m_rtFrameLength;

SetRectEmpty(&(pvi->rcSource));
SetRectEmpty(&(pvi->rcTarget));

pmt->SetType(&MEDIATYPE_Video);
pmt->SetFormatType(&FORMAT_VideoInfo);
// pmt->SetFormatType(&FORMAT_VideoInfo2);
pmt->SetTemporalCompression(FALSE);

const GUID SubTypeGUID = GetBitmapSubtype(&pvi->bmiHeader);
pmt->SetSubtype(&SubTypeGUID);
pmt->SetSampleSize(pvi->bmiHeader.biSizeImage);

[superrudi]

I did some debugging now and found in function CEncVideoFilter::DeliverBitstream that pBS->DataLength is only 522 for the first sample and the next ~100. For further testing I filled 100 lines of the bitmap with random pixels. Now the DataLength is much bigger and the filesize is growing. So I think, the file was small because of the uniform picture. But i still can't play the Video with VLC or MediaPlayer. And i can't connect the Async FileReader loading that file with the Intel H264 Decoder.

The next step is debugging the decoder. CH264DecVideoFilter::CheckInputType only get GUID_NULL in *mtIn->Subtype() when i try to connect the decoder with the async reader. I can see it in graphedit on the output-pin of the FileReader, too. Can I use FileWriter and FileSource(Async) for that?

Now I tried to connect the Encoder with the InfinitePinTee. The props.cBuffers in CEncoderInputPin::NotifyAllocator is only 1 and the Encoder needs nMin=7. What can I do to get higher cBuffers for InifinitePinTee?


Have a nice weekend,
Superrudi

[Eric_S_Intel]

It sounds like the async filter doesnt recognize the file format. Can you explain what you are trying to do with the graph? The first message didnt contain the decoder filter and now it does. Perhaps theres a better way to do it?

Regarding the Infinite T filter. We have ran across the problem before. We found that when we construct this graph: Camera->MJPEG Decompressor->Infinite T -> Encoder -> etc the Infinite T filter does not pass the MJPEG Decompressors allocator requests through to the encoder. It sends 1 instead. I suspect the same thing happening in your case. We were able to use the T filter if we just removed the : if(nMin > (mfxU32)props.cBuffers) check and just return S_OK. Not very elegant, I know. But since we dont have access to that filters source code, it was the best we could do. We tested that graph quite a bit and didnt run across any problems with allocations. Try it, and test.

You have a good weekend as well!

Eric

[superrudi]

This graph is only for testing. To get your filters running, the graph should be as simple as possible. So i tried one simple graph to save a mp4 file and one to replay the file. Our application graph is a little bit more complicated. It's a streaming application with preview, recording, motion detection and replay. In the last years we have used xvid or a mjpeg codec.

For me it sounds like the file format is not written correctly, because VLC and Windows MediaPlayer could not read the file, too. Can I check the correct file format somehow? With google I found mp4UI. On opening the file with that tool, i get the error "invalid atom size".

To connect the Encoder with the InfinitePinTee, I set cBuffers=7 in DecideBufferSize() of my source filter. Now it connects. I found the InfinitePinTee in the samples directory of the Windows SDK7.1. So I can try to get an elegant solution later.

[superrudi]

I can record files and play them now. The problem was GraphStudio, an alternative to graphedit. I had never problems with that, but it produces corrupt mp4 files in this case. With graphedit the saved files are ok.

Now i looked at CPU usage and i found out, that it is ~20% no matter if I activate the integrated GPU in the BIOS or not. How can I check, if the GPU is used for compression? What could be the reason, that the Intel GPU is not used?

[Eric_S_Intel]

Great. Seems like you are making progress. Thats good. However, 20% seems a bit high if the GPU was active. So, can you take a look at the CBaseEncoder::SetAcceleration() and see what impl is being returned? For the sake of completeness what is the version as well?

The HW lib is loaded by the MSDK dispatcher. The algorithm it uses to find the right implementation is documented in the mediasdk-distrib file in the \doc folder. The HW lib is only installed by the graphics driver it is not part of the MediaSDK download. If you havent done so, Id recommend updating your driver first.. then set about troubleshooting.

-Eric

[superrudi]

The graphics driver 15.22.50.2509 is installed on my system. SetAcceleration() is returning impl=0 and version Minor=1, Major=1, Version = 65537. Media SDK is Version 3.0 Beta 4.

Next step: from this thread http://software.intel.com/en-us/forums/showthread.php?t=81521 I got the hint to replace in CBaseEncoder::CBaseEncoder

sts = SetAcceleration(MFX_IMPL_AUTO)

with

sts = SetAcceleration(MFX_IMPL_HARDWARE_ANY);
if(sts != MFX_ERR_NONE)
sts = SetAcceleration(MFX_IMPL_AUTO);


Now the CPU is using only 5% for 1920x1080/20 fps. I tried it by hand and MFX_IMPL_HARDWARE3 is successful, don't know why it is not found with MFX_IMPL_AUTO.

When I build the graph with graphedit, the filters are connecting correctly. But when I use the following code in my application, the muxer don't connect with the file writer and i don't get an error:

CoCreateInstance(CLSID_MP4MuxerFilter, NULL, CLSCTX_INPROC, IID_IBaseFilter, (void **)&pMux);
pGraph->AddFilter(pMux, L"Muxer");
CoCreateInstance(CLSID_FileWriter, NULL, CLSCTX_INPROC, IID_IBaseFilter, (void **)&pFileWriter);
pGraph->AddFilter(pFileWriter, L"File Writer");
pBuild->RenderStream(0, &MEDIATYPE_Video, recordBranch, 0, pMux);
pBuild->RenderStream(0, &MEDIATYPE_Video, pMux, 0, pFileWriter);

What could be wrong?

[superrudi]

I tried to connect the filters with my own code instead of RenderStream. After searching the output pin of the muxer and the input pin of the file writer, I use

pGraph->Connect(pPinUp, pPinDown)

to connect the filters. Executing this function raised an exception:

Unhandled exception at ... in ...exe: 0xC0000005: Access violation reading location 0x00000000

I don't know why this happens and why it works with graphedit. What's the difference? With one of my filters I can reproduce this. After CBasePin::ReceiveConnection() is returned with NOERROR, the exception raised.

[Eric_S_Intel]

Hi again,

Ill code something up to simulate your graph building problem tonight, but in the meantime..

So we know this works fine in graphedit Ive checked it again today to ensure I didnt miss anything. So, question - when constructing the graph in code are you setting the filename the filewriter filter needs to output too? I noticed you need to QueryInterface the ISinkFilter->SetFilename method. Seems like this could cause an exception if it was overlooked. Can you confirm?

Thanks!

-Eric

[superrudi]

Yes, i set the filename through the interface IFileSinkFilter2. You can set and change the filename after connecting.

[Eric_S_Intel]

Have you compared your code against the MSDK sample "DShowPlayer"? This sample builds a graph programmatically - and connects the muxer to the filewriter when the user transcodes a file.Its running w/o issues.
-Eric

[superrudi]

Hi again,

my streaming server works now! I learned 3 things:

1) ICaptureGraphBuilder2::RenderStream() doesn't connect the MP4 Muxer with the FileWriter, you have to connect them with IGraphBuilder::ConnectDirect()
2) You have to set a filename before connecting the MP4 Muxer with the FileWriter (you can change it after connecting). With AVI Muxer you can do it after connecting.
3) When setting the filename to c:\tmp.mp4, the FileWriter won't connect on Win7. Perhaps file permission problem?

Next step: The streaming client...

[Eric_S_Intel]

Great News! Thanks for sharing your findings.

Not sure about the behavior of the filewriter filter, certainly windows permissions need to be considered.

-Eric

[superrudi]

Hi again,

everything is working now, but when I play a saved H264 file with

FileSource->Splitter->Decoder->ColorConverter->Renderer,

the picture is upside down (horizontal mirrored). When I replace my ColorConverter and my Renderer with the Enhanced Video Renderer, the picture is not flipped. I tried the following graph and everything was fine:

source->Encoder->Decoder->ColorConverter->Renderer

I can't connect ...Encoder->Muxer->Splitter for testing, so that's not the whole graph with all needed filters. If I replace my ColorConverter and my Renderer with the Enhanced Video Renderer, I get an error that I can't play the graph.

What is the difference between live view and file playback?

[superrudi]

Hi,

does'nt anybody know the difference between replay and live view from my last post?

Another proplem: I need to encode in the resolution 1080x1920. The CPU load with this resolution is 4-5 times higher than 1920x1080. m_pmfxENC->Init(&InitParams) in CBaseEncoder::InternalReset() returns MFX_WRN_PARTIAL_ACCELERATION. Can this been fixed somehow?

Best Regards,
SuperRudi

[IDZ_A_Intel]

Hi Superrudi,

As part of Media SDK 3.0 beta4 samples a new a new filter option is accessible via the m_bUseRGB32toNV12Plugin flag. Using thisflag RGB32 frames can be flipped if needed. Can you please explore this option to see if it helps. I suspect this may be the rootof upside down stream you encounter.

Regarding 1080x1920 encode. Media SDK and the driver does not support HW acceleration for that resolution, and will therefore fall back onSW encode, resulting in high CPU load. For more info see this post:

http://software.intel.com/en-us/forums/showpost.php?p=153211

Regards,

Petter