Screen capture issue on Win10

[Ramashankar]

Hi,
I have a Surface Pro3 (64GB) which I upgraded to Win10 OS just yesterday (earlier it had Win8.1). I had developed a screen capture app using screen capture plugin from Intel(R)_Media_SDK_2016.0.1 and Intel Media Samples 6.0.0.68.

After upgrading to Win10, I am observing a strange behaviour in screen capture. For all those desktop resolutions where it shows a black strip at top and bottom, or at left and right (for maintaining the aspect ratio), the starting few rows or columns respectively are filled with black strips. Actual screen data is not captured in these initial rows/column only. I verified it on 1920x1080 resolution. Same issue is occurring on a Win10 laptop also with resolution 1680x1050.

If I change to other resolution where desktop top left corner is fully aligned with screen's top left corner then screen is being captured perfectly. Also, this issue is not occurring on Win8.1 OS.

In one of my previous forum post (https://software.intel.com/en-us/forums/intel-media-sdk/topic/605345), I was informed that on Win10 Intel HW capture doesn't work and it falls back to Microsoft function. So do you think this issue is from Microsoft side?

Kindly suggest me what could be the cause of this issue and how we can resolve it.

Environment details:

- Media SDK version: Intel(R)_Media_SDK_2016.0.1

- sample code version: Intel Media Samples 6.0.0.68

- Driver version on surface: 20.19.15.4332 (SDK API version 1.17)

- Intel HD Graphics 4200

- OS: Windows10 64 bit

~

Ramashankar

[Harshdeep_B_Intel]

Hi Ramashankar,

Can you please share a reproducer (developed a screen capture app using screen capture plugin from Intel(R)_Media_SDK_2016.0.1 and Intel Media Samples 6.0.0.68) of your application using the samples. You can share the repro package directly via "private message". This can help us in further investigation and triage if this issue is either due to DXVA implementation on Win10 or the screen capture plugin itself. 

Thanks,

[Ramashankar]

Hi Harsh,

Thanks for your response. No specific reproducer required to check this issue, it can be reproduced with sample_decode.exe itself. I used Intel provided sample_decode.exe from [Intel Media Samples 6.0.0.68/_bin\win32] and capture plugin from [Intel(R)_Media_SDK_2016.0.1\bin\win32] path. I used following command line option:

[sample_decode.exe capture -w 1920 -h 1080 -d3d11 -p "22d62c07e672408fbb4cc20ed7a053e4" -o test1.yuv]

Please find attached 3 screenshots for this issue:

1. Screenshot_BeforeStart.png : captured just before running the exe

2. Screenshot_DuringRun.png : captured while exe was running

3. Screenshot_DumpPlayback.png: snapshot (with comment) of first frame of dump file (test1.yuv)

Please let me know if you need any other details in this regqard.

~ Thanks,

[Harshdeep_B_Intel]

Hi Ramashankar,

Thank you for sharing details and snapshots of the issue. I ran tests with sample_decode in capture mode on 4th/5th (Haswell/Broadwell) generation processors with Win10 installed and do not see or could reproduce this issue (black strips in captured frames, for 1st frame)  on latest versions of the graphics driver ( 15.40.4352:https://downloadcenter.intel.com/download/25644/Graphics-Intel-HD-Graphics-Driver-for-Windows-7-8-1-10-for-Intel-NUC and 15.40.4380:https://downloadcenter.intel.com/download/25749/Intel-Beta-Graphics-Driver-for-Windows-7-8-1-10-15-40-). Please try updating to latest driver from above link. Also, please note these drivers are generic drivers and could/will generate a warning during installation on your Surface Pro3, if that the case, I suggest contacting OEM (Microsoft) for drivers and the issue observed on Surface device. Snapshot attached. 

[Ramashankar]

Hi Harsh,

Thanks for your reply. It is strange that issue is not reproducible in your environment while it is always reproducible in my environment even after updating the latest driver 15.40.4380 [https://downloadcenter.intel.com/download/25749/Intel-Beta-Graphics-Driver-for-Windows-7-8-1-10-15-40-]. Refer the attached sys analyzer log.

So i want to confirm few things from your environment:

1. Are you using Surface device or any laptop/pc with Windows10?

2. What I observed in my environment is that: issue is reproducible only when desktop screen's left top coordinate is not starting from 0,0 i.e. it add some black strip padding to maintain the aspect ratio. So can you try it with any such resolution? E.g. in one of my test laptop having win10, issue does not occur with 1920x1080 resolution (probably because this resolution is scaled fully in my monitor's area). But if I change its resolution to 1680x1050 and setting of scaling is OFF then my desktop screen is padded with black strip in left and right side. Now in this case if i capture the screen then my captured frame is observing black strip in initial few columns.

To summarise it, wherever side (either left or top) OS is adding black strip to maintain the aspect ratio for that resolution, my captured screen's initial few columns or rows (respectively) are filled with black.

So please try with such resolution.

~

Thanks,

[Harshdeep_B_Intel]

Hi Ramashankar, 

Thank you for testing with latest driver and system analyzer log. Following are investigation notes. 

On Q(1),  I tried on a test pc and laptop with Windows 10 and exact same configuration as your surface (OEM) device using  the reproducer input steps you shared in previous comment : "[sample_decode.exe capture -w 1920 -h 1080 -d3d11 -p "22d62c07e672408fbb4cc20ed7a053e4" -o test1.yuv". I do not have access to exact system (OEM) machine as yours ( with Intel(R) Core(TM) i3). But, tried on a OEM (surface) device with Intel(R) Core(TM) i7 processor and could not reproduce same issue.

On q(2), Attached snapshots (from your previous comment) showed black strips for 1920x1080 resolution, but you mention "issue does not occur with 1920x1080 resolution (probably because this resolution is scaled fully in my monitor's area)" in above comment. Can you please clarify? I tried with 1680x1050 resolution with sample_decode, with command line :"sample_decode.exe capture -w 1680 -h 1050 -d3d11 -p "22d62c07e672408fbb4cc20ed7a053e4" -o test1.yuv", and do not see black strips in intial (first) frames of captured output. Refer to attached package (snapshot+output_captured). Let me know if I am missing something here and are you using default sample_decode (or a modified sample)? If, modified sample,then please attach modified reproducer sample.   

Can you please clarify "wherever side (either left or top) OS is adding black strip to maintain the aspect ratio for that resolution", what do you mean by that? As, I do not see any black strips captured in output for initial frames as OS is not adding any black strips to the output capture dump. Can you try on any other client (PC/laptop) system you might have with WIn10 installed? so we can isolate that this is not a specific OEM (Surface) issue.

Thanks, 

[Ramashankar]

Hi Harsh,

Thanks for your reply.

Can you please clarify "wherever side (either left or top) OS is adding black strip to maintain the aspect ratio for that resolution", what do you mean by that? 

I am sorry that I could not explain the scenario very well. So here are some new attachments from my today's test on a Windows10 laptop machine. Lets forget the surface device, issue can be reproduced on Windows10 laptop or desktop also. 

Please refer these attachment, I have added some comments in snapshot itself. Hope it would be easy to understand.

Test1 with 1920x1080 resolution: Desktop window is fully scaled on laptop screen, Issue did not occur.

Test2 with 1680x1050 resolution : Desktop window is not fully scaled to laptop screen, issue occurred (captured frames are cut from top, check in yuv dump) 

Test3 with 1600x900 resolution: Desktop window is central aligned to laptop screen, issue occurred (captured frames are cut from top and left both, check in yuv dump)

Test3 with 1600x900 resolution again: Desktop window is fully scaled to laptop screen this time, issue did not occur.

~ Thanks,

[Harshdeep_B_Intel]

Hi Ramashankar, 

Thank you for precise description of the scenario and the attachment. This definitely helps.  I see that changing display resolution settings (and when full scale to screen is not set in lower resolutions) is causing this behaviour. Let me investigate further with provide information and get back with updates soon. 

Thanks, 

[Ramashankar]

Hi Harsh,

Thanks for the update. Good to know that you could understand and able to reproduce the issue now. Looking forward for your further updates.

~ Thanks,

[Harshdeep_B_Intel]

Hi Ramashankar, 

We have triaged this issue and it is due to OS and have a bug sighting open. Will keep you posted whenever I have any updates. Also, I have sent a private message.

Thanks, 

[Ramashankar]

Hi Harsh,

Thanks for the updates. I have replied to your query in private message. I have one more query here, just for my curiosity:

Is this issue expected to be fixed in 'Intel Graphics driver' or in 'Media SDK Screen Capture Plugin' or in 'Media sdk sample code'?

As this issue is observed with Microsoft Desktop Duplication API also in exactly same manner, so I feel that root cause and fix might be common (either in Intel graphics driver or in MS DXGI components) 

~ Thanks,

[Harshdeep_B_Intel]

Hi Ramashankar, 

Thank you for your responding to further questions on private mesage. As, I mentioned previously the issue is at  OS level (specifically Desktop API), not in graphics driver or screen capture plugin on the sample code.  

Thanks, 

[Ramashankar]

Hi Harsh, 

OK, understood. I will wait for your further updates on solution of this issue.

Thanks,

[Chintan_M_]

Intel(R)_Media_SDK_2016.0.2\samples\_bin\x64>sample_decode.exe capture -w 400 -h 108 -p "22d62c07e672408fbb4cc20ed7a053e4" -o test1.yuv
plugin_loader.h :157 [INFO] Plugin was loaded from GUID

Return on error: error code -2, src\pipeline_decode.cpp 281

Return on error: error code 1,  src\sample_decode.cpp   501

plugin_loader.h :183 [INFO] MFXBaseUSER_UnLoad(session=0x0000000002868AC0), sts=0

Following error comes whenever I try to run anyone of the samples. Can you please assist?

[Ramashankar]

Hi Chintan,

As I don't have Intel(R)_Media_SDK_2016.0.2\samples installed on my system so unable to check the details. If you can share some code snapshot of around these lines here then we may find some relative info.

Return on error: error code -2, src\pipeline_decode.cpp 281

Return on error: error code 1,  src\sample_decode.cpp   501

[Chintan_M_]

Thanks Ramashankar for your reply.

This is the code around line 501 for sample_decode.cpp 

int main(int argc, char *argv[])
#endif
{
    sInputParams        Params; // input parameters from command line
    CDecodingPipeline   Pipeline;    // pipeline for decoding, includes input file reader, decoder and output file writer

    mfxStatus sts = MFX_ERR_NONE; // return value check

    sts = ParseInputString(argv, (mfxU8)argc, &Params);
    MSDK_CHECK_PARSE_RESULT(sts, MFX_ERR_NONE, 1);//line number 501 for sample_decode.cpp

    if (Params.bIsMVC)
        Pipeline.SetMultiView();

    sts = Pipeline.Init(&Params);
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, 1);

    // print stream info
    Pipeline.PrintInfo();

    msdk_printf(MSDK_STRING("Decoding started\n"));

    for (;;)
    {
        sts = Pipeline.RunDecoding();

        if (MFX_ERR_INCOMPATIBLE_VIDEO_PARAM == sts || MFX_ERR_DEVICE_LOST == sts || MFX_ERR_DEVICE_FAILED == sts)
        {
            if (MFX_ERR_INCOMPATIBLE_VIDEO_PARAM == sts)
            {
                msdk_printf(MSDK_STRING("\nERROR: Incompatible video parameters detected. Recovering...\n"));
            }
            else
            {
                msdk_printf(MSDK_STRING("\nERROR: Hardware device was lost or returned unexpected error. Recovering...\n"));
                sts = Pipeline.ResetDevice();
                MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, 1);
            }

            sts = Pipeline.ResetDecoder(&Params);
            MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, 1);
            continue;
        }
        else
        {
            MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, 1);
            break;
        }
    }

    msdk_printf(MSDK_STRING("\nDecoding finished\n"));

    return 0;
}

For code around line 281 in pipeline_decode.cpp please refer

// Populate parameters. Involves DecodeHeader call
    sts = InitMfxParams(pParams);
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    m_eWorkMode = pParams->mode;
    if (m_eWorkMode == MODE_FILE_DUMP) {
        // prepare YUV file writer
        sts = m_FileWriter.Init(pParams->strDstFile, pParams->numViews);
    } else if ((m_eWorkMode != MODE_PERFORMANCE) && (m_eWorkMode != MODE_RENDERING)) {
        msdk_printf(MSDK_STRING("error: unsupported work mode\n"));
        sts = MFX_ERR_UNSUPPORTED;
    }
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    // create device and allocator
    sts = CreateAllocator();
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    // in case of HW accelerated decode frames must be allocated prior to decoder initialization
    sts = AllocFrames();
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    sts = m_pmfxDEC->Init(&m_mfxVideoParams);
    if (MFX_WRN_PARTIAL_ACCELERATION == sts)
    {
        msdk_printf(MSDK_STRING("WARNING: partial acceleration\n"));
        MSDK_IGNORE_MFX_STS(sts, MFX_WRN_PARTIAL_ACCELERATION);
    }
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    if (m_eWorkMode == MODE_RENDERING)
    {
        sts = CreateRenderingWindow(pParams, m_bIsMVC && (m_memType == D3D9_MEMORY));
        MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);
    }

    return sts;
}

All the samples have been installed using following filename "MediaSamples_Windows_6.0.0.68.msi" provided by Intel

[Ramashankar]

Hi Chintan,

I have MediaSamples version 6.0.0.68 installed on my system so I could match the exact line of error in that sample (same as you provided above).  You are getting error in creating dump output file in pipeline_decode.cpp: 

sts = m_FileWriter.Init(pParams->strDstFile, pParams->numViews);

But as you are providing output file name correctly in your command line [sample_decode.exe capture -w 400 -h 108 -p "22d62c07e672408fbb4cc20ed7a053e4" -o test1.yuv] , so i think one possibility of issue can be the path from where you are running the sample app.

Do you have full access on the path? is the complete path name is very long? Please check on this point.

Also, to confirm that you have issue only in creating output file, please run these sample exe without giving output file option (-o) and check if you still getting error.

[Chintan_M_]

Hi Ramashankar,

Based on your suggestions I did following things:

1. Moved following files/folder just inside the E:\ drive sample_decode.exe (file), 22d62c07e672408fbb4cc20ed7a053e4 (folder)

2. Used following command without -o
E:\>sample_decode.exe capture -w 400 -h 108 -p 22d62c07e672408fbb4cc20ed7a053e4

After doing the above steps I got following prompt:

E:\>sample_decode.exe capture -w 400 -h 108 -p 22d62c07e672408fbb4cc20ed7a053e4
plugin_loader.h :157 [INFO] Plugin was loaded from GUID
WARNING: partial acceleration
Decoding Sample Version 6.0.0.68

Input video     CAPT
Output format   YUV420
Resolution      416x128
Crop X,Y,W,H    0,0,400,108
Frame rate      0.00
Memory type             system
MediaSDK impl           hw
MediaSDK version        1.19

Decoding started

I am grateful for your help.

~Chintan

[Ramashankar]

Hi Chintan,

Good to know that you are able to do the screen capturing now, and as I can see in your another post https://software.intel.com/en-us/forums/intel-media-sdk/topic/707004 you are able to save the dump file also. So go ahead with your experiment.

To see this screen dump file, either use FFMpeg's  ffplay.exe or you can use any other yuv player and just provide correct resolution (400x108 here) and raw video format (YUV420 here).

[LifeChariCute_L_]

If it is caused by Windows 10, you may roll back your OS to the former version or use other screen capture software to help you finish the job. For exmaple, you can try out Camtasia which is very famous in the filed. To record it:

1. Choose full screen recording

2. Click on Record- the red button

3. Click on "Finish" When you want.

The file will be save as Camrec if you use version before 8.4.

For more information about recording with Camtasia you may refer to this article:

http://www.videoconverterfactory.com/tips/camrec-to-avi.html

[Ramashankar]

Hi,

Thanks for your reply. This Camtasia looks good tool to use, but I need solution through some program/SDK/APIs which I can integrate in my code. As our product needs to support windows 10 OS also so rolling back to former OS will not help me here.

Thanks,