Unable to capture dxcap replay / best way to capture a frame which will time out GPU

[Simon_H_1]

Hi there,

I have a frame which repeatedly causes a timeout of the graphics driver and I would like to find out why. If I run with the WARP renderer it will eventually complete the frame, but it will take several minutes. I would really like to be able to capture this frame in GPA and find which draw call is the dodgy one.

So my questions would be:
1) can I capture several seconds worth of frames (hopefully including my bad frame), so that I can replay the capture and find the faulting frame and then the draw call?

2) I have managed to capture this event with dxcap. If I play back this capture it will do the appropriate bad thing on the failure frame. However if I try and run GPA's capture.exe (or the GUI tool) on dxcap nothing happens. The program silently quits after a few seconds of thinking. I can capture all other programs just fine. What can I do?

Cheers,

Simon

NB this frame will also time out NVIDIA GPUs too.

[Simon_H_1]

I've managed to get a captured frame! However it times out in GFA before it finishes rendering the erg tree view. I've been looking at this for days now - is there a way to somehow load my capture and only then step through the draw calls?

[Seth_S_Intel]

Hi Simon, 

Great questions!

1. There isn't a good way to capture a stream of frames.  But you may be able to use the triggers inside of GPA monitor to get the frame you need.   When the bad draw happens, does your frame rate or any other metric drastically change?  If so, you could trigger off that metric and GPA will capture that frame for you.  

2.  Glad you got the frame captured!! Sorry that GFA is timing out though.   Let's see if we can fix this for you.   If possible, could you private message me with the frame that is timing out?  Also, if you could post your Graphics Monitor -> About dialog, It will help me file a defect.   Unfortunately there is not a way to partially load the frame file. Is the scene extremely large?

Best,

Seth

[Simon_H_1]

Thanks!

1) yes, when using the WARP renderer it'll go from a hundred milliseconds to a couple of minutes per frame. As long as the event happens for a number of frames the trigger sounds the way to go.

2) Will do. I left it loading over night and I did manage to isolate the draw in the end. Supposedly a compute job took 91 million milliseconds to run. Err...!

I'll PM you with a capture and the details in a minute.

Thanks again!

Simon

[Seth_S_Intel]

Hi Simon, 

I got your frame file!  Let me know if you need help with setting those triggers.  

Also,  I saw that your about dialog said that you were using GPA 14.4.   We have fixed a lot of issues since this release, would you mind trying GPA  15.3?  software.intel.com/gpa 

I saw an error similar to this a while back and 15.3 seemed to fix it. :)

Best,

Seth

[Simon_H_1]

Yes I could have sworn I was using a far later version - a pre-release version you had supplied me before! Turns out that was on my other machine... When loading the file on that one (with NVIDIA-only graphics) it loads in a similar way - very very slowly!

After a couple of days debugging it now it turns out it may be a code-gen issue: we always get the same result but some machines run this code unbelievably slowly and others can do it in real-time!

(btw the other version I'm now loading the capture in is 15.3.249085)

So it looks like I'm in the clear now I know how to capture on a trigger. Although saying that is there a way to automatically capture what has been submitted if there is a GPU timeout?

[Seth_S_Intel]

Awesome.   Let me know if it doesn't turn out to be a code-gen issue and we can start the debug process on the GPA side. Also,  you may want to upgrade to the released version of 15.3 just for good measure. :) 

As for the triggers,  there isn't a way to trigger on a timeout, but you may be able to trigger the right frame off of EU stalls or Locks.   It will be different for each application, but you may want to give those a try.   

Best,

Seth

[Simon_H_1]

Out of interest, where exactly do I download the latest version? If I go to the downloads section and select the free INDE download for Windows, after running the installer I get 15.1.242923. When I then run GFA it prompts me again to upgrade! That upgrade link dumps you at the generic GPA landing page. Do I require the non-free versions to get 15.3?

Thanks!

[Seth_S_Intel]

Sorry about the confusion with INDE right now.  We are working to solve this.  Currently we offer a stand-alone version outside of INDE which is the most recently updated version of GPA.  You can get 15.3 for free from the GPA homepage - https://software.intel.com/gpa

You can then get access to all the GPA installers for our different host options and components.  Hope this helps! 

[Simon_H_1]

Thanks again for your help! Sorted. I may come back to you with more GPA problems :-)

Btw you mentioned code-gen. What do I do if I do find a code-gen problem? I'm seeing sin() on really large numbers (float) going wonky after a certain point and only with Intel.

HD 4000, driver version 10.18.10.4226. Even with the /Gis fxc option I get trouble.

Basically this:

float4 PS()

{

    int2 screen_pos(input.position.xy);

   float something = screen_pos.x * screen_pos.y;

   return float4(sin(something).xxx, 1);

}

[Seth_S_Intel]

Hi Simon, 

I have talked to a few people about this issue so far and it looks like it may be a driver defect on our part. 

A couple questions:

1. Could you upgrade your driver to the latest? - https://downloadcenter.intel.com/download/25313/Intel-HD-Graphics-Driver-for-Windows-7-8-1-10-3rd-Gen-BYT-
2. Also, could you post your about dialog here so I can file the driver bug properly. 

Thanks Simon and sorry for getting back to you a little late. 

Best,

Seth

[Simon_H_1]

Yo, sorry for the delay.

I've just upgraded to the latest HD 4000 driver and I can still get the same thing; version 10.18.10.4276.
I can send you a repro if you would like? (although that would take a little while)

Basically just draw a screen-sized quad with

float4 PS()

{

    int2 screen_pos(input.position.xy);

   float something = screen_pos.x * screen_pos.y;

   return float4(sin(something).xxx, 1);

}

and compare it with

float4 PS()

{

    int2 screen_pos(input.position.xy);

   float something = screen_pos.x * screen_pos.y;

   return float4(sin(fmod(something, 2 * 3.141592653589)).xxx, 1);

}

When using the WARP renderer, they both give the same image. But when running on the hardware there is a noticeable difference between the two.

Thank you :-)
Simon