Vulkan App exits with GPA

[GregConnected]

I have a Vulkan application I'd like to profile with GPA Trace.  When I try to start the application from Graphics Monitor, the application starts, then exits almost immediately with no visible error from either itself or Graphics Monitor.

However the application logs this error from vulkan validation layers:

Vulkan (Error) : Validation Error: [ VUID-vkCmdResetQueryPool-firstQuery-00797 ] Object 0: handle = 0xa7c5450000000023, type = VK_OBJECT_TYPE_QUERY_POOL; | MessageID = 0xd66ad571 | VkCmdResetQueryPool(): In Query VkQueryPool 0xa7c5450000000023[] the sum of firstQuery (10) + queryCount (2)is greater than the queryPool size (11). The Vulkan spec states: The sum of firstQuery and queryCount must be less than or equal to the number of queries in queryPool (https://vulkan.lunarg.com/doc/view/1.3.216.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdResetQueryPool-firstQuery-00797)

I can run this app in DX11, OpenGL 4.5 and Vulkan.  The test system contains an integrated Intel HD Graphics GPU and an nVidia Quadro GPU.

If I run with DX11 or GL on nVidia, the app runs and I see the profile overlay display. Running on intel GPU in any mode fails shortly after start.

[GregConnected]

Additional test details...

Updated intel graphics driver from 31.0.101.2114 to latest 31.0.101.2115 WHQL, recaptured with new driver.

Tried open frame on nVidia device instead of Intel.

No change, same error in Graphics Frame Analyzer.

[Pamela_H_Intel]

Thanks for hanging in there, Greg. I just reread the entire forum stream. So . . . embarrassingly, I have to say, I may have been misunderstanding the issue. I've been working on addressing a GPA issue. But your 2nd to last comment says that your workload works on NVIDIA (all 3 APIs), but not on Intel . . . so are you saying that the workload/game won't play at all if you disable your NVIDIA GPU? If that is the case, I will need to contact the Driver team. If it's a driver issue, GPA can't do anything to fix it. But we definitely want it fixed. Please let me know if, *without* GPA, your workload will not run.

[GregConnected]

Welcome back Pamela. That doesn't sound correct...

One of my earliest, or even first posts (09-23-2022 my time) mentions the app, when launched from Graphics Monitor, runs on nVidia in all modes but not intel.
However later, after updating GPA tools and Intel drivers, behavior improved somewhat.
Even later, I realized that it is non-release builds of my app that consistently fail to run in Vulkan mode on Intel only, but Release builds do run. (Side note: When Vulkan fails on non-release build, on Intel, the logged error messages have varied over time.)

Ignoring non-release builds, at the present time, I want to, and can, capture Vulkan Frames and Streams from Graphics Monitor on Intel. What I cannot do is view them in Graphics Frame Analyzer. The GFA app fails, with generic error message.

The fact that my dev machine is a nVidia + Intel GPU system just adds to the complexity of testing... To ensure a 100% Intel test, I must configure Windows (projector modes) or turn off my (main) external monitors and use the laptop display.
Otherwise either the wrong GPU will be selected, or the app will run at much lower speed as it's output is copied to the other GPU. (Task Manager shows this behavior as well as low frames per second.)
This is really just an annoyance or inconvenience. Attempting to configure Windows to use just one GPU instead of the other (via Windows, nVidia control panel or Intel control panel) does not appear to work as expected.
However DX11 and Vulkan APIs do allow adapter selection, which can work albeit at the lower performance depending on monitors.

So what I need is...
Intel Graphics Frame Analyzer to load a Vulkan Frame or Stream capture without error.
Non-release build of my app running Vulkan to launch from Graphics Monitor to capture Frames and Streams. (Because non-release builds use the Vulkan validation layers and debug object naming etc.)

I have no reason to believe the nVidia GPU or external monitors relate to this issue, unless perhaps the nVidia GPU causes an internal conflict within Intel Graphics Frame Analyzer regardless of configuration. Edit: I am able to capture a Stream from DX11 and load it in Graphics Monitor, so the issues appear to relate more to Vulkan.

[Pamela_H_Intel]

Okay - thanks for summarizing everything in one post.

I will update the Jira for our devs.

Also, do you know about selection of GPU in Graphics Frame Analyzer? After opening a stream (*before* opening a frame) in GFA's multi-frame view, you can select which GPU to play back on . . . in the upper left corner you will see the GPU that is selected. Click on it to see the dropdown of all GPUs available on your platform. Select the one to play back on. Then open a frame.

• opening a stream collects all the API calls and resource info
• opening a frame is when the calls are actually executed

[Pamela_H_Intel]

Greg - sorry for that last post. I just reread the whole stream and realized that I had already told you about selecting the GPU in GFA.

You are saying that even disabling a GPU in Windows Device Manager doesn't work as expected? That's crazy.

I want to run this by our Vulkan expert again . . . we shouldn't fail on non-release builds when we don't on release builds.

Pamela

[GregConnected]

Thanks for returning Pamela, I've just done some re-testing and have new results to report.

Testing with GPA v23.3 on Windows

gpa_23.3.1694629450_release-external_x64_standalone

First, I did disable nvidia GPU in Device Manager and it made no difference to the test behavior, compared to just selecting the intel adapter via Vulkan API and optionally disabling external monitors which affect performance.

I re-tested Graphics Monitor attempting to capture a Trace or Stream and then view in Graphics Trace Analyzer. (Frame capture is currently not supported for Vulkan.)

Because the behavior differs, I'm comparing Release and NonRelease builds of my application and Vulkan and DirectX code paths.

Lets start with the good and simple:

App Release build - DirectX 11
    Trace - Good (captures traces with each key press, doesn't terminate process. Captures have screenshot thumbnail.)
    Stream - Good

App Non Release build - DirectX 11
    Trace - Good
    Stream - Good

So DirectX 11 is behaving as expected. Since Intel with DX11 appears to work fine, I don't think there is any point using the nVidia GPU in tests. So now Vulkan...

App Release build - Vulkan
    Trace - Partially functioning
                  Process anomaly*
                  Terminates app on 1st trace capture
                  No screen thumbnail

    Stream - Silent fail (application log terminates abruptly with no errors)

App Non Release build - Vulkan
    Trace - Fails with application logging error from Vulkan debug layer:

Vulkan (Error) : Validation Error: [ VUID-vkCmdResetQueryPool-firstQuery-00797 ] Object 0: handle = 0xa7c5450000000023, type = VK_OBJECT_TYPE_QUERY_POOL; | MessageID = 0xd66ad571 | VkCmdResetQueryPool(): In Query VkQueryPool 0xa7c5450000000023[] the sum of firstQuery (10) + queryCount (2) is greater than the queryPool size (11). The Vulkan spec states: The sum of firstQuery and queryCount must be less than or equal to the number of queries in queryPool (https://vulkan.lunarg.com/doc/view/1.3.236.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdResetQueryPool-firstQuery-00797)

    Stream - Silent fail (application log terminates abruptly with no errors)

*Process anomaly:
The Graphics Monitor detects and switches to the wrong process, leaving the wrong command line in the 'Desktop Applications' item list.
For example, I select my app MyApp.exe but when launched, Graphics Monitor changes the item to OtherProcess.exe which is spawned by my app during startup. (Not as a launcher, but as a sub process.)
If I disable the 2nd process, the trace results look similar, so it appears to affect the Graphics Monitor hosting app more than the trace itself, though I am not certain.

So the NonRelease application build does not capture anything, and, same as previously, logs a Vulkan API error, which does not appear during normal (non instrumented) application execution.

The Release application build does capture something for Trace, but terminates upon first capture attempt.

The resultant capture does not include a screenshot thumbnail, instead shows generic icon:

Neither build will run with Stream capture. The application logs are terminated during startup with no hint why.

Graphics Trace Analyzer
    Now appears to work correctly. Trace loads without error. (Previous version failed with internal error).

I hope this description make sense regarding the current state of Vulkan with Graphics Monitor and Graphics Trace Analyzer.

[Pamela_H_Intel]

DON'T update your driver!! DX11 Driver issue currently: https://community.intel.com/t5/Intel-Graphics-Performance/DX11-Frames-Won-t-Open/m-p/1540074

[Pamela_H_Intel]

I originally thought this was only a Vulkan issue in our back end, but I'm starting to wonder if it's the front end also. So I'm roping in more folks. I'm glad you got back to me with updated info.

Meanwhile, in reading through old messages, I saw that you can get some info in System Analyzer "but want to capture traces and frames". Actually, you can capture traces and frames from System Analyzer.

• If you are in Stream mode in Graphics Monitor when you start System Analyzer, you can capture frames, at will. There's a little frame capture button on the upper left (if it's greyed out, then you are in Trace mode).
• If you are in Trace mode in GM, the little trace capture button is available.

Another cool feature of System Analyzer is that you can change the active metric set on the fly. So if you select metric set Render Basic (that's the default) in GM (under the Metrics tab) and then you start the capture window, then open SA, and then want to change metrics sets, you can choose any of the other available metrics sets. Here are some pics of how to change metrics sets https://community.intel.com/t5/Intel-Graphics-Performance/some-metrics-are-not-available-in-system-analyzer/td-p/1519899

Something to be wary of when using SA is that if your capture window is in the background, then you are not getting any data from it, as it is not rendering to screen . . . so make sure it's at least mostly in the foreground.

One more thing . . . you've used GPA Framework . . . we have a few more features. Check out these quick tip videos I'm made . . . maybe API Debug Layer or Dump State Layer could be useful (if the app is alive long enough).

https://www.intel.com/content/www/us/en/developer/videos/intel-gpa-framework-quick-tips.html - - oh, Dump State Layer video is not ready yet - we publish 1 of these videos/month.

[Pamela_H_Intel]

Greg,

My system loaded the old messages out of order so I noticed something I may have overlooked before . . . when I'd suggested using our cli, GPA Framework, you said you'd found that gpa-injector.exe was on your machine. The cli is our backend so parts are there when you install the front end (the GUI tools) but they may not be there in such a way that you can run them independently. If you have not downloaded GPA Framework, then it will likely not work correctly. Scroll down on the GPA Download page to see the GPA Framework download button.

https://www.intel.com/content/www/us/en/developer/tools/graphics-performance-analyzers/download.html

Pamela