I have run advisor on PVC on an OpenCL GPU app called CloverLeaf, which has many kernels. I am looking at the GPU Roofline Insights view. When I hover over the big, red dot in the GPU Roofline display, it says that it is advec_mom_vol_ocl_kernel, and it is bounded by HBM BW (utilization 21%). I am trying to understand what that means.
It seems to me, that to say the kernel is BW-limited, is to say that the kernel would run faster if the BW increased. But in fact, it is only using 21% of the existing BW. It is well below the HBM roofline.
Looking at the same kernel in Vtune, it says that it is in SBID stall almost half the time, and idle almost half the time. Only about 12% active. This would seem to indicate to me that it is HBM latency bound. That is, it is usually sitting around waiting for data to arrive from HBM memory. Not because of low BW, but because of high latency.
Does "bounded by HBM BW", in this context, really just mean "bounded by HBM"?
