- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I want to test the relationship between batch size and parallel computing on GPU. I did the following experiment
device: GPU(Intel UHD Graphics 630)
model input shape: (1,3,8,8)
input image shape:(8,8)
my command is like this:
python ./benchmark_app.py -m xxx_8x8_fp16_b1.xml -i 0801_8x8.png -d GPU -api sync -niter 1000 -b batchsize
we need to focus on latency so I set the -api synchronize mode
and only thing I need to do is to change the batch size, the test result is in the following chart
batchsize |
8 |
16 |
32 |
64 |
128 |
256 |
512 |
1024 |
1200 |
1280 |
1300 |
1500 |
1536 |
2000 |
2048 |
3000 |
3500 |
3700 |
3850 |
3900 |
4000 |
4096 |
Inputshape |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
8x8 |
GPU usage |
50% |
53.5% |
59% |
68% |
75% |
83% |
89% |
95% |
94% |
97% |
94% |
95% |
98% |
97% |
98% |
97.5 |
98% |
99% |
99% |
98% |
100% |
100% |
latency |
0.35 |
0.39 |
0.51 |
0.66 |
0.86 |
1.38 |
2.57 |
6.54 |
5.7 |
9.94 |
5.96 |
6.80 |
13.19 |
9.56 |
19.19 |
14.23 |
17.41 |
17.76 |
18.68 |
18.61 |
41.57 |
42.41 |
and look at the bold numbers. It's strange that the result doesn't show the regularity of the latency increases with the increase of batch size. and specifically, if the batch size is in a particular size, the GPU usage is slightly reduced and the latency reduced a lot
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Shirley,
We noticed that this is a duplicated issue from the following thread. If you need any additional information from Intel, please submit a new question since this thread is no longer being monitored.
Sincerely,
Peh
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Shirley,
We noticed that this is a duplicated issue from the following thread. If you need any additional information from Intel, please submit a new question since this thread is no longer being monitored.
Sincerely,
Peh
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page