offload large overhead

[King_Crimson]

Dear forum,

I have come across some problems when using offload pragma.

Here is the pseudo-code.

AllocateMemMIC(data); // nocopy(data:length(numElement) ALLOC RETAIN
AllocateMemMIC(result);
for(int i = 0; i < 4; ++i)
{
    UpdateData(data); // change data content
    
    MemcpyHostToMIC(data);

    OffloadCompute(data, result); // in(data:length(0) REUSE RETAIN), in(result:length(0) REUSE RETAIN)

    MemcpyMICToHost(result);

    UpdateResult(result); // accumulate result
}
FreeMemMIC(data);
FreeMemMIC(result);

I used OFFLOAD_REPORT for the timing information, and OFFLOAD_INIT=on_start to initialize the device earlier than main(). The strange thing is:

1. AllocateMemMIC() appears extremely slow. 2 GB memory would take ~7 seconds to allocate.

2. OffloadCompute() in loop No. 0 appears extremely slow, taking ~14 seconds, while the same call in the subsequent loop No. 1, 2, 3 only takes ~0.8 seconds.

Could anyone give me some hints? Many thanks.

BTW:

• MIC model --- 5110p
• compiler --- icpc 15.0.3
• mpss --- 3.5.1

[jimdempseyatthecove · Accepted solution]

The Linux process in the MIC, upon start, has a heap with addresses assigned but not mapped to physical RAM. Other than for the allocated node header (which the heap manager touches), the mapping does not occur on the allocation. Rather it occurs during runtime at the first time you touch the memory (page granularity). Each "first touch" to a page causes a page fault to the Linux O/S, which then maps the address to RAM (it may wipe the RAM too), then returns to the app. This repeats for each page in the allocation.

Jim Dempsey

[King_Crimson]

Lots of thanks for your insight. That explains why using large page size helps.

[jimdempseyatthecove]

>>That explains why using large page size helps.

Sometimes it doesn't. The cache system not only has a capacity in KB (multiples of cache line size) but also has an additional restriction on the number of different pages (held in TLB). Using Large Page Size may reduce the number of different pages that can be mapped (at each cache level). Therefore, while this may speed up "first touch" initialization, it may also slow down the application later on. Each application may have different page size requirements. You can find this out with testing.

Jim Dempsey

[TimP]

If each thread is initializing its own data page, the sequential effect on thread ramp up may limit effective parallel scaling.

Transparent huge pages may succeed in automatically finding much of the beneficial use of huge pages.

Mic has hardware provision for medium pages but somehow they didn't work out.