MIC_USE_2MB_BUFFERS after MPSS Gold Update 1

[Andrey_Vladimirov]

According to this document: http://software.intel.com/en-us/articles/optimizing-memory-bandwidth-on-stream-triad , "This update [KNC_gold_update_1-2.1.4982-15] has the “Transparent Huge pages” support which automatically promotes 4K pages to 2MB pages for stack and heap allocated data".

Does it mean that the programmer needs not worry about 2MB pages anymore? How does MPSS decide when to promote 4K pages to 2MB pages? Does the environment variable MIC_USE_2MB_BUFFERS still have an effect?

[Kevin_D_Intel]

I believe it works equivalent to under Linux on the host and that native apps benefit transparently but I’m not knowledgeable on the internal MPSS details myself.

The MIC_USE_2MB_BUFFERS is still supported and applicable for offload apps. That influences memory allocation for pointer variables transferred to the card under the offload models. The use is discussed further here.

[TimP]

As there are only 8 2MB pages available, "transparent huge pages, "  allowing the MPSS to use both page sizes, seems the practical choice for large applications.   Explicit switch to 2MB pages is recommended for certain benchmarks. 

This discussion http://lwn.net/Articles/423584/ gives the general picture about how it magically allocates useful huge pages without starving the application as to number of available pages.  You are probably getting some benefit from thp on the host side as well, where an attempt to use purely huge pages usually incurs a great increase in swapping activity and could not be profitable except with the use of fast storage such as SSD.

There is no connection between the use of huge pages inside the application and MIC_USE_2MB_BUFFERS for the offload data transfers.  I think efforts have been made to make the latter less critical, but it is still recommended if the transferred data sets exceed the theshold size you will set (100kB in the doc referred to above).

[Andrey_Vladimirov]

Thank you, Kevin and Tim!