I am looking for clarification as to whether it is possible to assign a specific number of 1G hugepages to a dedicated NUMA node. Our hardware platform has 2 NUMA nodes however our DPDK application runs only on the cores of a single NUMA node. We are finding we need to double our hugepage allocation given the kernel is uniformly distributing the hugepages across the 2 NUMA nodes. We have modified the kernel boot line in grub.conf as follows:
"default_hugepagesz=1G hugepagesz=1G hugepages=8"
My question is similar to that posed in the following link, however I do not see in the response how to change nr_hugepages.
/message/8787?_ga=1.189585131.1372410714.1393984191# 8787 https://embedded.communities.intel.com/message/8787?_ga=1.189585131.1372410714.1393984191# 8787
Any attempts at modifying /sys//devices/system/node/node[0|1]/hugepages/hugepages-1048576kB/nr_hugepages has been unsuccessful.
[root@box]# echo 0 > /sys/devices/system/node/node1/hugepages/hugepages-1048576kB/nr_hugepages
bash: echo: write error: Invalid argument
Any insight as to whether this is possible would be greatly appreciated,
Chapter 6 of http://dpdk.org/doc/intel/dpdk-start-linux-1.7.0.pdf http://dpdk.org/doc/intel/dpdk-start-linux-1.7.0.pdf has Quick Start Setup Script
Can you use 2 Meg instead of 1 G?
Because you will see at the end of 18.104.22.168 note saying with 1 G pages, it is not possible to reserve the hugepage memory after the system has booted.
And Note in section 22.214.171.124 indicates about in general equal reservation - can you see Documentation/Kernel-parameters.txt file in your Linux source tree for further details of these - "In the case of a dual-socket NUMA system, the number of hugepages reserved at boot time is generally divided equally between the two sockets (on the assumption that sufficient memory is present on both sockets).
However, in case you can use 2 Meg page size, in that case, the script can be useful to meet your requirement.
In the script, when you choose the option for "Setup hugepage mappings for NUMA systems" , it will ask you
Input the number of 2 MB pages for each node
then you will be able to reserve separately for node 0 and separately for node 1
Thanks. With 64 bit system, one can use either 2 Meg or 1 Gig.
In case your performance requirement if it can be met with 2 Meg pages, then with 2 Meg pages you can achieve what you are looking for.
Because with 1 Gig, Because you will see at the end of 126.96.36.199 note saying with 1 G pages, it is not possible to reserve the hugepage memory after the system has booted.