- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi all,
I am trying to run DPDK application on x86_64 Machine, On this process trying to do core isolation and assign one application to isolated Core.
While doing Core Isolation, achieved "Interrupt count increment stop on isolated core" in all interrupts except CAL (Function Call Interrupts)
Searched in google, but did not get enough data on CAL
While checking the CAL in the Books: Understanding Linux Kernel & Professional Linux Kernel Archiecture , did not get any info for CAL Interrupts generated.
Can you guys please explain reason to generate CAL and to stop/control for the same in x86_64 (IA64 or AMD64 ISA).
Thanks & Regards
Satish.G
- Tags:
- Intel® Advanced Vector Extensions (Intel® AVX)
- Intel® Streaming SIMD Extensions
- Parallel Computing
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Kumar,
I have the same question.
Moreover, in my case, the Local timer interrupts are still increasing on isolated cores.
How do you stop that?
Thanks & Regards,
Shujie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Even with NOHZ Full mode, you will still get timer interrupts every second.
There is a patch I believe that removed this, but I can't find it atm. Sorry this is not more help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It seems in Kernel 4.19 and 4.20, the residual 1HZ tick can be removed in NOHZ Full mode when running a task on the CPU.
I tried that, but the 1 HZ tick is still there. I am thinking maybe I missed some steps.
Here are what I did:
1. Set CONFIG_NOHZ_FULL=y, CONFIG_RCU_FAST_NO_HZ=y, CONFIG_RCU_NOCB_CPU=y
2. Set boot parameters: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash isolcpus=nohz,domain,1-3 nohz=on nohz_full=1-3 rcu_nocbs=1-3 intel_pstate=disable irqaffinity=0 nmi_watchdog=0 nosoftlockup
3. Modify /sys/devices/virtual/workqueue/cpumask to 1
4. systemctl stop irqbalance
5. Turn off turbo mode, P state, and hythreading in BIOS
Has anybody ever tried this? What did I miss?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That's similar to what we did on 3.10 kernel, but just left the 1Hz timer in there. We left turbo mode on, hyperthreading on, and Pstate to 1 to avoid the idle spin.
In Centos (RedHat) we also had to change the realtime timer scheduling to 100%, and set thread scheduling to FIFO with priority 99.
/proc/sys/kernel/sched_rt_runtime_us to -1
Have you tried executing irqbalance with a mask excluding your NOHZ cores after you stop the service ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
how did you change the realtime timer scheduling to 100%?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Set
/proc/sys/kernel/sched_rt_runtime_us to -1
and in your code, change your thread scheduling like this;
int policy; struct sched_param param; pthread_getschedparam(pthread_self(), &policy, ¶m); param.__sched_priority = 99; pthread_setschedparam(pthread_self(), SCHED_FIFO, ¶m);
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Kumar,
Have you found the solution to reduce the CAL interrupts? I have the same issue now.
Best,
Shujie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This topic needs to be raised :) - the first line of the google search for "Function Call Interrupts" and there is no solution.
I have the same problem - some jitter caused by CAL and I can't find manual how to tune it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Anyone has an updated on preventing CAL interrupts to be fired onto isolated cores ?
Even using a realtime kernel (Ubuntu for example) the CAL interrupts are still called on all cores causing a big (7-10 usec) interruption on every task that is executing on a given core.
![](/skins/images/45C6C2D737ED71F4C51F0145C8CB1E9C/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page