- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a Xeon Phi card running MPSS Gold update 3.
I want to use swap but mic_virtblk will not load into the kernel
message is cannot find mic_virtblk.
Do I need to use MPSS 3.2.3 to use swap?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did you read the article?
I suppose this is a potential hurdle in porting applications which needed a swap partition in the past, but it seems worth while to explore methods to reduce memory and disk usage (although the KNL would seem to change the rules again within 2 years).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Tried to follow the instructions in the article, but as I said in my original post
mic_virtblk.ko will not load into the mic kernel.
is there something else I need to do first that is missing from the article?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also,
This path is not present on my host machine.
/sys/devices/virtual/mic/mic0
up o
/sys/devices/virtual/mic
is present but only contains /ctrl and scif
do I have the correct version of MPSS to set up a swap space? (gold update 3)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Your contents for that directory appear to be normal even for current MPSS.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Unfortunately, your version of MPSS is too old for us to check. (We have no test systems with that old of an MPSS.)
Please update your version. If you still have an issue, let us know and we'll do what we can.
Regards
---
Taylor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Oh, and please wait for a few (business) days before updating your version of MPSS. ;-)
Regards
---
Taylor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Found this in the rel notes for gold update 3
Intel Tracking ID: 4845954 Affected OS: All Linux Description: [Driver] Virtualization is no longer working for coprocessors Notes: Investigating
Is this resolved in the latest version?
Phil Sharrock
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
According to the bug tracking system, this issue is resolved and closed as of this release.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Loaded MPSS 3.2.3.
Still will not load mic_virtblk.ko into kernel, same error as before.
Also when compiling on Phi card the linux command 'ar' (via busybox) does not accept
the -r option, this option is needed when adding items to archives.
Phil Sharrock
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sorry, I should have said closed as of the MPSS 3.3 release. And I guess I need to update that blog. The complete directions for using a virtio block device are now in the MPSS User's Guide, rather than the readme and some of the directory locations appear to have changed. Sigh.
As far as ar not having a replace option, I can see how that would be painful. I'll look into this.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Even following the instructions in the MPSS users guide, modprobe mic_virtblk fails to load the driver
I get the same error message file not found/not present
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Uh, oh. I think I know what is wrong. With 3.2 and continuing into 3.3, a number of files which are considered optional on the coprocessor have to moved to rpm files that must be explicitly added if you want to use them. I found mic_virtblk in mpss-modules-3.3-1.knightscorner.rpm. Under MPSS 3.3, to load these files you need to follow the instructions in section 11.3 in the MPSS User's Guide.
When I use these extra rpm files on the coprocessor, I prefer to follow section 11.3.3, making a directory containing only the rpm files I want uploaded to the coprocessor and including in these the files needed to run zypper on the coprocessor (see section 11.3). That way I can specify the directory name rather than the individual filenames on the micctrl command and not need to rerun the command each time I add or remove files from the directory. If, after I reboot, I log onto the coprocessor and find a directory called /RPMs-to-install, it means the rpm files did not successfully install. In this case, I install the files needed to run zypper and use zypper to check for any unsatisfied dependencies or errors, then fix those.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Tried to install the suggested rpm file.
Got the message already installed
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Swap on MIC works for me in MPSS 3.3. I only installed the RPMs inside of "mpss-3.3/" and the kernel module. The difference in my procedure versus Frances' article are:
- The path to the virtblk_file is different in MPSS 3.3
- I can format the drive from the host
- If you are changing virtblk_file, you should unload and reload module mic on the host
- There is no need to modprobe mic_virtblk, it loads automatically if virtblk_file existed when MPSS booted
- I used an LVM volume instead of a file on the host
Here is the sequence of commands that I used:
[root@c001-n002 ~]# service mpss unload Shutting down Intel(R) MPSS: [ OK ] Removing MIC Module: [ OK ] [root@c001-n002 ~]# modprobe mic [root@c001-n002 ~]# mkswap /dev/mapper/vg_phi-lvol0 # Formatting a logical volume for swap mkswap: /dev/mapper/vg_phi-lvol0: warning: don't erase bootbits sectors on whole disk. Use -f to force. Setting up swapspace version 1, size = 31457276 KiB no label, UUID=6b0609cc-4b8b-4bc4-8e35-7a0f52260ffa [root@c001-n002 ~]# echo /dev/mapper/vg_phi-lvol0 > /sys/class/mic/mic0/virtblk_file [root@c001-n002 ~]# service mpss start Starting Intel(R) MPSS: [ OK ] mic0: online (mode: linux image: /usr/share/mpss/boot/bzImage-knightscorner) mic1: online (mode: linux image: /usr/share/mpss/boot/bzImage-knightscorner) mic2: online (mode: linux image: /usr/share/mpss/boot/bzImage-knightscorner) mic3: online (mode: linux image: /usr/share/mpss/boot/bzImage-knightscorner) [root@c001-n002 ~]# ssh mic0 [root@c001-n002-mic0 ~]# swapon /dev/vda [root@c001-n002-mic0 ~]# cat /proc/swaps Filename Type Size Used Priority /dev/vda partition 31457276 0 -1 [root@c001-n002-mic0 ~]#
See also our article on file I/O on Xeon Phi for performance benchmarks: http://research.colfaxinternational.com/post/2014/07/28/io.aspx
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Quick update: I have just tested using a file instead of a logical volume on host. That configuration also worked for me.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Phil,
Any update? Did you get it to work?
Regards
--
Taylor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes that worked. swap now available.
now just got to stop the Phi card locking up when most of memory used.
no communication with card after a few minutes.
Phil
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, i'm trying to use virtblk with mpss 3.51 but
- no luck using the user guide method
- no luck using Andrey's method above
errors:
/dev/vda - - - No such file or directory
and when doing [root@mic0 ~]# modprobe mic_virtblk
FATAL: Error inserting mic_virtblk (/lib/modules/2.6.38.8+mpss3.5.1/extra/virtio/mic_virtblk.ko): No such device or address
Is there an updated procedure ? Many thanks in advance !
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
just tried it on my mpss 3.5.1 install and it "just works".
Did you install the mpss-modules package? this RPM includes the module mic_virtblk.ko:
[root@mic0 ~]# rpm -qf /lib/modules/2.6.38.8+mpss3.5.1/extra/virtio/mic_virtblk.ko mpss-modules-3.5.1-1.knightscorner
Also, make sure to initialize the swap device prior to using it:
[root@mic0 ~]# swapon /dev/vda swapon: /dev/vda: Invalid argument [root@mic0 ~]# mkswap /dev/vda Setting up swapspace version 1, size = 10485755904 bytes [root@mic0 ~]# swapon /dev/vda [root@mic0 ~]# cat /proc/swaps Filename Type Size Used Priority /dev/vda partition 10239996 0 -1
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I just double checked on a couple systems that I use and both had /lib/modules/2.6.38.8+mpss<version#>/extra/virtio/mic_virtio/mic_virtblk.ko installed by default. When the mic_virtblk kernel module loads, it creates /dev/vda. What modprobe is complaining about is not that it cannot find the kernel module but that it cannot find the device on the host that is providing the actual swap space. Without that space the module can't load.
So first check on the host to be sure the device you are using exists - /dev/<disk_name>, /dev/<disk_partition_name>, /dev/mapper/<logical_volume_name> or the file name , if you are using a file instead of an actual device.
Make sure /sys/class/mic/mic0/virtblk_file on the host exists and that the content of that file is the full path name to the device you are using.
If the file /sys/class/mic/mic0/virblk_file did not exist when the coprocessor was booted, stop and restart the mpss.
If all that checks out, mic_virtblk.ko really should load.

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page