Software Archive
Read-only legacy content
17061 Discussions

Enabling swap on mic with mpss-3.6 on centos7

Nirmal_P_
Beginner
490 Views

Trying to enable mic_virtblk on the host - the card always reports

[    7.481039] Module mic_virtblk loaded at 0xffffffffa007d000
[    7.488805] Have you set virtblk file?

Debugging the source of mpssd which sets up the virtio_device - the init_mic does not happen - the module returns false for is_upstream_driver - the path /sys/class/mic/scif exists on the system.

open("/sys/class/mic/scif", O_RDONLY)   = 9
close(9)                                = 0

if (is_upstream_driver())
                        pthread_create(&mpssdi->state_pth, NULL, init_mic, mic);
                else
                        pthread_create(&mpssdi->state_pth, NULL, mic_state, mic);

Looking at upstream_knc_driver - it seems to look at some path from /dev/vop_virtio which is a module in 4.x -

Can you please kindly confirm if mpss 3.6 supports this?  Which module handles passing the virtblk_file information in 3.x - is the source included to debug this?

I don't see anything on this in 3.6.1 release notes and 4.x modules don't look to be available at this time - does not look like mpssd on this system can initialize this.

Thanks in advance

 

0 Kudos
11 Replies
JJK
New Contributor III
490 Views

did you set the virtblk_file prior to booting up the Phi:

 echo /full/path/to/dummy/file > /sys/class/mic/mic0/virtblk_file

 

?  this works for me on CentOS 6 with mpss 3.6 ; I'm currently trying to set it up so that I can use a persistent root FS using this, including a swap partition. Once I've got it fool-proof I'll post a wiki on it. I'm not using CentOS/EL 7, of course.

 

0 Kudos
Nirmal_P_
Beginner
490 Views

Yes tried it and many different variations such as:

service mpss unload

<set up swap as lvm partition or a file> 

modprobe mic

echo <path> > /sys/class/mic/mic0/virtblk_file

service mpss start

and other variations that are posted online.  Could not get any of them to work.  That's when I took to debug into mpssd.  The included source does not look to do anything other than watch for status of the cards.  The cards are working fine otherwise (2 in the system). Will try to look up the mic.ko module.  Does the card expect to communicate with mpssd and get this in a structure?

0 Kudos
JJK
New Contributor III
490 Views

mpss does not do anything with the virtblk stuff - that's handled by the mic.ko driver on the host side (which provides the /sys/class/mic/mic0 dir struct) and by the Phi-side virtblk driver.

what are the permissions on the <path> file that you provide? also, check that the virtblk file is actually picked up by the mic.ko file by doing a

cat  /sys/class/mic/mic0/virtblk_file

 

Finally, make sure you're trying this on/with mic0 if you're specifying the above virtblk_file - you can set one per mic ;)

 

0 Kudos
Nirmal_P_
Beginner
490 Views

File is picked up correctly when I cat that path.  Definitely looking on mic0.

Are there specific permissions I need to set?

Presently everything is owned by root with rw.

The host log output does show a message that file is being changed - probably being output from mic.ko so it does not complain about the path. Anything else to check?

0 Kudos
JJK
New Contributor III
490 Views

err, I can't think of anything else to check. is SELinux enabled? can you disable it or set it to permissive?

 

0 Kudos
Nirmal_P_
Beginner
490 Views

Thanks for the pointers - it did not fix things.  I believe the problem may be in the mic.ko module - specifically [vhost/mic_blk.c] - centos 7 that I have is 3.10.0-229

#if (LINUX_VERSION_CODE >= KERNEL_VERSION(3,10,0))
        stat.size = (loff_t)0;  // CAZ TODO this is temportary until you fix this code.
        stat.mode = (umode_t)0;
#else
        ret = vfs_getattr(vblk->virtblk_file->f_path.mnt,
                                          vblk->virtblk_file->f_path.dentry, &stat);
#endif

The filesize is reported as 0 throwing that error on the mic.

 

0 Kudos
Nirmal_P_
Beginner
490 Views

That was the problem - the following fix takes care. Technically this call change is from kernel >= 3.9 [not 3.10].

438a439,441
> #if (LINUX_VERSION_CODE >= KERNEL_VERSION(3,10,0))
>     struct path p;
> #endif
481,482c484,486
<     stat.size = (loff_t)0;  // CAZ TODO this is temportary until you fix this code.
<     stat.mode = (umode_t)0;
---
>      p.mnt = vblk->virtblk_file->f_path.mnt;
>     p.dentry = vblk->virtblk_file->f_path.dentry;
>     ret = vfs_getattr(&p, &stat);   

0 Kudos
Garland_S_
Beginner
490 Views

I am having exactly the same problem (with mpss-3.6 on centos 7) but I do not understand the fix.  Does this require a change to the mic_virtblk module?

0 Kudos
Nirmal_P_
Beginner
490 Views

The kernel version is higher on centos 7 - the vfs_getattr API changed and hence it would pass the size as 0 causing the mic not to accept the file as a valid file.

Just apply the diff and rebuild mic.ko module and copy to the modules path on the host and reload the module - that should take care of the problem.

0 Kudos
Garland_S_
Beginner
490 Views

This worked, thanks. 

0 Kudos
Lazaro_S_
Beginner
490 Views

What exactly is the solution to this problem? I am having the same trouble with centos 7, but I don not know how to apply the change in code and to what file.

0 Kudos
Reply