SR-IOV on Xen SLES11 using 82576 Gigabit card

[idata]

I'm trying to start with basic functioning using a 82576 Gigabit card. I've tried searching for documents and nothing seems to be a solid reference for what I need.

I'm using a card that shows on igb load like this:

04:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0c)

05:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0c)

05:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0c)

06:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

06:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

07:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

07:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

I got (from sourceforge) igb-4.1.2.tar.gz

and from this site igbvf-2.0.4.tar.gz

"make install"ed both of these

My system install is suse -- 3.0.13-0.27-xen

I've looked at this a couple of times:

... I get to 4.4, but do not see any SR-IOV Virtual Functions listed

I was tempted by ethernet-x520-suse-linux-xen-tech-brief.pdf. Different hardware seems difficult to map to (my 82576 is not x520)

How can I get some virtual functions listed for me?

Help would be much appreciated. I would be willing to go to a different install, but it seems like a bad idea to go back to Red Hat 5.4 (because it is so old).

[Patrick_K_Intel1]

Thanx for posting to the forum.

It looks to me like you are not loading the driver with the parameters needed to tell it to run in SR-IOV mode. You can do this a couple of ways, one of which is documented in section 4.2 of my old blog you mention: .

The other way is manually when you load the driver:

modprobe igb max_vfs=7

This is of course all assuming you have a platform that supports SR-IOV and it is enabled :-)

Please try this out and let us know how it goes, and have fun with SR-IOV - it is cool stuff!

- Patrick

[idata]

I did try before using:

# modprobe igb max_vfs=7

I gather you are suggesting I must have some problem with my system having SR-IOV enabled.

Thank you for the reply. I'll try to look into that and I will follow up on any other suggestions.

[Patrick_K_Intel1]

Bummer, I was hoping it would be a quick fix.

SR-IOV requires several components, starting with BIOS support. You need to check to see (with your server vendor) if you server and its BIOS support SR-IOV and then figure out how to enable it.

[idata]

Here is what I have and what I see:

The system is a Dell Workstation T7500 with an official Mfg Date of 09/16/2011 ... which I got new probably about a year ago. I was thinking that a system that new would probably have SR-IOV support.

In the BIOS, I have (and had) enabled the two items under "Virtualization support"

The first they call "virtualization" and the second they call "VT for Direct I/O".

I still expect that this system should support what is needed (by age and system class).

Should loading igb automatically load igbvf ... or should I have igbvf loaded first?

Are the listings I have in /etc/modprobe.conf.local besides "options igb max_vfs=7" that important to see the VFs?

( blacklist igbvf

options loop max_loop=128 )

Is there some kind of debug print I can enable in one or both drivers to get at the snag?

Thanks again for your replies.

[Patrick_K_Intel1]

Extremely unlikely that a workstation BIOS would support SR-IOV at this time - maybe in a few years :-)

Currently it is a Server feature only, and it has only been in past year or so that it is more common that not in the server world.

If the platform supported SR-IOV, simply loading the drivers with the max_vfs should likley have made them show up.

So my guess, I'm sorry to say, is that your platform does not support SR-IOV.

- Patrick

[idata]

O.k., that sounds fair.

Can you tell me what intel motherboard(s) I could buy that would have full SR-IOV support? (I'm already trying to contact pre-sales support with that question ... but figured you might be able to help).

Thanks SO MUCH for helping. The bad news is it is not working now, the good news is it looks like I know what to do to get a working platform -- much better than wandering in the dark.

[Patrick_K_Intel1]

Would seem like I would have that information tucked away somewhere, however I don't.

http://ark.intel.com/ http://ark.intel.com/ lists all of our server boards, is a good place to start. I'd recommend going looking at boards that are 2011 and newer. I'll also go and pester some contacts to see if we have a list I can share.