topic MPSS 3.2 includes everything in Software Archive

Rebuilding MPSS-3.2 (Yocto) release from source

René_O_ — Tue, 08 Apr 2014 09:21:41 GMT

Dear community,

I think I have managed to go some steps forward to rebuild the Intel Xeon Phi MPSS-3.2 release (with Yocto, poky, bitbake ...) to add some required features in our installation. Now I'm stuck at the following with missing source code to rebuild the minimal mpss image:

$ bitbake mpss-image-minimal
WARNING: Host distribution "CentOS release 6.5 (Final)" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |########################################################################################################################################################################################################| ETA: 00:00:00
Loaded 1229 entries from dependency cache.

OE Build Configuration:
BB_VERSION = "1.15.1"
TARGET_ARCH = "k1om"
TARGET_OS = "linux"
MACHINE = "knightscorner"
DISTRO = "mpss"
DISTRO_VERSION = "3.2"
TUNE_FEATURES = "m64"
TARGET_FPU = ""
meta 
meta-yocto 
meta-mpss = "<unknown>:<unknown>"

NOTE: Resolving any missing task queue dependencies
ERROR: Nothing RPROVIDES 'mpss-psm' (but /tmp/poky/meta-mpss/recipes-mpss/tasks/mpss-ofed.bb RDEPENDS on or otherwise requires it)
NOTE: Runtime target 'mpss-psm' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['mpss-psm']
NOTE: Runtime target 'mpss-ofed' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['mpss-ofed', 'mpss-psm']
ERROR: Required build target 'mpss-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['mpss-image-minimal', 'mpss-ofed', 'mpss-psm']

Summary: There was 1 WARNING message shown.
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

The problem - I think - is the missing 'mpss-psm' recipe and source code in the released MPSS-3.2 source code packages.

Unfortunately, the documentation for system administration lacks behind the release of Yocto-based MPSS for months now (http://software.intel.com/en-us/articles/system-administration-for-the-intel-xeon-phi-coprocessor). Other technical information in the topic of the Yocto-based MPSS are still very rare, too. Documentation from other sources like success stories would be very nice.

Any help is appreciated. Creating something like a special support ticket for a required feature is not the way to go.

Just so you don't think we

Frances_R_Intel — Wed, 09 Apr 2014 16:18:05 GMT

Just so you don't think we have abandoned you, this question has been passed on to a more Yocto savvy person who will be getting back to you.

MPSS 3.2 includes everything

Evan_P_Intel — Fri, 11 Apr 2014 17:43:00 GMT

MPSS 3.2 includes everything necessary to re-build the card's opensource software stack, but some assembly is required--its completely undocumented state justified bug priority decisions which resulted in a few things not being fixed prior to the 3.2 release.

In our internal uses of Yocto, we have an additional, closed-source layer which we use to build a handful of proprietary components, mpss-psm among them; the bugs affecting you are generally erroneous references to closed-layer stuff. That's easy to work around, though--just delete the references.

Now, that approach does mean that the mpss-image-minimal you'll build won't have mpss-psm installed. However, you can manually edit the image afterward to install the binary RPM built by Intel. (Future releases of MPSS may simply require that as part of the OFED installation procedure.)

You're obviously past the first three steps, but I'll include them for completeness and so that you can see the assumptions my later script fragments make. Before proceeding, I recommend deleting your poky/build directory (which was created by sourcing oe-init-build-env)--having dirty work trees from before these kind of configuration changes may cause hard-to-diagnose build failures.

First, extract the source and apply MPSS's patches to Poky:

MPSS_TOP=`pwd`/build

tar -xf mpss-src-3.*.tar

mkdir -p $MPSS_TOP/sources
tar -xf mpss-downloadcache-3.*.tar -C $MPSS_TOP
tar -xf mpss-3.*/src/poky-???????.tar.bz2 -C $MPSS_TOP
tar -xf mpss-3.*/src/poky-patches-3.*.tar.bz2 -C $MPSS_TOP
tar -xf mpss-3.*/src/meta-mpss-3.*.tar.bz2 -C $MPSS_TOP
mv mpss-3.*/src/*.tar.bz2 $MPSS_TOP/sources

cd $MPSS_TOP/poky
for p in $MPSS_TOP/poky-patches/*.patch; do git apply $p; done

It's necessary to use "git apply" instead of merely "patch" because one of those patches creates a symlink at $MPSS_TOP/build/poky/meta/site/k1om-linux, which requires one of git's extensions to the patch file format. If you want, you can use "patch" provided you manually fix up the symlink after.

Second, initialize the Poky build environment:

. ./oe-init-build-env

Third, change Yocto's configuration as necessary to build MPSS. I use the following script, which edits the sample config files produced by sourcing oe-init-build-env:

icc_path=/path/to/where/you/installed/composer
icc_license=/path/to/composer/license/directory

numthreads=`grep ^processor /proc/cpuinfo | wc -l`
nummakejobs=$numthreads
sed -i.old conf/local.conf \
        -e '/^#BB_NUMBER_THREADS/ a BB_NUMBER_THREADS = "'$numthreads'"' \
        -e '/^#PARALLEL_MAKE/ a PARALLEL_MAKE = "-j '$nummakejobs'"' \
        -e '/^MACHINE/ a MACHINE = "knightscorner"' \
        -e '/^# DISTRO/ a DISTRO = "mpss"' \
        -e 's/^PACKAGE_CLASSES/#&/' \
        -e 's/^EXTRA_IMAGE_FEATURES/#&/' \
        -e 's/^USER_CLASSES/#&/' \
        -e '/^# CONF_VERSION/ i LICENSE_FLAGS_WHITELIST = "mpss intelsamplecode"' \
        -e '/^# CONF_VERSION/ i ICC_VERSION = "local"' \
        -e '/^# CONF_VERSION/ i ICC_INSTALL_PATH = "'"$icc_path"'"' \
        -e '/^# CONF_VERSION/ i INTEL_LICENSE_FILE = "'"$icc_license"'"' \
        -e '/^# CONF_VERSION/ i PREMIRRORS = ".*://.*/.* file://'"$MPSS_TOP"'/download-cache/ \\n"' \
        -e '/^# CONF_VERSION/ i MPSS_MIRROR = "file://'"$MPSS_TOP"'/sources"' \
        -e '/^# CONF_VERSION/ i BB_NO_NETWORK = "1"\n'

sed -i.old conf/bblayers.conf \
        -e 's|^\(\s*\)"$|\1'"$MPSS_TOP/meta-mpss"' \\\n&|'

Having a copy of Intel Composer XE is required to build MPSS; the most tested version is probably 2013 SP1. I recommend you "diff -u local.conf.old local.conf" to get a better feeling for how the above changes the sample configuration.

Fourth, paper over the aforementioned bugs in the meta-mpss layer:

sed -i $MPSS_TOP/meta-mpss/recipes-mpss/tasks/mpss-ofed.bb -e '/mpss-psm/ d'
sed -i $MPSS_TOP/meta-mpss/recipes-mpss/meta/meta-mpss.bb -e '/mpss-psm/ d'
sed -i $MPSS_TOP/meta-mpss/recipes-mpss/meta/meta-mpss.bb -e '/mpss-license/ d'
sed -i $MPSS_TOP/meta-mpss/recipes-mpss/meta/meta-mpss.bb -e '/mpss-sciftutorials/ d'
sed -i $MPSS_TOP/meta-mpss/recipes-mpss/micmgmt/mpss-micmgmt_mpss.bb \
        -e 's/ mpss-flash mpss-rasmm-kernel//'

In summary, disable building mpss-psm (which contains an OFED component), mpss-license (which installs a copy of the MPSS license text), and mpss-sciftutorials (which contains the SCIF tutorials). Then remove the install-time dependency on mpss-flash and mpss-rasmm-kernel (which contain proprietary firmware blobs) from mpss-micmgmt.

mpss-psm is proprietary closed-source
mpss-license and mpss-flash/mpss-rasmm-kernel don't work right when the closed layer isn't present
mpss-sciftutorials is...not even broken. No idea what happened there.

Fifth, disable the nativepkg variant--this ought to be the default, and I expect it will be in MPSS 3.3:

sed -i conf/local.conf \
        -e '/^# CONF_VERSION/ i NATIVEPKGS_BUILD_kernel = ""' \
        -e '/^# CONF_VERSION/ i NATIVEPKGS_BUILD_userland = ""'

We use nativepkg variants in our automation of host RPM production within our internal build infrastructure, so you probably don't care about it, and it's somewhat brittle.

At this point you should be able to proceed without any errors.

On the 24GB, 32-logical-core (1 socket, 16 cores * 2 hyperthreads each) machine I typically build with, "bitbake meta-mpss" takes about two hours from a pristine tree. The build process has a lot of bits that don't parallelize well, though.

(EDIT: Made it clear I'm recommending you delete poky/build before following the above procedure, not after.)

Was my response helpful?

Evan_P_Intel — Thu, 17 Apr 2014 19:46:44 GMT

Was my response helpful?

I should have also mentioned that, depending on the exact nature of your required feature, I might be able to suggest a simple way to add it without requiring you to rebuild the minimal image. Can you provide some details?

I was building kernel images

Kaicheng_Z_ — Mon, 21 Apr 2014 18:51:12 GMT

I was building kernel images for mic too. And Evan's post helped a lot.

The only problem is that there are some command used "--same-order" option with "-c" in command tar. I greped all "tar.*\\-ps" and changed those "-ps" to "-p", and everything ran smoothly.

It took me a lot of time to figure out how to configure a yocto build environment for mic before I saw this post. I think this post or sth. similar should has a link at mpss's download page.

thank you for the suggestion

BelindaLiviero — Mon, 21 Apr 2014 18:53:25 GMT

thank you for the suggestion on documentation - we will work on that

Dear Evan,

René_O_ — Wed, 23 Apr 2014 13:06:48 GMT

Dear Evan,

sorry for the late answer.

I can confirm, that your documentation works out of the box with minor tweaks:

1) The line

-e '/^# CONF_VERSION/ i PREMIRRORS = ".*://.*/.* file://'"$MPSS_TOP"'/download-cache/ \\n"' \

should be

-e '/^# CONF_VERSION/ i PREMIRRORS = ".*://.*/.* file://'"$MPSS_TOP"'/download-cache/"' \

2) The line

icc_path=/path/to/where/you/installed/composer

should be clarified with an example to (the composer base path)

icc_path=/opt/intel/composer_xe_2013_sp1.2.144

where bin/intel64/icc is appended. So a more general directory like /opt/intel/composerxe/ wouldn't work, because the bin/intel64 directory is not available there.

At the moment, I have a new 'problem': After building with bitbake meta-mpss, the package mpss-boot-files-3.2.1-1.glibc2.12.2.x86_64.rpm is not available, but after bitbake mpss-boot-files the RPM ./tmp/deploy/rpm/k1om/mpss-boot-files-3.2.1-1.k1om.rpm is ready. Why is there a difference in the package name, has it the same content and why is it not build with meta-mpss?

To answer your second question:

We would like to use domain-search in the MIC OS udhcpc DHCP client provided by the busybox executable (udhcpc -O search). In MPSS-2.x, busybox was compiled with the appropriate define/configuration ENABLE_FEATURE_UDHCP_RFC3397. This feature helps use to get a more flexible and dynamic resolv.conf configuration through DHCP in a subdomain based style.

Many thanks in advance!

Quote:René O. wrote:

Evan_P_Intel — Tue, 29 Apr 2014 00:37:10 GMT

René O. wrote:

After building with bitbake meta-mpss, the package mpss-boot-files-3.2.1-1.glibc2.12.2.x86_64.rpm is not available, but after bitbake mpss-boot-files the RPM ./tmp/deploy/rpm/k1om/mpss-boot-files-3.2.1-1.k1om.rpm is ready. Why is there a difference in the package name, has it the same content and why is it not build with meta-mpss?

Sorry, I should have covered this in my first response.

The short answer is that "bitbake mpss-boot-files" builds an RPM for $TARGET_ARCH==k1om, while what you're looking for is one built for $BUILD_ARCH==x86_64. That is accomplished with "bitbake nativepkg-mpss-boot-files".

The long answer is somewhat involved. First, recall that, by convention, "bitbake foo" almost always compiles "foo" for $MACHINE; if it's possible to compile "foo" for a different target (without changing $MACHINE, I mean), that's almost always indicated with a prefix or suffix that identifies the variant to be used—e.g. "foo-nativesdk" to build for $SDKMACHINE or "foo-native" to build for $BUILD_ARCH. Second, note that the meta-mpss layer includes a .bbclass that introduces a new "nativepkg" variant; this variant behaves similarly to the "native" variant except that it produces an RPM package ("native" does not), and it is this variant which is used to enable building mpss-boot-files for a different target (namely $BUILD_ARCH). As for why it's "nativepkg-mpss-boot-files" as opposed to "mpss-boot-files-nativepkg"—that's an artifact of the implementation; nativepkg.bbclass uses the same mechanism as multilib.bbclass, and in Yocto 1.2 such classes are identified with a prefix instead of a suffix. (More recent versions of Yocto use prefixes for everything, not just multilib-style variants.)

The reason it's not built when building meta-mpss is simpler; it's because that was disabled:

Evan Powers (Intel) wrote:

Fifth, disable the nativepkg variant—this ought to be the default, and I expect it will be in MPSS 3.3:
sed -i conf/local.conf \
        -e '/^# CONF_VERSION/ i NATIVEPKGS_BUILD_kernel = ""' \
        -e '/^# CONF_VERSION/ i NATIVEPKGS_BUILD_userland = ""'
We use nativepkg variants in our automation of host RPM production within our internal build infrastructure, so you probably don't care about it, and it's somewhat brittle.

Setting those two variables to the empty string effectively removes all nativepkg variants from meta-mpss's DEPENDS so they won't build automatically when building meta-mpss. (If you want to see the mechanics of how that happens, take a look at meta-mpss/recipes-mpss/meta/meta-mpss.{inc,bb}.) It's still possible to build them manually, however, and in the case of nativepkg-mpss-boot-files, there's little risk of it failing to work.

By contrast, the other nativepkg variants which would have otherwise been enabled are quite likely to break, for a variety of reasons. For some, it's because they require unusual things to be installed on the build host (e.g. the JRE); for others, it's because they haven't been audited for dependencies on our internal build environment.

For completeness, I should mention why the above doesn't affect mpss-sdk-k1om-*.x86_64.rpm or intel-composerxe-compat-k1om-*.x86_64.rpm, which are still built automatically when building meta-mpss. Disabling building nativepkg variants by default has no effect on them because they inherit from cross-canadian.bbclass instead of nativepkg.bbclass in order to access variables which would not otherwise be in scope. (Using cross-canadian.bbclass has its own caveats; of the RPMs in build/tmp/deploy/rpm/x86_64-nativesdk, only those two are safe to install on a host system without risk of confusing its RPM dependency solver.)

René O. wrote:

We would like to use domain-search in the MIC OS udhcpc DHCP client provided by the busybox executable (udhcpc -O search). In MPSS-2.x, busybox was compiled with the appropriate define/configuration ENABLE_FEATURE_UDHCP_RFC3397. This feature helps use to get a more flexible and dynamic resolv.conf configuration through DHCP in a subdomain based style.

Ah. For that purpose, "bitbake nativepkg-mpss-boot-files" probably is the simplest way to proceed, as you've no doubt decided already. If you had responded with something that could be accomplished by editing a run-time configuration file, or if you wanted to integrate a new software package which you had cross-compiled using the SDK (i.e. without integrating it into Yocto as a new bitbake recipe), I might have suggested approaches based on manipulating the Intel-built initramfs cpio archive....