Need "wrapper" to compile the Linux kernel with ICC

[TyMac]

I am a developer with an idea to start a new kernel source for Gentoo that is specifically for the Intel ICC compiler. Evidentally there is a "wrapper" that is needed to be applied to the kernel source first before ICC can correctly complie the Linux kernel. Where can I get this wrapper? (I have looked all over the web but the only yhing I can find is old documentation)

[TyMac]

I was talking with Daneil Robbins, creator of Gentoo Linux, and he said that we should try dailing down the ICC optimizations.

How do we do that when compiling the kernel?

[Feilong_H_Intel]

Quoting - thaidog

Confirmed. It appears that the kernel builds successfully with this modification. But I get stuck loading the kernel at a similar place:

root (hd0,0)

filesystem type is ext2fs, partition type 0x83

kernel /boot/kernel-2.6.25-intel-r7 root=/dev/hdb2

[Linux-bzImage, setup=0x3c00, size=0x2a5b64]

also, at the end of the make I never saw any output about making the bzImage

I'm afraid that it's hard to debug this runtime problem "remotely". Could you please post the first few lines (like above) from gcc kernel? Maybe we may find some obvious problem.

[newper]

Quoting - Feilong H (Intel)

Quoting - thaidog

Confirmed. It appears that the kernel builds successfully with this modification. But I get stuck loading the kernel at a similar place:

root (hd0,0)

filesystem type is ext2fs, partition type 0x83

kernel /boot/kernel-2.6.25-intel-r7 root=/dev/hdb2

[Linux-bzImage, setup=0x3c00, size=0x2a5b64]

also, at the end of the make I never saw any output about making the bzImage

I'm afraid that it's hard to debug this runtime problem "remotely". Could you please post the first few lines (like above) from gcc kernel? Maybe we may find some obvious problem.

gcc: [Linux-bzImage, setup=0x2a00, size=0x12c890]
icc: [Linux-bzImage, setup=0x3a00, size=0x163e90]

These two kernels are compiled with the same config file,the kernel version is 2.6.28,the gcc one works well,but the icc one can't boot.

[TyMac]

Quoting - newper

gcc: [Linux-bzImage, setup=0x2a00, size=0x12c890]
icc: [Linux-bzImage, setup=0x3a00, size=0x163e90]

These two kernels are compiled with the same config file,the kernel version is 2.6.28,the gcc one works well,but the icc one can't boot.

Thanks, and how do you get that info? I've been looking for a way to get that info.

[newper]

Quoting - thaidog

Quoting - newper

gcc: [Linux-bzImage, setup=0x2a00, size=0x12c890]
icc: [Linux-bzImage, setup=0x3a00, size=0x163e90]

These two kernels are compiled with the same config file,the kernel version is 2.6.28,the gcc one works well,but the icc one can't boot.

Thanks, and how do you get that info? I've been looking for a way to get that info.

I only use grub, like
grub
root (hd0,0)
kernel /boot/bzImage

[TyMac]

Quoting - newper

Quoting - thaidog

Quoting - newper

gcc: [Linux-bzImage, setup=0x2a00, size=0x12c890]
icc: [Linux-bzImage, setup=0x3a00, size=0x163e90]

These two kernels are compiled with the same config file,the kernel version is 2.6.28,the gcc one works well,but the icc one can't boot.

Thanks, and how do you get that info? I've been looking for a way to get that info.

I only use grub, like
grub
root (hd0,0)
kernel /boot/bzImage

I could never see it... it scrolled by to fast!

[TyMac]

Quoting - thaidog

I could never see it... it scrolled by to fast!

Well anyone on ideas as to what is causing this error?

[newper]

Quoting - thaidog

Quoting - thaidog

I could never see it... it scrolled by to fast!

Well anyone on ideas as to what is causing this error?

I find a way,it's very bad,but it works well on my machine.
I noticed that we changed mca.c for a little before,and I also noticed that the 2.6.18 kernel could be compiled well by the icc.But the 2.6.18 one doesn't have mca.c in that directory.I findout that the kernel doesn't contain mca.c in arch/i386/boot until 2.6.23.So I compiled the kernel using the following steps:
I download 2.6.22.19 version (the last stable version of 2.6.22 branch) and 2.6.28.2 (current stable version)
Unpack them and make some changes (only on the first page of this thread).
Generate config file for both kernels.
Run make HOSTCC=intelwraper CC=intelwrapper AR=xiar LD=xild vmlinux for these two kernels.
Then replace the vmlinux in 2.6.22 kernel directory with the one in 2.6.28 kernel directory.
Run make in 2.6.22 kernel directory ,the bzImage will be created.

I test it with icc 10.1.021

[TyMac]

Quoting - newper

Quoting - thaidog

Quoting - thaidog

I could never see it... it scrolled by to fast!

Well anyone on ideas as to what is causing this error?

I find a way,it's very bad,but it works well on my machine.
I noticed that we changed mca.c for a little before,and I also noticed that the 2.6.18 kernel could be compiled well by the icc.But the 2.6.18 one doesn't have mca.c in that directory.I findout that the kernel doesn't contain mca.c in arch/i386/boot until 2.6.23.So I compiled the kernel using the following steps:
I download 2.6.22.19 version (the last stable version of 2.6.22 branch) and 2.6.28.2 (current stable version)
Unpack them and make some changes (only on the first page of this thread).
Generate config file for both kernels.
Run make HOSTCC=intelwraper CC=intelwrapper AR=xiar LD=xild vmlinux for these two kernels.
Then replace the vmlinux in 2.6.22 kernel directory with the one in 2.6.28 kernel directory.
Run make in 2.6.22 kernel directory ,the bzImage will be created.

I test it with icc 10.1.021

Now that's a cool hack! Did the kernel actually boot this time?

[newper]

Quoting - thaidog

Quoting - newper

Quoting - thaidog

Quoting - thaidog

I could never see it... it scrolled by to fast!

Well anyone on ideas as to what is causing this error?

I find a way,it's very bad,but it works well on my machine.
I noticed that we changed mca.c for a little before,and I also noticed that the 2.6.18 kernel could be compiled well by the icc.But the 2.6.18 one doesn't have mca.c in that directory.I findout that the kernel doesn't contain mca.c in arch/i386/boot until 2.6.23.So I compiled the kernel using the following steps:
I download 2.6.22.19 version (the last stable version of 2.6.22 branch) and 2.6.28.2 (current stable version)
Unpack them and make some changes (only on the first page of this thread).
Generate config file for both kernels.
Run make HOSTCC=intelwraper CC=intelwrapper AR=xiar LD=xild vmlinux for these two kernels.
Then replace the vmlinux in 2.6.22 kernel directory with the one in 2.6.28 kernel directory.
Run make in 2.6.22 kernel directory ,the bzImage will be created.

I test it with icc 10.1.021

Now that's a cool hack! Did the kernel actually boot this time?

It works well
cat /proc/version
Linux version 2.6.28.2 (root@localhost) (Version 10.1 ) #1 Fri Jan 30 14:23:26 CST 2009

[TyMac]

Quoting - newper

Quoting - thaidog

Quoting - newper

Quoting - thaidog

Quoting - thaidog

I could never see it... it scrolled by to fast!

Well anyone on ideas as to what is causing this error?

I find a way,it's very bad,but it works well on my machine.
I noticed that we changed mca.c for a little before,and I also noticed that the 2.6.18 kernel could be compiled well by the icc.But the 2.6.18 one doesn't have mca.c in that directory.I findout that the kernel doesn't contain mca.c in arch/i386/boot until 2.6.23.So I compiled the kernel using the following steps:
I download 2.6.22.19 version (the last stable version of 2.6.22 branch) and 2.6.28.2 (current stable version)
Unpack them and make some changes (only on the first page of this thread).
Generate config file for both kernels.
Run make HOSTCC=intelwraper CC=intelwrapper AR=xiar LD=xild vmlinux for these two kernels.
Then replace the vmlinux in 2.6.22 kernel directory with the one in 2.6.28 kernel directory.
Run make in 2.6.22 kernel directory ,the bzImage will be created.

I test it with icc 10.1.021

Now that's a cool hack! Did the kernel actually boot this time?

It works well
cat /proc/version
Linux version 2.6.28.2 (root@localhost) (Version 10.1 ) #1 Fri Jan 30 14:23:26 CST 2009

This is seriously good news!! I would like to ask you if you wil help me spread word of your success gettting this to work... about a month ago I set up a mirror so that when the source could actually work it could put it there so that people and organizations could download it and use it for their systems. What do you think?

[newper]

I'm glad to help you,but it's difficult to compile a kernel using my 'hack'.If anybody wants to use my 'hack',he needs to download two kernel sources tarballs,compile the kernel over two times,replace ...
And I'm not sure it works on all the machines,because I replace a part of the new kernel with an old one(2.6.22).We need more tests.What do you think?

[TyMac]

Quoting - newper

I'm glad to help you,but it's difficult to compile a kernel using my 'hack'.If anybody wants to use my 'hack',he needs to download two kernel sources tarballs,compile the kernel over two times,replace ...
And I'm not sure it works on all the machines,because I replace a part of the new kernel with an old one(2.6.22).We need more tests.What do you think?

Yes I think you are right... but 2.6.22 would work without having to hack anything right? In any case we could tar ball your source and put it on the mirror for proof of concept until we get past all the tricky issues. I think the fact that you actually got it working will stir up serious interest! Is that ok with you? What cpu are you using or this?

Also i am running in to this error when I try to compile the 2.6.22 kernel:

icc: command line error: invalid argument for option '-m'
make[1]: ***[arch/i386/kernel/asm-offsets.s] Error 1
make: *** [prepare0] Error 2

[newper]

Quoting - thaidog

Quoting - newper

I'm glad to help you,but it's difficult to compile a kernel using my 'hack'.If anybody wants to use my 'hack',he needs to download two kernel sources tarballs,compile the kernel over two times,replace ...
And I'm not sure it works on all the machines,because I replace a part of the new kernel with an old one(2.6.22).We need more tests.What do you think?

Yes I think you are right... but 2.6.22 would work without having to hack anything right? In any case we could tar ball your source and put it on the mirror for proof of concept until we get past all the tricky issues. I think the fact that you actually got it working will stir up serious interest! Is that ok with you? What cpu are you using or this?

Also i am running in to this error when I try to compile the 2.6.22 kernel:

icc: command line error: invalid argument for option '-m'
make[1]: ***[arch/i386/kernel/asm-offsets.s] Error 1
make: *** [prepare0] Error 2

I changed the intel wrapper a little
Add | -maccumulate-outgoing-args after -ffreestanding

[newper]

The 2.6.22 kernel should be compiled and work fine after the change of the icc wrapper.But I didn't test the modules.
In fact,you successfully built the 2.6.25 kernel a month ago,but it's just the 'vmlinux' not 'bzImage',then we edit the mca.c file but it cannot boot.So my plan is to use 2.6.22's tools to create bzImage for 2.6.2x's vmlinux.
I noticed that there are a lot of asm codes in the tools which create bzImage after 2.6.23.Perhaps icc cannot handle asm codes well ?In icc 11.0,nearly all files in arch/x86/boot are failed to compile.I think that will explain why we compiled the kernel successfully after a lot of modify but cannot boot.

[TyMac]

Quoting - newper

Quoting - thaidog

Quoting - newper

I'm glad to help you,but it's difficult to compile a kernel using my 'hack'.If anybody wants to use my 'hack',he needs to download two kernel sources tarballs,compile the kernel over two times,replace ...
And I'm not sure it works on all the machines,because I replace a part of the new kernel with an old one(2.6.22).We need more tests.What do you think?

Yes I think you are right... but 2.6.22 would work without having to hack anything right? In any case we could tar ball your source and put it on the mirror for proof of concept until we get past all the tricky issues. I think the fact that you actually got it working will stir up serious interest! Is that ok with you? What cpu are you using or this?

Also i am running in to this error when I try to compile the 2.6.22 kernel:

icc: command line error: invalid argument for option '-m'
make[1]: ***[arch/i386/kernel/asm-offsets.s] Error 1
make: *** [prepare0] Error 2

I changed the intel wrapper a little
Add | -maccumulate-outgoing-args after -ffreestanding

hmm now I'm getting this error:

include/linux/compiler-intel.h(31): error #11: unrecognized preprocessing directive
#undef_compiler_offsetof
^

arch/i386/kernel/asm-offsets.c(29): warning #1419: external declaration in primary source file
void foo(void);
^

compilation aborted for arch/i386/kernel/asm-offsets.c (code 2)
make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 2
make: *** [prepare0] Error 2

I am using icc version 10.1.018 is that it? I changed the wrapper to look like this:

--------------------------------------------------------

# This is a wrapper script for icc 10.1 on IA-32

#--------------------------------------------------------

ARGS=$@

ICCARGS="-fno-builtin"



# For loop to change options of icc

for ARG in $@

do

case $ARG in

-fno-stack-protector )

ICCARGS="$ICCARGS -fno-stack-security-check"

;;



-Wno-pointer-sign | -msoft-float | -ffreestanding | -maccumulate-outgoing-args )

;;



* )

ICCARGS="$ICCARGS $ARG"

;;

esac

done



icc $ICCARGS

##########################################################

Does that look right? I only did that and the edits to the files on the first page like you said.

###########################

That ASM code error sounds right to me... Although I am not known as a "kernel hacker" ;) We had to change a little of that already. So do you think that the issue is isloated to the mca.o and arch/x86/boot ?

As soon as I can get a working copy up to my mirror I have a few tech writters who are interested in your success (everybody who contributed in this thread will get credit I promise!) I plan to make a few clusters with these kernels to test the performance when I get a chance. If you want you can reach me via e-mail: at tyler@mcadams.com It would be good to have your name so that you can recieve full credit - (granted you want credit that way)

I will give you a user account an password to my mirror if you would like also.

[newper]

#undef_compiler_offsetof
have a mistake?
it's #undef __compiler_offsetof

[TyMac]

Quoting - newper

#undef_compiler_offsetof
have a mistake?
it's #undef __compiler_offsetof

Sorry I'm a bit excited right now! It is up and working on my machine - IT'S A MIRACLE!!!

cat /proc/version
Linux version 2.6.22 (root@localhost) (Version 10.1 ) #2 SMP Wed Feb 4 00:26:42 EST 2009

Congradulations! Simply amazing!

[TyMac]

Quoting - newper

The 2.6.22 kernel should be compiled and work fine after the change of the icc wrapper.But I didn't test the modules.
In fact,you successfully built the 2.6.25 kernel a month ago,but it's just the 'vmlinux' not 'bzImage',then we edit the mca.c file but it cannot boot.So my plan is to use 2.6.22's tools to create bzImage for 2.6.2x's vmlinux.
I noticed that there are a lot of asm codes in the tools which create bzImage after 2.6.23.Perhaps icc cannot handle asm codes well ?In icc 11.0,nearly all files in arch/x86/boot are failed to compile.I think that will explain why we compiled the kernel successfully after a lot of modify but cannot boot.

Ok the 2.6.22 kernel is up at: www.linuxdna.com

Feilong any ideas about the ASM code that we are having issues with?

[Feilong_H_Intel]

Quoting - thaidog

Ok the 2.6.22 kernel is up at: www.linuxdna.com

Feilong any ideas about the ASM code that we are having issues with?

Glad to see you guys having the kernel running. Congratulations!

Regarding the asm code in arch/x86/boot, I didn't get a chance to compilethe files in that directorywith icc 11.0 yet. Do you have the error message for that problem? If you can show me the command line, that's even better.

[newper]

Quoting - Feilong H (Intel)

Quoting - thaidog

Ok the 2.6.22 kernel is up at: www.linuxdna.com

Feilong any ideas about the ASM code that we are having issues with?

Glad to see you guys having the kernel running. Congratulations!

Regarding the asm code in arch/x86/boot, I didn't get a chance to compilethe files in that directorywith icc 11.0 yet. Do you have the error message for that problem? If you can show me the command line, that's even better.

I need some time to get the error message ,but I can remember it's something like Internal Error (I didn't modify mca.a).And some other file in that directory like video-*.c also cannot be compiled by icc 11.0.069,it also says Internal Error