gcc cilk plus git repository corrupted?

[I-Ting_L_]

Hi,

I have cloned the gcc Cilk Plus git repo awhile back.  Today, when I tried to pull from the repo, it failed with the following error:

$ git pull
error: Unable to find 43269d882367dc4f0e4c2651d6625e1b7de78764 under http://gcc.gnu.org/git/gcc.git
Cannot obtain needed object 43269d882367dc4f0e4c2651d6625e1b7de78764
while processing commit d612d6bc401c95be8e7740ffea26a77845edd113.
error: Fetch failed.

 Since the repo was cloned awhile ago, I decided to try cloning it again, which again failed with the following error:

$ git clone http://gcc.gnu.org/git/gcc.git gcc-cilkplusCloning into gcc-cilkplus...
error: Unable to get pack file http://gcc.gnu.org/git/gcc.git/objects/pack/pack-0cfe761b4c3a62b686937aa26b6c944a3e7462e2.packtransfer closed with 83853301 bytes remaining to readerror: Unable to find 6540d658e4bda2dec5e2399ff1e42d8689b869ec under http://gcc.gnu.org/git/gcc.gitCannot obtain needed object 6540d658e4bda2dec5e2399ff1e42d8689b869ecwhile processing commit 791fd3d13f7aedd473f6c2db7d7ac5d4b7da747d.error: Fetch failed.

This occurred after 4 hours of downloading git objects, and I couldn't tell how much data was downloaded, since upon failing, git removed the local directory.  It seems that maybe some pack object on the remote git repo got corrupted.

I am wondering if there anything the Cilk Plus team can do to fix this.  I can try the svn repository, but it will be nice if I can simply git pull from my originally downloaded local git repository.

Thanks,
Angelina

[Balaji_I_Intel]

Hi Angelina,

      I believe this error is on the GCC side and not something we can fix. You can try and emailing the GCC developers in GCC mailing list and they maybe able to give you more ideas. I have had luck with using SVN. Please try that and let me know if it doesn't work.

Thanks,

Balaji V. Iyer.

[I-Ting_L_]

Hi Balaji,

Thanks.  Using svn indeed works, although the instruction on the web didn't work for me:

$ svn co http://gcc.gnu.org/svn/gcc/branches/cilkplus gcc-cilkplus
svn: Server sent unexpected return value (403 Forbidden) in response to PROPFIND request for '/svn/gcc/branches/cilkplus'

Replacing "http" with "svn" worked, however.

$ svn co svn://gcc.gnu.org/svn/gcc/branches/cilkplus gcc-cilkplus

Cheers,
Angelina 

[I-Ting_L_]

Hi Balaji,

Thanks.  Using svn indeed works, although the instruction on the web didn't work for me:

$ svn co http://gcc.gnu.org/svn/gcc/branches/cilkplus gcc-cilkplus
svn: Server sent unexpected return value (403 Forbidden) in response to PROPFIND request for '/svn/gcc/branches/cilkplus'

Replacing "http" with "svn" worked, however.

$ svn co svn://gcc.gnu.org/svn/gcc/branches/cilkplus gcc-cilkplus

Cheers,
Angelina 

[Balaji_I_Intel]

Hi Angelina,

     The fact that you had an issue with http://gcc.gnu.org/svn/... seem to suggest that you have some kind of http_proxy issue or some server issue on your side. I have seen that problem in the past when I move from one machine to another or if the proxy setup is incorrect.

Thanks,

Balaji V. Iyer.

PS. There is a git:// option for git repository. Here is a link for more information: http://gcc.gnu.org/wiki/GitMirror

[leoferres]

Angelina, regards from Chile!

Look, I tried myself to work with the git repo and it didn't work. Then I tried with the SVN and it *did* work, but found other things during the process of GCC Cilk installation. I had a few of my students also try and install it with various levels of success. Most of all, installing from git is impossible, as it is trying to make it work on a 32-bit machine (especially cilkscreen and cilkview). Barry then sent me a patch for the 32 bit version of cilkview and screen, I can pm them to you.

Anyway, here is a guide I wrote for my students (I'm making it public just in case someone finds it useful). It borrows heavily from the one at MIT (Tao's one) and from another one at Intel that I happened to have found. Please, if you succeed in installing everything, then congratulations, but if not, I could PM you with the tex source and you could write down your problem and how you solved it? My students have spent a good chunk of their summer attempting this exercise... I have installed it fine, so I guess I'm lucky.

Leo

[Balaji_I_Intel]

Hello Leo,

    I use GIT on a regular basis and I don't seem to be having too much trouble. Althrough, I agree SVN is relatively simpler. Can you please tell me what errors are you finding when using GIT? Also, I looked through your tutorial and I believe there is an easier method to install Cilkplus-GCC.

Let's assume you check out the sources inside a directory called "cilkplus-gcc" please do the following:

=====================================================================================================

cd cilkplus-gcc

./contrib/download_prerequisites    # Running this script assumes that your FTP and HTTP proxies are set correctly if necessary.

cd ..

mkdir b-gcc

cd b-gcc

../cilkplus-gcc/configure --prefix=<install_dir> --enable-languages="c,c++" --disable-multilib # only use disable multiliib if you need it.

make

make install

================================================================================

Thanks,

Balaji V. Iyer.

[leoferres]

That is wonderful, thanks Balaji. I will write it into the guide. You're absolutely right that's much simpler, I just didn't know this, and was installing gcc from (a very serious) scratch :) I was looking at the contrib/ folder and there are a bunch of scripts there that may also be helpful. One of them is gcc_update, will that also fetch the changes you make in the gcc tree? Is there a way to only update cilk?

I could never download the git repo. Ever. :) It just hang (not even counting the objects). Anyway, I find svn more intuitive. I will try again and let you know, if you're interested.

[Balaji_I_Intel]

I have not used gcc_update. For me, svn update or "git pull --rebase" would do the trick just fine.

If it is not even counting objects, then the connection between your computer and the gcc repo is not being made correctly. Most of the cases I have seen turned out to be a proxy issue. One thing I have noticed is that it requires a "static" proxy. For example (I am making up websites here), if your proxy setting is http://proxy-finder.my-domain.com which will find http://proxy.my-place.my-domain.com, then GIT seem to not work for me. Instead, I have to give the proxy as http://proxy.my-place.my-domain.com as the proxy website.

One great place to start would be your sys-admin to see if they have not blocked git accesses.

Thanks,

Balaji V. Iyer.

[I-Ting_L_]

Hi,

Balaji, thanks for pointing out that I may have the wrong proxy setting.  I had the env var http_proxy defined in my .bashrc, and by removing that, my git clone went much faster.  (Although somehow the svn http:// still doesn't work; I am not sure if there is a proxy setting used by machines on CSAIL network, but I am now able to use git.)   Also, I was able to successfully git clone this afternoon --- maybe the file system issue on gnu.org has been resolved. 

Ieoferres, thanks for sharing the experience.  One note about git --- I was experiencing really slow git clone when I had http_proxy set, and I am wondering if that's what you were experiencing.  Since there is 1.4GB worth of data to download, git clone may seem like it's hanged.  I used "du -ch" to check and make sure that the downloading was still happening.  Even when I removed the http_proxy setting, it still took me awhile to git clone (that depends on your network bandwidth).  Once git cloned successfully, I was able to use Balaji's instruction to install gcc successfully, although I had to use --disable-multilib in the configuration option.  When not used, it seems to fail to build for 32-bit (I am on a 64-bit machine, and it complained about incompatible libc) on my system.   

Cheers,
Angelina 

[Bradley_K_]

This problem is still present.  The problem is that the gnu.org git server is too slow, I think.

Here is how I can incrementally do a git clone

git init
git add .
git remote add cilkplus-gcc http://gcc.gnu.org/git/gcc.git
git pull http://gcc.gnu.org/git/gcc.git

I'm not sure the git add . is needed.

The git pull will get some data and then fail.  So then I do another git pull and another, and each time I get some more. I see errors such as

[bradley@bradley-x220 cilk]$ git pull http://gcc.gnu.org/git/gcc.git
error: Unable to find 7a9efecdddac2e80814504e2cae0adc1fe1594fa under http://gcc.gnu.org/git/gcc.git
Cannot obtain needed blob 7a9efecdddac2e80814504e2cae0adc1fe1594fa
while processing commit 1bb8a24aa5a1e2ae4273d085e931510e82c1c53f.
error: Fetch failed.

but eventually the errors seem to go away.  I haven't managed to get the whole thing yet, however...

Is development being managed in git or in svn?