topic in Intel® Moderncode for Parallel Architectures

igzip for VS10 C++?

David_L_6 — Tue, 30 Sep 2014 17:40:47 GMT

I was searching for a zlib-compatible compressor but faster, and came cross the paper describing igzip --

High Performance DEFLATE Compression on Intel Architecture Processors

igzip looks like exactly (!) what I am looking for. Compatible with zlib, but faster.

However, the downloadable source was for Linux. I need it for a VS10 C++ project. I have successfully (I think) compiled and assembled the desired modules (common, crc, crc_utils, hufftables, hufftables_c.cpp, igzip0c_body, igzip0c_finish, init_stream) into a .lib.

But when I attempt to link the library into my project, I get error LNK2019: unresolved external symbol fast_lz (and init_stream) from where they are called. I also have a "C" lz4 compression library linked into the project, and it works fine. I have spent 3 days playing with it, looking for the clue that will unlock the symbols, but no luck so far.

I get no other warnings and/or errors during the compiling/assembling of the library or project. Any help (especially if anybody has done the work to create a VS10 lib igzip project) would be appreciated!

Update:

David_L_6 — Thu, 02 Oct 2014 22:22:18 GMT

Update:

I was able to build the igzip library in my VC2010 C/C++ project. Yay!

1) I needed to invoke the YASM assembler in the project (thanks to Brian Gladman's "Building YASM with Microsoft Visual Studio 2010 - C/C++ v10") for the .ASM modules.

2) There were a few things to change between GCC and MSVC in the C (called .cpp, but they were C) modules. Ie: adding "typedef" to structure definitions and pulling the functions out of an .h file and creating a separate .c(pp) file for them.

But now I ran into an unrelated issue - the code uses the pclmulqdq instruction which my processor does not support.

I have looked for a .ASM macro to replace it (one supposedly resides in GAS, but I downloaded binutils 2.24 and did not find a reference to the instruction outside of the disassembler) but have had no luck. I do have the psuedo code and can build a 'C' version, but that is not optimal (and of course there will be no bugs in my code.... ya, right...).

Anybody out there with code/guidance?

Thanks,

Dave

very good news.

gvollant — Wed, 15 Oct 2014 21:20:37 GMT

very good news.

Is it possible you upload , send email or post on forum the full project?

I zipped up the VS10 igzip

David_L_6 — Sat, 18 Oct 2014 17:23:50 GMT

I zipped up the VS10 igzip library files and attached it (also put the zip at: http://www.cruzinsb.com/igzip_VS10/).

My focus was to use igzip to create zlib (not gzip) compressed files. So far I have been unsuccessful in replacing the gzip header/CRC with a zlib header/adler32. Getting closer though.

I believe the files zipped here have been returned to the successful gzip state with the zlib logic commented out.

If anyone has zlib-compatible versions of igzip, I'm all ears!

gvollant — Tue, 21 Oct 2014 12:15:21 GMT

thankyou a lot

igzip can create deflate (without header) or gzip

You probably need your code to add the (small) zlib header, which contain, I believe adler32 checksum

Any news to a fallback for older CPU (core 2 duo before 2009, I believe) with incompatible instruction ?

gvollant — Tue, 21 Oct 2014 12:31:08 GMT

By reading RFC950 and http://stackoverflow.com/questions/9050260/what-does-a-zlib-header-look-like

Just begin by 0x78 0x01 (ou 0x78 0x9c) then deflate data then adler32 checkum

gvollant — Tue, 21 Oct 2014 12:31:53 GMT

By reading RFC950 and http://stackoverflow.com/questions/9050260/what-does-a-zlib-header-look-like

Just begin by 0x78 0x01 (ou 0x78 0x9c) then deflate data then adler32 checkum