Utilizing multiple processor cores for faster TAR compressing on Linuxes.

From the forum http://www.linuxquestions.org/questions/linux-software-2/utilizing-multi-core-for-tar-gzip-bzip-compression-decompression-4175426075/

The gzip command itself is single threaded, but you can replace it on system with pigz, or provide extra arguments to the ‘tar’ command itself, which will give you a significant speed-up..

From the tar manpage:

-I, --use-compress-program PROG
      filter through PROG (must accept -d)

So, typical tar options to compress and extract files with tar plus pigz might be…

tar -I pigz -cf test.tar.gz *

tar -I pigz -xf test.tar.gz

Happy compressing!


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s