Linux.com

Search

Special Offers

Get special offers on:

Linux
Application Dev
Programming
Software

Email:

Forums: Programming and Development - How to use gzip on stdout

Print this topic | You are not subscribed
Forum Index » Forums » Programming and Development
Author Message
Joined: Sep 04, 2008
Posts: 1
Other Topics
Posted Sep 04, 2008 at 10:55:59 PM
Subject: How to use gzip on stdout

Does anyone know how I can use gzip to zip a large log file on the fly.

My simulation is currently logging a large file that I need for analysis at a later point. However the files are so huge that I may even run out of disk space. The content is mainly text so when compressed the files are smaller... I would like to gzip on the fly without having to dump it to a file and then create the archive.

This is what I am doing right now and would like to not store foo.log...
./sim <args> >foo.log 2>&1
and when its completed i do

tar -cf - foo.log | gzip > foo.tar.gz

Any suggetions?
Back to top Profile Email Website
proopnarine

Joined Apr 03, 2008
Posts: 402
Location:San Francisco

Other Topics
Posted: Sep 05, 2008 5:28:18 AM
Subject: How to use gzip on stdout

Hmmm, it seems that this should be possible. I know that this is done to some extent by Apache/Mozilla for serving pages.
http://www.mozilla.org/projects/apache/gzip/
Without being able to test this right now, what you could do is to first create a tar gzipped file, and then stream to the archive using tar. Look at the tar manual pages.
I have similar issues with very large simulation output files, so this would be a useful thing to figure out!

Take the red pill www.calacademy.org/blogs
Back to top Profile Email Website
tophandcwby
Joined Apr 10, 2008
Posts: 66

Other Topics
Posted: Sep 09, 2008 10:31:58 PM
Subject: How to use gzip on stdout

I'm not really following what your trying to do. I think what you want is
./sim <arg> 2>&1 | tar -czf foo.tar.gz
Back to top Profile Email Website
tophandcwby
Joined Apr 10, 2008
Posts: 66

Other Topics
Posted: Sep 09, 2008 10:53:05 PM
Subject: How to use gzip on stdout

Actually what I put in my previous post is wrong. Sorry about that.
Back to top Profile Email Website
tophandcwby
Joined Apr 10, 2008
Posts: 66

Other Topics
Posted: Sep 10, 2008 2:21:04 PM
Subject: How to use gzip on stdout

If all you want is a gzip file of a single log file try
./sim <arg> 2>&1 | gzip -c > foo.gz

Hope that works better than my first answer!
Back to top Profile Email Website
Forum Index » Forums » Programming and Development
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya