    <rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:admin="http://webns.net/mvcb/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:content="http://purl.org/rss/1.0/modules/content/">
     <channel>
        <title>Linux.com :: How to use gzip on stdout</title>
        <link>http://www.linux.com/archive/forums/topic/2711</link>
        <description>Everything Linux and Open Source</description>
        <dc:language>en-us</dc:language> 
        <dc:creator>Linux.com Admin</dc:creator> 
        <admin:generatorAgent rdf:resource="http://www.linux.com" /> 
        <admin:errorReportsTo rdf:resource="mailto:noreply@linux.com" />
       <sy:updatePeriod>hourly</sy:updatePeriod>
       <sy:updateFrequency>1</sy:updateFrequency>
       <docs>http://backend.userland.com/rss</docs>



<div class="xarbb-noteswrapper">
</div>

   <div class="xarbb-noteswrapper">
       <div class="xar-sub rightnote">
           <div>
                    <a href="http://www.linux.com/archive/forums/topic/2770">
Previous Topic</a>
|
                    <a href="http://www.linux.com/archive/forums/topic/2796">
Next Topic</a>
</div>

           <div>
                    <a href="http://www.linux.com/archive/forums/printtopic/2711?theme=print">
Print this topic</a> |
You are not subscribed            </div>
        </div>
        <div style="clear: both;"></div>
    </div>

    <h1>
        <a class="xar-title" href="http://www.linux.com/archive/forums/topic/2711">How to use gzip on stdout</a>
   </h1>


   <div class="xarbb-topiccontrols">
<form action="http://www.linux.com/archive/forums?func=jump" method="post">
   <label for="xarbb_jump_menu">Forum Jump</label>
    <select name="f" id="xarbb_jump_menu" onchange="this.form.submit();">
        <optgroup label="Options">
            <option value="5" selected="selected">
Please select forum            </option>
            <option value="5">--------------------</option>
        </optgroup>
        <optgroup label="Forums">
                <option value="1">New to Linux</option>
                <option value="2">Hardware</option>
                <option value="3">Linux Distributions</option>
                <option value="4">Applications</option>
                <option value="5">Programming and Development</option>
                <option value="6">Miscellaneous Discussion</option>
                <option value="7">Linux.com Writers Forum</option>
                <option value="8">Jobs</option>
        </optgroup>
    </select>
    <input type="submit" class="button" value="Jump" />
</form>
<div style="clear: both;"></div>
   </div>

   <div class="xarbb-topiccontrols">

<span class="xar-sub">
    <a href="http://www.linux.com/archive/forums">
Forum Index    </a>
    <span style="font-size: larger;">&#187;</span>
    <a href="http://www.linux.com/archive/forums/category/4181">
        Forums 
    </a>
    <span style="font-size: larger;">&#187;</span>
    <a href="http://www.linux.com/archive/forums/forum/5">
        Programming and Development   </a>
</span>
<div style="clear: both;"></div>
   </div>

<div class="xarbb-tablewrapper">



   <table>
       <tr>
           <th scope="col">
Author</th>
           <th scope="col">
Message</th>
       </tr>



            <tr>
                <td class="xar-norm author">
                    <div>
                        <a href="http://www.linux.com/archive/roles/409350">mitch</a>
                   </div>
                   <div class="xar-sub">
Joined: Sep 04, 2008<br/>Posts: 1                        <br />
                        <a href="http://www.linux.com/archive/forums?by=409350&amp;func=searchtopics">
Other Topics</a>
                   </div>
               </td>

               <td class="xar-norm message">
                   <div class="xarbb-messagecontrols">
                       <div class="leftnote">
                           <span class="xar-sub">
Posted Sep 04, 2008 at 10:55:59 PM</span>
                       </div>



<div style="float: left;">
                           <strong class="xar-sub">
Subject: How to use gzip on stdout                            </strong>
                        </div>
                        <div style="clear: both;"></div>
                    </div>

                    <div>
                        Does anyone know how I can use gzip to zip a large log file on the fly.

My simulation is currently logging a large file that I need for analysis at a later point. However the files are so huge that I may even run out of disk space. The content is mainly text so when compressed the files are smaller... I would like to gzip on the fly without having to dump it to a file and then create the archive.

This is what I am doing right now and would like to not store foo.log...
./sim &lt;args&gt; &gt;foo.log 2&gt;&amp;1 
and when its completed i do 

tar -cf - foo.log | gzip &gt; foo.tar.gz

Any suggetions?</div>

                </td>
            </tr>

            <tr>
                <td class="xar-norm author">
                    <a href="http://www.linux.com/archive/forums/topic/2711#top">
Back to top</a>
</td>
               <td class="xar-alt">

                        <a href="http://www.linux.com/archive/roles/409350">
                            <img src="modules/xarbb/xarimages/new/icon_profile.gif" alt="Profile" />
                        </a>
                        <a href="http://www.linux.com/archive/?module=roles&amp;func=email&amp;uid=409350">
                            <img src="modules/xarbb/xarimages/new/icon_email.gif" alt="Email" />
                       </a>
                            <a href="http://">
                                <img src="modules/xarbb/xarimages/new/icon_www.gif" alt="Website" />
                           </a>
</td>
           </tr>





           <tr>
               <td class="xar-norm author">
                   <strong>

                            <a href="http://www.linux.com/archive/roles/400849">
                                proopnarine                           </a>
</strong>

                   <span class="xar-sub">
<br />
<br />
Joined Apr 03, 2008<br/>Posts: 590<br />
Location:San Francisco<br />
                        <br />
                        <a href="http://www.linux.com/archive/forums?by=400849&amp;func=searchtopics">
Other Topics                        </a>
                    </span>
                </td>

                <td class="xar-accent message">
                   <div class="xarbb-messagecontrols">
                       <div class="leftnote">
                           <span class="xar-sub">
Posted:
                                Sep 05, 2008 5:28:18 AM                           </span>
                       </div>
                       <div class="rightnote">
</div>
                       <div style="float:left;">
                           <strong class="xar-sub">
Subject:
                                <a name="1197991" href="http://www.linux.com/archive/forums/topic/2711#1197991">
                                    How to use gzip on stdout                                </a>
                            </strong>
                        </div>
                        <div style="clear: both;"></div>
                    </div>

                    <div style="clear: left;">
                        Hmmm, it seems that this should be possible. I know that this is done to some extent by Apache/Mozilla for serving pages. 
http://www.mozilla.org/projects/apache/gzip/
Without being able to test this right now, what you could do is to first create a tar gzipped file, and then stream to the archive using tar. Look at the tar manual pages.
  I have similar issues with very large simulation output files, so this would be a useful thing to figure out!<p class="hiddensignature">
<p>Take the red pill</p>
<p><a href="http://www.calacademy.org/blogs">Climate Change Blog</a></p>
<a href="http://proopnarine.wordpress.com">Food Weblog</a></p>
                    </div>
                </td>
            </tr>

            <tr>
                <td class="xar-norm author">
                    <a href="http://www.linux.com/archive/forums/topic/2711#top">
Back to top</a>
</td>

               <td class="xar-alt">

                            <a href="http://www.linux.com/archive/roles/400849">
                                <img src="modules/xarbb/xarimages/new/icon_profile.gif" alt="Profile" />
                            </a>
                            <a href="http://www.linux.com/archive/?module=roles&amp;func=email&amp;uid=400849">
                                <img src="modules/xarbb/xarimages/new/icon_email.gif" alt="Email" />
                           </a>
                                <a href="http://zeus.calacademy.org/roopnarine/peter.html">
                                    <img src="modules/xarbb/xarimages/new/icon_www.gif" alt="Website" />
                               </a>
</td>
           </tr>

           <tr>
               <td class="xar-norm author">
                   <strong>

                            <a href="http://www.linux.com/archive/roles/401295">
                                tophandcwby                           </a>
</strong>

                   <span class="xar-sub">
<br />
Joined Apr 10, 2008<br/>Posts: 81<br />
                        <br />
                        <a href="http://www.linux.com/archive/forums?by=401295&amp;func=searchtopics">
Other Topics                        </a>
                    </span>
                </td>

                <td class="xar-norm message">
                   <div class="xarbb-messagecontrols">
                       <div class="leftnote">
                           <span class="xar-sub">
Posted:
                                Sep 09, 2008 10:31:58 PM                           </span>
                       </div>
                       <div class="rightnote">
</div>
                       <div style="float:left;">
                           <strong class="xar-sub">
Subject:
                                <a name="1198344" href="http://www.linux.com/archive/forums/topic/2711#1198344">
                                    How to use gzip on stdout                                </a>
                            </strong>
                        </div>
                        <div style="clear: both;"></div>
                    </div>

                    <div style="clear: left;">
                        I'm not really following what your trying to do. I think what you want is
./sim &lt;arg&gt;  2&gt;&amp;1 | tar -czf foo.tar.gz                    </div>
                </td>
            </tr>

            <tr>
                <td class="xar-norm author">
                    <a href="http://www.linux.com/archive/forums/topic/2711#top">
Back to top</a>
</td>

               <td class="xar-alt">

                            <a href="http://www.linux.com/archive/roles/401295">
                                <img src="modules/xarbb/xarimages/new/icon_profile.gif" alt="Profile" />
                            </a>
                            <a href="http://www.linux.com/archive/?module=roles&amp;func=email&amp;uid=401295">
                                <img src="modules/xarbb/xarimages/new/icon_email.gif" alt="Email" />
                           </a>
                                <a href="http://">
                                    <img src="modules/xarbb/xarimages/new/icon_www.gif" alt="Website" />
                               </a>
</td>
           </tr>

           <tr>
               <td class="xar-norm author">
                   <strong>

                            <a href="http://www.linux.com/archive/roles/401295">
                                tophandcwby                           </a>
</strong>

                   <span class="xar-sub">
<br />
Joined Apr 10, 2008<br/>Posts: 81<br />
                        <br />
                        <a href="http://www.linux.com/archive/forums?by=401295&amp;func=searchtopics">
Other Topics                        </a>
                    </span>
                </td>

                <td class="xar-accent message">
                   <div class="xarbb-messagecontrols">
                       <div class="leftnote">
                           <span class="xar-sub">
Posted:
                                Sep 09, 2008 10:53:05 PM                           </span>
                       </div>
                       <div class="rightnote">
</div>
                       <div style="float:left;">
                           <strong class="xar-sub">
Subject:
                                <a name="1198347" href="http://www.linux.com/archive/forums/topic/2711#1198347">
                                    How to use gzip on stdout                                </a>
                            </strong>
                        </div>
                        <div style="clear: both;"></div>
                    </div>

                    <div style="clear: left;">
                        Actually what I put in my previous post is wrong. Sorry about that.                    </div>
                </td>
            </tr>

            <tr>
                <td class="xar-norm author">
                    <a href="http://www.linux.com/archive/forums/topic/2711#top">
Back to top</a>
</td>

               <td class="xar-alt">

                            <a href="http://www.linux.com/archive/roles/401295">
                                <img src="modules/xarbb/xarimages/new/icon_profile.gif" alt="Profile" />
                            </a>
                            <a href="http://www.linux.com/archive/?module=roles&amp;func=email&amp;uid=401295">
                                <img src="modules/xarbb/xarimages/new/icon_email.gif" alt="Email" />
                           </a>
                                <a href="http://">
                                    <img src="modules/xarbb/xarimages/new/icon_www.gif" alt="Website" />
                               </a>
</td>
           </tr>

           <tr>
               <td class="xar-norm author">
                   <strong>

                            <a href="http://www.linux.com/archive/roles/401295">
                                tophandcwby                           </a>
</strong>

                   <span class="xar-sub">
<br />
Joined Apr 10, 2008<br/>Posts: 81<br />
                        <br />
                        <a href="http://www.linux.com/archive/forums?by=401295&amp;func=searchtopics">
Other Topics                        </a>
                    </span>
                </td>

                <td class="xar-norm message">
                   <div class="xarbb-messagecontrols">
                       <div class="leftnote">
                           <span class="xar-sub">
Posted:
                                Sep 10, 2008 2:21:04 PM                           </span>
                       </div>
                       <div class="rightnote">
</div>
                       <div style="float:left;">
                           <strong class="xar-sub">
Subject:
                                <a name="1198385" href="http://www.linux.com/archive/forums/topic/2711#1198385">
                                    How to use gzip on stdout                                </a>
                            </strong>
                        </div>
                        <div style="clear: both;"></div>
                    </div>

                    <div style="clear: left;">
                        If all you want is a gzip file of a single log file try
./sim &lt;arg&gt; 2&gt;&amp;1 |  gzip -c &gt; foo.gz

Hope that works better than my first answer!                    </div>
                </td>
            </tr>

            <tr>
                <td class="xar-norm author">
                    <a href="http://www.linux.com/archive/forums/topic/2711#top">
Back to top</a>
</td>

               <td class="xar-alt">

                            <a href="http://www.linux.com/archive/roles/401295">
                                <img src="modules/xarbb/xarimages/new/icon_profile.gif" alt="Profile" />
                            </a>
                            <a href="http://www.linux.com/archive/?module=roles&amp;func=email&amp;uid=401295">
                                <img src="modules/xarbb/xarimages/new/icon_email.gif" alt="Email" />
                           </a>
                                <a href="http://">
                                    <img src="modules/xarbb/xarimages/new/icon_www.gif" alt="Website" />
                               </a>
</td>
           </tr>
</table>
</div>

<div class="xarbb-topiccontrols">

<span class="xar-sub">
    <a href="http://www.linux.com/archive/forums">
Forum Index    </a>
    <span style="font-size: larger;">&#187;</span>
    <a href="http://www.linux.com/archive/forums/category/4181">
        Forums 
    </a>
    <span style="font-size: larger;">&#187;</span>
    <a href="http://www.linux.com/archive/forums/forum/5">
        Programming and Development   </a>
</span>

</div>



            </channel>
</rss>
