Home » Linux » How can I gzip standard in to a file and also print standard in to standard out?

How can I gzip standard in to a file and also print standard in to standard out?

Posted by: admin November 30, 2017 Leave a comment

Questions:

I want to execute a command, have the output of that command get gzip’d on the fly, and also echo/tee out the output of that command.

i.e., something like:

echo "hey hey, we're the monkees" | gzip --stdout > my_log.gz

Except when the line executes, I want to see this on standard out:

hey hey, we're the monkees
Answers:
echo "hey hey, we're the monkees" | tee /dev/tty | gzip --stdout > my_log.gz

As pointed out in the comments, /dev/stdout might work better than /dev/tty in some circumstances.

Questions:
Answers:

Another way (assuming a shell like bash or zsh):

echo "hey hey, we're the monkees" | tee >(gzip --stdout > my_log.gz)

The admittedly strange >() syntax basically does the following:

  • Create new FIFO (usually something in /tmp/)
  • Execute command inside () and bind the FIFO to stdin on that subcommand
  • Return FIFO filename to command line.

What tee ends up seeing, then, is something like:

tee /tmp/arjhaiX4

All gzip sees is its standard input.

For Bash, see man bash for details. It’s in the section on redirection. For Zsh, see man zshexpn under the heading “Process Substitution.”

As far as I can tell, the Korn Shell, variants of the classic Bourne Shell (including ash and dash), and the C Shell don’t support this syntax.

Questions:
Answers:

Have a nice cup of tee!

The tee command copies standard input
to standard output and also to any
files given as arguments. This is
useful when you want not only to send
some data down a pipe, but also to
save a copy

As I’m having a slow afternoon, here’s some gloriously illustrative ascii-art…

           +-----+                   +---+                  +-----+  
stdin ->   |cmd 1|    -> stdout ->   |tee|   ->  stdout  -> |cmd 2|
           +-----+                   +---+                  +-----+
                                       |
                                       v
                                     file

As greyfade demonstrates in another answer the ‘file’ need not be a regular file, but could be FIFO letting you pipe that tee’d output into a third command.

           +-----+                   +---+                  +-----+  
stdin ->   |cmd 1|    -> stdout ->   |tee|   ->  stdout  -> |cmd 2|
           +-----+                   +---+                  +-----+
                                       |
                                       v
                                     FIFO
                                       |
                                       v
                                    +-----+
                                    |cmd 3|
                                    +-----+

Questions:
Answers:

Just to post a way that doesn’t involve touching disk:

echo "hey hey, we're the monkees" | (exec 1>&3 && tee /proc/self/fd/3 | gzip --stdout > my_log.gz)

Leave a Reply

Your email address will not be published. Required fields are marked *