Archives & compression
Downloads, backups and log files arrive as .zip, .tar.gz, .tar.xz, .gz… Two different jobs hide behind those names: archiving (many files → one file) and compressing (one file → a smaller file). Once you can tell them apart, every download makes sense, and you'll never again unpack 200 files straight into your home folder.
You will learn
- Archive vs compress, and reading names like
.tar.xz gzip,xzandbzip2: speed vs size,-k,zcat/xzcattar: create (c), list (t), extract (x), withz/J/jand-Czip/unzipfor sharing with Windows and Macfile: what is this really? And how to spot a "tarbomb" before it goes off
Two jobs, many names
| Name | What it is | Unpack with |
|---|---|---|
.tar | archive only: many files glued together, not smaller | tar -xf |
.gz | ONE file compressed with gzip | gunzip (or gzip -d) |
.xz / .bz2 | ONE file compressed with xz / bzip2 | unxz / bunzip2 |
.tar.gz .tgz | a tar archive, then gzipped | tar -xf |
.tar.xz .tar.bz2 | a tar archive, then xz / bzip2 | tar -xf |
.zip | archive AND compression in one (Windows' favourite) | unzip |
Unix tools each do one job, so tar makes the archive and a compressor squeezes it. Modern tar -xf works out the compression by itself when extracting.
Compressing single files
gzip -k big.log # big.log.gz, and -k keeps the original xz -k big.log # big.log.xz: smaller, but slower to make ls -l big.log* # compare the sizes zcat big.log.gz | head # read it without unpacking (xzcat, bzcat for the others) gunzip big.log.gz # back to big.log
| Tool | Speed | Size | Where you meet it |
|---|---|---|---|
| gzip | fast | good | everywhere: web servers, log rotation, .tar.gz |
| bzip2 | slow | better | older downloads |
| xz | slowest to pack, quick to unpack | best | kernel sources, packages, software releases |
Text (logs, CSV, code) often shrinks to a tenth of its size. Photos, videos and anything already compressed barely shrink at all. Zipping a JPEG is a waste of time.
tar: the archiver
tar -czf backup.tar.gz Documents # c = create, z = gzip, f = file name (always last before the name!) tar -cJf backup.tar.xz Documents # J = xz (j = bzip2) tar -tf backup.tar.gz # t = list what's inside, without unpacking tar -xf backup.tar.gz # x = extract, here tar -xf backup.tar.gz -C /tmp/restore # …into another folder tar -caf site.tar.xz site/ # a = pick the compression from the name
Add v (verbose) to see file names fly by: tar -xvf. The Backups lesson uses tar for real backups: Linux Sysadmin, lesson 8.
A well-made archive has one folder at the top. A tarbomb has dozens of loose files that explode into whatever folder you're in. Always list first: tar -tf x.tar.gz | head. If there's no single top folder, make one and extract into it: mkdir x && tar -xf x.tar.gz -C x. Same for zip: unzip -l first, then unzip file.zip -d folder.
zip for everyone else
zip -r homework.zip Documents # -r = include folders' contents unzip -l website.zip # list unzip website.zip -d site # extract into a folder
Minimal installs have gzip, xz and tar, but not zip, unzip or bzip2: sudo dnf install zip unzip bzip2.
gzip, xz, bzip2 and tar are there. sudo apt install zip unzip for zip files.
What is this file, really?
file downloads/*
downloads/mystery: gzip compressed data, was "treasure.txt", from Unix downloads/website.zip: Zip archive data, at least v2.0 to extract downloads/logs.tar.xz: XZ compressed data, checksum CRC64
Linux doesn't care about extensions. file reads the first bytes (the "magic number") and tells you what's really inside, even when the name lies or is missing.
Practice: unpack the downloads folder 📦
~/downloads has a zip, some tarballs, a gzipped note, a file with no extension at all, and one archive that's a trap. There's also a big log in your home folder to squeeze.
Quick check
1. What's inside photos.tar.gz?
✓ Read the extensions right to left: gzip on the outside, tar inside.
2. Before extracting an archive from the internet, what do you run?
✓ Look first, then extract into a folder of its own if needed.
3. You gzip a folder of JPEG photos. Why is the result almost the same size?
✓ Compressing compressed data gains almost nothing. Just tar them.