Benchmarking Webp format (WIP)

Introduction

Webp is a image format created by Google in early 2010's, in this article, we will benchmark it against others lossless formats like avif, png, jpegxl.

The benchmark is done using the PNG format as input, and the output formats are Webp, Avif, and JpegXL.

Setup

Lossless compression

The dataset for the lossless benchmark is:

• SCD dataset (7597 PNG images (768x768), 4.2 GB)
• RSNA Breast Cancer dataset (547000 PNG images (1024x1024), 13.62 GB)
• CIFAR-10 dataset (60000 PNG images (32x32), 135 MB)
• Fashion MNIST dataset (70000 PNG images (28x28), 36 MB)

The PNG to Webp lossless compression command is:

cwebp -quiet -metadata all -lossless -exact -z <Compression> <Input> -o <Output>

The options are:

• -quiet: Quiet mode.
• -metadata all: Copy all metadata.
• -lossless: Lossless compression.
• -exact: Use exact mode, keep invisible pixels.
• -z <Compression>: Compression level, from 0 to 9, 9 best compression.
• -mt: Use multi-threading (we don't use it for the benchmark, gnu parallel is used).
• <Input>: Input PNG file.
• <Output>: Output Webp file.

For PNG to Avif lossless compression:

avifenc --jobs 1 --lossless --speed <Compression> --codec <codec> <Input> --output <Output>

The options are:

• --lossless: Lossless compression.
• --speed <Compression>: Compression level, from 0 to 9, 9 best compression.
• --jobs 1: Number of jobs.
• --codec: Use the aom codec, only codec available for lossless compression (11-2024).
• <Input>: Input PNG file.
• <Output>: Output Avif file.

For PNG to JPGXL lossless compression:

cjxl --num_threads 1 --quiet --effort <Compression> --distance 0.0 --brotli_effort 11 <Input> <Output>

The options are:

• --quiet: Quiet mode.
• --num_threads 1: Number of threads.
• --effort <Compression>: Compression level, from 0 to 9, 9 best compression.
• --distance 0.0: The quality, 0.0 is "mathematically lossless".
• --brotli_effort 11: Brotli compression level, 11 best compression.
• <Input>: Input PNG file.
• <Output>: Output JPGXL file.

For the final command, we use GNU Parallel to speed up the process (for Webp):

time find . -iname "*.png" -type f -print0 | parallel --jobs 16 --null cwebp -quiet -metadata all -lossless -exact -z 9 {} -o {.}.webp

The CMD to get the size of the output files is:

find . -type f -iname "*.webp" -printf "%s\n" | awk '{sum+=$1} END {printf "%.2f MiB\n", sum/1024/1024}'

Where {1} is the compression level from 0 to 9.

Results

For Ciphar-10:

Cleanup

find . -type f -name "*.webp" -delete

Sources

• cWebp
• Avif
• JpegXL