WebP Compression Study

Introduction

We do a comparative evaluation of WebP and JPEG image compression based on the new encoder that we released in libwebp. The goal is to run a set of easily repeatable experiments using publicly available tools and compare the image compression achieved by WebP and JPEG.

Our earlier comparative study of WebP focused on evaluating a large number of images and was conducted on JPEG sources that were crawled from the web. JPEG is a lossy compression format and introduces compression artifacts to the source. In addition, we used initially the Peak Signal-to-Noise Ratio or PSNR as a quality metric. PSNR has some downsides when used as a quality metric of images. Based on feedback from the community, we conduct the experiments on PNG images including images available in a few standard test suites. We use in this study Structural Similarity or SSIM as quality metric for this study. SSIM is considered to be a better visual quality metric.

Methodology

We describe two types of evaluations. In the first case, we study the additional compression achieved by WebP at the same quality level of JPEG. In particular, we generate WebP images of same quality (as per SSIM index) as the JPEG images and then compare the file sizes of WebP and JPEG images. In the second case, we analyze SSIM vs bits per pixel (bpp) plots for WebP and JPEG. These plots show the rate-distortion trade off for WebP and JPEG.

The data sets for the experiments consist of PNG images. These images were obtained from publicly accessible standard test image suites as well as a large collection of PNG images that were crawled from the web.

Each experiment consists of running a set of commands. Only publicly available command line tools are used to keep the experiments reasonably reproductible for others. The command line tools used for this experiment are:

  • identify and convert are command-line tools that are part of ImageMagick. We used version 6.5.7 for this analysis. We used convert to convert between PNG and PPM formats. We used identify to extract the width and height information of an image.

  • cwebp and dwebp are command-line tools that are part of the libwebp library we released in Q1 2011. We used version 0.1.2 of the library. We used cwebp and dwebp to encode and decode the WebP images. We used the default encoding and decoding parameters of these tools.

  • cjpeg and djpeg are command-line tools that are part of the libjpeg library. We used version 6b of the library. We used cjpeg and djpeg to encode and decode JPEG image. We used cjpeg with the -optimize flag to optimize for file size. We set all other parameters to their default values.

The quality of an image was measured using SSIM. As previously noted, SSIM is generally accepted as a better quality metric than PSNR. For the SSIM index computation, we used a publicly available implementation in C++. The reader may download the Python scripts we used in the experiments from the code repository for reference.

Image Data Sets

We chose the data sets to evaluate standard test image collections as well as on a large set of images. The following are the list of data sets used in the experiments.

  1. Lenna: widely used Lenna image (512 x 512 pixels).

  2. Kodak: 24 images from the Kodak true color image suite.

  3. Tecnick: 100 images from the collection available at Tecnick.com. We used the 100 original size RGB color images.

  4. Image_crawl: To test a large number of images, we collected a random sample of PNG images from Google Image Search web crawl database. Most of the crawled PNG images are icons, graphics, charts, scanned documents, etc. However most images in the standard test collections are like photographs, rather than computer generated images. To make this dataset similar in nature to the standard test suites, we ran a face detection algorithm over these PNG images and considered only those images (around 11,000) for this experiment, that passed this detection test.

Experiment 1: WebP vs. JPEG at Equal SSIM Index

In this experiment we generate JPEG and WebP images at comparable SSIM values and measure the ratio of file sizes. The steps of the experiment are as follows:

  1. Compress the source PNG image to JPEG with a fixed quality parameter Q and record the SSIM between the source PNG image and the compressed JPEG image.

  2. Compress the same source image to WebP with a quality parameter that achieves the above recorded SSIM as closely as possible.

We ran the above two steps on all the images in a dataset and then measured the average file size and SSIM for JPEG and WebP images.

The following tables show the results of this experiment for three different values (50, 75 and 95) of JPEG quality parameter Q. Quality parameters 50 and 95 represent the image compression at low and high bit-rates respectively. We have also included JPEG quality parameter 75 since it is the generally recommend quality level for JPEG images.

Table 1: Average file size for WebP and JPEG for the same SSIM index corresponding to JPEG Q=50

Lenna Kodak Tecnik Image_crawl
WebP: Average File Size
(Average SSIM)		 17.4 KB
(0.841)		 31.0 KB
(0.898)		 92.4 KB
(0.917)		 6.5 KB
(0.901)
JPEG: Average File Size
(Average SSIM)		 23.5 KB
(0.840)		 42.7 KB
(0.897)		 124.6 KB
(0.916)		 9.9 KB
(0.899)
Ratio of WebP to JPEG file size 0.74 0.72 0.74 0.66

Table 2: Average file size for WebP and JPEG for the same SSIM index corresponding to JPEG Q=75

Lenna Kodak Tecnik Image_crawl
WebP: Average File Size
(Average SSIM)		 26.7 KB
(0.864)		 46.5 KB
(0.932)		 139.0 KB
(0.939)		 9.9 KB
(0.930)
JPEG: Average File Size
(Average SSIM)		 37.0 KB
(0.863)		 66.0 KB
(0.931)		 191.0 KB
(0.938)		 14.4 KB
(0.929)
Ratio of WebP to JPEG file size 0.72 0.70 0.73 0.69

Table 3: Average file size for WebP and JPEG for same SSIM index corresponding to JPEG Q=95

Lenna Kodak Tecnik Image_crawl
WebP: Average File Size
(Average SSIM)		 74.3 KB
(0.910)		 118.0 KB
(0.978)		 356.0 KB
(0.970)		 23.4 KB
(0.969)
JPEG: Average File Size
(Average SSIM)		 104 KB
(0.907)		 162 KB
(0.976)		 492 KB
(0.970)		 31.3 KB
(0.968)
Ratio of WebP to JPEG file size 0.71 0.73 0.72 0.75

From the tables above, we can observe that WebP gives additional 25%-34% compression gains compared to JPEG at equal or slightly better SSIM index.

Experiment 2: SSIM vs. BPP Plots for WebP and JPEG

We study the rate-distortion trade-off of JPEG and WebP. In particular we study the SSIM vs. bits per pixel (bpp) plots for JPEG and WebP. We take the source PNG image, compress it to JPEG and WebP using all possible (0-100) quality values. Then for each quality value we plot the SSIM and bpp achieved for JPEG and WebP. Following figures show such SSIM vs bpp plots for the 3 images chosen from the 3 public data sets we have used.

Figure 1: SSIM vs. BPP for Lenna

ssim vs. bpp for Lenna

Figure 2: SSIM vs. BPP for kodim19.png from the Kodak dataset

ssim vs. bpp for Kodim19

Figure 3: SSIM vs. BPP for RGB_OR_1200x1200_061.png from the Tecnick dataset

ssim vs. bpp for RGB_OR_1200x1200

Additionally, we studied the average behavior of SSIM vs bpp plot for the Kodak and Tecnick data sets. To plot the average behavior, we compress all the images in a dataset using WebP and JPEG at 100 different quality values. We then plot the average of SSIM and bpp for each quality value. The image crawl data set consists of heterogeneous images that don't lend themselves to this data aggregation. The following plots show the average behavior of SSIM and bpp for the Kodak and Tecnick data sets respectively.

Figure 4: SSIM vs. BPP for the Kodak dataset

ssim vs. bpp for Kodak

Figure 5: SSIM vs. BPP for the Tecnick dataset

ssim vs. bpp for Tecnick dataset

Overall, from the above plots we can observe that WebP consistently requires less bits per pixel than JPEG to achieve the same SSIM index.

Conclusion

The study evaluated WebP compression in comparison to JPEG. We observed that the average WebP file size is 25%-34% smaller compared to JPEG file size at equivalent SSIM index. The SSIM vs bpp plots showed that WebP consistently required less bits per pixel than JPEG for the same SSIM index. These results indicate that WebP can provide significant compression improvements over JPEG.

1 Huynh-Thu, Q.; Ghanbari, M. (2008). "Scope of validity of PSNR in image/video quality assessment". Electronics Letters 44: 800-801.doi:10.1049/el:20080522.

2 Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error measurement to structural similarity," Image Processing, IEEE Transactions, Volume 13, Issue 4, pp. 600-612, 2004.