Archiving and Distribution of 2-D Geophysical Data Using Image Formats With Lossless Compression
Abstract
Certain types of two-dimensional (2-D) numerical remote sensing data can be losslessly and compactly compressed for archiving and distribution using standardized image formats. One common method for archiving and distributing data involves compressing data files using file compression utilities such as gzip and bzip2, which are widely available on UNIX and Linux operating systems. GZIP-compressed files and bzip2-compressed files must first be uncompressed before they can be read by a scientific application (e.g., MATLAB, IDL). Data stored using an image format, on the other hand, can be read directly by a scientific application supporting that format and, therefore, can be stored in compressed form, saving disk space. Moreover, wide use of image formats by data providers and wide support by scientific applications can reduce the need for providers of geophysical data to develop and maintain software customized for each type of dataset and reduce the need for users to develop and maintain or download and install such software. This letter demonstrates the utility of standardized image formats for losslessly compressing, archiving, and distributing 2-D geophysical data by comparing them with the traditional file compression utilities gzip and bzip2 on several types of remote sensing data. The formats studied include TIFF, PNG, lossless JPEG, JPEG-LS, and JPEG2000. PNG and TIFF are widely supported. JPEG2000 and JPEG-LS could become widely supported in the future. It is demonstrated that when the appropriate image format is selected, the compression ratios can be comparable to or better than those resulting from the use of file compression utilities. In particular, PNG, JPEG-LS, and JPEG2000 show promise for the types of data studied. Index Terms—Data archiving and distribution, data compression, image compression, image formats, JPEG-LS, JPEG2000, lossless compression, portable network graphics (PNG).
I. INTRODUCTION
DATA compression has become a very important issue in remote sensing with the growth in the amount of data being collected by satellites and in the amount of data being archived and distributed electronically. For example, the dataflow from the National Aeronautics and Space Administration (NASA) Aqua satellite can reach 89 GB/day [8]. Limits in bandwidth between satellites and ground-based processing and archiving stations, and between the data archive servers and potential users, can necessitate compression. A lot of remote sensing data is two-dimensional (2-D) (e.g., surface elevation and radiometric measurements) and can be Manuscript received July 28, 2004; revised Ocotber 21, 2004. Digital Object Identifier 10.1109/LGRS.2004.841422 treated as images, which raises the possibility of compressing and archiving such data by storing them using standardized image formats such as portable network graphics (PNG), JPEG, and JPEG2000. One common alternative method for archiving and distributing data involves compressing data using file compression utilities such as gzip and bzip2, which are widely available on UNIX and Linux operating systems. Files produced by gzip and bzip2 first have to be uncompressed to produce another file, which is then read by a scientific application. In contrast, data in image files can be read directly by scientific applications that support their formats. Such files can remain in compressed form, saving disk space. Using standardized images formats to archive and distribute data can also reduce the need for providers of data to develop and maintain specialized software to read the data and reduce the need for users to develop and maintain, or download and install, such software. The PNG, tagged image file format (TIFF), and JPEG image formats are supported by widely used scientific applications such as MATLAB and IDL. JPEG2000 and JPEG-LS could become widely supported in the future. This letter demonstrates the utility of standardized image formats for losslessly compressing, archiving, and distributing 2-D geophysical data by comparing them with the traditional file compression utilities gzip and bzip2 on several types of remote sensing data. The compression used in gzip is based on LZ77 compression [12], [20], while bzip2 relies on the Burrows–Wheeler transform, a block sorting transform [2], [4]. bzip2 is known to compress text files more compactly than gzip [7]. In general, lossy compression produces smaller files than lossless compression. However, lossless compression could be necessary if the type of information loss might obscure or distort information features in data. On the other hand, certain types of information loss could be tolerated. Cabrera-Mercader [3] developed a lossy hyperspectral data compression algorithm that estimates the ideal noiseless spectrum and then compresses only that part of the spectrum. This letter addresses only lossless compression.

Download full report
http://googleurl?sa=t&source=web&cd=2&ve...381350.pdf&ei=GSNTTp-SBcXrrQfhpt2qDg&usg=AFQjCNEO4QkTVQajzKOSvlGQoBoj7eHBOA