[attachment=13879]
ABSTRACT
A spatial noise shaping (SNS) method based on human visual sensitivity is presented. The method exploits the capability of frequency domain linear prediction for spatial envelope retrieval. It effectively shapes (or hides) the noise of an image in areas which are not sensitive to human vision so that the resultingimage is more pleasant to human eyes. The noise comes from the processing of the image, and it can be either separable like the additive noise pattern in image watermarking or non separable like the quantization noise in image coding. An application of the algorithm is demonstrated in the paper by using it to enhance image coders. Images decoded from the SNS incorporated coders have superior perceived quality than those without using SNS.
I. INTRODUCTION
NOISE appearance has always been unwelcome in many speech, audio, and image applications. Many studies have been done in the field of noise reduction, ranging from conventional median or Wiener filter types of algorithms [1] to recent wavelet denoising techniques [2], [3]. Although these methods are able to eliminate or reduce the amount of noise, some useful information in the host signal may be damaged by them as well, and the damage is usually proportional to the amount of noise reduced. This tradeoff constitutes the major challenge for these methods, and limits their usage. Noise shaping techniques are another approach of removing perceptible noise, and have been widely employed in many applications, such as coding (compression) [4], data hiding, and watermarking [5]. Unlike noise reduction methods, the purpose of all these techniques is not to reduce the noise but rather to shape the structure of the noise so that it becomes less perceptible in the final signal. Most of them shape noise by altering its spectrum, and hence called spectral noise shaping methods. These methods may be called spectral noise shaping methods. Many of them are application specific, and not applicable to other methods. Recently, a novel temporal noise shaping (TNS) method [6] was proposed, which adapts the temporal structure of the quantization noise to that of the host signal, therefore the masking effects of the human auditory system can be exploited. As a result, this new approach effectively reduces the pre-echo problem caused by the spread of quantization noise in the time domain within a transform window. It has been shown that TNS has contributed to the high performance of MPEG advanced audio coder (AAC) [7]. A few attempts [8]-[10] were reported with some success in shaping quantization noise spatially for image compression. A larger part of the noise could be spatially distributed to the textured part of the image by using filtered noise distribution makes it difficult to control the noise distribution accurately. The problem appeared to be resolved by optimizing the perceptual weighted quantization noise feedback [9], [10]. In addition, all these methods [8]-[10] were specifically developed for transform image coders, especially distributed to the textured part of the image by using filtered quantization noise feedback. However, the lack of known relationship between filter coefficients and noise distribution makes it difficult to control the noise distribution accurately. The problem appeared to be resolved by optimizing the perceptual weighted quantization noise feedback [9], [10]. The algorithm was shown to be effective in reducing mosquito noise in a JPEG image. However, since it is an iterative method, the computationis expensive. In addition, all these methods [8]-[10] were specifically developed for transform image coders, especially JPEG, but not directly applicable to other image processing applications, such as watermarking. In this paper, a generic spatial noise shaping (SNS) method has been developed for images to hide processing related noise (e.g., quantization) in areas which are not sensitive to human visual perception. Similar to TNS, the algorithm shapes the spatial structure of the processing noise to directly comply with human visual sensitivity. SNS runs open-loop linear prediction (LP) in the frequency domain instead of in the time domain, as compared to the conventional LP used in speech and image processing. It is well known that the operation of the conventional LP is in the time domain and it captures and/or shapes the spectrum of the signal (or noise). Conversely, SNS operates in the frequency domain, and shapes the spatial structure of the signal (or noise). The underlying principle or techniques developed for the algorithm is generic and SNS is applicable to various noise shaping tasks. The application of SNS to image coders is demonstrated in this paper. SNS can be conveniently used as an embedded or ad-hoc process in an image coder, although an embedded system will certainly provide a better result. This predictive analysis/synthesis process over frequency possesses two important properties which result in a decoded image with superior quality compared to the one without SNS. The first property is that since the analysis filter of SNS preserves some desirable spatial structure of the image in the filter, such as edges, so that they will not be damaged by the image coder and will be restored to the decoded image at the synthesis stage ofNS. The other unique property of SNS is that it effectively adapts the spatial structure of the quantization noise to that of the image masking profile, therefore, allows more efficient use of masking. As a result, given the same compression ratio, images produced by the coder with SNS possesses a cleaner and sharper appearance than those produced by the coder without SNS. This paper is organized as follows. The duality between the spectral and temporal noise shaping using linear prediction is discussed in Section II. The spatial shaping of noise according to human visual sensitivity is presented in Section III. The application of SNS is demonstrated and discussed in Section IV. Finally, the conclusion is given in Section V.
II. SPECTRAL VERSUS
TEMPORAL NOISE SHAPING
It has been shown [6] that given a real signal, x(t) the square of its Hilbert envelope
e(t) = ^{jx©.X*(^-f)d^}(i)
If x(f) is the Fourier Transform of x(t) then X(f) is the Fourier Transform of its analytic signal i.e., X(f) is a single sided spectrum defined as
{ 0, f<0
X(f) = { X(f) , f=0 (2)
{ 2X(f) , f>0
Equation (1) shows that the signal envelope directly relates to the autocorrelation function of its single sided spectrum, X(f). This relationship is the dual to the following well-known formula which relates the power spectrum density of a signal, Sxx(f) to its autocorrelation function in the time domain
Sxx(f)= ^(Ix(x). x*(x-t)dx} (3)
By taking advantage of this duality, some
well-established theories in time domain
linear prediction (LP) can be applied to the
frequency domain case. One of them is that
linear prediction coefficients (LPC) of the
time signal x(t) provide a good estimate of
its power spectrum, Sxx(f). As shown at the
topof Fig. 1, the time domain LP synthesis
filter which is an IIRfilter using LPC as the
filter taps captures the spectrum structure of
the host signal x(t). A nice property
of this filter is thatit can adapt the spectral structure of the processing noise, n(t) to that of the host signal, which is shown in the bottom of Fig. 1. Based on the duality of (1) and (3), the following deduction canbe made: If we apply the frequency domain linear prediction on the coefficients of X(f) which is the single sided spectrum of theAll opera-tions of the time domain LP, Fig. 1, are implemented in the time domain, but the noise shaping property is happening in the fre-quency domain. Conversely, all the operations of the frequency domain LP, Fig. 2, are carried out in the frequency domain,time signal x(t) the resulting frequency domain LPC will provide a good estimate of the envelope of the time signal x(t) This is shown at the top of Fig. 2. As shown at the bottom ofFig. 2, the frequency domain LP synthesis filter can adapt thetime structure of the processing noise to that of the host signal x(t) Note that Figs. 1 and 2 are duals to each other.