1 state of-the-art and trends in scalable video

1238 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 17, NO. 9, SEPTEMBER 2007
State-of-the-Art and Trends in Scalable Video
Compression With Wavelet-Based Approaches
Nicola Adami, Member, IEEE, Alberto Signoroni, Member, IEEE, and Riccardo Leonardi, Member, IEEE
(Invited Paper)
Abstract—Scalable Video Coding (SVC) differs form traditional
single point approaches mainly because it allows to encode in
a unique bit stream several working points corresponding to
different quality, picture size and frame rate. This work describes
the current state-of-the-art in SVC, focusing on wavelet based
motion-compensated approaches (WSVC). It reviews individual
components that have been designed to address the problem over
the years and how such components are typically combined to
achieve meaningful WSVC architectures. Coding schemes which
mainly differ from the space-time order in which the wavelet
transforms operate are here compared, discussing strengths and
weaknesses of the resulting implementations. An evaluation of
the achievable coding performances is provided considering the
reference architectures studied and developed by ISO/MPEG
in its exploration on WSVC. The paper also attempts to draw
a list of major differences between wavelet based solutions and
the SVC standard jointly targeted by ITU and ISO/MPEG. A
major emphasis is devoted to a promising WSVC solution, named
STP-tool, which presents architectural similarities with respect
to the SVC standard. The paper ends drawing some evolution
trends for WSVC systems and giving insights on video coding
applications which could benefit by a wavelet based approach.
Index Terms—Entropy coding, motion compensated tem-
poral filtering (MCTF), MPEG, Scalable Video Coding (SVC),
spatio–temporal multiresolution representations, video coding
architectures, video quality assessment, wavelets.
I. INTRODUCTION
TRADITIONAL single operating point video coding sys-
tems can be surpassed by using Scalable Video Coding
(SVC) architectures where, from a single bit stream, a number
of decodable streams can be extracted corresponding to various
operating points in terms of spatial resolution, temporal frame
rate or reconstruction accuracy. Different scalablity features can
coexist in a single video coded bit stream with coding perfor-
mance approaching the state-of-art single point coding tech-
niques. This has been more and more a trend in the last years
[1] and it has become a reality thanks to the development of
Manuscript received October 9, 2006; revised June 22, 2007. This paper was
recommended by Guest Editor T. Wiegand.
The authors are with the Department of Electronics for Automation, Faculty
of Engineering, University of Brescia, I-25123 Brescia, Italy (e-mail: alberto.
signoroni@ing.unibs.it).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TCSVT.2007.906828
SVC systems derived either from hybrid schemes [2] (used in all
MPEG-x or H.26x video coding standards) or from spatio–tem-
poral wavelet technologies [3]. A great part of this development
and relative debates regarding SVC requirements [4], applica-
tions and solutions has been carried out by people participating
in the ISO/MPEG standardization.
This paper offers an overview of existing SVC architectures
which are based on multiresolution spatio–temporal represen-
tation of video sequences. In particular, wavelet-based SVC
(WSVC) tools and systems will be classified and analyzed
with the following objectives: 1) to give a quite comprehen-
sive tutorial reference to those interested in the field; 2) to
analyze strong and weak points of the main WSVC tools and
architectures; 3) to synthesize and account for the exploration
activities made by the MPEG video group on WSVC and to
describe the issued reference platform; 4) to compare such
platform with the ongoing SVC standard (jointly targeted by
ITU and ISO/MPEG) in qualitative and quantitative terms; and
5) to discuss promising evolution paths and target applications
for WSVC. For this purpose the presentation has been struc-
tured as follows. In Section II, WSVC fundamentals as well
as basic and advanced tools which enable temporal, spatial
and quality salability are presented and discussed. Section III
shows how such components are typically combined to achieve
meaningful WSVC architectures, which typically differ from
the space-time order in which the wavelet transform operates,
discussing strengths and weaknesses of the resulting imple-
mentations. In the same section, an emphasis is placed on a
promising architecture which presents some similarities to the
SVC standard. Subsequently, Section IV explains the Wavelet
Video Reference architecture(s) studied by ISO/MPEG in its
exploration on Wavelet Video Compression. The paper attempts
as well (Section V) to draw a list of major differences between
such architecture(s) and tools with respect to the SVC reference
model, providing performance comparison in terms of coding
efficiency and giving a critical analysis of the presented results.
In Section VI, further investigations on WSVC critical aspects
are presented. A discussion on perspectives and applications
that appear better aimed to wavelet based approach is made
in Section VII. Finally, conclusions are derived. Note that,
for consistency with the other papers of this special issue,
the acronym SVC will be used without distinction to indicate
generic SVC systems and the SVC standard. The appropriate
meaning will emerge from the context.
1051-8215/$25.00 © 2007 IEEE

ADAMI et al.: STATE-OF-THE-ART AND TRENDS IN SCALABLE VIDEO COMPRESSION WITH WAVELET-BASED APPROACHES 1239
Fig. 1. SVC rationale. A unique bit stream is produced by the Coder. Decod-
able substreams can be obtained with simple extractions.
II. BASIC PRINCIPLES OF WAVELET-BASED
APPROACHES TO SVC
A. WSVC Fundamentals
Fig. 1, shows a typical SVC system which refers to the coding
of a video signal at an original CIF resolution and a framerate of
30 fps. In the example, the highest operating point and decoding
quality level correspond to a bit rate of 2 Mbps associated with
the data at the original spatio–temporal resolution. For a scaled
decoding, in terms of spatial, temporal and/or quality resolution,
the decoder only works on a portion of the originally coded bit
stream according to the specification of a desired working point.
Such a stream portion is extracted from the originally coded
stream by a functional block called extractor. As shown in Fig. 1
the extractor is arranged between the coder and the decoder. In
any practical application, it can be implemented as an indepen-
dent block within any node of the distribution chain or it can be
included in the decoder. The extractor receives the information
related to the desired working point—in the example of Fig. 1,
a lower spatial resolution QCIF, a lower frame rate (15 fps), and
a lower bit rate (quality) (150 kbps)— and extracts a decodable
bit stream matching or almost matching the working point spec-
ification. One of the main differences between an SVC system
and a transcoding solution is the low complexity of the ex-
tractor which does not require coding/decoding operations and
typically consists of simple parsing operations on the coded bit
stream. For practical purposes, let us consider the following ex-
ample where a 4CIF video is coded allowing to extract three
different spatial scalability layers and four embedded temporal
resolutions. In this case, a possible structure of the bit stream
generated by an SVC system is shown in Fig. 2. As indicated,
the usual partition in independently coded groups of pictures
(GOP) can be adopted for separating the various video frames.
The GOP header contains the fundamental characteristics of the
video and the pointers to the beginning of each independent
GOP bit stream. In this example, each GOP stream contains
three bit stream segments, one for each spatial level. Note that in
an ideal scalable bit stream, except for the lowest working point,
all information is incremental. This means, for example, that
in order to properly decode higher resolution (e.g., 4CIF), one
needs to use also the portions of the bit stream which represent
the lower spatial resolutions (e.g., QCIF and CIF). Given this
bit stream organization, for each spatial resolution level a tem-
poral segmentation of the bit stream can be further performed.
Accordingly, the 7.5 fps portion contains data that allow to re-
construct the lowest temporal resolution of the considered spa-
tial resolution reference video sequence. Similarly the 15, 30,
60 fps parts refer to information used to refine the frame-rate
until the allowed maximum. Since most successful video com-
pression schemes require the use of motion compensation (MC),
each temporal resolution level bit stream portion should include
the relative motion information (usually consisting of coded mo-
tion vector fields), as shown in Fig. 2. Further quality layers
can be obtained, for example, by arranging the signal repre-
sentation, for every temporal segment, in a real or virtual bit
plane progressive fashion (see Section II-B.3). With such a bit
stream, every allowed combination in terms of spatial, temporal
and quality scalability can be generally decided and achieved
according to the decoding capabilities. Fig. 3 provides a graph-
ical representation of a target bit rate extraction of a CIF 15 fps
coded sequence, from the original 4CIF 60 fps bit stream. Note
that corresponding parts of QCIF and CIF bit streams have been
extracted to reconstruct the sequence at the desired working
point.
A suitable property for concrete applications (e.g., for con-
tent distribution on heterogeneous networks) is the possibility
to adapt the bit stream according to a multiple (chained) ex-
traction path, from the highest towards lower operating points;
this has been called bit stream multiple adaptation [5]. Multiple
adaptations should be allowed, of course, along every advisable
path, and in general without any a-priori encoding settings. Mul-
tiple adaptation extraction differs from a single extraction only
in one aspect; for multiple adaptations the bit stream extraction
information must be conserved (i.e., processed and reinserted
throwing out what become useless) from the input bit stream.
Clearly, there is no need to reinsert (and spend bits for) the ex-
traction information if no further extraction is expected.
B. Tools Enabling Scalability
In the past two decades, there has been a proliferation of
activities focalized on theory and solutions for efficient (min-
imally redundant) multiresolution representation of signals. In
general, a multiresolution signal decomposition (transform) in-
herently enables a corresponding low to high resolution scal-
ability by the representation of the signal in the transformed
domain. The tools that enable scalable multidimensional (space-
time-quality) signal representation and coding are multidisci-
plinary. This fact is clearly depicted in Fig. 4 that gives a quick
overview of these tools and how they can be combined in order
to achieve the desired scalability feature: spatial, temporal, and
quality (S, T, and Q). In the following, each scalability dimen-
sion, S, T, and Q, will be considered and the most popular and ef-
fective solutions will be briefly described. In this context, Fig. 4
can be helpful in keeping the big picture when analyzing a spe-
cific tool. Due to lack of space to describe each tool in detail,
support references will be provided for the interested reader.
1) Spatial Scalability: Tools producing a multiresolution
representation of -dimensional signals can be of different
types. They can be linear or nonlinear, separable or not, re-
dundant or critically sampled. Linear decompositions have

Fig. 2. Example of a scalable bit stream structure.
Fig. 3. Example of a substream extraction: CIF at 15 fps is extracted from a 4CIF sequence at 60 fps.
Fig. 4. WSVC theoretical context.
been studied for a long time. The simplest mechanism to
perform a two resolution representation of a signal is based
on what here we call inter-scale prediction (ISP): an original
(full resolution) signal can be seen as a coarse resolution
signal , obtained by a decimation of (e.g., by filtering with
a lowpass filter and downsampling of factor 2), added to a
detail signal calculated as the difference of the original signal
and an interpolated version of (throughout
this document a symbol over a signal means that the detail
is not critically sampled). The Laplacian pyramid introduced
by Burt and Adelson [6] is an iterated version of such an ISP
mechanism resulting in a coarsest resolution signal and a
set of details at decreasing levels of spatial resolution
. Fig. 5 shows a Laplacian decomposition of the
1-D signal in -stages (from full resolution 0 to the coarsest
one ) where is implemented by upsampling by a factor
of 2 and interpolating with . The discrete wavelet transform
(DWT) is a linear operator which projects the original signal
in a set of multiresolution subspaces [7] allowing a critically
sampled (orthogonal or biorthogonal) representation of the
signal in the transformed domain and guaranteeing perfect
reconstruction synthesis. Similarly to the Laplacian pyramid,
the DWT decomposition of generates a coarse signal and
a series of detail signals at various levels of spatial reso-
lutions. By using the terminology of multirate filter banks [8],
the transformed domain signals are also called subbands. In

Fig. 5. n-level Laplacian pyramid analysis filter bank.
Fig. 6. n-level (bi)orthogonal DWT analysis/synthesis implemented with a
dyadic tree filter bank.
fact, the DWT can be implemented by means of a two-channel
filter bank, iterated on a dyadic tree path, as shown in Fig. 6,
[9], [10]. Symbols and represent respectively the lowpass
and highpass analysis filters, whereas and are the synthesis
for an orthogonal or biorthogonal wavelet decomposition.
Symmetric filter responses are preferred for visual data coding
applications due to their linear phase characteristics and can
be obtained only with biorthogonal decompositions (except for
length 2 filters, i.e., Haar basis). A popular filter pair for image
coding purposes is the one [10, p. 279], which gives an
analysis filter with 9 taps and a synthesis filter with 7 taps.
For multidimensional signals, e.g., images, separate filtering
on rows and columns are usually adopted to implement the so
called separable pyramidal and DWT decompositions. In the
case of separable 2-D-DWT on images this generates, at each
level of decomposition, one coarse subband , and three detail
subband that we continue to indicate altogether as .
Sweldens [11] introduced the lifting scheme as an alternative
spatial domain processing to construct multiresolution signal
representations. The lifting structure is depicted in Fig. 7 and re-
veals a quite intuitive spatial domain processing of the original
signal capable to generate a critically sampled represen-
tation of signal . According to this scheme, signal is split in
two polyphase components: the even and the odd samples
and . As the two components (each one is half the original
resolution) are correlated, a prediction can be performed be-
tween the two; odd samples can be predicted with an operator
which uses a neighborhood of even samples and that produces
the residue
(1)
The transform is not satisfactory from a multires-
olution representation point of view as the subsampled signal
could contain a lot of aliased components. As such, it should
be updated to a filtered version . The update step
(2)
Fig. 7. Dyadic decomposition by lifting steps: split, prediction P, and update
U.
can be constrained to do this job and it is structurally invertible
as well as the preceding stages. Perfect reconstruction is simply
guaranteed by
(3)
(4)
Daubechies and Sweldens [12] proved that every DWT can be
factorized in a chain of lifting steps.
The lifting structure is not limited to be an alternative and
computationally convenient form to implement a DWT trans-
form. Actually, it also allows nonlinear multiresolution decom-
positions. For example, lifting steps implementing a DWT can
be slightly modified by means of rounding operations in order
to perform a so called integer to integer (nonlinear) wavelet
transform [13]. Such a transform is useful as a component of
wavelet-based lossless coding systems. The kind and the degree
of introduced nonlinearity can depend on the structural or mor-
phological characteristics of the signal that one may want to cap-
ture [14], keeping in mind that nonlinear systems can have unex-
pected behaviors in terms of error propagation (which may turn
out to be a burden in presence of quantization of the transformed
signal). Moreover, the lifting structure has a fundamental role
for MC temporal filtering (MCTF) as we will see in the next
subsection.
2) Temporal Scalability: A key tool which enables temporal
scalability while exploiting temporal correlation is the MCTF.
An introduction to the subject can be found in [15]. After the
first work of Ohm proposing a motion compensated version of
the Haar transform [16], other studies [17], [18] began to show
that video coding systems exploiting this tool could be com-
petitive with respect to the classical and most successful hybrid
ones (based on a combination of block-based spatial DCT trans-
form coding and block-based temporal motion estimation and
compensation). Other works (e.g., [19] and [20]) focus on the
study of advanced MC lifting structures due to their versatility
and strength in increasing the coding performance. Basically,
an MCTF implementation by lifting steps can be represented
by the already seen (1)–(4) where index has now a temporal
meaning and where the prediction and update operators and
can be guided by motion information generated and coded after
a motion estimation (ME) stage becoming MCP and MPU oper-
ators respectively. Fig. 8 shows a conceptual block diagram of
this general idea. ME/MC are implemented according to a cer-
tain motion model and in the case of spatio–temporal multires-
olution SVC and WSVC systems they usually generate a set of
motion descriptions consisting of motion vector fields .
These are estimations of the trajectory of groups (usually rect-
angular blocks) of pixels between the temporal frames, at spatial

Fig. 8. Prediction and update lifting steps with explicit motion information
conveyed to the P and U operators.
level , involved in the th MCTF temporal decomposition level.
Haar filters based [16], [17], [21] and longer filters MCTF so-
lutions have been proposed [22]–[24]. For example MCTF im-
plementations of the transform exist [20]. and in gen-
eral longer filter solutions usually achieve better performance in
terms of prediction efficiency and visual quality at lower tem-
poral resolution frames, but make use of a bidirectional predic-
tion and thus motion information is more bitdemanding than
Haar based solutions. The two solutions recall unidirectional
and bidirectional block mode prediction used in hybrid schemes.
Similarly to advanced hybrid schemes, an adaptive selection
among the two modes is also possible in the case of MCTF,
leading to block-based adaptive MCTF solutions [25], [26]. Al-
ternatively, deformable motion models have also been explored
with promising results [27].
By exploiting the local adaptability of the MCP and MCU
operators and thanks to the transmitted (usually lossless coded)
information, the MCTF can also be aware of and
handle a series of issues affecting MC: 1) occlusion and uncov-
ered area problems can be handled recognizing unconnected
and multiple connected pixels and adapting the MCP and
MCU structure accordingly [22], [17], [21]; 2) in the case of
block-based ME/MC, blocking effects can be reduced in various
ways by considering adjacent blocks (spatio–temporal lifting)
realizing for example overlapped block-based MC (OBMC)
[21], [26]; and 3) when MVs are provided with fractional pel
precision the lifting structure can be modified to implement the
necessary pixel interpolations [22], [28], [20] while preserving
the temporal invertibility of the transform. It must also be noted
that MCU makes use of a reversed motion field with respect to
the one used by MCP. This may constitute a problem especially
in those areas where inversion cannot be performed exactly,
e.g., on occluded/uncovered areas, or does not fall on integer
pixel positions because fractional pel accuracy is involved. In
order not to duplicate the motion information and the associated
bitbudget, mechanisms to derive or predict MCU fields from
the MCP ones have been proposed [27], [29], [30] as well as
methods that identify inversion problems and adopt interpola-
tion based solutions (e.g., the Barbell lifting scheme [31]). With
lifting structures nondyadic temporal decomposition are also
possible [32], leading to temporal scalability factors different
from a power of two.
As already noticed, the update step could be omitted, without
impairing perfect reconstruction, generating a so called uncon-
strained MCTF (UMCTF) [33]. This guarantees the direct use
of the original frames for reduced frame-rates, then reduces the
complexity of the transform stage and presents the benefit of
eliminating ghosting artifacts. Ghosting artifacts can appear in
some regions of the lowpass temporal frames as a consequence
of unavoidable local MC failures (associated to scene changes,
unhandled object deformations, uncovered/occluded regions).
In such erroneously motion-compensated regions, the lowpass
temporal filtering updates the corresponding temporal subbands
with “ghosts” (which “should not exist” in the current frame)
coming from past or future frames. Depending on the entity of
the ghosts, this can locally impair the visual appearance. On the
other hand, for correctly motion compensated regions, the up-
date step has a beneficial temporal smoothing effect that can im-
prove visual perception at reduced frame-rates and also coding
performance. Reduction of ghosting artifacts is then an impor-
tant task and it has been obtained by the implementation of
MCTF adaptive update steps [34], [35] or by clipping the up-
date signal [36].
Either MCTF or UMCTF generate a temporal subband
hierarchy starting from higher temporal resolution to lower
ones. Alternatively, the hierarchical B-frames decomposition
(HBFD) [2] can be built enabling a similar temporal detail
structure but starting from the lower resolutions (predictions
between more distant frames). This allows closed-loop forms
of the temporal prediction [37]. Efficient MCTF implementa-
tions have also been studied to reduce system complexity [38],
memory usage [39], [23], and coding delay [40].
3) Quality Scalability: Among the best image compres-
sion schemes, wavelet-based ones currently provide for high
rate-distortion (R-D) performance while preserving a limited
computational complexity. They usually do not interfere with
spatial scalability requirements and allow a high degree of
quality scalability which, in many cases, consists in the possi-
bility of optimally truncating the coded bit stream at arbitrary
points (bit stream embedding). Most techniques that guarantee
an optimal bit stream embedding and consequently a bitwise
quality scalability are inspired from the zerotree idea first
introduced by Shapiro’s embedded zerotree wavelet (EZW)
technique [41] and then reformulated by Said and Pearlman
in their SPIHT algorithm [42]. A higher performance zerotree
inspired technique is the embedded zero-block coding (EZBC)
algorithm [43], where quad-tree partitioning and context mod-
eling of wavelet coefficients are well combined. The zerotree
idea can also be reformulated in a dual way, allowing to directly
build significant coefficients maps. To this end, a morphologic
processing based on connectivity analysis has been used and
also justified by the statistical evidence of the energy clustering
in the wavelet subbands [44]. The EMDC technique [45], [46]
exploits principles of morphological coding and guarantees,
for 2-D and 3-D data [47], coding performance comparable
or superior to state-of-the-art codecs, progressive decoding
and fractional bit plane embedding for a highly optimized bit
stream.
Another popular technique, which does not use the zerotree
hypothesis, is the embedded block coding with optimized trun-
cation (EBCOT) algorithm [48], adopted in the JPEG2000 stan-
dard [49], which combines layered block coding, fractional bit
planes [50], block-based R-D optimizations, and context-based
arithmetic coding to obtain good (adjustable) scalability prop-
erties and high coding efficiency.

Fig. 9. Basic WSVC schemes in a signal perspective. (a) t + 2D. (b) 2D + t.
Proposed techniques for coding coefficients of spatio–tem-
poral subbands generated by WSVC systems usually are
extensions of image coding techniques. Obviously such 3-D
extensions shall not interfere with temporal scalability require-
ments as well, and can be achieved by considering; 1) single
temporal frame coding with multiple frame statistical contexts
updating; 2) extended spatio–temporal nonsignificance (or
significance) prediction structures; and 3) spatio–temporal
statistical contexts, or a combination of them. Temporal ex-
tensions of SPIHT [18], EZBC [51], [52], EBCOT [36], [24],
and EMDC [53] have been proposed and used in WSVC video
coding systems.
III. CODING ARCHITECTURES FOR WSVC SYSTEMS
A. WSVC Notation
In a multidimensional multilevel spatio–temporal decom-
position there is a proliferation of signals and/or subbands.
To support our discussion we have adopted a notation that
guarantees easy referencing to the generated signals in a
general decomposition structure. We consider a generic
spatio–temporal signal , with spatial dimensions
. Signal which undergoes an -level mul-
tiresolution spatial transform is indicated with .
For nonredundant transforms, such as the DWT, the spa-
tially transformed signal actually consist of the subband set
, where superscript indi-
cates the signal representation at the coarsest spatial resolution
(lowpass) while, for each spatial level
indicates a critically sampled level of detail. In the case of
separable transforms (such as the DWT) each consists of
directional subbands. For redundant transforms (such
as the Laplacian pyramid) the transformed signal consists of
the signal set where
is an oversampled detail at level . With a similar notation a
spatio–temporal signal which undergoes a -level
multiresolution temporal transform is indicated as
. As before, superscripts and will be used
referring to the temporal dimension to indicate coarsest tem-
poral resolution, critically and not critically sampled temporal
details respectively. In the figures of the present section we will
graphically represent temporally transformed data subbands
by associating a gray level to them: the white corresponds to
while grey-tones get darker as the detail level goes from the
finest to the coarsest. Moreover, and indicate original and
quantized (reduced coefficient representation quality) signals
respectively. Symbol “ ” will be used as wild card to refer to
the whole set of spatio–temporal subbands of a given spatial or
temporal transform.
A decoded version of the original signal extracted and re-
constructed at given temporal resolution , spatial resolution
and reduced quality rate, will be indicated as .
Concerning ME/MC, given a certain motion model, a de-
scription is produced consisting of a set of motion vector fields
, where, and refer to various spatial and temporal
resolution levels respectively. As and are
obviously not independent, mechanisms of up- or down-conver-
sion, with or without refinement or approximation of the mo-
tion vector field, are usually implemented. Then, for example,
indicates that motion vectors at a
coarser resolution have been down-converted, by a suitable re-
duction of the block size and of the vector module, without ap-
proximation. Otherwise, in case of down-conversion with ap-
proximation the notation would have been used.
Moreover, indicates that motion
vectors at a second layer of finer resolution have been up-con-
verted twice with a refinement of the block partition and/or of
the motion vector value.
B. Basic WSVC Architectures
In Figs. 9 and 10, the main multiresolution decomposition
structures and corresponding WSVC architectures are shown.
Each one will be presented in the following paragraphs.
1) : Perhaps the most intuitive way to build a WSVC
system is to perform an MCTF in the original spatial domain
followed by a spatial transform on each temporal subband. This

Fig. 10. Pyramidal WSVC schemes with the pyramidal decomposition put (a) before and (b) after the temporal MCTF.
is usually denoted as a scheme, it guarantees critically
sampled subbands and it is represented in Fig. 9(a). Earlier
wavelet based coding systems were based on this scheme [16],
[17]. Many other wavelet based SVC systems are based on
the spatio–temporal decomposition, among which we
can cite [18]–[21], [31], [33], [51], [52], [54]–[58]. Despite
its conceptual simplicity, the solution presents some
relevant drawbacks especially for spatial scalability perfor-
mance. When only temporal and quality scalability (full spatial
resolution decoding) are required, the process is reversed
until reaching the desired frame-rate at resolution (partial
or complete MCTF inversion) and signal-to-noise ratio (SNR)
quality: where both direct
and inverse temporal transforms make use of the set of motion
vectors . Instead, if a lower spatial resolu-
tion version is needed the inversion process is incoherent with
respect to the forward decomposition. For clarity, let us con-
sider only spatial scalability, so that the spatially scaled signal
is obtained by . The problem
is due to the fact that the inverse MCTF transform is obtained
using motion vectors that are typically deduced from
the estimated ones by . 1) This cannot
guarantee perfect reconstruction because spatial and motion
compensated temporal transforms cannot be inverted without
most likely changing the result (due to the shift-variant na-
ture of the spatial transform) [15]. 2) These motion vectors
have not been estimated on data at the target resolution and
therefore they cannot be considered optimal for use, when
spatial subsampling has been made with nonalias free wavelet
ﬁlters. Moreover, the bit occupancy of full resolution motion
vectors penalizes the performance of spatially scaled decoding
especially at lower bit rates.

The above issues can only be mitigated in such a critically
sampled scheme but not structurally resolved. To re-
duce the MV bit load at full resolution scalable MV coding
can be used [59], [60], to reduce it at intermediate and lower
resolutions approximated MV down-conversions
can be adopted (e.g., from half-pel at spatial res-
olution again to half-pel at resolution instead of the
quarter-pel generated by the down-conversion). These two so-
lutions have the disadvantage of introducing some imprecisions
in the inverse temporal transform and their possible adoption
should be balanced in a R-D sense. The problem of the nonop-
timality of the down-conversion due to spatial aliasing, intro-
duced by the decimation, can be reduced by using (possibly
only for the needed resolutions) more selective wavelet filters
[61] or locally adaptive spectral shaping by acting on the quan-
tization parameters inside each spatial subband [62]. However,
such approaches can determine coding performance loss at full
resolution, because either the wavelet filters or coefficient quan-
tization laws are moved away from ideal conditions.
2) : In order to solve the spatial scalability perfor-
mance issues of schemes, a natural approach could
be to consider a scheme, where the spatial transform
is applied before the temporal one as shown in Fig. 9(b). The
resulting scheme is often called In-Band MCTF (IBMCTF)
[63] because estimation of is made and applied inde-
pendently on each spatial level in the spatial subband domain,
leading to a structurally scalable motion representation and
coding. With IBMCTF spatial and temporal scalability are
more decoupled with respect to the case. Signal recon-
struction after a combination of spatio–temporal and quality
scalability is given by
where the (partial) inverse temporal transform makes use of
, (a subset of) the same MVs
used for the direct decomposition.
Despite its conceptual simplicity the approach suf-
fers as well from the shift-variant (periodically shift-invariant)
nature of the DWT decomposition. In fact, ME in the critically
sampled detail subbands is not reliable because the coefficient
patterns of moving parts usually change from one frame to con-
secutive or preceding ones. Solutions to this issue have been
proposed [64], [65]; they foresee a frame expansion formula-
tion, at the expense of an increased complexity, to perform ME
and compensation in an overcomplete (shift-invariant) spatial
domain, while transmitting only decimated samples. Even so,
schemes demonstrated lower coding efficiency com-
pared to ones, especially at higher resolutions [66].
Motivations of this fact are probably due to the reduced effi-
ciency of MCTF when applied in the highpass spatial subband
domain. In addition, spatial inversion of detail subbands which
contain block-based MCTF coding artifacts visually smoothes
blocking effects but probably causes an overall visual quality
degradation.
Another possible explanation of a reduced coding efficiency
resides in the fact that parent-child spatial relationships, typ-
ically exploited during embedded wavelet coding or context
based arithmetic coding, are partially broken by independent
temporal filtering on the spatial levels.
3) Adaptive Architectures: Being characterized by different
advantages and disadvantages, and architec-
tures seem inadequate to give a final answer to the composite is-
sues arising in SVC applications. One possible approach comes
from schemes which try to combine the positive aspects of each
of them and propose adaptive spatio–temporal decompositions
optimized with respect to suitable criteria. In [67] and [68], the
authors suggest that content-adaptive versus
decomposition can improve coding performance. In particular,
they show how various kind of disadvantages of fixed transform
structures can be more or less preponderant depending on video
data content, especially on the accuracy of the derived motion
model. Therefore, local estimates of the motion accuracy allow
to adapt the spatio–temporal data decomposition structure in
order to preserve scalability while obtaining overall maximiza-
tion of coding performance.
Recently, another adaptive solution, named aceSVC [69],
[70], has been proposed where the decomposition adaptation is
driven by scalability requirements. This solution tries to achieve
maximum bit stream flexibility and adaptability to a selected
spatio–temporal and quality working point configuration. This
is obtained by arranging spatial and temporal decompositions
following the principles of a generalized spatio–temporal
scalability (GSTS) [71]. In aceSVC encoded bit stream, thanks
to the adapted decomposition structure, every coding layer is
efficiently used (without waste of bits) for the construction of
higher ones according to the predefined spatial, temporal and
quality working point configurations.
Another perspective to compensate for the versus
drawbacks can come from the pyramidal approaches,
as discussed in the reminder of this section.
4) Multiscale Pyramids: From the above discussion it is clear
that the spatial and temporal wavelet filtering cannot be decou-
pled because of the MC taking place. As a consequence, it is
not possible to encode different spatial resolution levels at once
(with only one MCTF) and thus, both lower and higher resolu-
tion frames or subband sequences should be MCT-Filtered. In
this perspective, a possibility for obtaining good performance in
terms of bit rate and scalability is to use coding schemes which
are usually referred as to or multiscale pyra-
mids [72]. In such schemes spatially scaled versions of video
signals are obtained starting from the higher resolution, while
the coding systems encompass ISP mechanisms in order to ex-
ploit the multiscale representation redundancy. These multires-
olution decompositions derive naturally from the first hierar-
chical representation technique introduced for images, namely
the Laplacian pyramid [6] (see Section II-B). So, even if from
an intuitive point of view schemes seems to be well
motivated, they have the typical disadvantage of overcomplete
transforms, namely those leading to a full size residual image, so
that coding efficiency is harder to achieve. As in the case of the
previously discussed and schemes, two different
multiscale pyramid based schemes are conceivable, depending
where the pyramidal residue generation is implemented, i.e., be-
fore or after the MCTF decomposition. These two schemes are
shown in Fig. 10. In both schemes the lowest spatial resolution
temporal subbands are the same, while higher reso-
lution residuals are generated differently.

Fig. 11. STP-tool scheme in a generated signals/subbands perspective.
The scheme of Fig. 10(a) suffers from the problems of
schemes, because ME is operated on spatial residuals, and to our
knowledge did not lead to efficient SVC implementations.
In the scheme depicted in Fig. 10(b) we first observe that
the higher resolution residuals , with ,
cannot be called since the spatial transform is not
exactly a Laplacian one because the temporal transform stages,
which are placed between the spatial down-sampling and the
prediction with interpolation stages, operate independently (on
separate originals). This fact can cause prediction inefficiencies
which may be due to motion model discrepancies among dif-
ferent resolutions or simply to the nonlinear and shift variant
nature of the MC even in the presence of similar motion models.
It is worth noting that the effect of these inefficiencies affects the
whole residue and then it worsens the problem of the increased
number of coefficients to code. Strategies to reduce these inef-
ficiencies are workable and we will shortly return on these is-
sues in the following subsection. Note that the scheme shown
in Fig. 10(b) can be used to represent the fundamental coding
principles underlying the SVC standard [2].
It is also important to mention that some recent works demon-
strate that using the frame theory [7] it is possible to improve the
coding performance associated to the Laplacian pyramid repre-
sentation [73], [74] and also to reduce its redundancy [75]. In
the future, these works may play a role in improving the spatial
scalability performance of the coding schemes described in this
paragraph.
C. STP-Tool Scheme
We here present a WSVC architecture that has been devel-
oped [76] and also adopted as a possible configuration of the
MPEG VidWav reference software [36] (see Section IV-B). This
scheme is based on a multiscale pyramid and differs from the
one of Fig. 10(b) in the ISP mechanism. The solution, depicted
in Fig. 11, is called STP-tool as it can be interpreted as a pre-
diction tool (ISP) for spatio–temporal subbands. After the repre-
sentation of the video at different scales, these undergo a normal
MCTF. The resulting signals are then passed to a further spa-
tial transform stage (typically one level of a wavelet transform)
before the ISP. This way, it is possible to implement a predic-
tion between two signals which are likely to bear similar pat-
terns in the spatio–temporal domain without the need to per-
form any interpolation. In particular, instead of the full reso-
lution residuals of Fig. 10(b) the spatio–temporal sub-
bands and residues are produced for
resolutions where are produced by
the coarse resolution adder (shown in Fig. 11) according to: a)
in a spatial open loop
architecture or b) in a
spatial closed loop architecture.
In an open-loop scheme and without the temporal transform,
the residual would be zero (if the same pre- and post-spatial
transforms were used). In this case, the scheme would be re-
duced to a critically sampled one. With the presence of the tem-
poral transform, things are more complicated due to the data-de-
pendent structure of the motion model and to the nonlinear and
time variant behavior of the MCTF. In general, it is not possible
to move the spatial transform stages forward and backward with
respect to the MCTF without changing the result. If the spatial
transform used to produce spatial subbands are the same, the
main factor that produces differences between the
and the signals can be identified in the incoher-
ences between the motion trajectories at different spatial reso-
lutions. On one hand, independent ME at various spatial resolu-
tion could lead to intra-layer optimized MC. On the other hand,
a motion model that captures the physical motion in the scene is
certainly effective for the MC task and reasonably should be self
similar across scales. Therefore, a desirable objective is to have
a hierarchical ME where high-resolution motion fields are ob-
tained by refinement of corresponding lower resolution ones (or
viceversa). Indeed this guarantees a certain degree of inter-scale
motion field coherence which makes the STP-tool prediction

Fig. 12. VidWav general framework.
more effective. A hierarchical motion model is a desired fea-
ture here and in addition it helps to minimize the motion vector
coding rate.
The STP-tool architecture presents another relevant and pecu-
liar aspect. Although the number of subband coefficients to code
is not reduced with respect to a pyramidal approach, different
(prediction) error sources are separated into different critically
sampled signals. This does not happen in a pyramidal schemes
where only full resolution residuals are produced [see
Fig. 10(b)].
To better understand these facts let us analyze sep-
arately each residual or detail component in the set
. In an open loop architecture:
• the residue signals account for the above men-
tioned inter-scale motion model discrepancies; while
• the temporal highpass, spatial highpass subbands
account for the intra-scale motion model
failure to completely absorb and represent the real motion
(occluded and uncovered areas and motion feature not
handled by the motion model); finally
• the temporal lowpass, spatial highpass subbands
contain the detail information to increase
the resolution of lowpass temporal frames.
Thus, maximizing the inter-scale motion coherence implies
minimizing the energy (and the coding cost) of . Note
that even if a perfect motion down-scaling is adopted, localized
residual components would be due to the shift variant proper-
ties of the MC especially where uncovered areas and occlusion
handling must be adopted. In other words, the coefficient in-
crease, determined by the presence of the residues ,
with respect to the critically sampled case is the price to pay for
allowing a suitable ME and MC across different scales.
In a closed-loop ISP architecture we also have the great ad-
vantage that the quantization errors related to resolution are
confined to signals of
resolution that contribute in a more or less predominant
way to the residues to code, that is
. This is of great advantage for the reduced number of
coefficients on which the closed-loop error is spread upon. Be-
sides, the signals are less structured, and thus more
difficult to code, with respect to and .
Therefore, the STP-tool scheme allows a useful separation of
the above signals, with the advantage that suitable coding strate-
gies can be designed for each source.
As for and schemes, the STP-tool solution may
suffer from the presence of spatial aliasing on reduced spatial
resolutions. If not controlled this may also create some inter-
scale discrepancies between the motion models. Again, a partial
solution can be found with the use of more selective decimation
wavelet filters, at least for decomposition levels corresponding
to decodable spatial resolutions.
The STP-tool solution has been tested on different platforms
with satisfactory performance, as described in the following sec-
tions.
IV. WSVC REFERENCE PLATFORM IN MPEG
In the past years, the ISO/MPEG standardization group initi-
ated an exploration activity to develop SVC tools and systems
mainly oriented to obtain spatio–temporal scalability. One of
the key issues of the conducted work was not to sacrifice coding
performance with respect to the state-of-the-art reference video
coding standard H.264/AVC.
In 2004, at Palma meeting, the ISO/MPEG group set up a
formal evaluation of SVC technologies. Several solutions [1],
[66], [77], [3], either based on wavelet spatio–temporal decom-
position or H.264/AVC technologies, had been proposed and
compared. The MPEG visual testing indicated that performance
of an H.264/AVC pyramid [25] appeared the most competitive
and it was selected as a starting point for the new standard. At the
next meeting in Hong Kong, MPEG jointly with the IEC/ITU-T
video group, started a joint standardization initiative accord-
ingly. A scalable reference model and software platform named
JSVM and derived from H.264/AVC technologies was adopted
[2]. During the above mentioned competitive phases, among
other wavelet based solutions, the platform submitted by Mi-
crosoft Research Asia (MSRA) was selected as the reference
to continue experiments on wavelet technologies. The MPEG
WSVC reference model and software (RM/RS) [78] will be in-
dicated, hereinafter, with the acronym VidWav (Video Wavelet).
The VidWav RM/RS has evolved seeing various components in-
tegrated, provided they would have given sufficient performance
improvement after appropriate core experiments.
In the paragraphs of the present section we mainly recall the
characteristics of the VidWav RM. Document [78] should be

taken as a reference to go into further details. In the next sec-
tion, we will discuss the main differences with respect to the
SVC standard. We will also provide some comparison results
between the two approaches.
A. VidWav: General Framework and Main Modules
The general framework supported by the VidWav RM
is shown in Fig. 12. With pre- and post-spatial decompo-
sition different WSVC configurations can be implemented:
and STP-tool. The main modules of the man-
ageable architectures are briefly considered in the following.
ME and Coding: A macroblock-based motion model was
adopted with H.264/AVC like partition patterns. To support spa-
tial scalability, macroblock size is scaled according to the dec-
imation ratio. Interpolation accuracy is also scaled. Therefore,
full pel estimation on 64 64 macroblocks at 4CIF resolution
can be used at quarter-pel precision on a 16 16 macroblock
basis for QCIF reconstructions. Hence, motion is estimated at
full resolution and kept coherent across scales by simply scaling
the motion field partition and the motion vector values. This
poses the limitations already discussed in Section III-B for the
architecture.
For each block there is the possibility to select a forward,
backward or bidirectional motion model. This requires a mode
selection information. Mode selection also determines the
block aspect ratio and informs about the possibility of deter-
mining current MV values from already estimated ones. Motion
vector modes and values are estimated using rate-constrained
lagrangian optimizations and coded with variable length and
predictive coding, similarly to what happens in H.264/AVC.
For MC filtering a connected/disconnected flag is included to
the motion information on units of 4 4 pixels. For 4:2:0 YUV
sequences, motion vectors for chroma components are half
those of the luminance.
Temporal Transform: Temporal transform modules imple-
ment a framewise motion compensated (or motion aligned)
wavelet transform on a lifting structure. The block-based tem-
poral model is adopted for temporal filter alignment. Motion
vectors are directly used for the motion aligned prediction
(MAP) lifting step while the motion field should be inverted
for motion aligned update (MAU). To this end, Barbell update
lifting [31] is adopted which directly uses original motion
vectors and bilinear interpolation to find values of update
signals from noninteger pixel values. In order to further reduce
blocking artifacts, overlapped block MC (OBMC) can be
used during the MAP. This is accomplished by multiplying
the reference signals by a pyramidal window with support
larger than 4 4 before performing the temporal filtering. The
update signal is also clipped according to a suitable threshold
in order to limit the visibility of possible ghosting artifacts.
Moreover, the implemented temporal transform is generally
locally and adaptively a Haar or a -taps depending whether
motion block modes are unidirectional (forward or backward)
or bidirectional.
Spatial Transform: A simple syntax allows to configure pre-
and post-spatial transforms on each frame or previously trans-
formed subband. This allows to implement both and
schemes. Different spatial decompositions are possible
Fig. 13. VidWaw framework in STP-tool configuration. The lowest spatial res-
olution layer is equivalent to a t+20Dscheme. Higher spatial layers implement
STP-tool ISP.
for different temporal subbands for possible adaptation to the
signal properties of the temporal subbands.
Entropy Coding: After the spatio–temporal modules the co-
efficients are coded with a 3-D (spatio–temporal) extension of
the EBCOT algorithm, called 3-D EBCOT. Each spatio–tem-
poral subband is divided into 3-D blocks which are coded in-
dependently. For each block, fractional bit plane coding and
spatio–temporal context based arithmetic coding are used.
B. VidWav STP-Tool Configuration
The VidWav reference software can be configured to follow
the STP-tool architecture of Fig. 11. A corresponding functional
block diagram of the resulting system is shown in Fig. 13. The
coding process starts from the lower resolution which acts as
a base layer for ISP. In particular, a closed loop ISP is im-
plemented as coded temporal subbands are saved and used for
STP-tool prediction for the higher resolution. STP-tool predic-
tion can be applied on all spatio–temporal subbands or only
to a subset of them. Decoded temporal subbands at a suitable
bit rate and temporal resolution are saved and used for ISP. At
present, no particular optimization exists in selecting the pre-
diction point. Obviously, it should be not too far from the max-
imum point for a certain temporal resolution at lower spatial
resolution in order to avoid bit stream spoilage (lower resolution
bit stream portions are useless for higher resolution prediction)
and prediction inefficiency (too large prediction residual). Spa-
tial transforms on the levels interested by STP-tool ISP can be
implemented either by using the three lifting steps (3LS) [61]
or the 9 7 filters. In order to better exploit inter scale redun-
dancy these filters should be of the same type of those used for
the generation of the reference videos for each of the considered
spatial resolutions. In this implementation, some important lim-
itations remain. These are mainly related to software integration
problems due to the data structures used by the original MSRA
software. Therefore, coherent and refinable bottom up ME and
coding are not present in the current VidWav software. When
STP-tool is used, three independent motion fields are estimated
independently. This introduces some undesired redundancy in
the bit stream which has a major impact for the lowest quality

point at higher spatial resolutions. In addition, it does not pro-
vide the motion field coherence across scales which was seen as
a desirable feature for good STP-tool prediction.
C. VidWav Additional Modules
Other modules have been incorporated to the baseline
VidWav reference platform after careful review of performance
improvement by appropriate testing conditions. Such tools can
be turned on or off through configuration parameters. More
detailed description of these tools can be found in [78].
Base Layer: The minimum spatial, temporal and quality res-
olution working point define a Base Layer on which additional
Enhancement Layers are built to provide scalability up to a de-
sired maximum spatial, temporal and quality resolution. This
tool enables the use of a base layer, in subband SVC,
which has been previously compressed with a different tech-
nique. In particular, the H.264/AVC stream can be used as base
layer by implementing a hierarchical B-frames decomposition
(HBFD) structure. This allows each B-picture of the base layer
to temporally match corresponding highpass subbands in the
enhancement layers. It therefore enables the frames to share
similar MCTF structure decomposition and this correlation can
be efficiently exploited. Motion information derived from the
base layer codec can also be used as additional candidate pre-
dictors for estimating motion information of the blocks in the
corresponding highpass temporal subbands of the enhancement
layers. Clearly, the base layer tool can significantly improve
the coding performances, at least of the lowest working point
which is now encoded with a nonscalable method. A second but
not less relevant provided benefit, is the backward compatibility
with preexisting nonscalable H.264/AVC decoder.
In-Band : The scheme is activated when
a nonidentity pre-spatial transform in Fig. 12 is set. In such
a configuration the MCTF is applied to each spatial subband
generated by the pre-spatial transform. In Section III-B.2 it has
been already evidenced that in-band scheme suffers a loss in
coding performance at full resolution when the MCTF is per-
formed independently on each subband. Conversely, if high res-
olution signals are involved in the prediction for low resolution,
drifting occurs when decoding low resolution video, because in
that case high resolution signals are not available at the decoder.
The VidWav RM adopts a technique [79] based on the use on
additional macroblock inter coding modes in the MCTF of the
spatial lowpass band. With respect to the pure scheme,
this method provides a better tradeoff between the lower resolu-
tion drifting error and the high resolution coding efficiency and
also improves the global coding performances.
Wavelet Ringing Reduction: In WSVC, ringing caused by
the loss of details in the high pass spatial subbands is a major
concern for the resulting visual quality [80]. Ringing is gener-
ated during inverse DWT because quantization-noise of lossily-
coded wavelet coefficients is spread by the reconstruction fil-
ters of the IDWT. The oscillating pattern of these filter impulse
responses at different reconstruction scales becomes objection-
able especially in flat luma and chroma regions. The ringing
magnitude is related to the quantization step-size. In order to re-
duce these ringing patterns, a 2-D bilateral filter applied to each
component of the output video is used.
Intra-Mode Prediction: The intra-prediction mode exploits
the spatial correlation within the frame in order to predict intra
areas before entropy coding. Similarly to the H.264/AVC ap-
proach, a macroblock can be predicted form the values of its
neighbors, i.e., by using spatial predictors.
V. WSVC AND SVC STANDARDIZATION REFERENCE
PLATFORMS: A COMPARISON
A. Architecture and Tools: Similarities and Differences
In this section, some differences and similarities between
JSVM and VidWav RM (see Fig. 13) frameworks will be
highlighted and discussed.
1) Single-Layer Coding Tools: We will start to analyze the
differences between the VidWav RM [36] and the JSVM [2]
when they are used to encode a single operating point, i.e., no
scalability features being considered. Broadly speaking this can
be seen as a comparison between a WSVC (which will
remain fully scalable at least for temporal and quality resolu-
tion) and something similar to H.264/AVC (possibly supporting
temporal scalability thanks to HBFD).
VidWav uses a block-based motion model. Block mode
types are similar to those of JSVM with the exception of the
Intra-mode which is not supported by VidWav. This is due to
the fact that the Intra mode determines blocks rewriting from
the original and prediction signal (highpass temporal subbands)
producing a mixed (original+residue) source which is detri-
mental for subsequent whole frame spatial transforms. This is
also one of the main differences for spatio–temporal redun-
dancy reduction between the two approaches. JSVM (as well
as all H.264/AVC and most of the best performing hybrid video
coding methods) operates in a local manner: single frames
are divided into macroblocks which are treated separately in
all the coding phases (spatial transform included). Instead,
the VidWav (as well as WSVC approaches) operates with a
global approach since the spatio–temporal transform is directly
applied to a group of frames. The fact that in the VidWav
framework a block-based motion model has been adopted is
only due to efficiency and the lack of effective implementations
of alternative solutions. However, this cannot be considered
an optimal approach. In fact, the presence of blockiness in the
highpass temporal subbands due to block-based predictions is
inconsistent and causes inefficiencies with respect to the sub-
sequent spatial transform. The energy spreading around block
boundaries, generates artificial high frequency components and
disturbs both zerotree and even block-based entropy coding
mechanisms which act on spatio–temporal subbands. On the
other hand blockiness is not a main issue when block-based
transforms, such as the DCT, are instead applied, as in the case
of JSVM. OBMC techniques can reduce the blockiness in the
highpass temporal subbands but do not remove it.
Contrarily to JSVM, the single layer VidWav frame-
work only supports open loop encoding/decoding. Thanks to the
closed loop encoding, JSVM can exploit an in-loop deblocking
filter, which makes a similar job with respect to the post-pro-
cessing deblocking (JSVM)/deringing (VidWav) filters with the
additional benefit of improving the quality of the reconstructed
signals used in closed-loop predictions.

Closed-loop configurations within WSVC systems would be
possible by adopting an HBFD like decomposition with the aim
of incorporating in-loop deringing strategies as well.
2) Scalable Coding Tools: We now consider spatial scala-
bility activated in both solutions, looking in particular to JSVM
and VidWav in STP-tool configuration. The latter can be consid-
ered as to the WSVC implementation closest to the JSVM one.
The main difference between the two systems in terms of a spa-
tial scalability mechanism is again related to the block-based
versus frame-based dichotomy. In JSVM, a decision on each
single macroblock determines whether the macroblock is en-
coded by using the information of its homologous at a lower
spatial resolution or not. In the latter case, one of the compat-
ible modality available for a single layer encoding will be used.
The decision, among all possible modes is globally regulated
by R-D optimization. On the contrary, ISP inside STP-tool ar-
chitecture is forced to be made at a frame level (with possible
decisions on which frames to involve).
Similarly to JSVM, STP-tool can use both closed and open
loop interlayer encoding while this is not true for or
configurations which can only operate in open loop mode.
B. Objective and Visual Result Comparisons
The comparison between DCT based SVC and WSVC sys-
tems is far from being an obvious task. Objective and subjec-
tive comparisons are critical even when the most “similar” ar-
chitectures are considered (e.g., JSVM and STP-tool), espe-
cially when comparisons are performed using decoded video
sequences at reduced spatial resolutions. In this case, since in
general different codecs use different down-sampling filters, the
coding–decoding processes will have different reconstruction
reference sequences, i.e., different down-sampled versions of
the same original. JSVM adopt the highly selective half-band
MPEG down-sampling filter, while WSVC systems are more or
less forced to use wavelet filter banks which in general offer less
selective half-band response and smoother transition bands. In
particular, in architectures wavelet down-sampling is a
requisite while in STP-tool ones (see Section III-C) one could
adopt a more free choice, at the expense of a less efficient ISP.
Visually, the reference sequences generated by wavelet filters
are in general more detailed but sometimes, depending on data
or visualization conditions, some spatial aliasing effects could
be seen. Therefore, visual benefits in using wavelet filters is con-
troversial while schemes adopting more selective filters can take
advantage, in terms of coding efficiency, of the lower data en-
tropy. Unfortunately, the MPEG down-sampling filter is not a
good lowpass wavelet filter because the corresponding highpass
filter have an undesired behavior in the stop-band [81]. Attempts
to design more selective wavelet filters have been made [61],
[81] with expected benefits in terms of visual and coding per-
formance for WSVC schemes.
Anyway, depending on the particular filter used for spatial
down-sampling, reduced spatial resolution decoded sequences
will differ even at full quality, impairing fair objective compar-
isons. For the above reasons peak SNR (PSNR) curves at inter-
mediate spatio–temporal resolution have been mainly used to
evaluate the coding performance of a single family of architec-
tures while for a direct comparison between two different sys-
Fig. 14. Combined Scalability example results on four test sequences: City,
Soccer, Crew, and Harbour, at different spatio–temporal resolutions and bit
rates, using encapsulated bit stream decoding as described in [82].
tems more time-demanding and costly visual comparisons have
been preferred. In the following we summarize the latest objec-
tive and subjective performance comparison issued by the work
on SVC systems as conducted in April 2006 by ISO/MPEG.
Finally, we will also return on the possibility of increasing ob-
jective comparisons fairness by proposing a common reference
solution.
1) Objective Comparison Results: Experiments have been
conducted under testing conditions described in document [82].
Three classes of experiments have been completed with the in-
tention of evaluating coding performance of different systems
and tool settings. The first class of experiments, named com-
bined scalability, aimed to test the combined use of spatial, tem-
poral and quality scalabilities according to shared scalability
requirements [4]. The comparison has been performed using
JSVM 4.0 and the VidWav RS 2.0 in STP-tool configuration.
Fig. 14 shows an example of the R-D curves where PSNR
are only shown. The example considers different sequences,
quality points and spatio–temporal resolutions, i.e., 4CIF-60 Hz,
CIF-7.5 Hz, QCIF-15 Hz (original resolution is 4CIF-60 Hz
in all cases). The complete set of results can be found in [3,
Annex 1].
As mentioned in Section IV-B, the VidWav STP-tool config-
uration implements an inefficient ME and coding that partially
justifies the performance differences visible in the above men-
tioned PSNR plots. The other two groups of experiments, spa-
tial scalability and SNR scalability, aimed to test the use and the
R-D performance of single scalability tools when other forms of
scalability are not requested. VidWav in STP-tool and in
configuration modes were tested. For the interested reader com-
plete results are reported in [3, Annex 2 and 3].
Where PSNR comparisons did not suffer from the different
original reference signal problems, mixed performance were ob-
served with a predominance of higher values for sequences ob-
tained using JSVM.
2) Subjective Comparison Results: Visual tests conducted
by ISO/MPEG included 12 expert viewers and have been per-
formed according to guidelines specified in [83]. On the basis
of these comparisons between VidWav RS 2.0 and JSVM 4.0
appear on average superior for JSVM 4.0, with marginal gains
in SNR conditions, and superior gains in combined scalability
settings. A full description of such test can be found in [3]

Fig. 15. Common (a) reference generation and (b) representation.
Annex 4]. It should be said that, from a qualitative point of
view, a skilled viewer could infer from which system the de-
coded image is generated. In fact, for motivations ascribable to
spatial filtering aspects, VidWav decoded frames are in general
more detailed with respect to JSVM ones, but bits not spent in
background details can be used by JSVM to better patch motion
model failures and then to reduce the impact of more objection-
able motion artifacts for the used test material.
3) Objective Measures With a Common Reference: A rea-
sonable way to approach a fair objective comparison, between
two systems that adopt different reference video sequences
and , generated from a same original sequence , is to create
a common weighted sequence . In partic-
ular, by selecting it can be easily verified that
. This means that and
are both equally disadvantaged by the creation of the mean ref-
erence . Fig. 15(a) shows the generation of the above three
signals using two different lowpass-filters and
which are assumed here to be a lowpass wavelet filter and the
MPEG down-sampling filter respectively. As already discussed
the output will have a more confined low-frequency content
with respect to . Fig. 15(b) is a projection of the decoding
process which shows vertically the video frame frequency con-
tent and horizontally other kind of differences (accounting for
filter response discrepancies and spatial aliasing). Such differ-
ences can be considered small since and are generated
starting from the same reference while targeting the same
down-sampling objective. Thus, can be considered halfway
on the above projection being a linear average. Now, as trans-
form based coding systems tend to reconstruct first the lower
spectral components of the signal, it is plausible that, at a cer-
tain bit rate (represented in Fig. 15(b) by the dashed bold line),
the VidWav reconstructed signal lies nearer to the JSVM refer-
ence than to its own reference. The converse is not possible and
this gives to the capability to compensate for this disparity.
Therefore, signal can reasonably be used as a common refer-
ence for PSNR comparisons of decoded sequences generated by
different SVC systems. These conjectures find a clear confirma-
tion in experimental measures as it can be seen e.g., in Fig. 16.
VI. FURTHER INVESTIGATIONS ON WSVC SCHEMES
In order to better identify the aspects or tools that could sig-
nificantly improve WSVC performances and to drive future in-
Fig. 16. PSNR comparison at QCIF resolution with the usage of own and
common reference.
vestigations and improvements more precisely, a series of basic
experiments has been conducted comparing JSVM and its most
similar WSVC codec, namely STP-tool.
In a first experiment a set of generic video sequences and still
images were selected with the intent to compare the two codecs
only in terms of intra coding performance. In other words, we
tested the ability of independently compressing a still image that
can be either a still image or frame randomly selected from a
video sequence. Each test image was coded using both codecs
in a single layer configuration repeating the procedure for a
range of different rate values. These values have been selected
in a way to have a reconstruction quality ranging from 30 to
40 dB. On the basis of the quality of decoded images the test
set would be divided into two groups. The first group included
all the images for which JSVM gives the best average coding
performance while the second includes those where PSNR is
in favor of the STP-tool codec. Not surprisingly, the first group
contains only frame of video sequences such as those used for
the MPEG comparisons and sequences acquired with low reso-
lution cameras. Conversely, the second group was characterized
by the presence of all the tested still images and high defini-
tion (HD) video sequences acquired with devices featuring the
latest digital technologies. Thus, most performing DCT-based
technologies appear to outperform wavelet-based ones for rel-
atively smooth signals and vice versa. This indicates that eli-
gible applications for WSVC are those that produce or use high-
definition/high-resolution content. The fact that wavelet-based
image (intra) coding can offer comparable if not better perfor-
mance when applied to HD content is compatible with the re-
sults reported by other independent studies, e.g., [84], [85]. In
[86], the STP-tool has been used to test the performances of
WSVC approaches for in home distribution of HD video se-
quence. As it can be noticed from Fig. 17 the SNR performance
of the proposed codec are comparable with those offered by
the state of the art of single point encoding techniques, e.g.,
H.264/AVC. Another set of tests has been conducted with the
intent of analyzing the coding efficiency of the multiresolution
to compress high pass MC temporal subbands (or hierarchical
B-frames residues). In this case coding efficiency was evaluated
by comparing the performance only on the residual image by
using the same predictor for both encoders and by changing the
filters of the spatial wavelet transform. The main consideration

Fig. 17. Quality (SNR) scalability results for the HD video sequence
CrowdRun.
arising from the obtained results is that commonly used filters,
such as the 9 7, 5 3 and 2 2, are not as effective on high
pass temporal subbands (or B-residues) as much as they are on
lowpass temporal subbands (or frames). This is mainly because
of the nature of the signal being considered which contains the
residue information produced by a local adaptation procedure
while the wavelet filter operates as a global transformation of
the considered temporal subband. This means, for example, that
if we want to encode a frame containing reconstruction errors
only for one macroblock after the spatial transformation these
errors will be spread across all the spatial subbands and the rela-
tive error support will also be extended because of filter lengths.
A more detailed analysis of the energy of spatial subbands re-
vealed that, contrarily to what happens for lowpass temporal
frames, most of the energy is located in the highest spatial res-
olution subbands. This is in accordance with the multiresolu-
tion interpretation for the used spatial transform but because of
their size these subbands cannot be efficiently encoded. Addi-
tionally, this prevents inter-band prediction mechanisms to cor-
rectly work.
VII. PERSPECTIVES
Although considerable efforts have been made to improve
WSVC solutions there are some reasons to say that we could
be at the beginning of a prolific research field with many pos-
sible applications. Present coding performances are not too far
from that of JSVM, which implements the technology defined in
the forthcoming SVC standard. Hybrid video coding (especially
based on block-based spatial DCT and block-based temporal
ME and MC) has experimented a strong increase in performance
as a consequence of the introduction of many optimized tools
and configuration parameters. While some of these solutions
have not yet been tested on a common reference architecture
such as VidWav, other were forcely discarded since not ade-
quate to the whole frame nature of the spatial wavelet trans-
form. For example advanced block-based predictions with mul-
tiple block-mode selection, advanced block-based ME, MC, and
coding can be fully adopted with block-based transforms while
they are not natural with full frame transforms. An example for
which an attempt has been made is represented by the “intra-pre-
diction” mode. This tool works remarkably well for JSVM but
if simply imported on a VidWav architecture, it does not give
the expected results. Wavelet based video coding systems would
require alternative prediction modes and motion models. Ex-
isting solutions appear still preliminary and, in order to maxi-
mize their benefit, they need to be effectively integrated in the
complete system. When compared to JSVM, existing wavelet
codecs lack several performance optimization tools. In partic-
ular R-D optimized solutions are hardly implementing those
tools which are not fully compatible with the whole frame ap-
proach used in WSVC systems. Therefore, both: 1) activities
directed on finding new tools and solutions; and 2) activities di-
rected on shaping and optimizing the use of existing tools, seem
to be important for the improvement and development of effec-
tive WSVC systems.
A. New Application Potentials
There are a number of functionalities and applications which
can be effectively targeted by WSVC technology and seem to
prospect comparable if not more natural solutions with respect
to their hybrid DCT+ME/MC SVC counterparts. A brief list is
reported here, while a more complete overview can be found
in [3].
• HD material storage and distribution. Thanks to the built-in
scalability brought by wavelets, it is possible to encode a
given content with nonpredefined scalability range and at
a quality up to near lossless. At the same time, a very low
definition decoding is possible allowing, a quick preview
of the content, for example.
• The possibility of using nondyadic wavelet decomposi-
tions [87], which enables the support of multiple HD for-
mats is an interesting feature not only for the aforemen-
tioned application but also for video surveillance and for
mobile video applications.
• WSVC with temporal filtering can also be adapted so that
some of the allowable operating points can be decoded by
J2K and MJ2K (MJ2K is only “intra” coded video using
J2K; if J2K compatibility is preserved MJ2K compatibility
will also remain).
• Enabling efficient similarity search in large video
databases. Different methods based on wavelets exist.
Instead of searching full resolution videos, one can search
low quality videos (spatially, temporally, and reduced
SNR), to accelerate the search operation. Starting from
salient points, on low quality videos, similarities can be
refined in space and time.
• Multiple Description Coding which would lead to better
error-resilience. By using the lifting scheme, it is easy to
replicate video data to transmit two similar bit streams.
Using intelligent splitting, one can decode independently
or jointly the two bit streams (spatially, temporally, and/or
SNR reduced).
• Space variant resolution adaptive decoding. When en-
coding the video material, it is possible to decode at a high
spatial resolution only a certain region while keeping at a
lower resolution the surrounding areas.

VIII. CONCLUSION
This paper has provided a picture of various tools that have
been designed in recent years for scalable video compression.
It has focused on multiresolution representations with the use
of the wavelet transform and its extensions to handle motion in
image sequences. A presentation of benefits of wavelet based
approaches has also been suggested for various application per-
spectives. The paper has also shown how it is possible to or-
ganize various multiresolution tools into different architecture
families depending on the (space/time) order in which the signal
transformation is operated. All such architectures make it pos-
sible to generate embedded bit streams that can be parsed to
handle combined quality, spatial and temporal scalability re-
quirements. This has been extensively simulated during the ex-
ploration activity on wavelet video coding as carried out effort
by ISO/MPEG, demonstrating that credible performance results
could be achieved. Nevertheless, when compared to the current
SVC standardization effort, which provides a pyramidal exten-
sion of MPEG-4/AVC, it appears that though close, the per-
formance of wavelet based video codecs remains inferior. This
seems partly due to:
• the level of fine (perceptual and/or R-D theoretic) opti-
mization that can be achieved for hybrid (DCT+ME/MC)
codecs because of the deep understanding on how all its
components interact;
• the spectral characteristics of standard resolution video
material on which standard multiresolution (wavelet
based) representation already exhibit lower performance
even in intra-mode (some conducted preliminary experi-
ments on high resolution material are showing increasing
performance for WT approaches);
• the inability of the experimented multiresolution (WT) ap-
proaches to effectively represent the information locality
in image sequences (block-based transform can be instead
more easily adapted to handle the nonstationarity of vi-
sual content, in particular by managing uncovered/covered
areas through a local effective representation of intracoded
blocks).
This last argument deserves probably some further consid-
erations. For image compression, WT based approaches are
showing quite competitive performance due to the energy com-
paction ability of the WT to handle piecewise polynomials that
are known to well describe many natural images. In video se-
quences, the adequacy of such model falls apart unless a precise
alignment of moving object trajectories can be achieved. This
might remain only a challenge, since as for any segmentation
problem, it is difficult to achieve it in a robust fashion, due to
the complex information modeling which is often necessary.
As an alternative, new transform families [88] could also be
studied so as to model the specificity of spatio–temporal visual
information.
This paper is an attempt to summarize the current under-
standing in wavelet video coding as carried out collectively in
the framework of the ISO/MPEG standardization. While it has
not yet been possible to establish wavelet video coding as an
alternative video coding standard for SVC, its performance ap-
pears on the rise when compared to previous attempts to estab-
lish credibly competitive video coding solutions with respect
to hybrid motion-compensated transform based approaches. We
hope that the picture offered will help many future researchers
to undertake work in the field and shape the future of wavelet
video compression.
ACKNOWLEDGMENT
The authors would like to acknowledge the efforts of col-
leagues and friends who have helped to establish the state
of the art in the field as described in this work. Particular
thanks are meant for all those institutions (academia, funding
organizations, research centers, and industries) which continue
to support work in this field and in particular the people who
have participated in the ISO/MPEG wavelet video coding
exploration activity during several years, e.g., C. Guillemot,
W.-J. Han, J. Ohm, S. Pateux, B. Pesquest-Popescu, J. Reichel,
M. Van der Shaar, D. Taubmann, J. Woods, J.-Z. Xu, just to
name a few.
REFERENCES
[1] “Registered Responses to the Call for Proposals on Scalable Video
Coding,” Munich, Germany, ISO/MPEG Video, Tech. Rep. M10569/
S01-S24, Mar. 2004.
[2] “Joint Scalable Video Model (JSVM) 6.0,” Montreux, Switzerland,
ISO/MPEG Video, Tech. Rep. N8015, Apr. 2006.
[3] R. Leonardi, T. Oelbaum, and J.-R. Ohm, “Status report on wavelet
video coding exploration,” Montreux, Switzerland, Tech. Rep. N8043,
Apr. 2006.
[4] “Applications and Requirements For Scalable Video Coding, ISO/IEC
JTC1/SC29/WG11, 71st MPEG Meeting,” Hong Kong, China, ISO/
MPEG Requirements, Tech. Rep. N6880, Jan. 2005.
[5] “Description of Core Experiments in MPEG-21 Scalable Video Coding
ISO/IEC JTC1/SC29/WG11, 69th MPEG Meeting Redmond,” WA,
USA, ISO/MPEG Video, Tech. Rep. N6521, Jul. 2004.
[6] P. Burt and E. Adelson, “The laplacian pyramid as a compact image
code,” IEEE Trans. Commun., vol. COM-31, no. 4, pp. 532–540, Apr.
1983.
[7] S. Mallat, A Wavelet Tour of Signal Processing. San Diego, CA: Aca-
demic, 1998.
[8] P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood
Cliffs, NJ: Prentice-Hall, 1992.
[9] M. Vetterli and J. Kova˘cevic´, Wavelets and Subband Coding.. Engle-
wood Cliffs, NJ: Prentice-Hall, 1995.
[10] I. Daubechies, Ten Lectures on Wavelets. Philadelphia, PA: SIAM,
1992.
[11] W. Sweldens, “The lifting scheme: A construction of second generation
wavelets,” SIAM J. Math. Anal., vol. 29, no. 2, pp. 511–546, 1997.
[12] I. Daubechies and W. Sweldens, “Factoring wavelet transforms into
lifting steps,” J. Fourier Anal. Appl., vol. 4, no. 3, pp. 247–269, 1998.
[13] A. Calderbank, I. Daubechies, W. Sweldensand, and B.-L. Yeo,
“Wavelet transforms that map integers to integers,” Appl. Comput.
Harmon. Anal., vol. 5, no. 3, pp. 332–369, 1998.
[14] H. Heijmans and J. Goutsias, “Multiresolution signal decomposition
schemes. Part 2: Morphological wavelets,” The Netherlands, CWI,
Amsterdam, Tech. Rep. PNA-R9905, Jun. 1999.
[15] J.-R. Ohm, Multimedia Communication Technology.. New York:
Springer Verlag, 2003.
[16] J. Ohm, “Three-dimensional subband coding with motion compensa-
tion,” IEEE Trans. Image Process., vol. 3, no. 9, pp. 559–571, Sep.
1994.
[17] S.-J. Choi and J. Woods, “Motion-compensated 3-D subband coding of
video,” IEEE Trans. Image Process., vol. 8, no. 2, pp. 155–167, Feb.
1999.
[18] B.-J. Kim, Z. Xiong, and W. Pearlman, “Low bit—Rate scalable
video coding with 3-D set partitioning in hierarchical trees (3D
SPIHT),” IEEE Trans. Circuits Syst. Video Technol., vol. 10, no. 8, pp.
1374–1387, Dec. 2000.
[19] A. Secker and D. Taubman, “Lifting-based invertible motion adaptive
transform (LIMAT) framework for highly scalable video compression,”
IEEE Trans. Image Processing, vol. 12, no. 12, pp. 1530–1542, Dec.
2003.

[20] G. Pau, C. Tillier, B. Pesquet-Popescu, and H. Heijmans, “Motion
compensation and scalability in lifting-based video coding,” Signal
Process.: Image Commun., vol. 19, no. 7, pp. 577–600, Aug. 2004.
[21] B. Pesquet-Popescu and V. Bottreau, “Three dimensional lifting
schemes for motion compensated video compression,” in Proc.
ICASSP, Salt Lake City, UT, May 2001, vol. 3, pp. 1793–1796.
[22] J. Ohm, “Motion-compensated wavelet lifting filters with flexible adap-
tation,” in Proc. Thyrrenian Int. Workshop Digital Commun. (IWDC),
Capri, Italy, Sep. 2002, pp. 113–120.
[23] Y. Zhan, M. Picard, B. Pesquet-Popescu, and H. Heijmans, “Long tem-
poral filters in lifting schemes for Scalable Video Coding,” ISO/IEC
JTC1/SC29/WG11, 61th MPEG Meeting, Klagenfurt, Austria, Tech.
Rep. M8680, Jul. 2002.
[24] J. Xu, Z. Xiong, S. Li, and Y.-Q. Zhang, “Three-dimensional embedded
subband coding with optimal truncation (3D ESCOT),” Appl. Comput.
Harmon. Anal., vol. 10, pp. 290–315, 2001.
[25] H. Schwarz, D. Marpe, and T. Wiegand, “Scalable extension of
H.264/AVC, ISO/IEC JTC1/SC29/WG11,” 68th MPEG Meeting
Munich, Germany, Tech. Rep. M10569/S03, Mar. 2004.
[26] R. Xiong, F. Wu, S. Li, Z. Xiong, and Y.-Q. Zhang, “Exploiting tem-
poral correlation with block-size adaptive motion alignment for 3-D
wavelet coding,” in Proc. Visual Commun. Image Process., San Jose,
CA, Jan. 2004, vol. 5308, pp. 144–155.
[27] A. Secker and D. Taubman, “Highly scalable video compression using
a lifting-based 3-D wavelet transform with deformable mesh motion
compensation,” in Proc. Int. Conf. Image Process. (ICIP), Sep. 2002,
vol. 3, pp. 749–752.
[28] L. Luo, F. Wu, S. Li, Z. Xiong, and Z. Zhuang, “Advanced motion
threading for 3-D wavelet video coding,” Signal Process.: Image
Commun., vol. 19, no. 7, pp. 601–616, 2004.
[29] D. Turaga, M. van der Schaar, and B. Pesquet-Popescu, “Temporal pre-
diction and differential coding of motion vectors in the MCTF frame-
work,” in Proc. Int. Conf. Image Process. (ICIP), Barcelona, Spain,
Sep. 2003, vol. 2, pp. 57–60.
[30] V. Valéntin, M. Cagnazzo, M. Antonini, and M. Barlaud, “Scalable
context-based motion vector coding for video compression,” in Proc.
Picture Coding Symp. (PCS), Apr. 2003, pp. 63–68.
[31] J. Xu, R. Xiong, B. Feng, G. Sullivan, M.-C. Lee, F. Wu, and S. Li,
“3D subband video coding using barbell lifting,” ISO/IEC JTC1/SC29/
WG11, 68th MPEG Meeting Munich, Germany, Tech. Rep. M10569/
S05, Mar. 2004.
[32] C. Tillier, B. Pesquet-Popescu, and M. van der Schaar, “Highly scalable
video coding by bidirectional predict-update 3-band schemes,” in Proc.
Int. Conf. Acoust. Speech Signal Process. (ICASSP), Montreal, Canada,
May 2004, vol. 3, pp. 125–128.
[33] M. V. der Schaar and D. Turaga, “Unconstrained motion compensated
temporal filtering (UMCTF) framework for wavelet video coding,” in
Proc. Int. Conf. Acoust. Speech and Signal Process. (ICASSP), Hong-
Kong, Apr. 2003, vol. 3, pp. 81–84.
[34] N. Mehrseresht and D. Taubman, “Adaptively weighted update steps
in motion compensated lifting based on scalable video compression,”
in Proc. Int. Conf. on Image Process. (ICIP), Barcelona, Spain, Sep.
2003, vol. 3, pp. 771–774.
[35] D. Maestroni, M. Tagliasacchi, and S. Tubaro, “In-band adaptive up-
date step based on local content activity,” in Proc. Visual Commun.
Image Process., Beijing, China, Jul. 2005, vol. 5960, pp. 169–176.
[36] “Wavelet Codec Reference Document and Software Manual,” ISO/IEC
JTC1/SC29/WG11, 73th MPEG Meeting, Poznan, Poland, ISO/MPEG
Video, Tech. Rep. N7334, Jul. 2005.
[37] H. Schwarz, D. Marpe, and T. Wiegand, “Analysis of hierarchical
B-pictures and MCTF,” presented at the nt. Conf. Multimedia Expo
(ICME), Toronto, ON, Canada, Jul. 2006.
[38] D. Turaga, M. van der Schaar, and B. Pesquet-Popescu, “Complexity
scalable motion compensated wavelet video encoding,” IEEE Trans.
Circuits Syst. Video Technol., vol. 15, no. 8, pp. 982–993, Aug. 2005.
[39] C. Parisot, M. Antonini, and M. Barlaud, “Motion-conpensated scan
based wavelet transform for video coding,” in Proc. Thyrrenian Int.
Workshop on Digital Commun. (IWDC), Capri, Italy, Sep. 2002, pp.
121–128.
[40] G. Pau, B. Pesquet-Popescu, M. van der Schaar, and J. Viéron, “Delay-
performance tradeoffs in motion-compensated scalable subband video
compression,” presented at the ACIVS, Brussels, Belgium, Sep. 2004.
[41] J. Shapiro, “Embedded image coding using zerotrees of wavelet coef-
ficients,” IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3445–3462,
Dec. 1993.
[42] A. Said and W. A. Pearlman, “A new fast and efficient image codec
based on set partitioning in hierarchical trees,” IEEE Trans. Circuits
Syst. Video Technol., vol. 6, no. 3, pp. 243–250, Jun. 1996.
[43] S.-T. Hsiang and J. W. Woods, “Embedded image coding using
zeroblocks of subband/wavelet coefficients and context modeling,”
in Proc. MPEG-4 Workshop Exhibition ISCAS, Geneva, Switzerland,
May 2000, vol. 3, pp. 662–665.
[44] S. D. Servetto, K. Ramchandran, and M. T. Orchard, “Image coding
based on a morphological representation of wavelet data,” IEEE Trans.
Image Process., vol. 8, no. 9, pp. 1161–1174, Sep. 1999.
[45] F. Lazzaroni, R. Leonardi, and A. Signoroni, “High-performance em-
bedded morphological wavelet coding,” IEEE Signal Process. Lett.,
vol. 10, no. 10, pp. 293–295, Oct. 2003.
[46] F. Lazzaroni, R. Leonardi, and A. Signoroni, “High-performance em-
bedded morphological wavelet coding,” in Proc. Thyrrenian Int. Work-
shop Digital Commun. (IWDC), Capri, Italy, Sep. 2002, pp. 319–326.
[47] F. Lazzaroni, A. Signoroni, and R. Leonardi, “Embedded morpholog-
ical dilation coding for 2-D and 3-D images,” in Proc. Visual Commun.
Image Process., San José, CA, Jan. 2002, vol. 4671, pp. 923–934.
[48] D. Taubman, “High performance scalable image compression with
EBCOT,” IEEE Trans. Image Process., vol. 9, no. 7, pp. 1158–1170,
Jul. 2000.
[49] D. Taubman, E. Ordentlich, M. Weinberger, and G. Seroussi, “Em-
bedded block coding in JPEG 2000,” Signal Process.: Image Commun.,
vol. 17, pp. 49–72, Jan. 2002.
[50] J. Li and S. Lei, “Rate-distortion optimized embedding,” in Proc. Pic-
ture Coding Symp., Sep. 1997, pp. 201–206.
[51] S.-T. Hsiang and J. Woods, “Embedded video coding using invert-
ible motion compensated 3-D subband/wavelet filter bank,” Signal
Process.: Image Commun., vol. 16, no. 8, pp. 705–724, May 2001.
[52] P. Chen and J. W. Woods, “Bidirectional MC-EZBC with lifting im-
plementation,” IEEE Trans. Circuits Syst. Video Technol., vol. 14, no.
10, pp. 1183–1194, Oct. 2004.
[53] N. Adami, M. Brescianini, M. Dalai, R. Leonardi, and A. Signoroni,
“A fully scalable video coder with inter-scale wavelet prediction and
morphological coding,” in Proc. Visual Commun. Image Process., Bei-
jing, China, Jul. 2005, vol. 5960, pp. 535–546.
[54] V. Bottreau, M. Benetiere, B. Felts, and B. Pesquet-Popescu, “A fully
scalable 3-D subband video codec,” in Proc. Int. Conf. Image Process.
(ICIP), Thessaloniki, Greece, Oct. 2001, vol. 2, pp. 1017–1020.
[55] K. Ho and D. Lun, “Efficient wavelet based temporally scalable video
coding,” in Proc. Int. Conf. Image Process. (ICIP), New York, USA,
Aug. 2002, vol. 1, pp. 881–884.
[56] N. Cammas, “Codage video scalable par maillages et ondelettes t +
2D” Ph.D. dissertation, Univ. Rennes, Rennes, France, Nov. 2004 [On-
line]. Available: http:www.inria.fr/rrrt/tu-1122.html
[57] G. Pau, “Ondlettes et décompositions spatio–temporelles avancées;
application au codage vidéo scalable,” Ph.D. dissertation, Ecole Nat.
Supér. Télecomun., Paris, France, 2007.
[58] M. Flierl and B. Girod, “Video coding with motion-compensated lifted
wavelet transforms,” Signal Process.: Image Commun., vol. 19, no. 7,
pp. 561–575, Aug. 2004.
[59] J. Barbarien, A. Munteanu, F. Verdicchio, Y. Andreopoulos, J. Cor-
nelis, and P. Schelkens, “Motion and texture rate—allocation for pre-
diction-based scalable motion-vector coding,” Signal Process.: Image
Commun., vol. 20, no. 4, pp. 315–342, Apr. 2005.
[60] M. Mrak, N. Sprljan, G. Abhayaratne, and E. Izquierdo, “Scalable
generation and coding of motion vectors for highly scalable video
coding,” presented at the Picture Coding Symp. (PCS), San Francisco,
CA, 2004.
[61] V. Bottreau, C. Guillemot, R. Ansari, and E. Francois, “SVC CE5:
Spatial Transform Using Three Lifting Steps Filters,” ISO/IEC JTC1/
SC29/WG11, 70th MPEG Meeting, Palma de Mallorca, Spain, Tech.
Rep. M11328, Oct. 2004.
[62] Y. Wu and J. Woods, “Aliasing Reduction for Scalable Sub-
band/Wavelet Video Coding,” ISO/IEC JTC1/SC29/WG11, 73rd
MPEG Meeting, Poznan, Poland, Tech. Rep. M12376, Jul. 2005.
[63] Y. Andreopoulos, A. Munteanu, J. Barbarien, M. van der Schaar, J.
Cornelis, and P. Schelkens, “Inband motion compensated temporal fil-
tering,” Signal Process.: Image Commun., vol. 19, no. 7, pp. 653–673,
Aug. 2004.
[64] H.-W. Park and H.-S. Kim, “Motion estimation using low-band-shift
method for wavelet-based moving-picture coding,” IEEE Trans. Image
Process., vol. 9, pp. 577–587, 2000.
[65] Y. Andreopoulos, A. Munteanu, G. V. D. Auwera, J. Cornelis, and
P. Schelkens, “Complete-to-overcomplete discrete wavelet transforms:
Theory and applications,” IEEE Trans. Signal Process., vol. 53, no. 4,
pp. 1398–1412, 2005.
[66] “Subjective test results for the CfP on Scalable Video Coding tech-
nology,” ISO/IEC JTC1/SC29/WG11, 68th MPEG Meeting, Munich,
Germany, Tech. Rep. N6383, Mar. 2004.

[67] N. Mehrseresht and D. Taubman, “An efficient content-adaptive MC
3D-DWT with enhanced spatial and temporal scalability,” in Proc. Int.
Conf. Image Process. (ICIP), Oct. 2004, vol. 2, pp. 1329–1332.
[68] D. Taubman, N. Mehrseresht, and R. Leung}, “SVC technical contribu-
tion: Overview of recent technology developments at UNSW ISO/IEC
JTC1/SC29/WG11,” 69th MPEG Meeting, Redmond, WA, USA, Tech.
Rep. M10868, Jul. 2004.
[69] N. Sprljan, M. Mrak, T. Zgaljic, and E. Izquierdo, “Software
proposal for Wavelet Video Coding Exploration Group ISO/IEC
JTC1/SC29/WG11,” 75th MPEG Meeting, Bangkok, Thailand, Tech.
Rep. M12941, Jan. 2006.
[70] M. Mrak, N. Sprljan, T. Zgaljic, N. Ramzan, S. Wan, and E. Izquierdo,
“Performance evidence of software proposal for Wavelet Video Coding
Exploration Group”, ISO/IEC JTC1/SC29/WG11, 6th MPEG Meeting,
Montreux, Switzerland, Tech. Rep. M13146, Apr. 2006.
[71] C. Ong, S. Shen, M. Lee, and Y. Honda, “Wavelet Video Coding—Gen-
eralized Spatial Temporal Scalability (GSTS),” ISO/IEC JTC1/SC29/
WG11, 72nd MPEG Meeting, Busan, Korea, Tech. Rep. M11952, Apr.
2005.
[72] “Scalable Video Model v2.0 ISO/IEC JTC1/SC29/WG11,” 69th
MPEG Meeting, Redmond, WA, USA, Tech. Rep. N6520, Jul. 2004,
ISO/MPEG Video.
[73] M. N. Do and M. Vetterli, “Framing pyramids,” IEEE Trans. Signal
Processing, vol. 51, no. 9, pp. 2329–2342, Sep. 2003.
[74] M. Flierl and P. Vandergheynst, “An improved pyramid for spatially
scalable video coding,” in Proc. Int. Conf. Image Process. (ICIP),
Genova, Italy, Sep. 2005, vol. 2, pp. 878–881.
[75] G. Rath and C. Guillemot, “Representing laplacian pyramids with
varying amount of redundancy,” presented at the Eur. Signal Process.
Conf. (EUSIPCO), Florence, Italy, Sep. 2006.
[76] N. Adami, M. Brescianini, R. Leonardi, and A. Signoroni, S¨ VC CE1:
STool—A native spatially scalable approach to SVC ISO/IEC JTC1/
SC29/WG11,” 70th MPEG Meeting, Palma de Mallorca, Spain, Tech.
Rep. M11368, Oct. 2004.
[77] “Report of the subjective quality evaluation for SVC CE1 ISO/IEC
JTC1/SC29/WG11,” 70th MPEG Meeting, Palma de Mallorca, Spain,
Tech. Rep. N6736, Oct. 2004, ISO/MPEG Video.
[78] R. Leonardi and S. Brangoulo, “Wavelet codec reference document
and software manual v2.0,” ISO/IEC JTC1/SC29/WG11, 74th MPEG
Meeting, Nice, France, Tech. Rep. N7573, Oct. 2005.
[79] D. Zhang, S. Jiao, J. Xu, F. Wu, W. Zhang, and H. Xiong, “Mode-
based temporal filtering for in-band wavelet video coding with spatial
scalability,” in Proc. Visual Commun. Image Process., Beijing, China,
Jul. 2005, vol. 5960, pp. 355–363.
[80] M. Beermann and M. Wien, “Wavelet video coding, EE4: Joint re-
duction of ringing and blocking, ISO/IEC JTC1/SC29/WG11,” 74th
MPEG Meeting, Nice, France, Tech. Rep. M12640, Oct. 2005.
[81] M. Li and T. Nguyen, “Optimal wavelet filter design in scalable video
coding,” in Proc. Int. Conf. Image Process. (ICIP), Genova, Italy, Sep.
2005, pp. 473–476.
[82] Description of testing in wavelet video coding ISO/IEC JTC1/SC29/
WG11,” 75th MPEG Meeting, Bangkok, Thailand, Tech. Rep. N7823,
Jan. 2006, ISO/MPEG Video.
[83] C. Fenimore, V. Baroncini, T. Oelbaum, and T. K. Tan, “Subjective
testing methodology in MPEG video verification,” in Proc. Appl. Dig-
ital Image Process. XXVII, Aug. 2004, vol. 5558, pp. 503–511.
[84] M. Ouaret, F. Dufaux, and T. Ebrahimi, “On comparing JPEG2000 and
intraframe AVC,” presented at the Appl. Digit. Image Process. XXIX,
Genova, Italy, Aug. 2006, 63120U.
[85] D. Marpe, S. Gordon, and T. Wiegand, “H.264/MPEG4-AVC fidelity
range extensions: Tools, profiles, performance, and application areas,”
in Proc. Int. Conf. Image Process. (ICIP), Genova, Italy, Sep. 2005,
vol. 1, pp. 593–596.
[86] L. Lima, F. Manerba, N. Adami, A. Signoroni, and R. Leonardi,
“Wavelet-based encoding for HD applications,” presented at the Proc.
Int. Conf. Multimedia Expo (ICME), Beijing, China, Jul. 2006.
[87] G. Pau and B. Pesquet-Popescu, “Image coding with rational spatial
scalability,” presented at the Proc. Eur. Signal Process. Conf. (EU-
SIPCO), Florence, Italy, Sep. 2006.
[88] B. Wang, Y. Wang, I. Selesnick, and A. Vetro, “An investigation of
3-D dual-tree wavelet transform for video coding,” in Proc. Int. Conf.
Image Process. (ICIP), Singapore, Oct. 2004, pp. 1317–1320.
Nicola Adami (M’01) received the Laurea degree in
electronic engineering and the Ph.D. degree in infor-
mation engineering from the University of Brescia,
Brescia, Italy, in 1998 and 2002, respectively.
From 2002 to the end of 2003, he was a Fellow
at the University of Brescia, where he has been an
Assistant Professor in the Engineering Faculty since
2003. His research interests are signal processing,
multimedia content analysis and representation,
and image and video coding. He has been involved
in several European granted projects and has been
participating to ISO/MPEG and JPEG standardization works.
Alberto Signoroni (M’03) received the Laurea de-
gree in electronic engineering and the Ph.D. in infor-
mation engineering from the University of Brescia,
Brescia, Italy, in 1997 and 2001, respectively.
Since March 2002, he has been an Assistant Pro-
fessor in Signal Processing and Telecommunications
at the Department of Electronics for the Automation,
University of Brescia. His research interests are in
image processing, image and video coding systems,
medical image coding, and medical image analysis.
Riccardo Leonardi (M’88) received the Diploma
and Ph.D. degrees in electrical engineering from the
Swiss Federal Institute of Technology, Lausanne,
Switzerland, in 1984 and 1987, respectively.
He spent one year (1987–1988) as a Postdoctoral
Fellow with the Information Research Laboratory,
University of California, Santa Barbara. From 1988
to 1991, he was a Member of Technical Staff at
AT&T Bell Laboratories, performing research ac-
tivities on image communication systems. In 1991,
he returned briefly to the Swiss Federal Institute
of Technology, Lausanne, to coordinate the research activities of the Signal
Processing Laboratory. Since February 1992, he has been with the University
of Brescia to lead research and teaching in the field of telecommunication. His
main research interests cover the field of digital signal processing applications,
with a specific expertise on visual communications, and content-based analysis
of audio–visual information. He has published more than 100 papers on these
topics. Since 1997, he acts also as an evaluator and auditor for the European
Union IST and COST programmes.

1 state of-the-art and trends in scalable video

More Related Content

What's hot (20)

Similar to 1 state of-the-art and trends in scalable video (20)

Recently uploaded (20)

1 state of-the-art and trends in scalable video