Introduction to the Java(TM) Advanced Imaging API

Introduction to the Java(TM) Advanced Imaging
API

Abstract

The Java TM Advanced Imaging (JAI) API is a cross-platform, flexible, extensible toolkit
for adding advanced image processing capabilities to applications for the Java platform.
It includes features such as tiled images, lazy evaluation, multi-resolution imaging,
meta-data handling, and network imaging. This course will introduce participants to the
imaging capabilities of the Java platform and provide practical examples of how to make
use of JAI in their applications.
The three major areas of JAI functionality will be described in detail: pixel-based, or
"rendered" imaging; resolution- and rendering-independent, or "renderable" imaging;
and networked, or "remote" imaging. In each of these areas, participants will learn both
how to use the standard capabilities of the API as well as how to write their own
extensions. An application example will be presented in depth.

Lecturer Information

Brian Burkhalter (Organizer)
Staff Engineer, Sun Microsystems, Inc.
Email: Brian.Burkhalter@eng.sun.com
Brian Burkhalter is the technical lead for core development of the Java Advanced
Imaging API. He has worked since 1996 in the Imaging Software Technologies group of
the Sun Microsystems Computer Systems division.
His previous work at Sun was primarily on the XIL imaging library. He has more than a
decade of experience in image processing and particularly in processing of remotely
sensed imagery and photogrammetry. He has represented the Java Advanced Imaging
API at the JavaOne developer’s conference and the Environmental Systems Research
Institute (ESRI) User’s Conference.
He holds a BA in Mathematics from the University of Chicago, a MSEE from the
University of Washington, and the Degree of Engineer (EE) from Stanford University.

Course Outline - Part 1

Introduction
Part 1 - JAI and Java 2D Imaging Basics

Introduction to JAI Features
Imaging in Java - the AWT and Java 2D APIs
Fundamental Java 2D image classes
Fundamental JAI image classes
Other fundamental JAI classes
JAI non-image classes
Instantiating a JAI operation
Duration: 30 minutes


Part 2 - Working with Rendered images and properties

The RenderedImage interface
Rendered image semantics
Importing an image
Displaying an image
Performing an operation on an image
Writing a new Rendered operation
Adding properties to images
Managing image properties


Part 3 - Working with Renderable images

Renderable image semantics
The RenderableImage interface
Building renderable chains
Rendering a chain
Editing a chain
Writing a new Renderable operation


Part 4 - Working with Remote images

Remote image sources
Overview of networking concepts in the Java platform
The RemoteImage interface


Part 5 - Using the JAI Image I/O Package (unofficial)

InputStream and extensions
The ImageCodec class
The ImageDecoder and ImageEncoder interfaces
JAI I/O operations
Adding new image decoders and encoders


Part 6 - Writing a JAI application

Setting up the JAI runtime environment
Integrating with the Swing toolkit
Integrating with browsers
An inside look at a JAI application

JAI and Java 2D Imaging Basics


JAI and Java 2D Imaging Basics - Outline

Imaging in the Java platform - the AWT and Java 2D APIs
Fundamental Java 2D image classes
Fundamental JAI image classes
Other fundamental JAI classes
JAI non-image classes
Instantiating a JAI operation


"Rendered" imaging - large/deep images
"Rendered" imaging - tiling/deferred execution
"Properties" - a framework for meta-data
"Renderable" imaging - multiresolution imaging
"Renderable" imaging - interactive editing
"Remote" images - imaging over the network
Extensibility

"Rendered" Imaging - Large/Deep Images

Large images
Varied pixel depths
Flexible formats
Float and double samples

"Rendered" Imaging -
Tiling/Deferred Execution

Images broken up into tiles
Image data computed only on demand
Computed tiles stored in a cache

"Properties" - A Framework for Metadata

Every image maintains a simple database of properties:

String getPropertyNames()
Object getProperty(String name)

PropertySource interface contains these methods
PropertyGenerator interface used to compute property values
dynamically

"Renderable" Imaging -
Multiresolution Imaging

RenderableImage == object that creates an image on demand
Resolution and rendering independence
Control rendering with a RenderContext object

"Renderable" Imaging - Interactive Editing

Building chains of operations
Chains may be reconfigured (change node’s sources)
Nodes may be reconfigured (change node’s parameters)
Chains are evaluated on demand

"Remote" Images - Imaging Over The Network

Standard implementation uses Remote Method Invocation (RMI)
Protocol-neutral interface
Transparently distribute processing across the network

Extensibility

Extensibility designed in from the start
Add new operators
Add new implementations of existing operators
Add new image formats
Write subclasses for interpolation, border extension, warping
Write subclasses for custom image representations
Write property (metadata) generators

Introduction to JAI Features (cont.)

JAI has classes that address a number of areas of functionality:
Imaging operations
Image collections
Image codecs
Interpolation/border extension/warping
Tile caching
Tile scheduling

Imaging Operations

Over 80 standard operations
Real and Complex arithmetic ops
Bitwise logical ops
Area ops: Convolve, Gradient
Geometric ops: Affine, Rotate, Scale, Shear, Translate
Statistics ops: Mean, Extrema, Histogram
Transform ops: DFT/IDFT, DCT/IDCT

Image Collections

Integrate with the Java Collection API
Operate on a Set, List, Vector, etc. of images
Provide specific image-oriented collections:

Pyramid (decimated and differences)
MIPMap (decimated)
Sequence (timestamped)
Stack (spatially notated)

Image Codecs

Read/write images in many standard formats
Add new encoders and decoders
Auto-detect image formats based on header bytes or full scan
Handle tiled file formats

Interpolation/Border Extension/Warping

Objects that perform common imaging sub-operations
Interpolation: compute a subpixel value
BorderExtender: produce data outside of an image for interpolators
Warp: compute a rectangle of warped source pixel positions
Leverage subclassing for extensibility
Detect known subclasses for speed

Tile Caching

When a tile is computed, cache it
When a tile is requested, check the cache
Provide a public interface, private implementation
New cache implementations are possible

Tile Scheduling

Feed tile computation to multiple threads
Analyze tile-to-tile dependencies, compute leaves first
Compute multiple tiles at once
Maximize tile reuse

Imaging in the Java Platform - the AWT and
Java 2D APIs

Java 1.0/1.1: the AWT

"push" model

Java 2 platform (a/k/a JDK TM 1.2): Java 2D

"immediate mode" model

Java Advanced Imaging

"pull" model

Java 1.0/1.1 - the AWT
A "Push" Model

java.awt.Image: opaque object, often native
ImageProducer/ImageConsumer:

"Push" pixels to a consumer
Good for Web graphics

image.getGraphics: perform drawing on an image

Can only draw on "off-screen" images
Simple raster drawing methods

Java 2 Platform (a/k/a JDK(TM) 1.2) - Java 2D
An "Immediate Mode" Model

java.awt.image.*: many new classes
BufferedImage: in-memory image, single tile
Immediate-mode operations (BufferedImageOp)
Accelerated display of common formats
RenderedImage/RenderableImage: expansion interfaces

Introduction to the Java(TM) Advanced Imaging API

Java Advanced Imaging
A "Pull" Model

javax.media.jai.PlanarImage (superclass)
TiledImage: in-memory image with multiple tiles
OpImage: procedurally-defined image
Compute tiles on demand - "pull" pixels from sources

Fundamental Java 2D Image Classes

DataBuffer
SampleModel
Raster
ColorModel
BufferedImage
RenderedImage
ParameterBlock
RenderedImageFactory (RIF)
RenderableImage

Java 2D Image Classes - DataBuffer

A place to store pixels
Arrays must have a specific type (no "void *" equivalent)

Java 2D currently supports only integral data types

Can have multiple "banks" per DataBuffer, offset per bank

Java 2D Image Classes - SampleModel

Describes the layout of pixels in a DataBuffer
Has a width and height
getPixels/setPixels

Extract/insert data from a simple external format

getPixelData/setPixelData

Extract/insert data in a more optimized format

Can recognize known subclasses, access pixels directly

Java 2D Image Classes -
ComponentSampleModel

Each sample lives in its own DataBuffer array element
Superclass for PixelInterleavedSampleModel and
BandedSampleModel
JAI prefers to work with ComponentSampleModel

PixelInterleavedSampleModel

Samples for a given pixel are adjacent
Get to next pixel by adding pixelStride to the index
Get to next line by adding scanlineStride to the index

Java 2D Image Classes - BandedSampleModel

Samples for a given pixel live in separate buffers
Get to next pixel by adding 1 to the index

MultiPixelPackedSampleModel

Each pixel has a single sample (e.g., gray scale or palette images)
Multiple pixels are stored in a single array element
Get to next sample by shifting a given number of bits

SinglePixelPackedSampleModel

Each pixel lives in its own DataBuffer array element
Samples for a given pixel are concatenated

Java 2D Image Classes - Raster

SampleModel + DataBuffer + (X, Y) Origin
Raster is read-only, WritableRaster is read/write
Constructed using factory methods defined in Raster
Can create a "child" Raster

Access a sub-rectangle/sub-band set, translate to a new origin

Java 2D Image Classes - ColorModel

Provides colorimetric interpretation of a pixel

Encapsulates translation of a pixel value to color components and
alpha
Requires a java.awt.color.ColorSpace object

Required for rendering to screen/printer or for color conversion
Assumes pixel components have been extracted using a SampleModel
Can use shifting and masking, lookup tables, arbitrary computation

Java 2D Image Classes - BufferedImage

Java 2D’s fundamental image type
Raster + ColorModel
A single-tile, in-memory image
Extends java.awt.Image and implements
WritableRenderedImage

Java 2D Image Classes - RenderedImage

Expansion interface representing an image with read-only data

Implemented by BufferedImage and JAI images

An image defined by dimensions, tile grid, SampleModel, ColorModel

The ColorModel may be null

Tiles retrieved by a getTile method

Arbitrary data regions obtained via getData or copyData

Provides interface for name-value style attributes (properties)
Both stored and procedural implementations possible
WritableRenderedImage subinterface allows editing, tracking of
edits

Java 2D Image Classes - ParameterBlock

Contains image sources and other parameters to an operation
Used in Java 2D for renderable operations
Used in JAI for all operations

Java 2D Image Classes - RenderedImageFactory

An interface containing a single method:
RenderedImage create(ParameterBlock paramBlock,
RenderingHints renderHints)
Used by Java 2D as part of RenderableImage instantiation
Used by JAI to instantiate a potential implementation of an operation
May return null if factory cannot produce result

Java 2D Image Classes - RenderableImage

A resolution- and rendering-independent image
Details are opaque
Produces renderings on demand
Will discuss in detail later

Fundamental JAI Image Classes

ImageJAI
PlanarImage
TiledImage
CollectionImage
OpImage
PointOpImage/AreaOpImage/UntiledOpImage/WarpOpImage

Fundamental JAI Image Classes - ImageJAI

Top-level JAI interface
Contains property query methods

String[] getPropertyNames()
String[] getPropertyNames(String prefix)
Object getProperty(String name)

Fundamental JAI Image Classes - PlanarImage

Top-level JAI image class
Implements java.awt.image.RenderedImage and ImageJAI
Maintains source/sink information
Adds "snapshot" capability
Adds "cobbling" methods
Adds "extension" methods
Concrete subclasses must implement getTile()

Fundamental JAI Image Classes - TiledImage

Straightforward in-memory tiled image
Extends PlanarImage
Implements java.awt.image.WritableRenderedImage
Implements createGraphics to allow arbitrary drawing

Fundamental JAI Image Classes -
CollectionImage

Uses standard Java Collection API classes

Set, List, Map, Tree, Vector, ...

Image->Image operations automatically generalize to
Collection->Collection
Collection->Image and Image->Collection ops also possible
Nested collections are supported.

Fundamental JAI Image Classes - OpImage

Extends PlanarImage
Implements getTile() to call computeTile(), cache result

computeTile() depends on computeRect()

Adds abstract source/dest geometric dependency methods
mapSourceRect() and mapDestRect()
Superclass for all image processing operators

Fundamental JAI Image Classes - OpImage
subclasses

PointOpImage
AreaOpImage
WarpOpImage
UntiledOpImage

PointOpImage

Each destination pixel depends only on its corresponding source pixel

mapDestRect() is the identity mapping.

No need to "cobble" sources
Can support in-place operation to reduce memory requirements

Fundamental JAI Image Classes - AreaOpImage

Each destination pixel depends on a fixed-size neighborhood around its
corresponding source pixel

mapDestRect() "pads" by a fixed amount on each side

Sources require some "cobbling" around tile edges
Subclass only needs to describe the neighborhood shape (padding)

WarpOpImage

Each destination pixels depends on a neighborhood around its
corresponding source pixel
Destination-to-source correspondence may be complex

Backward mapping defined by a Warp object
Required padding defined by an Interpolation object

Provides a simple interface for porting, ops may need to override for
performance

UntiledOpImage

Each destination pixel depends on all source pixels

computeTile() implemented in terms of abstract
computeImage()

Best to cobble source once
Examples: Fourier transforms

Other Fundamental JAI Classes

JAI
OperationRegistry
RenderedOp

Other Fundamental JAI Classes - JAI

A set of static create() methods for instantiating ops
Instances contain references to:

An OperationRegistry
A TileCache
A TileScheduler
A set of rendering hints (java.awt.RenderingHints)

Other Fundamental JAI Classes -
OperationRegistry

Maps operation names to a sequence of RenderedImageFactory
instances
Each RIF is called in turn to produce a rendering
First one to succeed "wins"
New RIFs may be added for new or existing operations
Preferences between RIFs may be modified

Other Fundamental JAI Classes - RenderedOp

A lightweight object containing:

An operation name
A ParameterBlock with sources and parameters
A reference to an OperationRegistry

Used to create graphs of operations
RenderedOp graphs can be transferred over the network, re-rendered
using classes available on the other host
A RenderedOp acquires a rendering on demand (just-in-time)

JAI Non-image Classes

Histogram
KernelJAI
LookupTableJAI

ColorCube

PerspectiveTransform
ROI

ROIShape

Instantiating a JAI Operation

User calls JAI.create(name, params, hints)

Parameters checked for validity
Returns a RenderedOp storing the name, parameters, hints,
registry

Pixel data or other information requested from the RenderedOp

RenderedOp calls OperationRegistry.create
OperationRegistry walks the prioritized list of RIFs, calls
create() on each one
RIF create parses arguments
-> returns null if it cannot handle the case
-> returns a new OpImage subclass (or any RenderedImage)

Instantiating a JAI Rendered Chain

Working With Rendered Images and Properties


Working With Rendered Images and Properties -
Outline

The RenderedImage interface
Rendered image semantics
Importing an image
Displaying an image
Performing an operation on an image
Writing a new Rendered operation
Adding properties to images
Managing image properties

The RenderedImage Interface

Implemented by BufferedImage in Java 2D and PlanarImage in JAI
getMinX()/getMinY()/getWidth()/getHeight()
getTileGrid{X,Y}Offset()/getTileWidth()/getTileHeight()
getSampleModel()
getColorModel()
getTile() returns a single tile as a Raster

getData() returns a rectangular region as a Raster
copyData() fills or returns a WritableRaster

Property accessor methods exist

RenderedImage Semantics

Has fixed bounds, pixel resolution
May contain multiple tiles
Each tile has a common SampleModel, ColorModel

The ColorModel may be null

Tiles may extend past the image bounds
Tiles are accessed procedurally, may be computed on demand

Importing an Image

AWTImage operation brings old-style images into JAI
Can use a BufferedImage or other RenderedImage directly
JAI automatically "wraps" an external RenderedImage with a
PlanarImage
Standard set of image readers:

BMP, GIF, FlashPix, JFIF, PNG, PNM, TIFF

Simple interface:
String filename = "image.png";
JAI.create("fileload", filename);

Displaying an Image

Java 2D handles basic image display via Graphics2D:
drawRenderedImage(RenderedImage, AffineTransform)

Future acceleration possible via GraphicsJAI wrapper class
CanvasJAI class automates use of GraphicsJAI

Performing an Operation on an Image

Syntax:
JAI.create(String name,
ParameterBlock pb,
RenderingHints rh)
Operation name is located using the OperationRegistry
ParameterBlock references the sources and parameters
Rendering hints contain "optional" info:

ImageLayout - output image bounds, tile grid, SampleModel
(incl. data type), ColorModel
Which TileCache instance to use
Which BorderExtender instance to use

Code Example

Perform a Rendered operation, set output SampleModel:
RenderedImage src1, src2;
ParameterBlock pb = new ParameterBlock();
pb.addSource(src1);
pb.addSource(src2);
pb.add(param1);
pb.add(param2);
pb.add(param3);

ImageLayout layout = new ImageLayout();
layout.setSampleModel(sampleModel);
RenderingHints rh =
new RenderingHints(JAI.KEY_IMAGE_LAYOUT,
layout);

RenderedImage dst = JAI.create("opname", pb, rh);

Writing a new Rendered Operation

Subclass OpImage
Implement one of: getTile/computeTile/computeRect
getTile: handle all processing
computeTile

Allow OpImage to cache the output

computeRect(Raster[], WritableRaster, Rectangle)

Allow OpImage to cobble sources

computeRect(PlanarImage[], WritableRaster,
Rectangle)

Use uncobbled sources

Implement RIF or CRIF as appropriate
Implement OperationDescriptor
Register OperationDescriptor, (C)RIF with OperationRegistry

Dealing With Tiled Sources

Cobbling (copying regions that cross tiles)
Iterators

Cobbling Example (Point Operator)

Source has X tile splits every 100th pixel, destination has X tile splits
every 64th pixel

Cobbling Example (Area Operator)

Each dest pixel X requires source pixels [X - lpad, X + rpad]
Compute dest in four pieces

Large central area does not require cobbling

Iterator Example

Construct an iterator for each source image
Iterator will handle source tile crossings automatically
Iterators currently have lower performance

Adding Properties to Images

File formats may contain meta-information that is decoded as properties
Properties may be set on a node prior to rendering

Managing Image Properties

Image properties on a RenderedOp node come from five sources:
Properties of the node’s sources
Properties of the node’s rendering
Synthetic properties based on the rendering (width, height, ...)
Locally-set properties (set before rendering the node)
PropertyGenerator objects attached to the node

Can access the other four types of sources

Managing Image Properties (cont.)

Writing a PropertyGenerator

A PropertyGenerator computes a named property on behalf of a
Rendered or Renderable operation node
May be set to automatically attach to nodes performing a given operation
Implement getPropertyNames, getProperty
Example: DFT op has a property generator that tells whether the output
image is Real or Complex
Computation is based on same property in source, parameter of the op

Working With Renderable Images


Working With Renderable Images - Outline

Renderable image semantics
The RenderableImage interface
Building renderable chains
Rendering a chain
Editing a chain
Writing a new Renderable operation

Renderable Image Semantics

Abstract description of a scene
Floating point coordinate system

Default of 1.0 high by aspect ratio wide

Two way flow of information
Lightweight layer

The RenderableImage Interface

Implemented by RenderableImageOp (Java 2D) and RenderableOp
(JAI)
getMinX(), getMinY(), getWidth(), getHeight() (float)
create{Scaled,Default}Rendering()
getParameterBlock()
getSources()
isDynamic()
setParameterBlock() (RenderableImageOp & RenderableOp)

RenderContext

Defined in Java 2D package java.awt.image.renderable
Provides context in which rendering will be used:

User to device transform (Renderable -> Rendered Coordinates)
as a java.awt.geom.AffineTransform
Area of interest (in Renderable coordinates) as a
java.awt.Shape
Rendering hints (tweak rendering)

The ContextualRenderedImageFactory (CRIF)
Interface

Defined in Java 2D package java.awt.image.renderable
Plays role analogous to that of RenderedImageFactory
mapRenderContext() transforms RenderContext for inputs
create() Same role as RIF.create
isDynamic() indicates if host can cache previous results.

Building Renderable Chains

Similar approach as for RenderedOps
Sources and parameters put in ParameterBlock
RenderableOp uses OperationRegistry to get CRIF

CRIFs do not have preferences

JAI.createRenderable(...)
Custom implementation of RenderableImage - do whatever...

Rendering a Chain

createRendering() Methods.
Mechanics

createRendering Calls

createRendering(RenderContext renderContext)

Primary interface, uses RenderContext to generate a
RenderedImage

createScaledRendering(int w, int h, RenderingHints
hints)

Computes appropriate user to device transform for requested
image size

createDefaultRendering()

Computes a ’default’ size image

Renderable Rendering Mechanics

Update RenderContext for each source

mapRenderContext method in CRIF

Call createRendering() for each source
Apply your operation, if any, to resultant RenderedImages

create() method in CRIF

Return your operation

Instantiating a JAI Renderable Chain

Editing a Chain

Edits to a RenderableImage chain are seen in the result of the next
createRendering call
setParameterBlock

Change sources
Change parameters

isDynamic:

Subsequent createRendering calls may differ, even if
ParameterBlock has not changed

Writing a new Renderable Operation

RenderableImageOp
ContextualRenderedImageFactory (CRIF)
Implement isDynamic()
Implement Rectangle2D getBounds2D(ParameterBlock
paramBlock)

see Rectangle2D.Float and Rectangle2D.Double

Implement mapRenderContext()
Implement create()

Implementing mapRenderContext

Decode ParameterBlock
Decode Hints (from dest)
Check Properties (from source)
Create RenderContext for input.

mapRenderContext Example

public RenderContext
mapRenderContext(int i, RenderContext rc,
ParameterBlock pb,
RenderableImage parent) {
if (i > 1) {
throw new IllegalArgumentException
("Composite only takes two inputs");
}

RenderContext ret = (RenderContext)rc.clone();
Shape rcShape = rc.getAreaOfInterest();
Shape inShape = (Shape)pb.getObjectParameter(i);

ret.setAreaOfInterest(intersect(inShape, rcShape));
return ret;
}

Implementing create

Figure out what you were going for in mapRenderContext
Construct RenderedImage based on inputs
Often passes off work to RIF

create Example

public RenderedImage create(RenderContext rc,
ParameterBlock srcPB) {
AffineTransform at = rc.getTransform();
Shape aoi = rc.getAreaOfInterest();

aoi = at.createTransformedShape(aoi);

ParameterBlock pb = new ParameterBlock();
for (int i=0; i < srcPB.getNumSources(); i++) {
RenderedImage ri = srcPB.getRenderedSource(0);
Shape clip = (Shape)srcPB.getObjectParameter(0);

clip = at.createTransformedShape(clip);
clip = intersect(clip, aoi);

pb.addSource(ri);
pb.add(clip);
}
return create(pb, rc.getRenderingHints());
}

Renderable Summary

Framework for resolution independent imaging.
Promotes two way flow of information
Lightweight layer

Working With Remote Images


Working With Remote Images - Outline

Remote image sources
Overview of networking concepts in the Java platform
The RemoteImage interface

Remote Image Sources

Images from network file systems, URLs, sockets, etc.

Make connection, use InputStream abstraction

Images computed by a remote JAI server

Use Remote Method Invocation to make image appear local

Overview of Networking Concepts in the Java
Platform

java.io package: input and output streams, filters

Streams may be chained together to add functionality

java.net package: sockets, URLs
java.rmi package: create Java technology objects on another host

Use serialization to define the wire protocol
Transfer deep copies to the remote host
Maintain a local stub object

The RemoteImage Interface

Extends PlanarImage
Constructor specifies the server where it will live
Evaluates a RenderedOp or RenderableOp chain
Chain sent to server using Serialization, RMI
Multiple remote servers may be used

Data may pass between servers without returning to client

Example of Using RemoteImage

Using RemoteImage is straightforward:
PlanarImage source0, source1;
RenderedOp op = JAI.create("add", source0, source1);
RemoteImage im = new RemoteImage(serverName, op);
Now im can be used like any other image

Using RemoteImage Efficiently

Ensure sources can be copied to the server efficiently

URL and FileLoad ops are good candidates

Ensure output can be sent back to the local host efficiently

Use RenderableOp chains to get only the necessary resolution

Using the JAI Image I/O Package (unofficial)


Using the JAI Image I/O Package (unofficial) -
Outline

InputStream and extensions
The ImageCodec class
The ImageDecoder and ImageEncoder interfaces
JAI I/O operations
Adding new image decoders and encoders

The JAI Image I/O Package (unofficial)

JAI provides an unofficial image I/O package

This is to be superseded by a separate image I/O package
New image I/O package initially separate, later in JRE (JDK)

Supports reading and writing common image formats
New formats may be added
Codecs are based around standard InputStream/OutputStream
classes
Package may be used separately from the rest of JAI
A new standard extension will subsume this package

The InputStream Abstraction

The InputStream class represents a source of bytes
Some InputStream methods:

read: read bytes
skip: skip bytes
mark/reset/markSupported: return to a previous point
File sources: stream = new FileInputStream(filename);
Network sources: stream = Socket.getInputStream();

Adding Functionality to InputStream

ByteArrayInputStream: read from data already in memory
FileInputStream: read from a local file on disk
DataInputStream: read from another stream, adding methods:

readChar, readInt, readFloat, readDouble, etc.

BufferedInputStream: read from another stream, perform buffering

The ImageCodec Classes (unofficial)

ImageCodec: registry of classes supporting a given format
ImageDecoder: abstract superclass for image decoders
ImageDecoderParam: abstract superclass for decoding parameters
ImageEncoder: abstract superclass for image encoders
ImageEncoderParam: abstract superclass for encoding parameters

The ImageCodec Class (unofficial)

One subclass of ImageCodec for each supported format
Has a set of static factory methods:

Query codecs
Register/unregister codecs
Locate codecs that support a particular format
Each format implements instance methods for its format:

Get information about the format
Recognize its format in an InputStream
Create ImageDecoder and ImageEncoder instances

The ImageCodec Class (unofficial)

ImageCodec is generally used statically:
public ImageDecoder
getDecoder(InputStream src) {
String[] names =
ImageCodec.getDecoderNames(src);
System.out.println("Image is in " +
names[0] + " format.");
return ImageCodec.createImageDecoder(names[0],
src,
null);
}

The ImageDecoder and ImageEncoder
Interfaces (unofficial)

No concrete public implementations
Set parameters, perform decoding or encoding
Each codec has its own unique parameter class

"Do the right thing" if no parameter supplied
Set some basic parameters without knowing about the specific
implementation

Working With ImageCodecs (unofficial)

Decoding a stream is simple:
public RenderedImage
decodeImage(InputStream src) {
ImageDecoder dec = getDecoder(src);
return dec.decodeAsRenderedImage();
}

public void
encodeImage(RenderedImage im,
OutputStream dst,
String format) {
ImageEncoder enc =
ImageCodec.createImageEncoder(format,
dst,
null);
enc.encode(im);
dst.flush();
}

The JAI "fileload," "url," and "stream"
Operations

JAI has standard operators to simplify the image loading process:
String filename;
im = JAI.create("fileload", filename, hints);
URL url;
im = JAI.create("url", url, hints);
SeekableStream stream; // unofficial
im = JAI.create("stream", stream, hints);

Writing a new Codec (unofficial)

Write an ImageDecoder subclass
Write an ImageEncoder subclass
Write an ImageCodec subclass
Register the new ImageCodec

Write an ImageDecoder Subclass (unofficial)

Implement a decode method
Typically, return a new object that implements RenderedImage
Simple case: return value is a BufferedImage
More sophisticated: object implements RenderedImage, responds to
getTile() requests
This approach allows use of sources such as tiled TIFF, FlashPIX, NITF

Write an ImageEncoder Subclass (unofficial)

Implement an encode method
Encoder uses image format (bit depth, ColorModel) and encoder param
to determine output format

Write an ImageCodec Subclass (unofficial)

Implement instance methods:

getFormatName
getNumHeaderBytes
isFormatRecognized
createImageDecoder
createImageEncoder
canEncodeImage(RenderedImage, ImageEncodeParam)

Register the new ImageCodec (unofficial)

ImageCodec.registerCodec(new XXXCodec());

Writing an Application With JAI


Writing an Application With JAI - Outline

Setting up the JAI runtime environment
Integrating with the Swing toolkit
Integrating with browsers
An inside look at a JAI application

Setting up the JAI Runtime Environment

Add JAI .jar files to classpath when compiling and running

jai_core.jar
jai_codec.jar (name will change)
mlibwrapper_jai.jar (optional)

Place the directory containing native libraries on the library search path

Windows TM : PATH variable
Solaris TM : LD_LIBRARY_PATH variable

jai_core.jar contains a registry initialization file
mlibwrapper_jai.jar contains "glue" for native acceleration, may
be deleted
jai_codec.jar may be used without jai_core.jar for image I/O

Integrating With the Swing Toolkit

Can draw to a Swing canvas using normal Graphics2D methods:

drawRenderedImage()
drawRenderableImage()

Swing has an Icon interface that draws an image on demand

IconJAI class demonstrates how to integrate with JAI

BufferedImage is a WritableRenderedImage

Can be constructed from a Raster
Can be populated with one of the CopyData methods or drawing.

Example DisplayJAI to be discussed

Integrating With Browsers

JAI requires the Java 2 platform.

Sun’s Java Plug-in supports the Java 2 platform

Security issues (Native code)

An Inside Look at an Application Using JAI

Describe application
Describe operator
Operator implementation

OpImage subclass
Image factory classes: RIF and CRIF
OperationDescriptor

Swing display component
Example Application

OperationRegistry configuration
Use of Swing display component
Use of new operator

Describe Application

Simple image viewer and subsampler
Loads image from a URL

URL entered in a JTextField

Subsamples image using new operator

Subsampling factor set via a JSlider

Uses Rendered layer

Describe Operator

Subsampling operator "Subsample"

specific implementation for byte data only
integral subsampling factors
averages over adaptive source window

Supports rendered and renderable layers
Preferences can be set for interplay with "Scale" operator

"Scale" can backup "Subsample"
"Subsample" can be used for "Scale" in some cases

Operator Implementation

OpImage subclass
OperationDescriptor

OpImage subclass

Performs actual computation
Derives from OpImage hierarchy in javax.media.jai

OpImage subclass - constructor

public class SubsampleOpImage extends WarpOpImage {

// [...]

SubsampleOpImage(RenderedImage source,
BorderExtender extender,
TileCache cache,
ImageLayout layout,
int scaleX,
int scaleY) {
super(source, extender, cache, // fwd to superclass
setLayout(layout, source, scaleX, scaleY), // set bounds
createWarp(scaleX, scaleY), // Warp object
new InterpolationAverage(scaleX, scaleY), // Interpolation obj.
true); // cobble sources

OpImage subclass - constructor (continued)
private static final ImageLayout setLayout(ImageLayout layout,
RenderedImage source,
int scaleX,
int scaleY) {
if(scaleX <= 0 || scaleY <= 0) {
throw new IllegalArgumentException("Non-positive scale factor.");
}

if(layout != null &&
(layout.getValidMask() &
(ImageLayout.MIN_X_MASK | ImageLayout.MIN_Y_MASK |
ImageLayout.WIDTH_MASK | ImageLayout.HEIGHT_MASK)) != 0) {
return layout; // bounds set in layout parameter
}

ImageLayout il = layout == null ?
new ImageLayout() : (ImageLayout)layout.clone();

// Set the bounds to the scaled source bounds.
Rectangle bounds =
forwardMapRect(source.getMinX(), source.getMinY(),
source.getWidth(), source.getHeight(),
scaleX, scaleY);
il.setMinX(bounds.x);
il.setMinY(bounds.y);
il.setWidth(bounds.width);
il.setHeight(bounds.height);

return il;
}

OpImage subclass - forward mapping method
public Rectangle mapSourceRect(Rectangle sourceRect,
int sourceIndex) {
if (sourceIndex != 0) { // this image only has one source
throw new IllegalArgumentException("Non-zero source index.");
}

return forwardMapRect(sourceRect.x + interp.getRightPadding(),
sourceRect.y + interp.getBottomPadding(),
sourceRect.width - interp.getWidth(),
sourceRect.height - interp.getHeight(),
scaleX, scaleY);
}

private static final Rectangle forwardMapRect(int x, int y, int w, int h,
int scaleX, int scaleY) {
float sx = 1.0F/scaleX;
float sy = 1.0F/scaleY;

x = Math.round(x*sx);
y = Math.round(y*sy);

return new Rectangle(x, y,
Math.round((x + w)*sx) - x,
Math.round((y + h)*sy) - y);
}

OpImage subclass - backward mapping method

public Rectangle mapDestRect(Rectangle destRect,
int sourceIndex) {
if (sourceIndex != 0) { // this image only has one source
throw new IllegalArgumentException("Non-zero source index.");
}

return new Rectangle(destRect.x*scaleX - interp.getLeftPadding(),
destRect.y*scaleY - interp.getTopPadding(),
destRect.width*scaleX + interp.getWidth(),
destRect.height*scaleY + interp.getHeight());
}

OpImage subclass - computation method

public void computeRect(Raster [] sources,
WritableRaster dest,
Rectangle destRect) {
// Get RasterAccessor tags (initialized in OpImage superclass).
RasterFormatTag[] formatTags = getFormatTags();

// Get destination accessor.
RasterAccessor dst = new RasterAccessor(dest, destRect,
formatTags[1],
getColorModel());

if(dst.getDataType() != DataBuffer.TYPE_BYTE) {
throw new IllegalArgumentException("Supports byte data only.");
}

// Get source accessor.
RasterAccessor src = new RasterAccessor(sources[0],
mapDestRect(destRect, 0),
formatTags[0],
getSource(0).getColorModel());

OpImage subclass - computation method (continued)
// Get dimensions.
int dwidth = dst.getWidth();
int dheight = dst.getHeight();
int dnumBands = dst.getNumBands();

// Get destination data array references and strides.
byte dstDataArrays[][] = dst.getByteDataArrays();
int dstBandOffsets[] = dst.getBandOffsets();
int dstPixelStride = dst.getPixelStride();
int dstScanlineStride = dst.getScanlineStride();

// Get source data array references and strides.
byte srcDataArrays[][] = src.getByteDataArrays();
int srcBandOffsets[] = src.getBandOffsets();
int srcPixelStride = src.getPixelStride();
int srcScanlineStride = src.getScanlineStride();

// Compute scaled source strides.
int srcPixelStrideScaled = scaleX * srcPixelStride;
int srcScanlineStrideScaled = scaleY * srcScanlineStride;

// Get the dimensions of the averaging area and cache their product.
int numCols = interp.getWidth();
int numRows = interp.getHeight();
float denom = numCols*numRows;

for (int k = 0; k < dnumBands; k++) {
byte dstData[] = dstDataArrays[k];
byte srcData[] = srcDataArrays[k];
int srcScanlineOffset = srcBandOffsets[k];
int dstScanlineOffset = dstBandOffsets[k];

for (int j = 0; j < dheight; j++) {
int srcPixelOffset = srcScanlineOffset;
int dstPixelOffset = dstScanlineOffset;

for (int i = 0; i < dwidth; i++) {
int imageVerticalOffset = srcPixelOffset;

// Average the source over the scale-dependent window.
int sum = 0;
for (int u = 0; u < numRows; u++) {
int imageOffset = imageVerticalOffset;
for (int v = 0; v < numCols; v++) {
sum += (int)(srcData[imageOffset]&0xff);
imageOffset += srcPixelStride;
}
imageVerticalOffset += srcScanlineStride;
}

dstData[dstPixelOffset] = (byte)(sum / denom + 0.5F);


srcPixelOffset += srcPixelStrideScaled;
dstPixelOffset += dstPixelStride;
}
srcScanlineOffset += srcScanlineStrideScaled;
dstScanlineOffset += dstScanlineStride;
}
}

// If the RasterAccessor set up a temporary write buffer for the
// operator, tell it to copy that data to the destination Raster.
if (dst.isDataCopy()) {
dst.clampDataArrays();
dst.copyDataToRaster();
}
}


Registered with OperationRegistry
RIF creates an instance of the OpImage subclass

RIF Implementation

public class SubsampleCRIF implements ContextualRenderedImageFactory {
/** Default constructor. */
public SubsampleCRIF() {}

public RenderedImage create(ParameterBlock paramBlock,
RenderingHints renderHints) {
RenderedImage source = paramBlock.getRenderedSource(0);

BorderExtender extender = renderHints == null ? null :
(BorderExtender)renderHints.get(JAI.KEY_BORDER_EXTENDER);
TileCache cache = renderHints == null ? null :
(TileCache)renderHints.get(JAI.KEY_TILE_CACHE);
ImageLayout layout = renderHints == null ? null :
(ImageLayout)renderHints.get(JAI.KEY_IMAGE_LAYOUT);

int scaleX = paramBlock.getIntParameter(0);
int scaleY = paramBlock.getIntParameter(1);

return new SubsampleOpImage(source, extender, cache, layout,
scaleX, scaleY);
}

// [...]
}

OperationDescriptor

Specifies:

the behavior of the operator
what layers it supports (rendered/renderable)
number and types of sources
parameters

Provides:

default initialization of some parameters
validation of parameters
property generation

Registered with OperationRegistry

OperationDescriptor Implementation
/**
* An OperationDescriptor describing the "Subsample" operation.
*
* The "Subsample" operation subsamples an image by integral factors.
* The value of each destination pixel is calculated by averaging all source
* pixel values in a scale-dependent rectangular window around the backward
* mapped position. The furnished scale factors express the ratio of the
* source dimensions to the destination dimensions.
*/
public class SubsampleDescriptor extends OperationDescriptorImpl {

/**
* The resource strings that provide the general documentation
* and specify the parameter list for this operation.
*/
private static final String[][] resources = {
{"GlobalName", "Subsample"},
{"LocalName", "Subsample"},
{"Vendor", "org.s2000.courses.jai"},
{"Description", "Subsamples an image."},
{"DocURL", "http://org.s2000.courses/jai/SubsampleDescriptor.html"},
{"Version", "0.0"},
{"arg0Desc", "The X scale factor."},
{"arg1Desc", "The Y scale factor."}
};

(continued)

/** The parameter class list for this operation. */
private static final Class[] paramClasses = {
java.lang.Integer.class, java.lang.Integer.class
};

/** The parameter name list for this operation. */
private static final String[] paramNames = {
"scaleX", "scaleY"
};

/** The parameter default value list for this operation. */
private static final Object[] paramDefaults = {
new Integer(2), new Integer(2)
};

/** Constructor. */
public SubsampleDescriptor() {
super(resources, 1, paramClasses, paramNames, paramDefaults);
}

(continued)

/** Returns true since renderable operation is supported. */
public boolean isRenderableSupported() {
return true;
}

/**
* Returns an array of PropertyGenerators implementing
* property inheritance for the "Subsample" operation.
*
* @return An array of property generators.
*/
public PropertyGenerator[] getPropertyGenerators() {
PropertyGenerator[] pg = new PropertyGenerator[1];
pg[0] = new SubsamplePropertyGenerator();
return pg;
}

(continued)

protected boolean validateParameters(ParameterBlock args,
StringBuffer msg) {
if (!super.validateParameters(args, msg)) {
return false;
}

int scaleX = args.getIntParameter(0);
int scaleY = args.getIntParameter(1);
if (scaleX <= 0 || scaleY <= 0) {
msg.append(getName() +
" operation requires positive scale factors.");
return false;
}

return true;
}

public Number getParamMinValue(int index) {
if (index == 0 || index == 1) {
return new Integer(1);
} else {
throw new ArrayIndexOutOfBoundsException();
}
}
}

Swing image display component

Swing component able to contain an image
The size of the image and size of the container can be different
The image can be positioned within the container
Extends JPanel in order to support layout management

Swing image display component implementation

public class DisplayJAI extends JPanel {

/** image to display */
protected RenderedImage source = null;

/** image origin relative to panel origin */
protected int originX = 0;
protected int originY = 0;

public DisplayJAI(RenderedImage image) {
super();
setLayout(null);
source = image;
setPreferredSize(new Dimension(source.getWidth(),
source.getHeight()));
}

Swing image display component implementation
(continued)

public synchronized void paintComponent(Graphics g) {

Graphics2D g2d = (Graphics2D)g;

// empty component (no image)
if ( source == null ) {
g2d.setColor(getBackground());
g2d.fillRect(0, 0, getWidth(), getHeight());
return;
}

// account for borders
Insets insets = getInsets();
int tx = insets.left + originX;
int ty = insets.top + originY;

// clear damaged component area
Rectangle clipBounds = g2d.getClipBounds();
g2d.setColor(getBackground());
g2d.fillRect(clipBounds.x,
clipBounds.y,
clipBounds.width,
clipBounds.height);

// Translation moves the entire image within the container
g2d.drawRenderedImage(source,
AffineTransform.getTranslateInstance(tx, ty));
}

Example Application

Use of Swing display component
Use of new operator

Class Definition

public class JAIExample extends JFrame
implements ActionListener, ChangeListener {

private static Dimension screenSize =
Toolkit.getDefaultToolkit().getScreenSize();

private static String DEFAULT_URL_SPEC =
"http://istserver.eng/images/tiff/Barn_Doorknob.tiff";

private static RenderingHints renderHints =
new RenderingHints(JAI.KEY_BORDER_EXTENDER,
BorderExtender.createInstance(
BorderExtender.BORDER_COPY));

// Instance variables.
JTextField textField = null;
JSlider slider = null;
RenderedImage source = null;
DisplayJAI display = null;


static {
OperationRegistry registry =
JAI.getDefaultInstance().getOperationRegistry();

registry.registerOperationDescriptor(new SubsampleDescriptor(),
"Subsample");

ContextualRenderedImageFactory subsampleCRIF = new SubsampleCRIF();
registry.registerCRIF("Subsample", subsampleCRIF);
registry.registerRIF("Subsample", "org.s2000.courses.jai",
subsampleCRIF);

RenderedImageFactory subsampleByScaleRIF = new SubsampleByScaleRIF();
registry.registerRIF("Subsample", "org.s2000.courses.jai",
subsampleByScaleRIF);
registry.setRIFPreference("Subsample", "org.s2000.courses.jai",
subsampleCRIF, subsampleByScaleRIF);

registry.registerRIF("Scale", "org.s2000.courses.jai",
new ScaleBySubsampleRIF());
registry.setProductPreference("Scale", "org.s2000.courses.jai",
"com.sun.media.jai");
}

Class construction

public JAIExample(String title) {
super(title);

// Add anonymous inner class to listen for windowClosing event.
// [...]

// Set layout and add URL text field and scale factor slider.
Container contentPane = getContentPane();
contentPane.setLayout(new BorderLayout());
contentPane.add(textField = new JTextField(DEFAULT_URL_SPEC),
BorderLayout.NORTH);
contentPane.add(slider = new JSlider(1, 10, 1),
BorderLayout.CENTER);

// Load default image if possible.
try {
source = JAI.create("url", new URL(textField.getText()));
} catch(Exception e) {
}

// Create image display panel and add it to the frame.
display = source != null ? new DisplayJAI(source) : new DisplayJAI();
contentPane.add(new JScrollPane(display), BorderLayout.SOUTH);

// Listen for events from the slider and the text field.
slider.addChangeListener(this);
textField.addActionListener(this);

pack();
show();
}

ActionListener: actionPerformed()
URL url = new URL(textField.getText());

Dimension size = getSize();
int wOld = source.getWidth();
int hOld = source.getHeight();

source = JAI.create("url", url);

if(source == null) return;

int wNew = source.getWidth();
int hNew = source.getHeight();

Dimension d = getSize();
d.setSize(Math.min(d.width += wNew - wOld,
screenSize.width),
Math.min(d.height += hNew - hOld,
screenSize.height));
setSize(d);

display.setPreferredSize(new Dimension(wNew, hNew));
display.set(source);

slider.setValue(1); // reset scale factor

pack();
show();

ChangeListener: stateChanged()

/** Handle JSlider event: subsample source according to slider valer. */
public void stateChanged(ChangeEvent e) {
Object o = e.getSource();
if(o.equals(slider)) {
int scale = slider.getValue();

ParameterBlock paramBlock =
(new ParameterBlock()).addSource(source).add(scale).add(scale);

RenderedImage image =
JAI.create("subsample", paramBlock, renderHints);

display.set(image);
}
}

Conclusion

JAI 1.0 released June 1999

Latest update is JAI 1.0.2 released December 1999

Visit the JAI home page for the latest information and downloads
http://java.sun.com/products/java-media/jai/index.html

Send questions to the jai-interest@java.sun.com mailing list

See home page for details on how to subscribe

Send comments (private) to jai-comments@sun.com

Trademark Notices

Sun, the Sun logo, Sun Microsystems, Solaris, Java, JDK, and the Java
Coffee Cup logo are trademarks or registered trademarks of Sun
Microsystems, Inc. in the United States and other countries.
Windows is a trademark of Microsoft, Inc.

Introduction to the Java(TM) Advanced Imaging API

More Related Content

What's hot (20)

Viewers also liked (8)

Similar to Introduction to the Java(TM) Advanced Imaging API (20)

More from white paper (20)

Recently uploaded (20)

Introduction to the Java(TM) Advanced Imaging API