Seeing Like Software
0 likesā¢857 views
The document discusses the evolution and applications of computer vision technologies, particularly focusing on the Kinect and related software tools like OpenCV. It provides an overview of various matrix operations, image processing techniques, and programming methods for computer vision. Additionally, it references several resources and projects relevant to the development and exploration of these technologies.
![Overview of OpenCV toolkit
OpenCV 2.3 Cheat Sheet (C++)
The OpenCV C++ reference manual is here:
http: // opencv. willowgarage. com/ documentation/ cpp/ .
Use Quick Search to ļ¬nd descriptions of the particular
functions and classes
Key OpenCV Classes
Point Template 2D point class
Point3 Template 3D point class
Size Template size (width, height) class
Vec Template short vector class
Matx Template small matrix class
Scalar 4-element vector
Rect Rectangle
Range Integer value range
Mat 2D or multi-dimensional dense array
(can be used to store matrices, images,
histograms, feature descriptors, voxel
volumes etc.)
SparseMat Multi-dimensional sparse array
Ptr Template smart pointer class
Matrix Basics
Create a matrix
Mat image(240, 320, CV 8UC3);
[Re]allocate a pre-declared matrix
image.create(480, 640, CV 8UC3);
Create a matrix initialized with a constant
Mat A33(3, 3, CV 32F, Scalar(5));
Mat B33(3, 3, CV 32F); B33 = Scalar(5);
Mat C33 = Mat::ones(3, 3, CV 32F)*5.;
Mat D33 = Mat::zeros(3, 3, CV 32F) + 5.;
Create a matrix initialized with speciļ¬ed values
double a = CV PI/3;
Mat A22 = (Mat <float>(2, 2) <<
cos(a), -sin(a), sin(a), cos(a));
float B22data[] = {cos(a), -sin(a), sin(a), cos(a)};
Mat B22 = Mat(2, 2, CV 32F, B22data).clone();
Initialize a random matrix
randu(image, Scalar(0), Scalar(256)); // uniform dist
randn(image, Scalar(128), Scalar(10)); // Gaussian dist
Convert matrix to/from other structures
(without copying the data)
Mat image alias = image;
float* Idata=new float[480*640*3];
Mat I(480, 640, CV 32FC3, Idata);
vector<Point> iptvec(10);
Mat iP(iptvec); // iP ā 10x1 CV 32SC2 matrix
IplImage* oldC0 = cvCreateImage(cvSize(320,240),16,1);
Mat newC = cvarrToMat(oldC0);
IplImage oldC1 = newC; CvMat oldC2 = newC;
... (with copying the data)
Mat newC2 = cvarrToMat(oldC0).clone();
vector<Point2f> ptvec = Mat <Point2f>(iP);
Access matrix elements
A33.at<float>(i,j) = A33.at<float>(j,i)+1;
Mat dyImage(image.size(), image.type());
for(int y = 1; y < image.rows-1; y++) {
Vec3b* prevRow = image.ptr<Vec3b>(y-1);
Vec3b* nextRow = image.ptr<Vec3b>(y+1);
for(int x = 0; y < image.cols; x++)
for(int c = 0; c < 3; c++)
dyImage.at<Vec3b>(y,x)[c] =
saturate cast<uchar>(
nextRow[x][c] - prevRow[x][c]);
}
Mat <Vec3b>::iterator it = image.begin<Vec3b>(),
itEnd = image.end<Vec3b>();
for(; it != itEnd; ++it)
(*it)[1] ^= 255;
Matrix Manipulations: Copying,
Shuļ¬ing, Part Access
src.copyTo(dst) Copy matrix to another one
src.convertTo(dst,type,scale,shift) Scale and convert to
another datatype
m.clone() Make deep copy of a matrix
m.reshape(nch,nrows) Change matrix dimensions and/or num-
ber of channels without copying data
m.row(i), m.col(i) Take a matrix row/column
m.rowRange(Range(i1,i2))
m.colRange(Range(j1,j2))
Take a matrix row/column span
m.diag(i) Take a matrix diagonal
m(Range(i1,i2),Range(j1,j2)),
m(roi)
Take a submatrix
m.repeat(ny,nx) Make a bigger matrix from a smaller one
flip(src,dst,dir) Reverse the order of matrix rows and/or
columns
split(...) Split multi-channel matrix into separate
channels
merge(...) Make a multi-channel matrix out of the
separate channels
mixChannels(...) Generalized form of split() and merge()
randShuffle(...) Randomly shuļ¬e matrix elements
Example 1. Smooth image ROI in-place
Mat imgroi = image(Rect(10, 20, 100, 100));
GaussianBlur(imgroi, imgroi, Size(5, 5), 1.2, 1.2);
Example 2. Somewhere in a linear algebra algorithm
m.row(i) += m.row(j)*alpha;
Example 3. Copy image ROI to another image with conversion
Rect r(1, 1, 10, 20);
Mat dstroi = dst(Rect(0,10,r.width,r.height));
src(r).convertTo(dstroi, dstroi.type(), 1, 0);
Simple Matrix Operations
OpenCV implements most common arithmetical, logical and
other matrix operations, such as
ā¢ add(), subtract(), multiply(), divide(), absdiff(),
bitwise and(), bitwise or(), bitwise xor(), max(),
min(), compare()
ā correspondingly, addition, subtraction, element-wise
multiplication ... comparison of two matrices or a
matrix and a scalar.
Example. Alpha compositing function:
void alphaCompose(const Mat& rgba1,
const Mat& rgba2, Mat& rgba dest)
{
Mat a1(rgba1.size(), rgba1.type()), ra1;
Mat a2(rgba2.size(), rgba2.type());
int mixch[]={3, 0, 3, 1, 3, 2, 3, 3};
mixChannels(&rgba1, 1, &a1, 1, mixch, 4);
mixChannels(&rgba2, 1, &a2, 1, mixch, 4);
subtract(Scalar::all(255), a1, ra1);
bitwise or(a1, Scalar(0,0,0,255), a1);
bitwise or(a2, Scalar(0,0,0,255), a2);
multiply(a2, ra1, a2, 1./255);
multiply(a1, rgba1, a1, 1./255);
multiply(a2, rgba2, a2, 1./255);
add(a1, a2, rgba dest);
}
ā¢ sum(), mean(), meanStdDev(), norm(), countNonZero(),
minMaxLoc(),
ā various statistics of matrix elements.
ā¢ exp(), log(), pow(), sqrt(), cartToPolar(),
polarToCart()
ā the classical math functions.
ā¢ scaleAdd(), transpose(), gemm(), invert(), solve(),
determinant(), trace() eigen(), SVD,
ā the algebraic functions + SVD class.
ā¢ dft(), idft(), dct(), idct(),
ā discrete Fourier and cosine transformations
For some operations a more convenient algebraic notation can
be used, for example:
Mat delta = (J.t()*J + lambda*
Mat::eye(J.cols, J.cols, J.type()))
.inv(CV SVD)*(J.t()*err);
implements the core of Levenberg-Marquardt optimization
algorithm.
Image Processsing
Filtering
filter2D() Non-separable linear ļ¬lter
sepFilter2D() Separable linear ļ¬lter
boxFilter(),
GaussianBlur(),
medianBlur(),
bilateralFilter()
Smooth the image with one of the linear
or non-linear ļ¬lters
Sobel(), Scharr() Compute the spatial image derivatives
Laplacian() compute Laplacian: āI = ā2
I
āx2 + ā2
I
āy2
erode(), dilate() Morphological operations
1](https://image.slidesharecdn.com/slideshareversion-130624110329-phpapp01/85/Seeing-Like-Software-19-320.jpg)
Seeing Like Software
- 1. S T U D I O S T U D I O Seeing Like Software Andrew Lovett-Barron @readywater
- 2. The machines can see
- 4. CV in the day-to-day
- 6. Rafael Lozano Hemmer Computer Vision and Art
- 13. http://autoponics.org/?p=147 Getting past the Kinect Hack
- 14. Cheap distance and shape
- 15. Victor Castaneda, Nassir Navab Kinect Programming for Computer Vision Summer Term 2011 Kinect and Structured Light
- 16. Victor Castaneda, Nassir Navab Kinect Programming for Computer Vision Summer Term 2011 Other types of Depth Camera
- 17. Sensors live through Software
- 18. OpenCV toolkits
- 19. Overview of OpenCV toolkit OpenCV 2.3 Cheat Sheet (C++) The OpenCV C++ reference manual is here: http: // opencv. willowgarage. com/ documentation/ cpp/ . Use Quick Search to ļ¬nd descriptions of the particular functions and classes Key OpenCV Classes Point Template 2D point class Point3 Template 3D point class Size Template size (width, height) class Vec Template short vector class Matx Template small matrix class Scalar 4-element vector Rect Rectangle Range Integer value range Mat 2D or multi-dimensional dense array (can be used to store matrices, images, histograms, feature descriptors, voxel volumes etc.) SparseMat Multi-dimensional sparse array Ptr Template smart pointer class Matrix Basics Create a matrix Mat image(240, 320, CV 8UC3); [Re]allocate a pre-declared matrix image.create(480, 640, CV 8UC3); Create a matrix initialized with a constant Mat A33(3, 3, CV 32F, Scalar(5)); Mat B33(3, 3, CV 32F); B33 = Scalar(5); Mat C33 = Mat::ones(3, 3, CV 32F)*5.; Mat D33 = Mat::zeros(3, 3, CV 32F) + 5.; Create a matrix initialized with speciļ¬ed values double a = CV PI/3; Mat A22 = (Mat <float>(2, 2) << cos(a), -sin(a), sin(a), cos(a)); float B22data[] = {cos(a), -sin(a), sin(a), cos(a)}; Mat B22 = Mat(2, 2, CV 32F, B22data).clone(); Initialize a random matrix randu(image, Scalar(0), Scalar(256)); // uniform dist randn(image, Scalar(128), Scalar(10)); // Gaussian dist Convert matrix to/from other structures (without copying the data) Mat image alias = image; float* Idata=new float[480*640*3]; Mat I(480, 640, CV 32FC3, Idata); vector<Point> iptvec(10); Mat iP(iptvec); // iP ā 10x1 CV 32SC2 matrix IplImage* oldC0 = cvCreateImage(cvSize(320,240),16,1); Mat newC = cvarrToMat(oldC0); IplImage oldC1 = newC; CvMat oldC2 = newC; ... (with copying the data) Mat newC2 = cvarrToMat(oldC0).clone(); vector<Point2f> ptvec = Mat <Point2f>(iP); Access matrix elements A33.at<float>(i,j) = A33.at<float>(j,i)+1; Mat dyImage(image.size(), image.type()); for(int y = 1; y < image.rows-1; y++) { Vec3b* prevRow = image.ptr<Vec3b>(y-1); Vec3b* nextRow = image.ptr<Vec3b>(y+1); for(int x = 0; y < image.cols; x++) for(int c = 0; c < 3; c++) dyImage.at<Vec3b>(y,x)[c] = saturate cast<uchar>( nextRow[x][c] - prevRow[x][c]); } Mat <Vec3b>::iterator it = image.begin<Vec3b>(), itEnd = image.end<Vec3b>(); for(; it != itEnd; ++it) (*it)[1] ^= 255; Matrix Manipulations: Copying, Shuļ¬ing, Part Access src.copyTo(dst) Copy matrix to another one src.convertTo(dst,type,scale,shift) Scale and convert to another datatype m.clone() Make deep copy of a matrix m.reshape(nch,nrows) Change matrix dimensions and/or num- ber of channels without copying data m.row(i), m.col(i) Take a matrix row/column m.rowRange(Range(i1,i2)) m.colRange(Range(j1,j2)) Take a matrix row/column span m.diag(i) Take a matrix diagonal m(Range(i1,i2),Range(j1,j2)), m(roi) Take a submatrix m.repeat(ny,nx) Make a bigger matrix from a smaller one flip(src,dst,dir) Reverse the order of matrix rows and/or columns split(...) Split multi-channel matrix into separate channels merge(...) Make a multi-channel matrix out of the separate channels mixChannels(...) Generalized form of split() and merge() randShuffle(...) Randomly shuļ¬e matrix elements Example 1. Smooth image ROI in-place Mat imgroi = image(Rect(10, 20, 100, 100)); GaussianBlur(imgroi, imgroi, Size(5, 5), 1.2, 1.2); Example 2. Somewhere in a linear algebra algorithm m.row(i) += m.row(j)*alpha; Example 3. Copy image ROI to another image with conversion Rect r(1, 1, 10, 20); Mat dstroi = dst(Rect(0,10,r.width,r.height)); src(r).convertTo(dstroi, dstroi.type(), 1, 0); Simple Matrix Operations OpenCV implements most common arithmetical, logical and other matrix operations, such as ā¢ add(), subtract(), multiply(), divide(), absdiff(), bitwise and(), bitwise or(), bitwise xor(), max(), min(), compare() ā correspondingly, addition, subtraction, element-wise multiplication ... comparison of two matrices or a matrix and a scalar. Example. Alpha compositing function: void alphaCompose(const Mat& rgba1, const Mat& rgba2, Mat& rgba dest) { Mat a1(rgba1.size(), rgba1.type()), ra1; Mat a2(rgba2.size(), rgba2.type()); int mixch[]={3, 0, 3, 1, 3, 2, 3, 3}; mixChannels(&rgba1, 1, &a1, 1, mixch, 4); mixChannels(&rgba2, 1, &a2, 1, mixch, 4); subtract(Scalar::all(255), a1, ra1); bitwise or(a1, Scalar(0,0,0,255), a1); bitwise or(a2, Scalar(0,0,0,255), a2); multiply(a2, ra1, a2, 1./255); multiply(a1, rgba1, a1, 1./255); multiply(a2, rgba2, a2, 1./255); add(a1, a2, rgba dest); } ā¢ sum(), mean(), meanStdDev(), norm(), countNonZero(), minMaxLoc(), ā various statistics of matrix elements. ā¢ exp(), log(), pow(), sqrt(), cartToPolar(), polarToCart() ā the classical math functions. ā¢ scaleAdd(), transpose(), gemm(), invert(), solve(), determinant(), trace() eigen(), SVD, ā the algebraic functions + SVD class. ā¢ dft(), idft(), dct(), idct(), ā discrete Fourier and cosine transformations For some operations a more convenient algebraic notation can be used, for example: Mat delta = (J.t()*J + lambda* Mat::eye(J.cols, J.cols, J.type())) .inv(CV SVD)*(J.t()*err); implements the core of Levenberg-Marquardt optimization algorithm. Image Processsing Filtering filter2D() Non-separable linear ļ¬lter sepFilter2D() Separable linear ļ¬lter boxFilter(), GaussianBlur(), medianBlur(), bilateralFilter() Smooth the image with one of the linear or non-linear ļ¬lters Sobel(), Scharr() Compute the spatial image derivatives Laplacian() compute Laplacian: āI = ā2 I āx2 + ā2 I āy2 erode(), dilate() Morphological operations 1
- 20. How to navigate openCV
- 21. http://whiteglovetracking.com/ by Evan Roth Focus and Regions of Interest
- 22. Image credit: Kineme Processing to Analysis
- 23. Haar based detection by Adam Harvey Methods of Analysis
- 25. Alternatives to C++ for openCV
- 26. Starting off with OF
- 28. Kyle McDonald showing off ofxCv OfxCv
- 30. Design process
- 34. Learn to see the invisible
- 35. S T U D I O S T U D I O Andrew Lovett-Barron @readywater http://andrewlb.com [email protected]