Abstract image of a woman sitting on books and studying on a laptop

Convolution as matrix multiplication

Download as PPTX, PDF
E
Edwin Efraín Jiménez Lepe

- Convolution can be implemented as matrix multiplication by rearranging the input and weights through techniques like im2col. - Backpropagation through a convolutional layer involves computing the gradient with respect to the weights (d_w) and inputs (d_x) by treating convolution as a matrix multiplication without any weight rotations. - Computing d_x involves performing a full convolution between the gradient of the loss with respect to the outputs (d_y) and the weights, without any transformations to the weights.

Software
Related topics:
Convolutional Neural Networks
1 of 22
Downloaded 140 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Convolution as matrix multiplication • Edwin Efraín Jiménez Lepe
16 24 32 47 18 26 68 12 9 Input 0 1 -1 0 2 3 4 5 W1 W2 ∗ = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 Im2col (input) 0 5 1 3 -1 4 0 2 x W1 W2 = 23 353 50 535 -14 354 -14 248 Rearrange 23 -14 50 -14 353 354 535 248 FeedForward Applying kernel rotation
16 24 32 47 18 26 68 12 9 Input 0 1 -1 0 2 3 4 5 W1 W2 ∗ = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 Im2col (input) 0 5 1 3 -1 4 0 2 x W1 W2 = 23 353 50 535 -14 354 -14 248 Rearrange 24 -13 51 -13 353 354 535 248 Now with bias 1 1 1 1 1 0 FeedForward
16 24 32 47 18 26 68 12 9 Input 0 0 -2.94504954e-05 0 d_y ∗ = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 Im2col (input) 0 0 -2e-05 6e-06 0 0 0 0 x Im2col(d_y) -1.38417328e-03 3.00583533 e-04 -2.00263369e-03 4.34886814 e-04 -5.30108917e-04 1.15117098 e-04 -3.53405945e-04 7.67447318 e-05 Rearrange -1.38417328e-03 -5.30108917e-04 -2.00263369e-03 3.53405945e-04 d_w = input * d_y The update correspond to the Rotated kernel BackPropagation d_w 0 0 6.39539432e-06 0 -1.38417328e-03 -5.30108917e-04 -2.00263369e-03 3.53405945e-04
0 0 -2.94504954e-05 0 d_y d_x = d_y * w (without rotation) BackPropagation 0 0 -6.39539432e-06 0 We need full convolution And keep kernel unrotated 0 0 0 0 0 0 0 0 0 -2.94504954e-05 0 0 0 0 0 0 d_y 0 0 0 0 0 0 0 0 0 6.39539432e-06 0 0 0 0 0 0 0 1 -1 0 2 3 4 5 W1 W2 ∗ 0 1 -1 0 2 3 4 5 W1 W2 ∗
d_x = d_y * w (without rotation) BackPropagation 0 0 0 0 0 0 0 0 0 -2.94504954e-05 0 0 0 0 0 0 d_y 0 0 0 0 0 0 0 0 0 6.39539432e-06 0 0 0 0 0 0 = 0 1 -1 0 2 3 4 5 W1 W2 ∗ 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 0 -1 1 0 x 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 2 4 3 5 x T T
d_x = d_y * w (without rotation) BackPropagation 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 0 -1 1 0 x 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 2 4 3 5 x T T = 0 0 -0.2945e-04 0 0.2945e-04 0 0 0 0 0 0.3198e-04 0.1919-04 0 0.2558-04 0.1279-04 0 0 0
d_x = d_y * w (without rotation) BackPropagation 0 0 -0.2945e-04 0 0.2945e-04 0 0 0 0 0 0.3198e-04 0.1919-04 0 0.2558-04 0.1279-04 0 0 0 + = 0 0.3198e-04 -0.1026e-04 0 0.5503e-04 0.1279-04 0 0 0 reshape 0 0 0 0.3198e-04 0.5503e-04 0 -0.1026e-04 0.1279-04 0
d_x = d_y * w (without rotation) BackPropagation 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 0 -1 1 0 x 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 2 4 3 5 T = In fact, we can do it in just one operation 0 0.3198e-04 -0.1026e-04 0 0.5503e-04 0.1279-04 0 0 0 Notice, every channel of delta is multiplied by the correspondent filter that generates it
A multi-channel example 16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,3,3) (2,3,2,2) Output= (2,2,2) = 2171 2170 5954 2064 13042 13575 11023 6425 Applying theano convolution (which rotates Automatically the filters)
A multi-channel example (vectorized) 16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,3,3) (2,3,2,2) = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 0 60 1 32 -1 22 0 18 5 35 3 7 4 46 2 23 16 78 68 20 24 81 -2 42 x T
A multi-channel example (vectorized) 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 x = T 0 60 1 32 -1 22 0 18 5 35 3 7 4 46 2 23 16 78 68 20 24 81 -2 42 2171 13042 5954 11023 2170 13575 2064 6425 Channel 1 Channel 2 Rearrange 2171 2170 5954 2064 13042 13575 11023 6425
Backpropagation .1678 .098 .002 .246 0.5 0.67 0.21 0.487 Imagine we got the next error from an up-layer And we want to propagate it to the correspondent layer (input of convolution) We need to compute d_y * w (without rotation) But is a ‘full’ convolution, so we add 1 zero padding to d_y 0 0 0 0 0 .1678 .098 0 0 .002 .246 0 0 0 0 0 0 0 0 0 0 0.5 .67 0 0 .21 .487 0 0 0 0 0
Backpropagation d_y-1=d_y * w (without rotation) 0 0 0 0 0 .1678 .098 0 0 .002 .246 0 0 0 0 0 0 0 0 0 0 0.5 .67 0 0 .21 .487 0 0 0 0 0 im2col 0 0 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 0 0 0 0 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 0 0
Backpropagation d_y-1=d_y * w (without rotation) 0 0 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 0 0 0 0 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 0 0 T Notice, every channel of delta is multiplied by the correspondent filter that generates it 0 -1 1 0 2 4 3 5 -2 24 68 16 18 22 32 60 23 46 7 35 42 81 20 78 x = 30 18.339 41.6848 28.7678 11.3634 37.8224 6.722 1.476 4.336 51.0322 47.6112 98.3552 64.376 44.7626 99.7084 19.61 8.981 35.284 14.642 31.212 56.622 22.528 38.992 73.295 8.766 11.693 19.962
Backpropagation d_y-1=d_y * w (without rotation) 30 18.339 41.6848 28.7678 11.3634 37.8224 6.722 1.476 4.336 51.0322 47.6112 98.3552 64.376 44.7626 99.7084 19.61 8.981 35.284 14.642 31.212 56.622 22.528 38.992 73.295 8.766 11.693 19.962 rearrange 30 51.0322 14.642 28.7678 64.376 22.528 6.722 19.61 8.766 18.339 47.6112 31.212 11.3634 44.7626 38.992 1.476 8.981 11.693 41.6848 98.3552 56.622 37.8224 99.7084 73.295 4.336 35.284 19.962
Backpropagation (no vectorized) d_y-1=d_y * w (without rotation) .1678 .098 .002 .246 0.5 0.67 0.21 0.487 (2,2,2) 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (2,3,2,2) Transpose dimensions 0 and 1 .1678 .098 .002 .246 0.5 0.67 0.21 0.487 (2,2,2) 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,2,2,2) Filter 3
Backpropagation (no vectorized, full convolution) d_y-1=d_y * w (without rotation) .1678 .098 .002 .246 0.5 0.67 0.21 0.487 (2,2,2) 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,2,2,2) Filter 3 = 30 51.0322 14.642 28.7678 64.376 22.528 6.722 19.61 8.766 18.339 47.6112 31.212 11.3634 44.7626 38.992 1.476 8.981 11.693 41.6848 98.3552 56.622 37.8224 99.7084 73.295 4.336 35.284 19.962
Backpropagation d_w=input * d_y16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 (3,3,3) ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = d_y (2,2,2) Dimensions do not match, So it is telling us that we need to Apply both filters to any cannel of the input 16 24 32 47 18 26 68 12 9 ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = 9.5588 13.5952 12.7386 7.8064 42.716 49.882 55.684 33.323 26 57 43 24 21 12 02 11 19 ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = 15.1628 16.7726 8.7952 9.3958 66.457 67.564 31.847 30.103 ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = 9.1104 12.9086 6.8332 5.4248 44.252 44.674 33.744 21.991 18 47 21 4 6 12 81 22 13
Backpropagation d_w=input * d_y = 33.832 43.2764 28.367 22.627 153.425 162.12 121.275 85.417 Error associated with rotated kernel, it means We need to rotate this result to update the unrotated kernel 9.5588 13.5952 12.7386 7.8064 42.716 49.882 55.684 33.323 15.1628 16.7726 8.7952 9.3958 66.457 67.564 31.847 30.103 9.1104 12.9086 6.8332 5.4248 44.252 44.674 33.744 21.991 + +
Backpropagation vectorized d_w=input * d_y (without rotate d_y) 16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 (3,3,3) ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = d_y (2,2,2) Dimensions do not match, So it is telling us that we need to Apply both filters to any cannel of the input 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 T x .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487
Backpropagation vectorized d_w=input * d_y (without rotate d_y) 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 T x .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487 = 33.832 153.425 28.367 121.275 43.2764 162.12 22.627 85.417 33.832 43.2764 28.367 22.627 153.425 162.12 121.275 85.417 rearrange

More Related Content

PDF
Control systems formula book
Hussain K
 
PDF
ARM Architecture Instruction Set
Dwight Sabio
 
PDF
Microcontroller pic 16 f877 registers memory ports
Nilesh Bhaskarrao Bahadure
 
PDF
GPIO In Arm cortex-m4 tiva-c
Zakaria Gomaa
 
PPTX
Arm instruction set
PriyangaKR1
 
PPTX
Synchronous down counter
RONAK SUTARIYA
 
PPTX
Stack in 8085 microprocessor
hepzijustin
 
PPTX
Arm cortex-m4 programmer model
Mohammed Gomaa
 
Control systems formula book
Hussain K
 
ARM Architecture Instruction Set
Dwight Sabio
 
Microcontroller pic 16 f877 registers memory ports
Nilesh Bhaskarrao Bahadure
 
GPIO In Arm cortex-m4 tiva-c
Zakaria Gomaa
 
Arm instruction set
PriyangaKR1
 
Synchronous down counter
RONAK SUTARIYA
 
Stack in 8085 microprocessor
hepzijustin
 
Arm cortex-m4 programmer model
Mohammed Gomaa
 

What's hot (20)

PPTX
Shifters
Rabindranath Tagore University, Bhopal
 
PPT
Arithmetic & logical operations in 8051
Jay Patel
 
PPTX
Artifical intelligence in agriculture
YogeshDadhich4
 
PPTX
QRT-PPT-2019.pptx
AMBIKABHANDARI5
 
PPTX
Smart farming ppt.
SirikornManeecharoen
 
PDF
Memory segmentation-of-8086
mudulin
 
DOCX
Steps for design of butterworth and chebyshev filter
tamil arasan
 
PPTX
Presentation On RAWE
Agriculture
 
PPTX
Radix-2 DIT FFT
Sarang Joshi
 
PPTX
Intel 8051 Programming in C
Sudhanshu Janwadkar
 
PPTX
Latches and flip flop
Shuaib Hotak
 
DOCX
READY / RAWE report
AsgarHaidery
 
PPTX
Memory banking-of-8086-final
Estiak Khan
 
PPTX
8254 Programmable Interval Timer by vijay
Vijay Kumar
 
PDF
ARM CORTEX M3 PPT
Gaurav Verma
 
PPT
Data representation and Arithmetic Algorithms
Mumthas Shaikh
 
PPTX
Full custom digital ic design of priority encoder
Vishesh Thakur
 
DOCX
Modified booth
sravan kumar y
 
PPTX
ARM Processors
Mathivanan Natarajan
 
PDF
Advanced microprocessor
Shehrevar Davierwala
 
Shifters
Rabindranath Tagore University, Bhopal
 
Arithmetic & logical operations in 8051
Jay Patel
 
Artifical intelligence in agriculture
YogeshDadhich4
 
QRT-PPT-2019.pptx
AMBIKABHANDARI5
 
Smart farming ppt.
SirikornManeecharoen
 
Memory segmentation-of-8086
mudulin
 
Steps for design of butterworth and chebyshev filter
tamil arasan
 
Presentation On RAWE
Agriculture
 
Radix-2 DIT FFT
Sarang Joshi
 
Intel 8051 Programming in C
Sudhanshu Janwadkar
 
Latches and flip flop
Shuaib Hotak
 
READY / RAWE report
AsgarHaidery
 
Memory banking-of-8086-final
Estiak Khan
 
8254 Programmable Interval Timer by vijay
Vijay Kumar
 
ARM CORTEX M3 PPT
Gaurav Verma
 
Data representation and Arithmetic Algorithms
Mumthas Shaikh
 
Full custom digital ic design of priority encoder
Vishesh Thakur
 
Modified booth
sravan kumar y
 
ARM Processors
Mathivanan Natarajan
 
Advanced microprocessor
Shehrevar Davierwala
 
Ad

Viewers also liked (11)

PDF
Introduction to Convolutional Neural Networks
Hannes Hapke
 
PPTX
Lecture 29 Convolutional Neural Networks - Computer Vision Spring2015
Jia-Bin Huang
 
PDF
Backpropagation in Convolutional Neural Network
Hiroshi Kuwajima
 
PDF
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
GeeksLab Odessa
 
PPTX
Neuroevolution and deep learing
Accenture
 
PDF
Deep Convolutional Neural Networks - Overview
Keunwoo Choi
 
PDF
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...
Madhumita Tamhane
 
PPTX
Introduction to CNN
Shuai Zhang
 
PDF
101: Convolutional Neural Networks
Mad Scientists
 
PDF
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
PDF
Deep Learning - Convolutional Neural Networks
Christian Perone
 
Introduction to Convolutional Neural Networks
Hannes Hapke
 
Lecture 29 Convolutional Neural Networks - Computer Vision Spring2015
Jia-Bin Huang
 
Backpropagation in Convolutional Neural Network
Hiroshi Kuwajima
 
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
GeeksLab Odessa
 
Neuroevolution and deep learing
Accenture
 
Deep Convolutional Neural Networks - Overview
Keunwoo Choi
 
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...
Madhumita Tamhane
 
Introduction to CNN
Shuai Zhang
 
101: Convolutional Neural Networks
Mad Scientists
 
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Deep Learning - Convolutional Neural Networks
Christian Perone
 
Ad

Similar to Convolution as matrix multiplication (20)

PDF
Ch02
Paralafakyou Mens
 
PDF
Capítulo 02 considerações estatísticas
Jhayson Carvalho
 
DOCX
Calculation template
Yekian Ian
 
PDF
Solucionario_Diseno_en_Ingenieria_Mecani (1).pdf
DannyCoronel5
 
PDF
Solucionario_Diseno_en_Ingenieria_Mecani.pdf
franciscoantoniomonr1
 
DOCX
Ejerciciooo3
DanielRojas118002
 
DOCX
project designa.docx
Mahamad Jawhar
 
PDF
Solution manual for Precalculus, 4th Edition Cynthia Y. Young
testbankconsult
 
PDF
Budynas sm ch20
luisorellano125
 
PDF
Deber 9
AldoChandiAngulo
 
DOCX
Examen final
Jcabreraes
 
PDF
Solution manual For College Algebra and Trigonometry 5e Young
testbankeyes
 
DOC
Examples on total consolidation
Malika khalil
 
PDF
Gaussian Elimination
ZunAib Ali
 
PDF
Shi20396 ch03
Paralafakyou Mens
 
PDF
130 problemas dispositivos electronicos lopez meza brayan
brandwin marcelo lavado
 
PPT
Math 6 (Please download first to activate the different animation settings)
Eddie Abug
 
PDF
Budynas sm ch01
Engel Villanueva
 
PDF
Perfect method for Frames
andreslahe
 
PDF
Coeficiente de correlacion lineal 5 1
Poncho Garcia
 
Ch02
Paralafakyou Mens
 
Capítulo 02 considerações estatísticas
Jhayson Carvalho
 
Calculation template
Yekian Ian
 
Solucionario_Diseno_en_Ingenieria_Mecani (1).pdf
DannyCoronel5
 
Solucionario_Diseno_en_Ingenieria_Mecani.pdf
franciscoantoniomonr1
 
Ejerciciooo3
DanielRojas118002
 
project designa.docx
Mahamad Jawhar
 
Solution manual for Precalculus, 4th Edition Cynthia Y. Young
testbankconsult
 
Budynas sm ch20
luisorellano125
 
Deber 9
AldoChandiAngulo
 
Examen final
Jcabreraes
 
Solution manual For College Algebra and Trigonometry 5e Young
testbankeyes
 
Examples on total consolidation
Malika khalil
 
Gaussian Elimination
ZunAib Ali
 
Shi20396 ch03
Paralafakyou Mens
 
130 problemas dispositivos electronicos lopez meza brayan
brandwin marcelo lavado
 
Math 6 (Please download first to activate the different animation settings)
Eddie Abug
 
Budynas sm ch01
Engel Villanueva
 
Perfect method for Frames
andreslahe
 
Coeficiente de correlacion lineal 5 1
Poncho Garcia
 

Recently uploaded (20)

PDF
Odoo Construction Management System by CandidRoot
CandidRoot Solutions Private Limited
 
PDF
Top 10 Project Management Software for Small Teams in 2025.pdf
Tech Qafila
 
PPTX
StacksandQueuesCLASS 12 COMPUTER SCIENCE.pptx
manivarsh
 
PPTX
AI Tools Revolutionizing Software Development Workflows
amngrg0511
 
PDF
Cloud Native Aachen Meetup - Aug 21, 2025
roehlclemens
 
PPTX
Human-Computer Interaction for Lecture 2
ssuserca7fe9
 
PPTX
Human-Computer Interaction for Lecture 1
ssuserca7fe9
 
PDF
Mobile App for Guard Tour and Reporting.pdf
ecomme9
 
PPTX
Beige and Black Minimalist Project Deck Presentation (1).pptx
omchoksi99
 
PPTX
Post-Migration Optimization Playbook: Getting the Most Out of Your New Adobe ...
bbedford2
 
PPTX
WJQSJXNAZJVCVSAXJHBZKSJXKJKXJSBHJBJEHHJB
soniatharv2010
 
PDF
Building an Inclusive Web Accessibility Made Simple with Accessibility Analyzer
Accessibility Analyzer
 
PDF
SOFTWARE ENGINEERING Software Engineering (3rd Edition) by K.K. Aggarwal & Yo...
yaderyeka
 
PDF
WhatsApp Chatbots The Key to Scalable Customer Support.pdf
Quick Message
 
PPTX
MCP empowers AI Agents from Zero to Production
innovaterz
 
PDF
Crypto Loss And Recovery Guide By Expert Recovery Agency.
DRA | Trusted Crypto Recovery Expert |
 
PPT
3.Software Design for software engineering
anant5150ca6
 
PDF
Sanket Mhaiskar Resume - Senior Software Engineer (Backend, AI)
Sanket Mhaiskar
 
PPTX
HackYourBrain__UtrechtJUG__11092025.pptx
SimonedeGijt
 
PDF
Ragic Data Security Overview: Certifications, Compliance, and Network Safegua...
Ragic
 
Odoo Construction Management System by CandidRoot
CandidRoot Solutions Private Limited
 
Top 10 Project Management Software for Small Teams in 2025.pdf
Tech Qafila
 
StacksandQueuesCLASS 12 COMPUTER SCIENCE.pptx
manivarsh
 
AI Tools Revolutionizing Software Development Workflows
amngrg0511
 
Cloud Native Aachen Meetup - Aug 21, 2025
roehlclemens
 
Human-Computer Interaction for Lecture 2
ssuserca7fe9
 
Human-Computer Interaction for Lecture 1
ssuserca7fe9
 
Mobile App for Guard Tour and Reporting.pdf
ecomme9
 
Beige and Black Minimalist Project Deck Presentation (1).pptx
omchoksi99
 
Post-Migration Optimization Playbook: Getting the Most Out of Your New Adobe ...
bbedford2
 
WJQSJXNAZJVCVSAXJHBZKSJXKJKXJSBHJBJEHHJB
soniatharv2010
 
Building an Inclusive Web Accessibility Made Simple with Accessibility Analyzer
Accessibility Analyzer
 
SOFTWARE ENGINEERING Software Engineering (3rd Edition) by K.K. Aggarwal & Yo...
yaderyeka
 
WhatsApp Chatbots The Key to Scalable Customer Support.pdf
Quick Message
 
MCP empowers AI Agents from Zero to Production
innovaterz
 
Crypto Loss And Recovery Guide By Expert Recovery Agency.
DRA | Trusted Crypto Recovery Expert |
 
3.Software Design for software engineering
anant5150ca6
 
Sanket Mhaiskar Resume - Senior Software Engineer (Backend, AI)
Sanket Mhaiskar
 
HackYourBrain__UtrechtJUG__11092025.pptx
SimonedeGijt
 
Ragic Data Security Overview: Certifications, Compliance, and Network Safegua...
Ragic
 

Convolution as matrix multiplication

  • 1. Convolution as matrix multiplication • Edwin Efraín Jiménez Lepe
  • 2. 16 24 32 47 18 26 68 12 9 Input 0 1 -1 0 2 3 4 5 W1 W2 ∗ = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 Im2col (input) 0 5 1 3 -1 4 0 2 x W1 W2 = 23 353 50 535 -14 354 -14 248 Rearrange 23 -14 50 -14 353 354 535 248 FeedForward Applying kernel rotation
  • 3. 16 24 32 47 18 26 68 12 9 Input 0 1 -1 0 2 3 4 5 W1 W2 ∗ = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 Im2col (input) 0 5 1 3 -1 4 0 2 x W1 W2 = 23 353 50 535 -14 354 -14 248 Rearrange 24 -13 51 -13 353 354 535 248 Now with bias 1 1 1 1 1 0 FeedForward
  • 4. 16 24 32 47 18 26 68 12 9 Input 0 0 -2.94504954e-05 0 d_y ∗ = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 Im2col (input) 0 0 -2e-05 6e-06 0 0 0 0 x Im2col(d_y) -1.38417328e-03 3.00583533 e-04 -2.00263369e-03 4.34886814 e-04 -5.30108917e-04 1.15117098 e-04 -3.53405945e-04 7.67447318 e-05 Rearrange -1.38417328e-03 -5.30108917e-04 -2.00263369e-03 3.53405945e-04 d_w = input * d_y The update correspond to the Rotated kernel BackPropagation d_w 0 0 6.39539432e-06 0 -1.38417328e-03 -5.30108917e-04 -2.00263369e-03 3.53405945e-04
  • 5. 0 0 -2.94504954e-05 0 d_y d_x = d_y * w (without rotation) BackPropagation 0 0 -6.39539432e-06 0 We need full convolution And keep kernel unrotated 0 0 0 0 0 0 0 0 0 -2.94504954e-05 0 0 0 0 0 0 d_y 0 0 0 0 0 0 0 0 0 6.39539432e-06 0 0 0 0 0 0 0 1 -1 0 2 3 4 5 W1 W2 ∗ 0 1 -1 0 2 3 4 5 W1 W2 ∗
  • 6. d_x = d_y * w (without rotation) BackPropagation 0 0 0 0 0 0 0 0 0 -2.94504954e-05 0 0 0 0 0 0 d_y 0 0 0 0 0 0 0 0 0 6.39539432e-06 0 0 0 0 0 0 = 0 1 -1 0 2 3 4 5 W1 W2 ∗ 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 0 -1 1 0 x 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 2 4 3 5 x T T
  • 7. d_x = d_y * w (without rotation) BackPropagation 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 0 -1 1 0 x 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 2 4 3 5 x T T = 0 0 -0.2945e-04 0 0.2945e-04 0 0 0 0 0 0.3198e-04 0.1919-04 0 0.2558-04 0.1279-04 0 0 0
  • 8. d_x = d_y * w (without rotation) BackPropagation 0 0 -0.2945e-04 0 0.2945e-04 0 0 0 0 0 0.3198e-04 0.1919-04 0 0.2558-04 0.1279-04 0 0 0 + = 0 0.3198e-04 -0.1026e-04 0 0.5503e-04 0.1279-04 0 0 0 reshape 0 0 0 0.3198e-04 0.5503e-04 0 -0.1026e-04 0.1279-04 0
  • 9. d_x = d_y * w (without rotation) BackPropagation 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 -2.94e-05 0 0 0 0 0 0 0 0 -1 1 0 x 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 6.395e-06 0 0 0 0 0 0 0 2 4 3 5 T = In fact, we can do it in just one operation 0 0.3198e-04 -0.1026e-04 0 0.5503e-04 0.1279-04 0 0 0 Notice, every channel of delta is multiplied by the correspondent filter that generates it
  • 10. A multi-channel example 16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,3,3) (2,3,2,2) Output= (2,2,2) = 2171 2170 5954 2064 13042 13575 11023 6425 Applying theano convolution (which rotates Automatically the filters)
  • 11. A multi-channel example (vectorized) 16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,3,3) (2,3,2,2) = 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 0 60 1 32 -1 22 0 18 5 35 3 7 4 46 2 23 16 78 68 20 24 81 -2 42 x T
  • 12. A multi-channel example (vectorized) 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 x = T 0 60 1 32 -1 22 0 18 5 35 3 7 4 46 2 23 16 78 68 20 24 81 -2 42 2171 13042 5954 11023 2170 13575 2064 6425 Channel 1 Channel 2 Rearrange 2171 2170 5954 2064 13042 13575 11023 6425
  • 13. Backpropagation .1678 .098 .002 .246 0.5 0.67 0.21 0.487 Imagine we got the next error from an up-layer And we want to propagate it to the correspondent layer (input of convolution) We need to compute d_y * w (without rotation) But is a ‘full’ convolution, so we add 1 zero padding to d_y 0 0 0 0 0 .1678 .098 0 0 .002 .246 0 0 0 0 0 0 0 0 0 0 0.5 .67 0 0 .21 .487 0 0 0 0 0
  • 14. Backpropagation d_y-1=d_y * w (without rotation) 0 0 0 0 0 .1678 .098 0 0 .002 .246 0 0 0 0 0 0 0 0 0 0 0.5 .67 0 0 .21 .487 0 0 0 0 0 im2col 0 0 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 0 0 0 0 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 0 0
  • 15. Backpropagation d_y-1=d_y * w (without rotation) 0 0 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 .1678 .002 0 .098 .246 0 0 0 .1678 .002 0 .098 .246 0 0 0 0 0 0 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 .5 .21 0 .67 .487 0 0 0 .5 .21 0 .67 .487 0 0 0 0 T Notice, every channel of delta is multiplied by the correspondent filter that generates it 0 -1 1 0 2 4 3 5 -2 24 68 16 18 22 32 60 23 46 7 35 42 81 20 78 x = 30 18.339 41.6848 28.7678 11.3634 37.8224 6.722 1.476 4.336 51.0322 47.6112 98.3552 64.376 44.7626 99.7084 19.61 8.981 35.284 14.642 31.212 56.622 22.528 38.992 73.295 8.766 11.693 19.962
  • 16. Backpropagation d_y-1=d_y * w (without rotation) 30 18.339 41.6848 28.7678 11.3634 37.8224 6.722 1.476 4.336 51.0322 47.6112 98.3552 64.376 44.7626 99.7084 19.61 8.981 35.284 14.642 31.212 56.622 22.528 38.992 73.295 8.766 11.693 19.962 rearrange 30 51.0322 14.642 28.7678 64.376 22.528 6.722 19.61 8.766 18.339 47.6112 31.212 11.3634 44.7626 38.992 1.476 8.981 11.693 41.6848 98.3552 56.622 37.8224 99.7084 73.295 4.336 35.284 19.962
  • 17. Backpropagation (no vectorized) d_y-1=d_y * w (without rotation) .1678 .098 .002 .246 0.5 0.67 0.21 0.487 (2,2,2) 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (2,3,2,2) Transpose dimensions 0 and 1 .1678 .098 .002 .246 0.5 0.67 0.21 0.487 (2,2,2) 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,2,2,2) Filter 3
  • 18. Backpropagation (no vectorized, full convolution) d_y-1=d_y * w (without rotation) .1678 .098 .002 .246 0.5 0.67 0.21 0.487 (2,2,2) 0 1 -1 0 2 3 4 5 Filter 1 Filter 2 ∗ -2 68 24 16 18 32 22 60 23 7 46 35 42 20 81 78 (3,2,2,2) Filter 3 = 30 51.0322 14.642 28.7678 64.376 22.528 6.722 19.61 8.766 18.339 47.6112 31.212 11.3634 44.7626 38.992 1.476 8.981 11.693 41.6848 98.3552 56.622 37.8224 99.7084 73.295 4.336 35.284 19.962
  • 19. Backpropagation d_w=input * d_y16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 (3,3,3) ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = d_y (2,2,2) Dimensions do not match, So it is telling us that we need to Apply both filters to any cannel of the input 16 24 32 47 18 26 68 12 9 ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = 9.5588 13.5952 12.7386 7.8064 42.716 49.882 55.684 33.323 26 57 43 24 21 12 02 11 19 ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = 15.1628 16.7726 8.7952 9.3958 66.457 67.564 31.847 30.103 ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = 9.1104 12.9086 6.8332 5.4248 44.252 44.674 33.744 21.991 18 47 21 4 6 12 81 22 13
  • 20. Backpropagation d_w=input * d_y = 33.832 43.2764 28.367 22.627 153.425 162.12 121.275 85.417 Error associated with rotated kernel, it means We need to rotate this result to update the unrotated kernel 9.5588 13.5952 12.7386 7.8064 42.716 49.882 55.684 33.323 15.1628 16.7726 8.7952 9.3958 66.457 67.564 31.847 30.103 9.1104 12.9086 6.8332 5.4248 44.252 44.674 33.744 21.991 + +
  • 21. Backpropagation vectorized d_w=input * d_y (without rotate d_y) 16 24 32 47 18 26 68 12 9 Input 26 57 43 24 21 12 02 11 19 18 47 21 4 6 12 81 22 13 (3,3,3) ∗ .1678 .098 .002 .246 0.5 0.67 0.21 0.487 = d_y (2,2,2) Dimensions do not match, So it is telling us that we need to Apply both filters to any cannel of the input 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 T x .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487
  • 22. Backpropagation vectorized d_w=input * d_y (without rotate d_y) 16 47 24 18 47 68 18 12 24 18 32 26 18 12 26 9 26 24 57 21 24 2 21 11 57 21 43 12 21 11 12 19 18 4 47 6 4 81 6 22 47 6 21 12 6 22 12 13 T x .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487 .1678 0.5 .002 0.21 .098 0.67 .246 0.487 = 33.832 153.425 28.367 121.275 43.2764 162.12 22.627 85.417 33.832 43.2764 28.367 22.627 153.425 162.12 121.275 85.417 rearrange