1. Fundamentals of CNNs and RNNs

This is a note from coursera lecture : https://www.coursera.org/learn/cnns-and-rnns

 

• Is CNN fully connected neural network or no?

The image above is showing that left is fully connected neural network, right is not. The right image is CNN neural network.

CNN is same structure as fully connected neural network but some of weight of connections are zero.

So it is partially connected neural network having shared weight.

In the right image, same color lines have same connectin weight.

 

• CNN is specialized model for sequantial data.

 

• What is sequantial data?

The data created by human such as image, speech, text, etc.

If the data location is important, it is sequential. The data has different meaning by its location.

 

• CNN is originated by finding local features.

For example, computer cannot compute both images are same. But if image splits into small sizes, it can be matched. These splited images are local feature.

 

• what is local features?

Definition of local feature, in other words, the local feature what I want to find. Filter or mask.

 

• feature map

When 100% match between the local feature and image, it results the biggest positive number.

Vice versa, if there is unmatched between the local feature and image, it results the biggest negative number.

 

• convolution

The operation to find the loal feature.

 

• The size of output image is decided by kernel size.

So, size of input(image size) is diferrent with size of output.

output size=image size-kernel size+1

For example,

image size	kernel size	output size
(8 x 8)	(3 x 3)	(6 x 6)
(8 x 8)	(5 x 5)	(4 x 4)

 

• With zero padding, size of input(image size) can be same as size of output. Set the zero padding is user's option, not mandatory.

image size	kernel size	before zero padding output size	zero padding	after zero padding output size
(8 x 8)	(3 x 3)	(6 x 6)	padding=1	(8 x 8)
(8 x 8)	(5 x 5)	(4 x 4)	padding=2	(8 x 8)

 

• threshold

Change negative number to zero(ReLU) in the feafure map.

ReLU(x)=max(0,x)

 

• 1 color image is combined 3 channels(R, G, B).

The number of input channel and the number of filter channel should be same.

The number of feature map(output) is only one.

 

• pooling

Downsampling from feature map, in other words, reduce the size of feature map.

Mostly max pooling is used common.

The reason why pooling is reduce the operation consumption.