The first time I encounter this word after started reading OverFeat paper. I skipped this word until I started to understand the architecture of the paper and also I found it difficult to understand some articles where I just pretend to fully understand this term and go on with the article.
The overlap is term in a CNN design where it is mostly have to do with stride of filter. mostly in papers when defining the architecture, the author will not simply use the term “stride” just because to elaborate the actual reason behind it.
we all know that to calculate an output using a CNN, we simply use:
((input_size + 2 * padding_size-filter_size))/Stride)+1
where if we keep lower the stride , the more computations is needed and more we can improve the resolution. This is where the overlap term comes because a lower stride filter can overuse pixels in a input for an corresponding output.
lets understand it by an example:
as we can see and input with 4x4 size convolve with 2X2 filter size and apply the above formula we can get an 3X3 size output. the yellow pixels indicate that these are only used once for output (the filter windows is gone through on this only one time),the orange color pixels are used twice and the red pixels are used 4 times. this is an example of overlapping here.
now how can we make it non-overlapping or zero overlap?
the answer is by simply making the stride equal to the size of filter (in this above case we make the stride=2 ). keep in mind that Width and Hight of the filter should be equal in this case.
As we can observe, the output shrinks and in mostly cases we loses some resolution. but with some good hyperparameter, we still mostly perceive great model accuracy.
here are some data to distinguish and understand the the overlapping and Non-overlapping .
