What is the meaning of “stationarity of statistics” and “locality of pixel dependencies”?Where can I find the proof of the universal approximation theorem?Is the traditional meaning of “strong AI” outmoded?What is the difference between a Convolutional Neural Network and a regular Neural Network?What is the difference between a receptive field and a feature map?What is a bad local minimum in machine learning?Pixel-Level Detection of Each Object of the Same Class In an ImageWhat is the difference between latent and embedding spaces?How to debug and find neurons that most influenced a pixel in the output image?What is “dense” in DensePose?Understanding the reconstruction loss in the paper “Anomaly Detection using Deep Learning based Image Completion”What is the meaning of the words 'bias' and 'variance' in RL?
Tikzpicture doesn't display correctly
Do 3/8 (37.5%) of Quadratics Have No x-Intercepts?
What is a good example for artistic ND filter applications?
How did the SysRq key get onto modern keyboards if it's rarely used?
Why does the Eurostar not show youth pricing?
Convert graph format for Mathematica graph functions
Is there a word to describe someone who is, or the state of being, content with hanging around others without interacting with them?
Is it safe if the neutral lead is exposed and disconnected?
Newton's cradles
Piece of chess engine, which accomplishes move generation
What clothes would flying-people wear?
Why were contact sensors put on three of the Lunar Module's four legs? Did they ever bend and stick out sideways?
Should I accept an invitation to give a talk from someone who might review my proposal?
Blank spaces in a font
How to efficiently shred a lot of cabbage?
Would people understand me speaking German all over Europe?
Did Vladimir Lenin have a cat?
Alternatives to minimizing loss in regression
Narset, Parter of Veils interaction with Aria of Flame
Semen retention is a important thing in Martial arts?
What is the meaning of "stationarity of statistics" and "locality of pixel dependencies"?
Unknown indication below upper stave
How do I make my photos have more impact?
8086 stack segment and avoiding overflow in interrupts
What is the meaning of “stationarity of statistics” and “locality of pixel dependencies”?
Where can I find the proof of the universal approximation theorem?Is the traditional meaning of “strong AI” outmoded?What is the difference between a Convolutional Neural Network and a regular Neural Network?What is the difference between a receptive field and a feature map?What is a bad local minimum in machine learning?Pixel-Level Detection of Each Object of the Same Class In an ImageWhat is the difference between latent and embedding spaces?How to debug and find neurons that most influenced a pixel in the output image?What is “dense” in DensePose?Understanding the reconstruction loss in the paper “Anomaly Detection using Deep Learning based Image Completion”What is the meaning of the words 'bias' and 'variance' in RL?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
$begingroup$
I'm reading the ImageNet Classification with Deep Convolutional Neural Networks paper by Krizhevsky et al, and came across these lines in the Intro paragraph:
Their (convolutional neural networks') capacity can be controlled by varying their depth and breadth, and they also make strong and mostly correct assumptions about the nature of images (namely, stationarity of statistics and locality of pixel dependencies). Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse.
What's meant by "stationarity of statistics" and "locality of pixel dependencies"? Also, what's the basis of saying that CNN's theoretically best performance is only slightly worse than that of feedforward NN?
convolutional-neural-networks terminology papers
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I'm reading the ImageNet Classification with Deep Convolutional Neural Networks paper by Krizhevsky et al, and came across these lines in the Intro paragraph:
Their (convolutional neural networks') capacity can be controlled by varying their depth and breadth, and they also make strong and mostly correct assumptions about the nature of images (namely, stationarity of statistics and locality of pixel dependencies). Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse.
What's meant by "stationarity of statistics" and "locality of pixel dependencies"? Also, what's the basis of saying that CNN's theoretically best performance is only slightly worse than that of feedforward NN?
convolutional-neural-networks terminology papers
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I'm reading the ImageNet Classification with Deep Convolutional Neural Networks paper by Krizhevsky et al, and came across these lines in the Intro paragraph:
Their (convolutional neural networks') capacity can be controlled by varying their depth and breadth, and they also make strong and mostly correct assumptions about the nature of images (namely, stationarity of statistics and locality of pixel dependencies). Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse.
What's meant by "stationarity of statistics" and "locality of pixel dependencies"? Also, what's the basis of saying that CNN's theoretically best performance is only slightly worse than that of feedforward NN?
convolutional-neural-networks terminology papers
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I'm reading the ImageNet Classification with Deep Convolutional Neural Networks paper by Krizhevsky et al, and came across these lines in the Intro paragraph:
Their (convolutional neural networks') capacity can be controlled by varying their depth and breadth, and they also make strong and mostly correct assumptions about the nature of images (namely, stationarity of statistics and locality of pixel dependencies). Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse.
What's meant by "stationarity of statistics" and "locality of pixel dependencies"? Also, what's the basis of saying that CNN's theoretically best performance is only slightly worse than that of feedforward NN?
convolutional-neural-networks terminology papers
convolutional-neural-networks terminology papers
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
edited 3 hours ago
nbro
5,7274 gold badges15 silver badges32 bronze badges
5,7274 gold badges15 silver badges32 bronze badges
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
asked 8 hours ago
Shirish KulhariShirish Kulhari
1211 bronze badge
1211 bronze badge
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Shirish Kulhari is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
locality of pixel dependencies probably means that neighboring pixels tend to be correlated, while faraway pixels are usually not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in practice, it is usually chosen to be fixed and rectangular (or squared).
stationarity of statistics probably means that the values of the pixels do not change over time.
With while their theoretically-best performance is likely to be only slightly worse the authors probably thought that, theoretically, CNNs are not as powerful as feedforward neural networks. However, both CNNs and FFNNs are universal function approximators (but, at the time, nobody probably had yet investigated seriously the theoretical powerfulness of CNNs).
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
$endgroup$
$begingroup$
I really think in this case stationarity isnt regarding time, but spatial location (given no time exists-- im assuming you meant over the dataset) I think theyre saying alot of local statistics are common across multiple areas of the image (therefore a single convolved filter can be a useful featurization of multiple fields rather than just one)
$endgroup$
– mshlis
1 hour ago
$begingroup$
@mshlis You might be right. This was my interpretation, given the usual meaning of stationarity (e.g. in reinforcement learning). However, if your interpretation is correct, then their terminology is highly confusing or misleading.
$endgroup$
– nbro
59 mins ago
$begingroup$
I completely agree, generally stationarity when referring to images draws on the distributions of the pixels themselves, but here it seems to be used based on the dependencies, treating the moving indices of the convolution to parametrize some from of process.
$endgroup$
– mshlis
29 mins ago
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "658"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Shirish Kulhari is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fai.stackexchange.com%2fquestions%2f13683%2fwhat-is-the-meaning-of-stationarity-of-statistics-and-locality-of-pixel-depen%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
locality of pixel dependencies probably means that neighboring pixels tend to be correlated, while faraway pixels are usually not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in practice, it is usually chosen to be fixed and rectangular (or squared).
stationarity of statistics probably means that the values of the pixels do not change over time.
With while their theoretically-best performance is likely to be only slightly worse the authors probably thought that, theoretically, CNNs are not as powerful as feedforward neural networks. However, both CNNs and FFNNs are universal function approximators (but, at the time, nobody probably had yet investigated seriously the theoretical powerfulness of CNNs).
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
$endgroup$
$begingroup$
I really think in this case stationarity isnt regarding time, but spatial location (given no time exists-- im assuming you meant over the dataset) I think theyre saying alot of local statistics are common across multiple areas of the image (therefore a single convolved filter can be a useful featurization of multiple fields rather than just one)
$endgroup$
– mshlis
1 hour ago
$begingroup$
@mshlis You might be right. This was my interpretation, given the usual meaning of stationarity (e.g. in reinforcement learning). However, if your interpretation is correct, then their terminology is highly confusing or misleading.
$endgroup$
– nbro
59 mins ago
$begingroup$
I completely agree, generally stationarity when referring to images draws on the distributions of the pixels themselves, but here it seems to be used based on the dependencies, treating the moving indices of the convolution to parametrize some from of process.
$endgroup$
– mshlis
29 mins ago
add a comment |
$begingroup$
locality of pixel dependencies probably means that neighboring pixels tend to be correlated, while faraway pixels are usually not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in practice, it is usually chosen to be fixed and rectangular (or squared).
stationarity of statistics probably means that the values of the pixels do not change over time.
With while their theoretically-best performance is likely to be only slightly worse the authors probably thought that, theoretically, CNNs are not as powerful as feedforward neural networks. However, both CNNs and FFNNs are universal function approximators (but, at the time, nobody probably had yet investigated seriously the theoretical powerfulness of CNNs).
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
$endgroup$
$begingroup$
I really think in this case stationarity isnt regarding time, but spatial location (given no time exists-- im assuming you meant over the dataset) I think theyre saying alot of local statistics are common across multiple areas of the image (therefore a single convolved filter can be a useful featurization of multiple fields rather than just one)
$endgroup$
– mshlis
1 hour ago
$begingroup$
@mshlis You might be right. This was my interpretation, given the usual meaning of stationarity (e.g. in reinforcement learning). However, if your interpretation is correct, then their terminology is highly confusing or misleading.
$endgroup$
– nbro
59 mins ago
$begingroup$
I completely agree, generally stationarity when referring to images draws on the distributions of the pixels themselves, but here it seems to be used based on the dependencies, treating the moving indices of the convolution to parametrize some from of process.
$endgroup$
– mshlis
29 mins ago
add a comment |
$begingroup$
locality of pixel dependencies probably means that neighboring pixels tend to be correlated, while faraway pixels are usually not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in practice, it is usually chosen to be fixed and rectangular (or squared).
stationarity of statistics probably means that the values of the pixels do not change over time.
With while their theoretically-best performance is likely to be only slightly worse the authors probably thought that, theoretically, CNNs are not as powerful as feedforward neural networks. However, both CNNs and FFNNs are universal function approximators (but, at the time, nobody probably had yet investigated seriously the theoretical powerfulness of CNNs).
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
$endgroup$
locality of pixel dependencies probably means that neighboring pixels tend to be correlated, while faraway pixels are usually not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in practice, it is usually chosen to be fixed and rectangular (or squared).
stationarity of statistics probably means that the values of the pixels do not change over time.
With while their theoretically-best performance is likely to be only slightly worse the authors probably thought that, theoretically, CNNs are not as powerful as feedforward neural networks. However, both CNNs and FFNNs are universal function approximators (but, at the time, nobody probably had yet investigated seriously the theoretical powerfulness of CNNs).
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
answered 4 hours ago
nbronbro
5,7274 gold badges15 silver badges32 bronze badges
5,7274 gold badges15 silver badges32 bronze badges
$begingroup$
I really think in this case stationarity isnt regarding time, but spatial location (given no time exists-- im assuming you meant over the dataset) I think theyre saying alot of local statistics are common across multiple areas of the image (therefore a single convolved filter can be a useful featurization of multiple fields rather than just one)
$endgroup$
– mshlis
1 hour ago
$begingroup$
@mshlis You might be right. This was my interpretation, given the usual meaning of stationarity (e.g. in reinforcement learning). However, if your interpretation is correct, then their terminology is highly confusing or misleading.
$endgroup$
– nbro
59 mins ago
$begingroup$
I completely agree, generally stationarity when referring to images draws on the distributions of the pixels themselves, but here it seems to be used based on the dependencies, treating the moving indices of the convolution to parametrize some from of process.
$endgroup$
– mshlis
29 mins ago
add a comment |
$begingroup$
I really think in this case stationarity isnt regarding time, but spatial location (given no time exists-- im assuming you meant over the dataset) I think theyre saying alot of local statistics are common across multiple areas of the image (therefore a single convolved filter can be a useful featurization of multiple fields rather than just one)
$endgroup$
– mshlis
1 hour ago
$begingroup$
@mshlis You might be right. This was my interpretation, given the usual meaning of stationarity (e.g. in reinforcement learning). However, if your interpretation is correct, then their terminology is highly confusing or misleading.
$endgroup$
– nbro
59 mins ago
$begingroup$
I completely agree, generally stationarity when referring to images draws on the distributions of the pixels themselves, but here it seems to be used based on the dependencies, treating the moving indices of the convolution to parametrize some from of process.
$endgroup$
– mshlis
29 mins ago
$begingroup$
I really think in this case stationarity isnt regarding time, but spatial location (given no time exists-- im assuming you meant over the dataset) I think theyre saying alot of local statistics are common across multiple areas of the image (therefore a single convolved filter can be a useful featurization of multiple fields rather than just one)
$endgroup$
– mshlis
1 hour ago
$begingroup$
I really think in this case stationarity isnt regarding time, but spatial location (given no time exists-- im assuming you meant over the dataset) I think theyre saying alot of local statistics are common across multiple areas of the image (therefore a single convolved filter can be a useful featurization of multiple fields rather than just one)
$endgroup$
– mshlis
1 hour ago
$begingroup$
@mshlis You might be right. This was my interpretation, given the usual meaning of stationarity (e.g. in reinforcement learning). However, if your interpretation is correct, then their terminology is highly confusing or misleading.
$endgroup$
– nbro
59 mins ago
$begingroup$
@mshlis You might be right. This was my interpretation, given the usual meaning of stationarity (e.g. in reinforcement learning). However, if your interpretation is correct, then their terminology is highly confusing or misleading.
$endgroup$
– nbro
59 mins ago
$begingroup$
I completely agree, generally stationarity when referring to images draws on the distributions of the pixels themselves, but here it seems to be used based on the dependencies, treating the moving indices of the convolution to parametrize some from of process.
$endgroup$
– mshlis
29 mins ago
$begingroup$
I completely agree, generally stationarity when referring to images draws on the distributions of the pixels themselves, but here it seems to be used based on the dependencies, treating the moving indices of the convolution to parametrize some from of process.
$endgroup$
– mshlis
29 mins ago
add a comment |
Shirish Kulhari is a new contributor. Be nice, and check out our Code of Conduct.
Shirish Kulhari is a new contributor. Be nice, and check out our Code of Conduct.
Shirish Kulhari is a new contributor. Be nice, and check out our Code of Conduct.
Shirish Kulhari is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Artificial Intelligence Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fai.stackexchange.com%2fquestions%2f13683%2fwhat-is-the-meaning-of-stationarity-of-statistics-and-locality-of-pixel-depen%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
