What makes linear regression with polynomial features curvy?
Will TSA allow me to carry a Continuous Positive Airway Pressure (CPAP) device?
correct term describing the action of sending a brand-new ship out into its first seafaring trip
What are the words for people who cause trouble believing they know better?
What happens if you do emergency landing on a US base in middle of the ocean?
Is it OK to bring delicacies from hometown as tokens of gratitude for an out-of-town interview?
I wrote a scene that the majority of my readers loved. How do I get back to that place while writing my new book?
Accidentally renamed tar.gz file to a non tar.gz file, will my file be messed up
Pronoun introduced before its antecedent
Movie where a boy is transported into the future by an alien spaceship
Is it possible for people to live in the eye of a permanent hypercane?
How to make thick Asian sauces?
You've spoiled/damaged the card
Convert camelCase and PascalCase to Title Case
Is the capacitor drawn or wired wrongly?
Pros and cons of writing a book review?
Comma Code - Ch. 4 Automate the Boring Stuff
Does the growth of home value benefit from compound interest?
Why is Colorado so different politically from nearby states?
Can a magnetic field of an object be stronger than its gravity?
If Boris Johnson were prosecuted and convicted of lying about Brexit, can that be used to cancel Brexit?
Could the Missouri River be running while Lake Michigan was frozen several meters deep?
What are they doing to this poor rocket?
1980s (or earlier) book where people live a long time but they have short memories
In this example, which path would a monster affected by the Dissonant Whispers spell take?
What makes linear regression with polynomial features curvy?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
$begingroup$
The following is my understanding of what happens: if I take a "two dimensional problem" e.g. I have X as inputs and Y as the outcome and I add a feature x^2. This gives a problem an additional dimension and the linear fit on the x and y values define a line as well as the linear fit on x^2 and y values and the two lines define a plane which is the best fit. Is this correct? How does this translate back to the 2 dimensional space? Does this somehow show up in two dimensions as curvy? How?
regression polynomial
asked 10 hours ago
user412953user412953
212
$endgroup$
add a comment |
$begingroup$
The following is my understanding of what happens: if I take a "two dimensional problem" e.g. I have X as inputs and Y as the outcome and I add a feature x^2. This gives a problem an additional dimension and the linear fit on the x and y values define a line as well as the linear fit on x^2 and y values and the two lines define a plane which is the best fit. Is this correct? How does this translate back to the 2 dimensional space? Does this somehow show up in two dimensions as curvy? How?
regression polynomial
asked 10 hours ago
user412953user412953
212
$endgroup$
1
$begingroup$
$x^2$ is not an additional dimension because it's determined by $x$. the dimensions must be independent to some degree at least
$endgroup$
– Aksakal
10 hours ago
2
$begingroup$
@Aksakal In the sense of dimensions of the column space of the model matrix, though, $x^2$ usually does introduce an additional dimension. That seems like a natural and useful way to understand this question.
$endgroup$
– whuber♦
10 hours ago
$begingroup$
If we're thinking in terms of the design matrix $X$ that has observations as rows and variables as columns, then $x^2$ has its own column and in this regard adds a dimension. for instance, a covariance matrix $ptimes p$ will one more dimension. moreover, in many cases the matix will even retain its rank $p$ despite $x^2$ being dependent on $x$, because it is not linearly dependent. that's why polynomial regression often works. however, it may sometimes fail due to collinearity or condition.
$endgroup$
– Aksakal
9 hours ago
$begingroup$
I would suggest using orthogonal polynomials though. they're free of dependency issues of simple polynomials
$endgroup$
– Aksakal
9 hours ago
add a comment |
$begingroup$
The following is my understanding of what happens: if I take a "two dimensional problem" e.g. I have X as inputs and Y as the outcome and I add a feature x^2. This gives a problem an additional dimension and the linear fit on the x and y values define a line as well as the linear fit on x^2 and y values and the two lines define a plane which is the best fit. Is this correct? How does this translate back to the 2 dimensional space? Does this somehow show up in two dimensions as curvy? How?
regression polynomial
asked 10 hours ago
user412953user412953
212
$endgroup$
The following is my understanding of what happens: if I take a "two dimensional problem" e.g. I have X as inputs and Y as the outcome and I add a feature x^2. This gives a problem an additional dimension and the linear fit on the x and y values define a line as well as the linear fit on x^2 and y values and the two lines define a plane which is the best fit. Is this correct? How does this translate back to the 2 dimensional space? Does this somehow show up in two dimensions as curvy? How?
regression polynomial
regression polynomial
asked 10 hours ago
user412953user412953
212
asked 10 hours ago
user412953user412953
212
asked 10 hours ago
user412953user412953
212
asked 10 hours ago
user412953user412953
212
asked 10 hours ago
user412953user412953
212
212
1
$begingroup$
$x^2$ is not an additional dimension because it's determined by $x$. the dimensions must be independent to some degree at least
$endgroup$
– Aksakal
10 hours ago
2
$begingroup$
@Aksakal In the sense of dimensions of the column space of the model matrix, though, $x^2$ usually does introduce an additional dimension. That seems like a natural and useful way to understand this question.
$endgroup$
– whuber♦
10 hours ago
$begingroup$
If we're thinking in terms of the design matrix $X$ that has observations as rows and variables as columns, then $x^2$ has its own column and in this regard adds a dimension. for instance, a covariance matrix $ptimes p$ will one more dimension. moreover, in many cases the matix will even retain its rank $p$ despite $x^2$ being dependent on $x$, because it is not linearly dependent. that's why polynomial regression often works. however, it may sometimes fail due to collinearity or condition.
$endgroup$
– Aksakal
9 hours ago
$begingroup$
I would suggest using orthogonal polynomials though. they're free of dependency issues of simple polynomials
$endgroup$
– Aksakal
9 hours ago
add a comment |
1
$begingroup$
$x^2$ is not an additional dimension because it's determined by $x$. the dimensions must be independent to some degree at least
$endgroup$
– Aksakal
10 hours ago
2
$begingroup$
@Aksakal In the sense of dimensions of the column space of the model matrix, though, $x^2$ usually does introduce an additional dimension. That seems like a natural and useful way to understand this question.
$endgroup$
– whuber♦
10 hours ago
$begingroup$
If we're thinking in terms of the design matrix $X$ that has observations as rows and variables as columns, then $x^2$ has its own column and in this regard adds a dimension. for instance, a covariance matrix $ptimes p$ will one more dimension. moreover, in many cases the matix will even retain its rank $p$ despite $x^2$ being dependent on $x$, because it is not linearly dependent. that's why polynomial regression often works. however, it may sometimes fail due to collinearity or condition.
$endgroup$
– Aksakal
9 hours ago
$begingroup$
I would suggest using orthogonal polynomials though. they're free of dependency issues of simple polynomials
$endgroup$
– Aksakal
9 hours ago
1
1
$begingroup$
$x^2$ is not an additional dimension because it's determined by $x$. the dimensions must be independent to some degree at least
$endgroup$
– Aksakal
10 hours ago
$begingroup$
$x^2$ is not an additional dimension because it's determined by $x$. the dimensions must be independent to some degree at least
$endgroup$
– Aksakal
10 hours ago
2
2
$begingroup$
@Aksakal In the sense of dimensions of the column space of the model matrix, though, $x^2$ usually does introduce an additional dimension. That seems like a natural and useful way to understand this question.
$endgroup$
– whuber♦
10 hours ago
$begingroup$
@Aksakal In the sense of dimensions of the column space of the model matrix, though, $x^2$ usually does introduce an additional dimension. That seems like a natural and useful way to understand this question.
$endgroup$
– whuber♦
10 hours ago
$begingroup$
If we're thinking in terms of the design matrix $X$ that has observations as rows and variables as columns, then $x^2$ has its own column and in this regard adds a dimension. for instance, a covariance matrix $ptimes p$ will one more dimension. moreover, in many cases the matix will even retain its rank $p$ despite $x^2$ being dependent on $x$, because it is not linearly dependent. that's why polynomial regression often works. however, it may sometimes fail due to collinearity or condition.
$endgroup$
– Aksakal
9 hours ago
$begingroup$
If we're thinking in terms of the design matrix $X$ that has observations as rows and variables as columns, then $x^2$ has its own column and in this regard adds a dimension. for instance, a covariance matrix $ptimes p$ will one more dimension. moreover, in many cases the matix will even retain its rank $p$ despite $x^2$ being dependent on $x$, because it is not linearly dependent. that's why polynomial regression often works. however, it may sometimes fail due to collinearity or condition.
$endgroup$
– Aksakal
9 hours ago
$begingroup$
I would suggest using orthogonal polynomials though. they're free of dependency issues of simple polynomials
$endgroup$
– Aksakal
9 hours ago
$begingroup$
I would suggest using orthogonal polynomials though. they're free of dependency issues of simple polynomials
$endgroup$
– Aksakal
9 hours ago
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
This is a piece of a plane in 3D.
Here is the same plane with coordinates shown and a set of points selected along its $x$ axis.
The third coordinate is used to plot the squares of these $x$ values, producing points along a parabola at the base of the coordinate box.
A vertical "curtain" through the parabola intersects the plane at all the points directly above the parabola. This intersection is a curve.
A polynomial model supposes the response $y$ (graphed in the vertical direction) differs from the height of this plane by random amounts. The values of $y$ corresponding to these $x$ coordinates are shown as red dots.
Consequently, the $(x,y)$ points lie along a curve--this projection--rather than a line, even though the model of the response is based on the plane originally shown.
Moral
When the explanatory variables clearly lie on a curve, the responses will appear to lie on a curve, too.
answered 8 hours ago
whuber♦whuber
209k34459836
$endgroup$
add a comment |
$begingroup$
If you have a single independent variable x and a single dependent variable y, then "y = f(x)" is usually considered as two dimensional even if the relationship between these two variables is complicated. As a hypothetical example, if an experimental model is "pressure = a * temperature + b * log(temperature) - c * sine(temperature)" there are only two variables, temperature and pressure. For this reason, such a relationship could be plotted as a curved line on a flat plane.
If the model had two independent variables, such as "pressure = a * log(temperature) - b * exp(altitude)", this has the form of "z = f(x,y)" and could be plotted as a 3D surface.
answered 10 hours ago
James PhillipsJames Phillips
505257
$endgroup$
1
$begingroup$
Ah, I think I got it, thanks a lot!
$endgroup$
– user412953
10 hours ago
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f411043%2fwhat-makes-linear-regression-with-polynomial-features-curvy%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
This is a piece of a plane in 3D.
Here is the same plane with coordinates shown and a set of points selected along its $x$ axis.
The third coordinate is used to plot the squares of these $x$ values, producing points along a parabola at the base of the coordinate box.
A vertical "curtain" through the parabola intersects the plane at all the points directly above the parabola. This intersection is a curve.
A polynomial model supposes the response $y$ (graphed in the vertical direction) differs from the height of this plane by random amounts. The values of $y$ corresponding to these $x$ coordinates are shown as red dots.
Consequently, the $(x,y)$ points lie along a curve--this projection--rather than a line, even though the model of the response is based on the plane originally shown.
Moral
When the explanatory variables clearly lie on a curve, the responses will appear to lie on a curve, too.
answered 8 hours ago
whuber♦whuber
209k34459836
$endgroup$
add a comment |
$begingroup$
This is a piece of a plane in 3D.
Here is the same plane with coordinates shown and a set of points selected along its $x$ axis.
The third coordinate is used to plot the squares of these $x$ values, producing points along a parabola at the base of the coordinate box.
A vertical "curtain" through the parabola intersects the plane at all the points directly above the parabola. This intersection is a curve.
A polynomial model supposes the response $y$ (graphed in the vertical direction) differs from the height of this plane by random amounts. The values of $y$ corresponding to these $x$ coordinates are shown as red dots.
Consequently, the $(x,y)$ points lie along a curve--this projection--rather than a line, even though the model of the response is based on the plane originally shown.
Moral
When the explanatory variables clearly lie on a curve, the responses will appear to lie on a curve, too.
answered 8 hours ago
whuber♦whuber
209k34459836
$endgroup$
add a comment |
$begingroup$
This is a piece of a plane in 3D.
Here is the same plane with coordinates shown and a set of points selected along its $x$ axis.
The third coordinate is used to plot the squares of these $x$ values, producing points along a parabola at the base of the coordinate box.
A vertical "curtain" through the parabola intersects the plane at all the points directly above the parabola. This intersection is a curve.
A polynomial model supposes the response $y$ (graphed in the vertical direction) differs from the height of this plane by random amounts. The values of $y$ corresponding to these $x$ coordinates are shown as red dots.
Consequently, the $(x,y)$ points lie along a curve--this projection--rather than a line, even though the model of the response is based on the plane originally shown.
Moral
When the explanatory variables clearly lie on a curve, the responses will appear to lie on a curve, too.
answered 8 hours ago
whuber♦whuber
209k34459836
$endgroup$
This is a piece of a plane in 3D.
Here is the same plane with coordinates shown and a set of points selected along its $x$ axis.
The third coordinate is used to plot the squares of these $x$ values, producing points along a parabola at the base of the coordinate box.
A vertical "curtain" through the parabola intersects the plane at all the points directly above the parabola. This intersection is a curve.
A polynomial model supposes the response $y$ (graphed in the vertical direction) differs from the height of this plane by random amounts. The values of $y$ corresponding to these $x$ coordinates are shown as red dots.
Consequently, the $(x,y)$ points lie along a curve--this projection--rather than a line, even though the model of the response is based on the plane originally shown.
Moral
When the explanatory variables clearly lie on a curve, the responses will appear to lie on a curve, too.
answered 8 hours ago
whuber♦whuber
209k34459836
answered 8 hours ago
whuber♦whuber
209k34459836
answered 8 hours ago
whuber♦whuber
209k34459836
answered 8 hours ago
whuber♦whuber
209k34459836
209k34459836
add a comment |
add a comment |
$begingroup$
If you have a single independent variable x and a single dependent variable y, then "y = f(x)" is usually considered as two dimensional even if the relationship between these two variables is complicated. As a hypothetical example, if an experimental model is "pressure = a * temperature + b * log(temperature) - c * sine(temperature)" there are only two variables, temperature and pressure. For this reason, such a relationship could be plotted as a curved line on a flat plane.
If the model had two independent variables, such as "pressure = a * log(temperature) - b * exp(altitude)", this has the form of "z = f(x,y)" and could be plotted as a 3D surface.
answered 10 hours ago
James PhillipsJames Phillips
505257
$endgroup$
1
$begingroup$
Ah, I think I got it, thanks a lot!
$endgroup$
– user412953
10 hours ago
add a comment |
$begingroup$
If you have a single independent variable x and a single dependent variable y, then "y = f(x)" is usually considered as two dimensional even if the relationship between these two variables is complicated. As a hypothetical example, if an experimental model is "pressure = a * temperature + b * log(temperature) - c * sine(temperature)" there are only two variables, temperature and pressure. For this reason, such a relationship could be plotted as a curved line on a flat plane.
If the model had two independent variables, such as "pressure = a * log(temperature) - b * exp(altitude)", this has the form of "z = f(x,y)" and could be plotted as a 3D surface.
answered 10 hours ago
James PhillipsJames Phillips
505257
$endgroup$
1
$begingroup$
Ah, I think I got it, thanks a lot!
$endgroup$
– user412953
10 hours ago
add a comment |
$begingroup$
If you have a single independent variable x and a single dependent variable y, then "y = f(x)" is usually considered as two dimensional even if the relationship between these two variables is complicated. As a hypothetical example, if an experimental model is "pressure = a * temperature + b * log(temperature) - c * sine(temperature)" there are only two variables, temperature and pressure. For this reason, such a relationship could be plotted as a curved line on a flat plane.
If the model had two independent variables, such as "pressure = a * log(temperature) - b * exp(altitude)", this has the form of "z = f(x,y)" and could be plotted as a 3D surface.
answered 10 hours ago
James PhillipsJames Phillips
505257
$endgroup$
If you have a single independent variable x and a single dependent variable y, then "y = f(x)" is usually considered as two dimensional even if the relationship between these two variables is complicated. As a hypothetical example, if an experimental model is "pressure = a * temperature + b * log(temperature) - c * sine(temperature)" there are only two variables, temperature and pressure. For this reason, such a relationship could be plotted as a curved line on a flat plane.
If the model had two independent variables, such as "pressure = a * log(temperature) - b * exp(altitude)", this has the form of "z = f(x,y)" and could be plotted as a 3D surface.
answered 10 hours ago
James PhillipsJames Phillips
505257
answered 10 hours ago
James PhillipsJames Phillips
505257
answered 10 hours ago
James PhillipsJames Phillips
505257
answered 10 hours ago
James PhillipsJames Phillips
505257
505257
1
$begingroup$
Ah, I think I got it, thanks a lot!
$endgroup$
– user412953
10 hours ago
add a comment |
1
$begingroup$
Ah, I think I got it, thanks a lot!
$endgroup$
– user412953
10 hours ago
1
1
$begingroup$
Ah, I think I got it, thanks a lot!
$endgroup$
– user412953
10 hours ago
$begingroup$
Ah, I think I got it, thanks a lot!
$endgroup$
– user412953
10 hours ago
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f411043%2fwhat-makes-linear-regression-with-polynomial-features-curvy%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
$x^2$ is not an additional dimension because it's determined by $x$. the dimensions must be independent to some degree at least
$endgroup$
– Aksakal
10 hours ago
2
$begingroup$
@Aksakal In the sense of dimensions of the column space of the model matrix, though, $x^2$ usually does introduce an additional dimension. That seems like a natural and useful way to understand this question.
$endgroup$
– whuber♦
10 hours ago
$begingroup$
If we're thinking in terms of the design matrix $X$ that has observations as rows and variables as columns, then $x^2$ has its own column and in this regard adds a dimension. for instance, a covariance matrix $ptimes p$ will one more dimension. moreover, in many cases the matix will even retain its rank $p$ despite $x^2$ being dependent on $x$, because it is not linearly dependent. that's why polynomial regression often works. however, it may sometimes fail due to collinearity or condition.
$endgroup$
– Aksakal
9 hours ago
$begingroup$
I would suggest using orthogonal polynomials though. they're free of dependency issues of simple polynomials
$endgroup$
– Aksakal
9 hours ago