Linear Regression Is Actually a Projection Problem (Part ...

What’s Happening

Here’s the thing: The Vector View of Least Squares.

The post Linear Regression Is Actually a Projection Problem (Part 2: From Projections to Predictions) appeared first on Towards Data Science. We think that linear regression is about fitting a line to data. (we’re not making this up)

But mathematically, that’s not what it’s doing.

The Details

It is finding the closest possible vector to your target within the space spanned by features. To understand this, we need to change how we look at our data.

In Part 1 , weve got a basic idea of what a vector is and explored the concepts of dot products and projections. Now, lets apply these concepts to solve a linear regression problem.

Why This Matters

Image by Author The Usual Way: Feature Space When we try to understand linear regression, we generally start with a scatter plot drawn between the independent and dependent variables. Each point on this plot represents a single row of data. We then try to fit a line through these points, with the goal of minimizing the sum of squared residuals.

This adds to the ongoing AI race that’s captivating the tech world.

Key Takeaways

• As we already discussed in my earlier multiple linear regression (MLR) blog, this is the standard way to understand the problem.
• This is what we call as a feature space.
• Image by Author After doing all that process, we get a value for the slope and intercept.

The Bottom Line

A Shift in Perspective Lets look at our data. Now, instead of considering Price and Size as axes, lets consider each house as an axis.

What do you think about all this?