Decision Boundary

The decision boundary is the set of points where model confidence is exactly at threshold. For logistic regression with default threshold 0.5, this is where z = w⃗·x⃗ + b = 0.

Everything on one side is predicted positive, everything on the other side negative.

• Linear features -> line/plane/hyperplane boundary.
• Engineered polynomial features -> curved boundary in original input space.

Important distinction: the boundary is determined by learned parameters, while threshold determines how probabilities are mapped to classes. Changing threshold moves operational decisions even when parameters stay fixed.

Production implications:

• For imbalanced classes, threshold 0.5 is often suboptimal.
• Boundary quality must be judged with precision/recall trade-offs, not accuracy alone.
• Calibration matters: two models can share similar boundary accuracy but very different probability reliability.

Core geometric intuition: training does not directly draw a line; it optimises parameters so boundary placement minimizes loss under data constraints.

Deepening Notes

Source-backed reinforcement: these points are extracted from the session source note to strengthen your theory intuition.

• Remember that the cost function gives you a way to measure how well a specific set of parameters fits the training data.
• In this video, we'll look at how the squared error cost function is not an ideal cost function for logistic regression.
• The cost on a certain set of parameters, w and b, is equal to 1 over m times the sum of all the training examples of the loss on the training examples.
• In the upcoming optional lab, you'll get to take a look at how the squared error cost function doesn't work very well for classification, because you see that the surface plot results in a very wiggly costs surface with many local minima.
• We'll also figure out a simpler way to write out the cost function, which will then later allow us to run gradient descent to find good parameters for logistic regression.

Interview-Ready Deepening

Source-backed reinforcement: these points add detail beyond short-duration UI hints and emphasize production tradeoffs.

• Where the model draws the line between classes — linear and non-linear boundaries.
• Email spam: decision boundary in 2D feature space (word count vs. link count). The line separates spam (high links, many words) from legitimate email. A curved boundary might separate better if spam has a non-linear pattern.
• The decision boundary is the set of points where model confidence is exactly at threshold.
• Calibration matters: two models can share similar boundary accuracy but very different probability reliability.
• Important distinction: the boundary is determined by learned parameters, while threshold determines how probabilities are mapped to classes.
• Core geometric intuition: training does not directly draw a line; it optimises parameters so boundary placement minimizes loss under data constraints.
• Recall for linear regression, this is the squared error cost function.
• When y is equal to 1, the loss function incentivizes or nurtures, or helps push the algorithm to make more accurate predictions because the loss is lowest, when it predicts values close to 1.

Tradeoffs You Should Be Able to Explain

• More expressive models improve fit but can reduce interpretability and raise overfitting risk.
• Higher optimization speed can reduce training time but may increase instability if learning dynamics are not monitored.
• Feature-rich pipelines improve performance ceilings but increase maintenance and monitoring complexity.

First-time learner note: Read each model as a dataflow system: inputs become representations, representations become scores, and scores become decisions through a chosen loss and thresholding policy.

Production note: Track three things relentlessly in ML systems: data shape contracts, evaluation methodology, and the operational meaning of the model's errors. Most expensive failures come from one of those three.