Skip to content
Concept-Lab
← Machine Learning🧠 30 / 114
Machine Learning

Anomaly Detection Algorithm

Fit one Gaussian per feature, multiply densities into p(x), then classify with epsilon threshold.

Core Theory

Algorithm structure: for each feature x_j, estimate mu_j and sigma_j^2 from training data, then compute per-feature Gaussian probabilities and multiply them into p(x).

p(x) = product_j p(x_j; mu_j, sigma_j^2)

Decision rule: if p(x) < epsilon, mark as anomaly; otherwise treat as normal.

Core intuition: one strongly unusual feature can push the product probability down sharply, making the whole example stand out.

Assumption: this factorization corresponds to conditional independence approximation. Even when that assumption is imperfect, the approach can still be practical with good features.

Failure mode: correlated features and poor feature engineering can make p(x) poorly calibrated, causing both misses and noisy alerts.

Interview-Ready Deepening

Source-backed reinforcement: these points add detail beyond short-duration UI hints and emphasize production tradeoffs.

  • Fit one Gaussian per feature, multiply densities into p(x), then classify with epsilon threshold.
  • Algorithm structure: for each feature x_j, estimate mu_j and sigma_j^2 from training data, then compute per-feature Gaussian probabilities and multiply them into p(x).
  • Understanding statistical independence is not needed to fully complete this class and also, be able to very effectively use anomaly detection algorithm.
  • But it turns out this algorithm often works fine even that the features are not actually statistically independent.
  • Decision rule: if p(x) < epsilon, mark as anomaly; otherwise treat as normal.
  • To model p(x2) x2 is a totally different feature measuring the vibrations of the airplane engine.
  • Core intuition: one strongly unusual feature can push the product probability down sharply, making the whole example stand out.
  • Even when that assumption is imperfect, the approach can still be practical with good features.

Tradeoffs You Should Be Able to Explain

  • More expressive models improve fit but can reduce interpretability and raise overfitting risk.
  • Higher optimization speed can reduce training time but may increase instability if learning dynamics are not monitored.
  • Feature-rich pipelines improve performance ceilings but increase maintenance and monitoring complexity.

First-time learner note: Read each model as a dataflow system: inputs become representations, representations become scores, and scores become decisions through a chosen loss and thresholding policy.

Production note: Track three things relentlessly in ML systems: data shape contracts, evaluation methodology, and the operational meaning of the model's errors. Most expensive failures come from one of those three.

🧾 Comprehensive Coverage

Exhaustive coverage points to ensure complete topic understanding without missing core concepts.

Loading interactive module...

💡 Concrete Example

Engine monitoring with features [heat, vibration]: - Fit Gaussian for heat and another for vibration. - New engine has moderate heat but extreme vibration. - p(vibration) is very low, so p(x) becomes small. - If p(x) < epsilon, flag for quality inspection.

🧠 Beginner-Friendly Examples

Guided Starter Example

Engine monitoring with features [heat, vibration]: - Fit Gaussian for heat and another for vibration. - New engine has moderate heat but extreme vibration. - p(vibration) is very low, so p(x) becomes small. - If p(x) < epsilon, flag for quality inspection.

Source-grounded Practical Scenario

Fit one Gaussian per feature, multiply densities into p(x), then classify with epsilon threshold.

Source-grounded Practical Scenario

Algorithm structure: for each feature x_j, estimate mu_j and sigma_j^2 from training data, then compute per-feature Gaussian probabilities and multiply them into p(x).

🧭 Architecture Flow

Loading interactive module...

🎬 Interactive Visualization

🛠 Interactive Tool

🧪 Interactive Sessions

  1. Concept Drill: Manipulate key parameters and observe behavior shifts for Anomaly Detection Algorithm.
  2. Failure Mode Lab: Trigger an edge case and explain remediation decisions.
  3. Architecture Reorder Exercise: Reorder 5 flow steps into the correct production sequence.

💻 Code Walkthrough

Concept-to-code walkthrough checklist for this topic.

  1. Define input/output contract before reading implementation details.
  2. Map each conceptual step to one concrete function/class decision.
  3. Call out one tradeoff and one failure mode in interview wording.

🎯 Interview Prep

Questions an interviewer is likely to ask about this topic. Think through your answer before reading the senior angle.

  • Q1[beginner] Why does multiplying per-feature probabilities help anomaly scoring?
    Strong answer structure: define the concept in one sentence, ground it in a concrete scenario (Fit one Gaussian per feature, multiply densities into p(x), then classify with epsilon threshold.), then explain one tradeoff (More expressive models improve fit but can reduce interpretability and raise overfitting risk.) and how you'd monitor it in production.
  • Q2[intermediate] What is the role of epsilon in the classifier?
    Strong answer structure: define the concept in one sentence, ground it in a concrete scenario (Fit one Gaussian per feature, multiply densities into p(x), then classify with epsilon threshold.), then explain one tradeoff (More expressive models improve fit but can reduce interpretability and raise overfitting risk.) and how you'd monitor it in production.
  • Q3[expert] How do correlated features affect this simple model?
    Strong answer structure: define the concept in one sentence, ground it in a concrete scenario (Fit one Gaussian per feature, multiply densities into p(x), then classify with epsilon threshold.), then explain one tradeoff (More expressive models improve fit but can reduce interpretability and raise overfitting risk.) and how you'd monitor it in production.
  • Q4[expert] How would you explain this in a production interview with tradeoffs?
    Discuss this as a baseline detector: simple, fast, explainable. Then mention that feature quality and threshold policy usually matter more than mathematical sophistication.
🏆 Senior answer angle — click to reveal
Use the tier progression: beginner correctness -> intermediate tradeoffs -> expert production constraints and incident readiness.

📚 Revision Flash Cards

Test yourself before moving on. Flip each card to check your understanding — great for quick revision before an interview.

Loading interactive module...