outlier detection

Neural network balanced weight for imbalanced classification

Neural Network Model Balanced Weight For Imbalanced Classification In Keras

When using a neural network model to classify imbalanced data, we can adjust the balanced weight for the cost function to give more attention to the minority class. Python’s Keras library has a built-in option called class_weight to help us achieve this quickly. One benefit of using the balanced weight adjustment is that we can use the …

Neural Network Model Balanced Weight For Imbalanced Classification In Keras Read More »

Isolation Forest For Anomaly Detection And Imbalanced Classification

Isolation Forest For Anomaly Detection

Isolation forest uses the number of tree splits to identify anomalies or minority classes in an imbalanced dataset. The idea is that anomaly data points take fewer splits because the density around the anomalies is low. Python’s sklearn library has an implementation for the isolation forest model. Isolation forest is an unsupervised algorithm, where the …

Isolation Forest For Anomaly Detection Read More »

One Class SVM Anomaly Detection

One-Class Support Vector Machine (SVM) For Anomaly Detection

One-Class Support Vector Machine (SVM) is an unsupervised model for anomaly or outlier detection. Unlike the regular supervised SVM, the one-class SVM does not have target labels for the model training process. Instead, it learns the boundary for the normal data points and identifies the data outside the border to be anomalies. In this post, …

One-Class Support Vector Machine (SVM) For Anomaly Detection Read More »

Autoencoder For Anomaly Detection Using Tensorflow Keras

Autoencoder For Anomaly Detection Using Tensorflow Keras

Autoencoder is an unsupervised neural network model that uses reconstruction error to detect anomalies or outliers. The reconstruction error is the difference between the reconstructed data and the input data. Autoencoder uses only normal data to train the model and all data to make predictions. Therefore, we expect outliers to have higher reconstruction errors because …

Autoencoder For Anomaly Detection Using Tensorflow Keras Read More »

Local Outlier Factor (LOF) For Anomaly Detection

Local Outlier Factor (LOF) For Anomaly Detection

Local Outlier Factor (LOF) is an unsupervised model for outlier detection. It compares the local density of each data point with its neighbors and identifies the data points with a lower density as anomalies or outliers. In this tutorial, we will talk about Resources for this post: Step 1: Import Libraries The first step is …

Local Outlier Factor (LOF) For Anomaly Detection Read More »

How to detect outliers | Data Science Interview Questions and Answers

How to detect outliers | Data Science Interview Questions and Answers

Welcome to GrabNGoInfo! In this tutorial, we will talk about how to answer the data science interview question about outlier detection. The tutorial covers the general strategies of answering the question, and provides example questions and answers. Resources for this post: Strategies The strategy for the outlier detection question is divide and conquer. We divide …

How to detect outliers | Data Science Interview Questions and Answers Read More »

Gaussian Mixture Model (GMM) for Anomaly Detection. Predict anomalies from a Gaussian Mixture Model (GMM) using percentage threshold and value threshold, and improve anomaly prediction performance

Gaussian Mixture Model (GMM) for Anomaly Detection

Gaussian Mixture Model (GMM) is a probabilistic clustering model that assumes each data point belongs to a Gaussian distribution. Anomaly detection is the process of identifying unusual data points. Gaussian Mixture Model (GMM) detects outliers by identifying the data points in low-density regions [1]. In this tutorial, we will use Python’s sklearn library to implement Gaussian Mixture …

Gaussian Mixture Model (GMM) for Anomaly Detection Read More »