What is Local Interpretable Machine Learning in xAI?
Local Interpretable Machine Learning (LIME) is an essential element of Explainable AI (xAI) to find out a reasonable explanation of the decision-making processes of machine learning models. Instead of attempting to understand the daunting task of the global behaviour of a model across its entire input space, LIME Interpretation offers a more strategic approach by explaining individual predictions.
This "zooming in" approach to specific examples creates a clearer and more intuitive understanding of the reasoning behind a LIME Interpretable machine learning model's decision. The goal is to model the behaviour of the original model as closely as possible in a small neighbourhood around a specific data point with a simpler, more understandable model.
LIME is a tool for LIME Interpretation in explainable AI and it exemplifies this local neighbourhood approach. It builds a simpler, more interpretable model (which is normally a linear model) around the instance of interest. The local Interpretable model's purpose is to replicate the behaviour of the original, complicated model in that particular neighbourhood. LIME Interpretation allows us to identify which features had the greatest impact on a model's decision for a given instance, which is essential for debugging the model error, building trust in a model's output and uncovering biases or unexpected sensitivities.
By using LIME interpretable machine learning, users can manipulate the features in a single record and see how those changes influence the final prediction, which is interactive and facilitates understanding of model behaviour. Ultimately, LIME interpretation in explainable AI provides a human-understandable interpretation for an individual case, and why a decision was made for a specific case improves understanding and ultimately confidence. Rather than conceptualising how a loan application model works overall, LIME Interpretation machine learning provides an interpretation for why one particular customer was denied.
Why is LIME important?
The critical importance of LIME Interpretation arises from its unique position to illuminate the workings of complicated, often cryptic, "black-box" machine learning models. As important as accuracy often is in machine learning capabilities, that measure is rarely sufficient. In many applications in which we apply models in the real world, we want to know beyond simply understanding the accuracy of a model.
We may want to understand why it made that prediction, what the most significant factors were and how those factors come together to produce that prediction. That’s why LIME interpretability considers other factors related to model accountability is most critical in high-stakes decisions when there is significant cost to decisions made by models.
LIME interpretability in machine learning fills a critically important role in this area to understand enough to look into the black box to understand the causes for a given outcome. There is something inherently important about transparency in black box models, particularly in high-stakes situations when accountability and transparency of models are critical.
Consider these examples:
- Medical Diagnosis: I cannot imagine the level of comfort a physician feels in accepting the output of a machine learning model that predicts a health-related diagnosis (e.g., a certain type of cancer). LIME interpretation might help them understand the features that made the prediction (e.g., the symptoms, lab results, imaging, etc.) and provide a way to evaluate the model's reasoning.
- Loan Applications: If someone has had their loan application denied, seeing the rationale is an important feature for understanding what specific financial factors they may need to improve to be considered again in the future. LIME Interpretable machine learning will indicate the most important factors in the decision, such as credit score, the debt-to-income ratio and employment history.
- Criminal Justice: In predictive policing or risk assessment tools used in justice systems, understanding the guidelines as to why a model gave an individual a given risk score is necessary when thinking about fairness and discrimination. LIME explanation machine learning identifies which of those factors were flagged to help analyze and address any bias in the model.
- Autonomous Vehicles: If an autonomous vehicle crashes, understanding the rationale behind why the driverless car swerved in a given direction (e.g., right or left) will be essential to determine liability and improve the safety of the system.
Given the valuable information that LIME interpretability provides concerning the decision by an AI system and allowing the user to confirm the logic behind the predicted output, trust and responsible design and deployment of AI systems are bolstered. Moreover, by allowing users or stakeholders to question the predicted output that may be incorrect, flawed and biased, LIME Interpretation in explainable AI ensures that the process of using AI has fairness and accountability within it. Without a visible and constraining rationale for the model output, the adoption of xAI with local interpretation in critical situations would be very unlikely because of a distrust.
Local interpretable models in machine learning
Local interpretable models signify a paradigm shift in thinking about interpreting complex machine learning models. Instead of attempting to understand the entire complexity of a global model with a single explanation (which may not maybe possible), local interpretable model agnostic explanations emphasize that local models are intended to provide simplified representations of the model, around a particular region of the input space. Local interpretable models also only attempt to explain the behaviors of the model around a particular data point or a narrow cluster of data points of similar observation. Local interpretable model agnostic explanation uses input information to try to predict the behaviors of the original model in that local boundary - in essence, providing guidance about which features are most responsible for that piece of data/information.
LIME interpretation makes use of this concept well. It builds a locally weighted linear model (or some other simple, interpretable model) that rationalises and approximates the behaviour of a black-box model within a very small neighbourhood of the instance being explained. The local interpretable model agnostic explanation model is not intended to be an identical replica of the original model; indeed, it could be more different than similar as a 'simple'/ 'interpretation' model is designed to replicate the behaviour in the local neighbourhood of the black-box model.
The process consists of the following steps:
- Perturb: produce a series of modified observations from the original observation.
- Predict: provide the perturbed data set to the original complex model, which in turn produces a series of predictions.
- Weight: place weights onto the perturbed data based on the distance to the original observation, whereby the closer the observation, the greater the weight.
- Train: to the perturbation data and predictions; use the respective weights in training the simple model (s) (e.g., a linear model).
In sum, the resulting coefficients from the linear model then indicate the respective importance of each feature's contribution to the black-box model's prediction associated with that given instance or observation. The local interpretable model agnostic explanation provides the user with a better insight into the rationale associated with the model prediction without needing to understand the global behaviour of the model.
Model-agnostic interpretability methods
Methodologies categorized as model-agnostic interpretability represent one of the strongest and most flexible ways of Interpretable machine learning method. These methods can be applied to any machine learning algorithm regardless of its architecture, complexity or training process; unlike model-specific methods that are based on the model's internals (e.g., model weights in a linear regression model or the structure of a decision tree).
Model-agnostic interpretability addresses the model from the "black box" perspective, where the only concern is the input-output relationship of the sometimes hopelessly complex model. They allow us to interpret the model's predictions without having to depend on the model's internals like parameters, gradients, and so on. Model-agnostic interpretability treats the model merely as a mechanism to learn about the model's processes by observing its predictions for different inputs.
An example of model-agnostic interpretability is LIME Interpretable machine learning. LIME interpretation does not make any assumptions about the model being explained. All LIME Interpretation in explainable AI needs is a model that can input and view output data pairs. Therefore, this model-agnostic nature makes LIME interpretation a very flexible approach to understanding many model types, including deep neural networks, support vector machines, random forests, gradient boosting machines and even black-box proprietary or custom-built models. The existence of model-agnostic interpretability methods (LIME specifically) is essential to practitioners when they are working with many models of varying complexity. In the real world, we often need to piece together a working collection of models to make predictions.
Model-agnostic interpretability methods - Using LIME Interpretation in explainable AI to document model uncertainty
LIME interpretation has the added bonus: LIME interpretable machine learning excels at understanding opaque 'black-box' models where the internal decision-making processes are completely hidden from view and very difficult to make sense of otherwise. In these types of situations, ordinary strategies for investigating the model parameters or what is going on internally in a black box model are not possible. However, LIME interpretation successfully gets around this problem by viewing the black box model as a function that feeds inputs into outputs. Accordingly, it uses the black box function to create explanations but does not need the internal workings of the model at all.
Model Black – Box interpretability methods using LIME
LIME is used to interpret predictions made by complex black-box machine learning models. It explains individual predictions by approximating the model locally with a simpler and interpretable model.
Input Perturbation:
LIME interpretable machine learning first perturbs the input instance and creates a set of perturbed input instances in the form of slightly altered input values. These perturbations can be any means of perturbing the input values, such as noise factor, masking features or otherwise making small adjustments to the value.
Black-Box Prediction:
Next, these perturbed inputs are fed through the black box model and the model's predictions are collected. This ultimately creates a new dataset of perturbed inputs and model predictions.
Interpretable Model Trained:
Finally, LIME interpretation takes the perturbed data and their respective predictions and fits a simple, interpretable machine learning model to the perturbed data. Interpretable machine learning models can be linear models, decision trees, etc. The simple, interpretable machine learning model will only be a useful and effective approximation of the black box model's behavior locally, around the original input instance. The perturbations that are closer to the original data point will be weighted more in fitting, meaning the interpretable machine learning model is really learning the local behaviour of the black box model.
Feature Importance Extraction:
The trained, interpretable machine learning model's coefficients or feature significance indicate the importance of each feature in driving the prediction of the black-box model for that instance. These feature importance values give a human-based, understandable justification for the black-box model's prediction.
This method uses the black-box model as an oracle, asking it questions through inputs and learning to approximate the oracle's calculations through a simpler, interpretable machine learning model. LIME interpretation allows us to avoid having to get into the very complicated internal structure of a black-box model, and so it has value in understanding and ultimately trusting these models in a real-world situation.
Explainable AI with LIME implementation
An explainable AI with LIME implementation follows a deliberate series of steps to provide meaningful explanations to individual predictions. The basic procedure is common across data types, but the particular details of perturbation and interpretable machine learning model choice may differ.
Below are the basic steps:
- Instance Selection: The process starts with selecting an instance (data point) in which the explainable AI is desired. Your instance could be a customer profile, a specific medical image of a patient or a text-based review from a customer.
- Neighbourhood Generation: The key to LIME's local interpretable model agnostic explanation approximation is sampling a number of new data points that simulate the "neighbourhood" of the original instance. This simulation is generated from perturbing the original instance in a controlled way. The method of perturbation is based on the data type.
- Tabular Data: Features are perturbed by either randomly sampling from the marginal distributions of each feature or distorting the feature values by adding random noise in interpretable machine learning model.
- Image Data: The image is broken up into superpixels (regions of pixels that share similarity in colour) and the perturbation is randomly turning off some or all of the superpixels.
- Text Data: The text is perturbed through randomly deleting words or parts of sentences.
- Prediction with Black-Box Model: Each of these perturbed instances is passed through the black-box model and predictions are recorded for each instance in an interpretable machine learning model. This provides input-output pairs that reflect the black-box model's behaviour in relation to the original instance.
- Fitting a Simple Model: A simple, interpretable machine learning model (such as a linear model or sparse linear model) is fitted to the perturbed data and predictions. A key part of this step is the weighting for the perturbed data points. Instances that are closer to the original instance (the one being explained) have more weight; the reasoning for fitting a weighted model is that the interpretable machine learning model is only supposed to approximate the behaviour of the black-box model in the neighbourhood of the instance being explained.
- Obtaining the Explanation: Finally, the coefficients or feature importance from the fitted interpretable machine learning model are used to uncover which features contributed to the prediction. The coefficients represent how much the prediction was influenced by the features from the interpretable machine learning model, which can only approximate the black-box model locally. The sign of the coefficient indicates the direction of influence (positive or negative).
Implementation details can vary a lot depending on the data you are working with, as previously mentioned. The next sections will provide a slightly greater detail when using LIME Interpretation in explainable AI using tabular, image and text data.
LIME Implementation with Tabular Data
Implementing LIME interpretation for tabular data concerns both how we should perturb the data and which interpretable model we should then fit, based on our data. We can describe the process in a fair amount of detail:
Data Preprocessing:
Prior to using LIME Interpretation in explainable AI, tabular data has often been pre-processed in a way that transforms this data into something more suitable for our analysis work. Pre-processing may include scaling, normalizing and transforming categorical features into a numeric representation. This level of pre-processing allows making sure that the features have similar scales and categorical features are numerically represented.
Changing Features:
Continuous features are typically changed by random noise that is sampled from a normal distribution with zero mean and standard deviation proportional to the range of the feature of the LIME explanation machine learning. Categorical features of the LIME interpretability are changed by sampling random values from the possible categories of that feature.
The Distance Metric:
A distance metric explains how similar our changed instances are to the instance we are trying to explain or understand. There are many ways of measuring distances, the most common being Euclidean distance, cosine similarity and Manhattan distance. The distance metric we select will likely affect the LIME explanation machine learning we derive.
Interpretable Model:
The interpretable machine learning model is usually a linear model, such as a linear regression model or a sparse linear model. Sparse linear models are usually preferred, because they will select the features for you and will yield simpler and therefore easier to explain, local interpretable model agnostic explanations.
Feature Importance Visualization:
The coefficients of the linear model describe the contribution of each of the features to producing the prediction. The feature importance of the LIME interpretation can be visualized in bar plots, pie charts, etc. The user can then look through these visualizations to see what is contributing most to the prediction.
Libraries like Python have useful functions and classes to wrap LIME Interpretable machine learning with tabular data, which perform all the steps above for you. Interpretable machine learning models in Python almost cover everything about Local Interpretable Model-Agnostic explanations with suitable examples and applications.
LIME Implementation with Image Data
LIME implementation on image data is different from text data because the perturbation of the image considers the image as a set of spatially structured pixels. There are some additional considerations and approaches to perturbation of the image and the visualization of the lime explanation machine learning.
Superpixels Segmentation:
The first thing in Lime Implementation that happens is that the image is segmented using superpixels, which are small, contiguous pixel regions with similar characteristics in the image (e.g., colour, texture). Superpixels are better than pixels in that they take more meaningful local patterns and structures into consideration. The most commonly used superpixels segmentation algorithm in LIME Interpretation in explainable AI is also known as SLIC (Simple Linear Iterative Clustering).
Perturbation:
The next step in Lime Implementation is to perturb the image by turning off or occluding some of the superpixels. The replacement of pixels in the selected superpixels with neutral colour (e.g., grey, black) is the more customary practice. The number of superpixels that are turned off is a tunable hyperparameter.
Interpretable Model:
The interpretable machine learning model is trained on the perturbed image and their predictions. The input to the interpretable machine learning model is typically a vector representing the presence or absence of each of the superpixels in the perturbed image.
Explanation Visualization:
The coefficients of the interpretable machine learning model indicate the contribution of superpixels to the prediction. The coefficients of the interpretable machine learning model are visualized by highlighting the important superpixels in the original image and the user can visually inspect the highlighted superpixels to see the parts of the image that drove the prediction.
For example, if the model predicted that the image contained a cat, LIME interpretation would highlight the superpixels that corresponded to the face, ears and body of the cat. The highlighted superpixels provide a visual representation of the model’s reasoning.
LIME Implementation with Text Data
LIME interpretation has a similar application to text data; however, it is worth stressing that text data is sequential and the order of the words in the text can be very important.
Text Preprocessing:
The first step to Lime Implementation with text data are pre-processing by tokenizing the text (converting text into separate words or phrases), removing stop words (words that are common words such as: the, a, is) and stemming or lemmatizing the words (reducing the word to its root form).
Perturbation:
Text is perturbed by randomly removing words or phrases from the text. Once again, the number of words or phrases removed is a tunable hyperparameter.
Interpretable Model:
After the interpretable machine learning model is trained on the perturbed text and their predictions. The interpretable machine learning model input is typically a vector representing the presence or absence of each of the words or phrases from the perturbed text input. Instead, the local interpretable machine learning model can represent the term weights based on the importance in the document using TF-IDF (Term Frequency-Inverse Document Frequency).
Explanation Visualization:
The coefficients of the interpretable machine learning model represent the influence of each word or phrase on the prediction. The coefficients of the local interpretable machine learning model are visualized by bringing attention to the important words or phrases in the original text. Once again, the user can check the highlighted words or phrases to see what parts of the text were most consequential to the prediction. For example, if the model classified a review as positive, LIME interpretation would highlight positive words like "amazing", "delicious", "highly recommended" and "excellent service".
Examples of Local Interpretable Model-Agnostic Explanations
Fraud Detection:
A credit card transaction is flagged as potentially fraudulent. LIME explanation machine learning explains that the factors that contributed to this were that the transaction amount was much higher than the customer’s average transaction amount, and it was made in a foreign country that the customer had never traveled to and the transaction was made at an odd time of day. These three factors LIME interpretation provided, give clear reasoning as to why the model flagged the transaction as fraud, allowing fraud analysts to make a risk decision.
Medical Diagnosis:
A model predicts that a patient has a high risk of diabetes. LIME explanation machine learning explains the reasons with the high BMI, glucose levels, family history of diabetes and lack of exercise. LIME interpretation explained the model’s reasoning to the doctors so they could inform the patient of lifestyle changes and preventive measures.
Sentiment Analysis:
A customer review is predicted as negative. LIME interpretation in explainable AI gives the features that contributed to the negative prediction, such as the words "terrible," "unreliable," "disappointed," and "waste of money." LIME interpretation helped the business understand the customer dissatisfaction causing the negative rating and provided areas for improvement on their product or service.
Image Classification:
A model classifies an image of a dog. LIME explanation machine learning explains the parts of the image that matter most to the model are the dog’s head, ears and paws. This clarification provides users the assurance that the model is on attention to the relevant features of the image and not irrelevant background.
