Explainable AI (XAI) - A Perspective
4 likes7,506 views
The document discusses explainable AI (XAI), emphasizing its importance in understanding AI decision-making through three key pillars: reasonable AI, understandable AI, and traceable AI. It outlines the limitations and challenges of XAI, including feature importance, domain nuances, and the complexities of different model types. The text also highlights business benefits of XAI, such as improved performance, decision-making, and regulatory compliance.
Explainable AI (XAI) - A Perspective
- 1. Explainable AI (XAI) – A Perspective Author – Saurabh Kaushik Twitter - @saurabhkaushik
- 2. 2 AI - How does it Work? Audience Measurement System Accounting Compliance System Output: This Customer will Churn Output: This Transaction will be Risky. How does it work?
- 3. 3 AI as Black Box Black Box AI • Why did the AI system do that? • Why didn’t the AI system do something else? • When did the AI system succeed? • When did the AI system fail? • When does the AI system give enough confidence in the decision that you can trust it? • How can the AI system correct an error?
- 4. 4 XAI – Full Circle of Life World Data Black Box Model Explainable Technique Captures Trains Interprets Augment
- 5. 5 Explainable AI – Three Key Pillars Explainable AI Reasonable AI: •The ability to understand the reasoning behind each individual prediction Understandable AI: •The ability to fully understand the model upon which the AI decision- making is based Traceable AI: •The ability to trace prediction process from logic of math algo to nature of data.
- 6. 6 Principals for XAI Designing • AI with an eye to societal impact Defining • Standards for the provenance, use, and securing of data sets Testing • AI extensively before release Using • AI transparently Monitoring • AI rigorously after release Fostering • Workforce training and retraining Protecting • Data privacy Establishing • Tools and standards for auditing algorithms
- 7. 7 Limitations for XAI • Most model analysis stays in finding the important features, but they largely ignore the interactions between the features.Feature importance • Each business problem has its domain nuances and has defining contribution to the Explainability. If model analysis process has less of it, the outcome of explanation will be blurred or too generalized.Problem domain • Preprocessing (Dimensionality Reduction, Word Vectorization, etc.) obscure the original human meaning and makes data less informative.Data preprocessing • During feature analysis and selection process, most of the correlations are either dropped to improve accuracy or swamped by other powerful features with more predictive powers. This hides the latent correlation factors from model and leads to incorrect explanation. Correlated input features • Straightforward Binary or Ordinal Classification and Linear Regression models are easier to explain as they have natural direction for decision. However, models like multi class classification do not have inherent order are difficult to explain, unless it comes down to ‘One vs All’ model. Type of prediction
- 8. 8 Business Benefits from XAI Optimize Model performance Decision making Retain Control Safety Maintain Trust Ethics Comply Accountability Regulation “Explainable AI is Responsible AI” “Right to Explanation” – GDPR
- 9. 9 Current State of AI Models w.r.t. XAI vs Performance • Regression – Multinomial Equation can be messy • Random Forrest – Multiple Tree and their Data Set and Voting • SVM – Kernel and Data Partition effect on Feature • K Mean – Nature of Centroid don’t describe cluster well. • NN – Hidden Nodes and their way of creating features
- 10. 10 Techniques for XAI • Deals with inner working of Algo/Model to interpret its results Model Specific Techniques • Deals with analyzing the feature and its relationships with its output. Model Agnostic Techniques Data Model Explanation Prediction
- 11. 11 Model Specific vs Agnostic – Approach Linear Regression Decision Tree SVM Random Forest Neural Network Linear Regression Decision Tree SVM Random Forest Neural Network Accuracy Explainability Explainability Accuracy Algo - SLIMs, GAINs, Rules Algo - LIME Model Specific Approach Model Agnostic Approach
- 12. 12 XAI - Model Specific Techniques • Decision Tree is most interpretable algorithm because its simple information gain mechanism of building the tree. Random Forest on other has multiple small tree with dataset variation and votes for final decision, which is based on majority. • There are open source project – Tree Interpreter available but deep domain and deep algorithm knowledge are required to study its output. Tree Interpretation • SLIM is a discrete optimization problem that minimizes the 0-1 loss to encourage a high level of accuracy, regularizes the L0-norm to encourage a high level of sparsity, and constrains coefficients to a set of interpretable values. Supersparse Linear Integer Models (SLIM) • Deep Lift - A method for decomposing the output prediction of a neural network on a specific input by backpropagation the contributions of all neurons in the network to every feature of the input. Deep LIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. Using this, This can give important sequence of input data. Deep Neural Network
- 13. 13 XAI - Model Agnostic Techniques • It try to analyze the effect of single input feature alteration on its model output. This given linear approximation of model response. • This approach is often extended to Partial Dependence Plots (PDP) or Individual Conditional Expectation (ICE) Plots to give global graphical representation of single Sensitivity Analysis • It try to find out Feature Importance by capturing Feature Interaction (Correlation and Covariance Analysis) between features and output using a linear model between Features. It used Perturbance technique which observer Prediction deviation based on one feature modification. Local Interpretable Model Explanation – LIME • SHAP is an Additive feature attribution methods which assigns a value to each feature for each prediction (i.e. feature attribution); the higher the value, the larger the feature’s attribution to the specific prediction. Shapley Addictive Explanations – SHAP
- 14. 14 Future Status of XAI Systems
- 15. Thank You Author – Saurabh Kaushik Twitter - @saurabhkaushik
- 16. 16 References • https://www.darpa.mil/program/explainable-artificial-intelligence • https://ai.intel.com/the-challenges-and-opportunities-of-explainable-ai/ • https://arxiv.org/pdf/1708.08296.pdf • https://bons.ai/blog/explainable-ai-machine-learning-systems • https://medium.com/@BonsaiAI/explainable-ai-3-deep-explanations-approaches-to-xai-1807e251e537 • https://www.pwc.com/us/en/services/consulting/library/artificial-intelligence-predictions/explainable- ai.html • https://aibusiness.com/explainable-algorithms-rainbird-ai/ • https://www.accenture.com/us-en/blogs/blogs-why-explainable-ai-must-central-responsible-ai • https://www.accenture.com/us-en/blogs/blogs-responsible-ai