Explaining job recommendations: a human-centred perspective
0 likes49 views
This document summarizes Katrien Verbert's presentation on explaining job recommendations from a human-centered perspective. The presentation discusses (1) the need to explain job recommendation models to increase user trust and acceptance, (2) using explanation methods like visualizations to enable user interaction with explanations and improve models, and (3) designing explanations of a job recommendation system to increase user empowerment, clarify recommendations, and support job mediators. The research aims to balance explanation, exploration, and actionable insights when interacting with recommender systems.
![Support the dialogue between domain expert and laymen
14
Human-in-the-loop
Sven Charleer, Andrew Vande Moere, Joris Klerkx, Katrien Verbert, and Tinne De Laet. 2017. Learning
Analytics Dashboards to Support Adviser-Student Dialogue. IEEE Transactions on Learning Technologies
(2017), 1–12.
“… the expert can become the
intermediary between the [system] and
the [end-user] in order to avoid
misinterpretation and incorrect
decisions on behalf of the data… “](https://image.slidesharecdn.com/xai-hr-220923135626-77475239/85/Explaining-job-recommendations-a-human-centred-perspective-14-320.jpg)
![Design goals
16
[DG1] Control the message
[DG2] Clarify the recommendations
[DG3] Support the mediator](https://image.slidesharecdn.com/xai-hr-220923135626-77475239/85/Explaining-job-recommendations-a-human-centred-perspective-16-320.jpg)
![[DG1] control the message
20
Two themes
(1) Customization
(2) Importance of the human factor](https://image.slidesharecdn.com/xai-hr-220923135626-77475239/85/Explaining-job-recommendations-a-human-centred-perspective-20-320.jpg)
![[DG2] Clarify recommendations
21
Two themes
(1) Understanding the visualisation
(2) Convincing power](https://image.slidesharecdn.com/xai-hr-220923135626-77475239/85/Explaining-job-recommendations-a-human-centred-perspective-21-320.jpg)
![[DG3] Support the mediator
Useful cases
¤ Orientation
¤ Job mobility
22](https://image.slidesharecdn.com/xai-hr-220923135626-77475239/85/Explaining-job-recommendations-a-human-centred-perspective-22-320.jpg)
![Labor Market Explorer Design Goals
31
[DG1] Exploration/Control
Job seekers should be able to control
recommendations and filter out the information
flow coming from the recommender engine by
prioritizing specific items of interest.
[DG2] Explanations
Recommendations and matching scores should be
explained, and details should be provided on-
demand.
[DG3] Actionable Insights
The interface should provide actionable insights to
help job-seekers find new or more job
recommendations from different perspectives.](https://image.slidesharecdn.com/xai-hr-220923135626-77475239/85/Explaining-job-recommendations-a-human-centred-perspective-31-320.jpg)
Explaining job recommendations: a human-centred perspective
- 1. Explaining job recommendations: a human-centred perspective FEAST @ ECML-PKDD - 23 Sept 2022 Katrien Verbert Augment/HCI - KU Leuven @katrien_v
- 2. Human-Computer Interaction group Explainable AI - recommender systems – visualization – intelligent user interfaces Learning analytics & human resources Media consumption Precision agriculture Healthcare Augment Katrien Verbert ARIA Adalberto Simeone Computer Graphics Phil Dutré LIIR Sien Moens E-media Vero Vanden Abeele Luc Geurts Kathrin Gerling
- 3. Augment/HCI team Robin De Croon Postdoc researcher Katrien Verbert Professor Francisco Gutiérrez Postdoc researcher Tom Broos PhD researcher Nyi Nyi Htun Postdoc researcher Houda Lamqaddam Postdoc researcher Oscar Alvarado Postdoc researcher https://augment.cs.kuleuven.be/ Diego Rojo Carcia PhD researcher Maxwell Szymanski PhD researcher Jeroen Ooge PhD researcher Aditya Bhattacharya PhD researcher Ivania Donoso Guzmán PhD researcher 3
- 4. q Explaining model outcomes to increase user trust and acceptance q Enable users to interact with the explanation process to improve the model Research objectives Models
- 5. 5
- 6. Collaborative filtering – Content-based filtering Knowledge-based filtering - Hybrid Recommendation techniques
- 7. 7 Bostandjiev, S., O'Donovan, J., & Höllerer, T. (2013, March). LinkedVis: exploring social and semantic career recommendations. In Proceedings of the 2013 international conference on Intelligent user interfaces (pp. 107-116).
- 8. Explaining prediction models 8 Gutiérrez, F., Ochoa, X., Seipp, K., Broos, T., & Verbert, K. (2019, September). Benefits and trade-offs of different model representations in decision support systems for non-expert users. In IFIP Conference on Human-Computer Interaction (pp. 576-597).
- 10. Human resources Job mediators ¤ Supporting dialogue ¤ Explaining predications Job seekers ¤ Supporting job seekers ¤ Explaining recommendations 10
- 11. Predictions for finding jobs 11
- 12. Predicting duration to find a job 12 Context: Three years of data, 700 000 job seekers. Key Issues: Missing data, prediction trust issues, job seeker motivation, lack of control.
- 13. Goal 13 Design explanations to increase:
- 14. Support the dialogue between domain expert and laymen 14 Human-in-the-loop Sven Charleer, Andrew Vande Moere, Joris Klerkx, Katrien Verbert, and Tinne De Laet. 2017. Learning Analytics Dashboards to Support Adviser-Student Dialogue. IEEE Transactions on Learning Technologies (2017), 1–12. “… the expert can become the intermediary between the [system] and the [end-user] in order to avoid misinterpretation and incorrect decisions on behalf of the data… “
- 15. Customer journey approach Observation of hands-on time Observations of individual mediation sessions Questionnaire Preliminary study
- 16. Design goals 16 [DG1] Control the message [DG2] Clarify the recommendations [DG3] Support the mediator
- 17. 17
- 18. Design and development 18 Forest plot Circles chart Barchart
- 19. Evaluation 19 Years of experience: (M = 9, SD = 4.3) Six mediators dealt only with higher education job seekers. Four with secondary to higher education. Two dealth with job seekers without technical/professional education. Semi-structured interviews 1) Feedback on parameter visuals. 2) Interaction feedback with the working prototype dashboard. Qualitative evaluation with expert users: (N = 12, 10f, age: M= 40.7, SD = 9.4)
- 20. [DG1] control the message 20 Two themes (1) Customization (2) Importance of the human factor
- 21. [DG2] Clarify recommendations 21 Two themes (1) Understanding the visualisation (2) Convincing power
- 22. [DG3] Support the mediator Useful cases ¤ Orientation ¤ Job mobility 22
- 23. Take away messages ¤ Key difference between actionable and non-actionable parameters. ¤ Need for customization and contextualization. ¤ The human expert plays a crucial role when interpreting and relaying in the predicted or recommended output. 23 Charleer S., Gutiérrez Hernández F., Verbert K. (2019). Supporting job mediator and job seeker through an actionable dashboard. In: Proceedings of the 24th IUI conference on Intelligent User Interfaces Presented at the ACM IUI 2019, Los Angeles, USA. (Core: A)
- 24. Human resources Job mediators ¤ Supporting dialogue ¤ Explaining predications Job seekers ¤ Supporting job seekers ¤ Explaining recommendations 24
- 25. Explaining job recommendations 25 25 • Abundant overload of job vacancies • Dynamic Labor Market: need to support job mobility • Providing effective recommendations particularly challenging. • Need for: increased diversity explanations user control exploration
- 26. Approach Explaining job recommendations to show competence match Support exploration and user control over broad and diverse recommendations
- 27. Explaining job recommendations 27 Gutiérrez, F., Charleer, Sven, De Croon, Robin, Nyi Nyi Htun, Goetschalckx, Gerd, & Verbert, Katrien. (2019) “Explaining and exploring job recommendations: a user-driven approach for interacting with knowledge-based job recommender systems”. In Proceedings of the 13th ACM Conference on Recommender Systems. ACM, 2019
- 28. Methods 28
- 29. 2 9 Ranking of parameters as voted by participants
- 30. 3 0
- 31. Labor Market Explorer Design Goals 31 [DG1] Exploration/Control Job seekers should be able to control recommendations and filter out the information flow coming from the recommender engine by prioritizing specific items of interest. [DG2] Explanations Recommendations and matching scores should be explained, and details should be provided on- demand. [DG3] Actionable Insights The interface should provide actionable insights to help job-seekers find new or more job recommendations from different perspectives.
- 32. 32
- 33. Final Evaluation 33 66 job seekers (age 33.9 ± 9.5, 18F) 8 Training Programs, 4 Groups, 1 Hour. 1 2 3 4 5 6 7 8 ResQue Questionnaire + two open questions. Users explored the tool freely. All interactions were logged.
- 34. 34 Results
- 35. 35 Results
- 36. Results ¤ Explanations contribute to support user empowerment. ¤ A diverse set of actionable insights were also mentioned by participants. ¤ Participants in the technical group engaged more with all the different features of the dashboard. ¤ Non-native speakers, sales and construction groups engaged more with the map. ¤ The table overview was perceived as very useful by all user groups, but the interaction may need further simplification for some users. 36
- 37. Research challenges ¤ Insight vs. information overload ¤ Visual representations often difficult for non-expert users ¤ Limitations of user studies 37
- 38. Next steps ¤ Personalisation ¤ Conversational explanation methods ¤ Interactive explanation methods 38
- 39. Peter Brusliovsky Nava Tintarev Cristina Conati Denis Parra Collaborations Bart Knijnenburg Jurgen Ziegler