DRONE: A Tool to Detect and Repair Directive Defects in Java APIs Documentation
1 like452 views
DRONE is a tool that detects and repairs defects in API documentation by analyzing documentation directives and source code. It uses natural language processing techniques to extract constraints from documentation and builds abstract syntax trees from code to classify parameter usage. DRONE was evaluated on 8 Java libraries and Android APIs, detecting 1291 real defects with a precision over 76%, recall over 84%, and F-measure over 80%. It also provides repair recommendations for the detected defects based on predefined templates.
DRONE: A Tool to Detect and Repair Directive Defects in Java APIs Documentation
- 1. DRONE: A Tool to Detect and Repair Directive Defects in Java APIs Documentation Sebastiano Panichella Xin Yan Taolue Chen Harald Gall Yu Zhou
- 2. Outline 2 DRONE Context: Proposed Solution based on: Case Study: APIs Usage in OSS and Industrial Projects - NLP Approaches; - Static Analysis Techniques; - Logic reasoning; Assessment of DRONE on Documentation and Code 8 Java Libraries Documentation Code
- 3. Open Source (OS) and Industrial Projects 3 “Social networks like Facebook or Pinterest, or utilities like Google Maps or Dropbox are popular examples of APIs providers.”
- 4. Open Source (OS) and Industrial Projects 4 “Social networks like Facebook or Pinterest, or utilities like Google Maps or Dropbox are popular examples of APIs providers.”
- 5. Open Source (OS) and Industrial Projects 5“APIs are great time savers for Developers…”
- 6. 6 Difficult to Understand describe Information Needed Ideally…. “API documents represent one of the most important references for developers…” Source Code APIs Documents Application Programming Interface (API) Documents
- 7. 7 Software Changes over the Time “…as consequence the original documentation tend to be incomplete and inconsistent with the source code…” Insufficient Information Source Code Difficult to Understand APIs Documents Inconsistent withComing back to the reality... Inconsistent/ Incomplete
- 8. 8 Source Code APIs Documents Inconsistent/ Incomplete API Document Defect: Example 1 Class: TextLayout. Method: getBlackBoxBounds(int firstEndpoint, int secondEndpoint) API Document JDK-1.8
- 9. 9 Source Code APIs Documents Inconsistent/ Incomplete API Document Defect: Example 1 Class: TextLayout. Method: getBlackBoxBounds(int firstEndpoint, int secondEndpoint) API Document JDK-1.8 https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html
- 10. 10 Source Code APIs Documents Inconsistent/ Incomplete API Document Defect: Example 1 API Document JDK-1.8 Class: TextLayout. Method: getBlackBoxBounds(int firstEndpoint, int secondEndpoint) https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html
- 11. 11 Source Code APIs Documents Inconsistent/ Incomplete API Document Defect: Example 1 API Document JDK-1.8 Class: TextLayout. Method: getBlackBoxBounds(int firstEndpoint, int secondEndpoint) https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html
- 12. 12 Source Code APIs Documents Inconsistent/ Incomplete API Document Defect: Example 1 API Document JDK-1.8 Class: TextLayout. Method: getBlackBoxBounds(int firstEndpoint, int secondEndpoint) ————————————-—————————————- ————————————-—————————————- https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html
- 13. 13 Source Code APIs Documents Inconsistent/ Incomplete API Document Defect: Example 1 API Document JDK-1.8 Class: TextLayout. Method: getBlackBoxBounds(int firstEndpoint, int secondEndpoint) ————————————-—————————————- ————————————-—————————————- https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html
- 14. 14 Source Code APIs Documents Incomplete API Document Defect: Example 1 API Document JDK-1.8 Class: TextLayout. Method: getBlackBoxBounds(int firstEndpoint, int secondEndpoint) ————————————-—————————————- ————————————-—————————————- https://docs.oracle.com/javase/8/docs/api/java/awt/font/TextLayout.html
- 15. ————————————-—————————————- ————————————-—————————————- 15 Source Code APIs Documents Inconsistent API Document Defect: Example 2 API Document JDK-1.8 Class: InputEvent Method: getMaskForButton(int button) https://docs.oracle.com/javase/8/docs/api/java/awt/event/InputEvent.html
- 16. 16 Source Code APIs Documents Inconsistent API Document Defect: Example 2 API Document JDK-1.8 Class: InputEvent Method: getMaskForButton(int button) ————————————-—————————————- ————————————-—————————————- https://docs.oracle.com/javase/8/docs/api/java/awt/event/InputEvent.html
- 17. 17 Source Code APIs Documents Inconsistent API Document Defects are Frequent API Document JDK-1.8 Class: InputEvent Method: getMaskForButton(int button) ————————————-—————————————- ————————————-—————————————- https://docs.oracle.com/javase/8/docs/api/java/awt/event/InputEvent.html “…and tend to be discovered and fixed after long time…” http://stackoverflow.com/questions/2967303/inconsistency-in-java-util-concurrent-future
- 18. 18 DRONE Detect and Repair of dOcumentatioN dEfects) Code API Document Software Artifacts AST Parsing Pre-Process and POS Tagging Defect Reports Control Flow- Based Constraint Analysis SMT Solver Dependency Parsing and Pattern Analysis Code Constraint FOL Generating Doc Constraint FOL Generating HeuristicsHeuristics
- 19. 19 DRONE DetectoR of dOcumentatioN dEfects
- 20. 20 DRONE DetectoR of dOcumentatioN dEfects “…we consider 4 cases of parameter usage constraints”
- 21. 21 DRONE DetectoR of dOcumentatioN dEfects “…we consider 4 cases of parameter usage constraints” 1) “Nullness not allowed” 2) “Nullness allowed” NULL
- 22. 22 DRONE DetectoR of dOcumentatioN dEfects “…we consider 4 cases of parameter usage constraints” 1) “Nullness not allowed” 2) “Nullness allowed” 3) “Type restriction” 4) “Range limitation” NULL T
- 23. Previous Work ICSE 2017 TSE 2018
- 24. 24 Step 1. Construct AST
- 25. 25 Step 1. Construct AST Source Code
- 26. 26 Step 1. Construct AST Source Code c c(i+1) ……. …….
- 27. 27 Step 1. Construct AST 1) for each method “m” the 2) its code call graph G c(i+1); c = { call(m,c) } c(i+1) ……. ……. “…and extracts” 27
- 28. 28 Step 1. Construct AST 28 1) for each method “m” the 2) its code call graph G c(i+1); c = { call(m,c) } c(i+1) ……. “…and extracts”
- 29. 29 Step 1. Construct AST 29
- 30. 30 Step 1. Construct AST 30 c For each
- 31. 31 Step 1. Construct AST 31 c For each we collect
- 32. 32 Step 1. Construct AST 32 c(i+1);c(i+1) ……. ……. c For each we classify the 1) “Nullness not allowed” 2) “Nullness allowed” 3) “Type restriction” NULL T
- 33. 33 Step 1. Construct AST 33 c(i+1);c(i+1) ……. ……. 1) “Nullness not allowed” 2) “Nullness allowed” 3) “Type restriction” NULL T 4) “Range limitation”
- 34. 34 Extract Constraints from Directives in API Documents Natural Language Parsing
- 35. 35 Extract Constraints from Directives in API Documents Natural Language Parsing “Recurrent Linguistic Patterns (LPs)…” Di Sorbo et al. ASE 2015
- 36. 36 Example “ @exception NullPointerException if the item’s value is equal to < code> null< /code>” “Recurrent Linguistic Patterns (LPs)…” Di Sorbo et al. ASE 2015 Class: InputEvent java.awt.Choice Method: addItem(String item)
- 37. 37 Example “ @exception NullPointerException if the item’s value is equal to < code> null< /code>” {@exception NullPointerException } Class: InputEvent java.awt.Choice Method: addItem(String item) “Linguistic Pattern (LP)…” “item equal to null” Di Sorbo et al. ASE 2015
- 38. 38 Definition of NLP Heuristic “Linguistic Pattern (LP)…” “item equal to null” Steps: 1) Considering the relevant details; 2) Generalizing some information; 3) Ignoring useless information NLP Heuristic: Di Sorbo et al. ASE 2015 NLP Heuristic: 1) if the item’s value is equal to null”
- 39. 39 Definition of NLP Heuristic {@exception NullPointerException } “Linguistic Pattern (LP)…” “item equal to null” Steps: 1) Considering the relevant details; 2) Generalizing some information; 3) Ignoring useless information NLP Heuristic: 1) if the item’s value is equal to null” Di Sorbo et al. ASE 2015
- 40. 40 Definition of NLP Heuristic {@exception NullPointerException } “Linguistic Pattern (LP)…” “item equal to null” Steps: 1) Considering the relevant details; 2) Generalizing some information; 3) Ignoring useless information NLP Heuristic: 1) if the item’s value is equal to null” 2) if the (subj)’s value is equal to null” Di Sorbo et al. ASE 2015
- 41. 2) if the (subj)’s value is equal to null” 41 Definition of NLP Heuristic {@exception NullPointerException } “Linguistic Pattern (LP)…” “item equal to null” Steps: 1) Considering the relevant details; 2) Generalizing some information; 3) Ignoring useless information NLP Heuristic: 1) if the item’s value is equal to null” 3) if the (subj)’s value is equal to null” Di Sorbo et al. ASE 2015
- 42. 2) if the (subj)’s value is equal to null” 42 Definition of NLP Heuristic “Linguistic Pattern (LP)…” “item equal to null” NLP Heuristic: 1) if the item’s value is equal to null” 3) if the (subj)’s value is equal to null” 64 NLP Heuristics Steps: 1) Considering the relevant details; 2) Generalizing some information; 3) Ignoring useless information Di Sorbo et al. ASE 2015
- 43. 3) if the (subj)’s value is equal to null” 2) if the (subj)’s value is equal to null” 43 NLP Heuristic: 1) if the item’s value is equal to null” 64 NLP Heuristics 3) if the (subj)’s value is equal to null” Steps: 1) Considering the relevant details; 2) Generalizing some information; 3) Ignoring useless information Generation of FOL Constraints
- 44. 3) if the (subj)’s value is equal to null” 2) if the (subj)’s value is equal to null” 44 Generation of FOL Constraints NLP Heuristic: 1) if the item’s value is equal to null” 64 NLP Heuristics 3) if the (subj)’s value is equal to null” (subj) = null Steps: 1) Considering the relevant details; 2) Generalizing some information; 3) Ignoring useless information
- 45. 45 ? SMT Solver
- 46. 46 DRONE Demo DRONE UI—defect detection view !
- 47. 47 Defects Repair:! Based on the defect report generated from the previous part, DRONE gives repair recommendations based on the predefined templates.! Repair Recommendation Templates! DRONE Demo
- 48. 48 DRONE DRONE UI—defect repair view !
- 50. 50 Two Case Studies Case Study 1 8 JDK Libraries Case Study 2 Android APIs
- 51. 51 Case Study 1 8 JDK Libraries Android APIs
- 52. 52 8 JDK Libraries Results of Case Study 1: JDK APIs Case Study 1 DRONE ? 1540 Potential Defects Android APIs
- 53. 53 8 JDK Libraries Results of Case Study 1: JDK APIs Case Study 1 1291 Real Defects DRONE ? 1540 Potential Defects Android APIs
- 54. 54 8 JDK Libraries Results of Case Study 1: JDK APIs Case Study 1 1291 Real Defects DRONE ? 1540 Potential Defects Precision > 0.76 Recall > 0.84 F-measure > 0.80 Android APIs
- 55. 55 Case Study 2 Questions Designed to Evaluate Repair Recommendation
- 56. 56 Case Study 2 Result Distribution Questions Designed to Evaluate Repair Recommendation
- 57. Conclusion & Future Work “API documents represent one of the most important references for developers…”
- 58. Conclusion & Future Work Documentation Code “API documents represent one of the most important references for developers…” “Analyzing APIs Documentation and Code to Detect Directive Defects”. ICSE 2017 DRONE
- 59. Conclusion & Future Work Documentation Code With DRONE we analyzed 8 java libraries and The Android APIs detecting around 1291 of API documentation defects, providing repair solutions for them. with high precision, recall, and F- measure “API documents represent one of the most important references for developers…” DRONE