Advanced automated visual testing with Selenium
Download as PPTX, PDF3 likes942 views
The document discusses the importance of automated visual testing in software development, particularly in relation to various web browsers, devices, and screen resolutions. It outlines the methods, tools, and technologies involved in conducting these tests, including the use of image comparison engines and the role of different UI testing tools. The presentation emphasizes the necessity for fast feedback and high-quality assurance as release cycles continue to shorten in the industry.
Advanced automated visual testing with Selenium
- 1. Adam Carmi (@carmiadam) Co-Founder and VP R&D at Applitools ADVANCED AUTOMATED VISUAL TESTING WITH SELENIUM
- 2. YOU CAN AND SHOULD AUTOMATE YOUR VISUAL TESTS!
- 4. AGENDA Why automated visual testing? Tools & Technology Where does it fit? Q & A
- 5. WHAT IS VISUAL TESTING? A quality assurance activity aimed to verify that a Graphical User Interface appears correctly to users
- 10. WHY AUTOMATE? The test matrix is too big to cover manually Web browsers Devices Operating systems Screen resolutions Responsive design L10n 3rd Party upgrades
- 11. WHY AUTOMATE? Mobile apps quality is critical Harder to roll back changes Can’t push daily Updates take battery and data Higher quality bar
- 12. WHY AUTOMATE? Release cycles keep getting shorter
- 13. AGENDA Why automated visual testing? Tools & Technology Where does it fit? Q & A
- 14. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 15. DEMOWebDriverCSS
- 16. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 24. AND MORE… 1 pixel offsets in element positioning Dynamic content Moving elements Images of different size Performance
- 27. ImageMagick A powerful command line tool for image processing. Fuzzing is used to eliminate slight color differences An error ratio is usually used to determine a match $ compare –metric AE –fuzz 5% img1.png img2.png diff.png 2246
- 28. THE JAVASCRIPT ENGINES Resemble.js Pixel by pixel + error ratio + anti-aliasing http://huddle.github.io/Resemble.js Blink-Diff Pixel by pixel + error ratio + anti-aliasing + perceptual color distance computation https://github.com/yahoo/blink-diff Looks-Same Pixel by pixel + perceptual color distance computation + ignore Caret https://github.com/gemini-testing/looks-same
- 29. APPLITOOLS EYES Handles anti-aliasing, pixel offsets, color similarity, and image scaling Dynamic and moving content Compare images of different sizes No error ratio configuration required Validates full UI pages Fast! Supports layout / structure matching
- 31. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 32. QUICK FEEDBACK TOOLS MOTIVATION Get fast feedback on code changes Run tests locally in the background SETUP Render screenshots with a headless browser PhatomJS, SlimerJS Configuration file driven tests CONS Partial coverage Chrome, IE? Old browser layout engine versions Limited navigation
- 33. VISUAL COVERAGE TOOLS MOTIVATION Verify that the app / site looks right in all real execution environments Test as many UI states as possible Execute many tests in parallel SETUP Render screenshots using real browsers on a variety of operating systems and devices in parallel WebDriver, Grid Code driven tests WebDriver, DSLCONS Requires test infrastructure Implement and maintain test code
- 34. SELENIUM BASED TOOLS CODE / SCRIPT ● Needle (Python WD) ● WebDriverCss (JS WebDriverIO) ● Gemini (JS DSL) ● Selenium Visual Diff (Java WD) ● VisualCeption (PHP CodeCeption) ● Pix-Diff (JS Protractor) ● Shoov (JS WebDriverIO) ● Vizregress (.NET WD) ● Rspec Page Regression (Capibara) ● Applitools Eyes (All WD + Appium) ● Fighting Layout Bugs (Java WebDriver) ● Huxley (Record Playback) CONFIGURATION ● Viff ● CSS Visual Test ● GreenOnion ● Wraith-Selenium
- 35. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 36. REPORT DIFFERENCES As files on the file system WebDriverCSS
- 37. REPORT DIFFERENCES As a gallery Selenium Visual Diff
- 38. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 42. AGENDA Why automated visual testing? Tools & Technology Where does it fit? Q & A
- 43. WHERE DOES IT FIT? • Component s • Code review • Developers • Designers • QA Visual testing of frontend components by frontend developers
- 44. WHERE DOES IT FIT? • Pages • Page sections • Developers • Designers • QA • Others Full or partial validation of application screens A collaboration tool
- 45. WHERE DOES IT FIT? • Staging vs. Production • Ops • QA Validate your staging deployment using your production deployment as a baseline
- 46. WHERE DOES IT FIT? • Monitoring • Ops • QA No missing resources in production No corruption due to 3rd party data No corruption due to browser / OS upgrades
- 47. Adam Carmi (@carmiadam) Co-Founder and VP R&D at Applitools QUESTIONS?