SlideShare a Scribd company logo
Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
ADVANCED AUTOMATED
VISUAL TESTING WITH
SELENIUM
YOU CAN AND SHOULD
AUTOMATE YOUR
VISUAL TESTS!
Advanced automated visual testing with Selenium
AGENDA
 Why automated visual testing?
 Tools & Technology
 Where does it fit?
 Q & A
WHAT IS VISUAL TESTING?
A quality assurance activity aimed to verify that a Graphical
User Interface appears correctly to users
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
WHY AUTOMATE?
The test matrix is too big to cover manually
 Web browsers
 Devices
 Operating systems
 Screen resolutions
 Responsive design
 L10n
 3rd Party upgrades
WHY AUTOMATE?
Mobile apps quality is critical
 Harder to roll back changes
 Can’t push daily
 Updates take battery and data
 Higher quality bar
WHY AUTOMATE?
Release cycles keep getting shorter
AGENDA
 Why automated visual testing?
 Tools & Technology
 Where does it fit?
 Q & A
HOW DOES IT WORK?
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
DEMOWebDriverCSS
HOW DOES IT WORK?
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
Advanced automated visual testing with Selenium
AND MORE…
 1 pixel offsets in element positioning
 Dynamic content
 Moving elements
 Images of different size
 Performance
Advanced automated visual testing with Selenium
IMAGE COMPARISON
ENGINES
ImageMagick
 A powerful command line tool for image processing.
 Fuzzing is used to eliminate slight color differences
 An error ratio is usually used to determine a match
$ compare –metric AE –fuzz 5% img1.png img2.png diff.png
2246
THE JAVASCRIPT ENGINES
 Resemble.js
Pixel by pixel + error ratio + anti-aliasing
http://huddle.github.io/Resemble.js
 Blink-Diff
Pixel by pixel + error ratio + anti-aliasing + perceptual color distance
computation
https://github.com/yahoo/blink-diff
 Looks-Same
Pixel by pixel + perceptual color distance computation + ignore Caret
https://github.com/gemini-testing/looks-same
APPLITOOLS EYES
 Handles anti-aliasing, pixel offsets, color similarity, and image scaling
 Dynamic and moving content
 Compare images of different sizes
 No error ratio configuration required
 Validates full UI pages
 Fast!
 Supports layout / structure matching
DEMOVISUAL LAYOUT MATCHING
HOW DOES IT WORK?
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
QUICK FEEDBACK TOOLS
MOTIVATION
 Get fast feedback on code changes
 Run tests locally in the background
SETUP
 Render screenshots with a headless
browser
 PhatomJS, SlimerJS
 Configuration file driven tests
CONS
 Partial coverage
 Chrome, IE?
 Old browser layout engine versions
 Limited navigation
VISUAL COVERAGE TOOLS
MOTIVATION
 Verify that the app / site looks right in
all real execution environments
 Test as many UI states as possible
 Execute many tests in parallel
SETUP
 Render screenshots using real browsers
on a variety of operating systems and
devices in parallel
 WebDriver, Grid
 Code driven tests
 WebDriver, DSLCONS
 Requires test infrastructure
 Implement and maintain test code
SELENIUM BASED TOOLS
CODE / SCRIPT
● Needle (Python WD)
● WebDriverCss (JS WebDriverIO)
● Gemini (JS DSL)
● Selenium Visual Diff (Java WD)
● VisualCeption (PHP CodeCeption)
● Pix-Diff (JS Protractor)
● Shoov (JS WebDriverIO)
● Vizregress (.NET WD)
● Rspec Page Regression (Capibara)
● Applitools Eyes (All WD + Appium)
● Fighting Layout Bugs (Java WebDriver)
● Huxley (Record Playback)
CONFIGURATION
● Viff
● CSS Visual Test
● GreenOnion
● Wraith-Selenium
HOW DOES IT WORK?
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
REPORT DIFFERENCES
As files on the file system
WebDriverCSS
REPORT DIFFERENCES
As a gallery
Selenium Visual Diff
HOW DOES IT WORK?
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
UPDATE THE BASELINE
Gemini
Advanced automated visual testing with Selenium
DEMOAUTOMATED MAINTENANCE
AGENDA
 Why automated visual testing?
 Tools & Technology
 Where does it fit?
 Q & A
WHERE DOES IT FIT?
• Component
s
• Code
review
• Developers
• Designers
• QA
Visual testing of frontend components by
frontend developers
WHERE DOES IT FIT?
• Pages
• Page
sections
• Developers
• Designers
• QA
• Others
Full or partial validation of application screens
A collaboration tool
WHERE DOES IT FIT?
• Staging vs.
Production
• Ops
• QA
Validate your staging deployment using your
production deployment as a baseline
WHERE DOES IT FIT?
• Monitoring
• Ops
• QA
No missing resources in production
No corruption due to 3rd party data
No corruption due to browser / OS upgrades
Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
QUESTIONS?
Ad

Recommended

Advanced Automated Visual Testing
Advanced Automated Visual Testing
adamcarmi
 
Sauce Labs+Applitools - Automated Visual Testing in the Cloud
Sauce Labs+Applitools - Automated Visual Testing in the Cloud
Sauce Labs
 
Automated Visual Testing That Does Not Suck
Automated Visual Testing That Does Not Suck
adamcarmi
 
Selenium conference, 2016
Selenium conference, 2016
Pooja Shah
 
Advanced Appium
Advanced Appium
Dan Cuellar
 
Selenium Based Visual Test Automation
Selenium Based Visual Test Automation
adamcarmi
 
Visual Automation Framework via Screenshot Comparison
Visual Automation Framework via Screenshot Comparison
Mek Srunyu Stittri
 
Selenium Camp 2016
Selenium Camp 2016
Dan Cuellar
 
Appium
Appium
Deepshikha Singh
 
What's New With Appium? From 1.0 to Now
What's New With Appium? From 1.0 to Now
Sauce Labs
 
Nakal think test_2015
Nakal think test_2015
Rajdeep Varma
 
Appium Mobile Test Automation like WebDriver
Appium Mobile Test Automation like WebDriver
Andrii Dzynia
 
Android UI Testing with Appium
Android UI Testing with Appium
Luke Maung
 
Advanced Appium: SeleniumConf UK 2016
Advanced Appium: SeleniumConf UK 2016
Dan Cuellar
 
Selenium, Appium, and Robots!
Selenium, Appium, and Robots!
hugs
 
Introduction To Appium With Robotframework
Introduction To Appium With Robotframework
Syam Sasi
 
SeConf_Nov2016_London
SeConf_Nov2016_London
Pooja Shah
 
Selenium Testing on Chrome - Google DevFest Armenia 2015
Selenium Testing on Chrome - Google DevFest Armenia 2015
Sargis Sargsyan
 
Wheat - Mobile functional test automation
Wheat - Mobile functional test automation
Sunny Tambi
 
Appium overview
Appium overview
Abhishek Yadav
 
Appium
Appium
Keshav Kashyap
 
Improving Android app testing with Appium and Sauce Labs
Improving Android app testing with Appium and Sauce Labs
Isaac Murchie
 
Appium solution
Appium solution
Nael Abd Eljawad
 
The wild wild west of Selenium Capabilities
The wild wild west of Selenium Capabilities
Adi Ofri
 
Advanced Appium Tips & Tricks with Jonathan Lipps
Advanced Appium Tips & Tricks with Jonathan Lipps
Perfecto by Perforce
 
Testing on Mobile Devices with Location Services
Testing on Mobile Devices with Location Services
Sauce Labs
 
Testing Native iOS Apps with Appium
Testing Native iOS Apps with Appium
Sauce Labs
 
Cross Platform Appium Tests: How To
Cross Platform Appium Tests: How To
GlobalLogic Ukraine
 
Brace yourself from automation death trap
Brace yourself from automation death trap
Prasad Kalgutkar
 
Testing desktop apps with selenium
Testing desktop apps with selenium
Filip Braun
 

More Related Content

What's hot (20)

Appium
Appium
Deepshikha Singh
 
What's New With Appium? From 1.0 to Now
What's New With Appium? From 1.0 to Now
Sauce Labs
 
Nakal think test_2015
Nakal think test_2015
Rajdeep Varma
 
Appium Mobile Test Automation like WebDriver
Appium Mobile Test Automation like WebDriver
Andrii Dzynia
 
Android UI Testing with Appium
Android UI Testing with Appium
Luke Maung
 
Advanced Appium: SeleniumConf UK 2016
Advanced Appium: SeleniumConf UK 2016
Dan Cuellar
 
Selenium, Appium, and Robots!
Selenium, Appium, and Robots!
hugs
 
Introduction To Appium With Robotframework
Introduction To Appium With Robotframework
Syam Sasi
 
SeConf_Nov2016_London
SeConf_Nov2016_London
Pooja Shah
 
Selenium Testing on Chrome - Google DevFest Armenia 2015
Selenium Testing on Chrome - Google DevFest Armenia 2015
Sargis Sargsyan
 
Wheat - Mobile functional test automation
Wheat - Mobile functional test automation
Sunny Tambi
 
Appium overview
Appium overview
Abhishek Yadav
 
Appium
Appium
Keshav Kashyap
 
Improving Android app testing with Appium and Sauce Labs
Improving Android app testing with Appium and Sauce Labs
Isaac Murchie
 
Appium solution
Appium solution
Nael Abd Eljawad
 
The wild wild west of Selenium Capabilities
The wild wild west of Selenium Capabilities
Adi Ofri
 
Advanced Appium Tips & Tricks with Jonathan Lipps
Advanced Appium Tips & Tricks with Jonathan Lipps
Perfecto by Perforce
 
Testing on Mobile Devices with Location Services
Testing on Mobile Devices with Location Services
Sauce Labs
 
Testing Native iOS Apps with Appium
Testing Native iOS Apps with Appium
Sauce Labs
 
Cross Platform Appium Tests: How To
Cross Platform Appium Tests: How To
GlobalLogic Ukraine
 
What's New With Appium? From 1.0 to Now
What's New With Appium? From 1.0 to Now
Sauce Labs
 
Nakal think test_2015
Nakal think test_2015
Rajdeep Varma
 
Appium Mobile Test Automation like WebDriver
Appium Mobile Test Automation like WebDriver
Andrii Dzynia
 
Android UI Testing with Appium
Android UI Testing with Appium
Luke Maung
 
Advanced Appium: SeleniumConf UK 2016
Advanced Appium: SeleniumConf UK 2016
Dan Cuellar
 
Selenium, Appium, and Robots!
Selenium, Appium, and Robots!
hugs
 
Introduction To Appium With Robotframework
Introduction To Appium With Robotframework
Syam Sasi
 
SeConf_Nov2016_London
SeConf_Nov2016_London
Pooja Shah
 
Selenium Testing on Chrome - Google DevFest Armenia 2015
Selenium Testing on Chrome - Google DevFest Armenia 2015
Sargis Sargsyan
 
Wheat - Mobile functional test automation
Wheat - Mobile functional test automation
Sunny Tambi
 
Improving Android app testing with Appium and Sauce Labs
Improving Android app testing with Appium and Sauce Labs
Isaac Murchie
 
The wild wild west of Selenium Capabilities
The wild wild west of Selenium Capabilities
Adi Ofri
 
Advanced Appium Tips & Tricks with Jonathan Lipps
Advanced Appium Tips & Tricks with Jonathan Lipps
Perfecto by Perforce
 
Testing on Mobile Devices with Location Services
Testing on Mobile Devices with Location Services
Sauce Labs
 
Testing Native iOS Apps with Appium
Testing Native iOS Apps with Appium
Sauce Labs
 
Cross Platform Appium Tests: How To
Cross Platform Appium Tests: How To
GlobalLogic Ukraine
 

Viewers also liked (20)

Brace yourself from automation death trap
Brace yourself from automation death trap
Prasad Kalgutkar
 
Testing desktop apps with selenium
Testing desktop apps with selenium
Filip Braun
 
Selenium for Jobseekers
Selenium for Jobseekers
Seshu Madhav Chaturvedula
 
Better Page Object Handling with Loadable Component Pattern
Better Page Object Handling with Loadable Component Pattern
Sargis Sargsyan
 
What you can do with WordPress Heartbeat API
What you can do with WordPress Heartbeat API
Tabitha Chapman
 
Page Objects Done Right - selenium conference 2014
Page Objects Done Right - selenium conference 2014
Oren Rubin
 
Automated Unit Testing
Automated Unit Testing
Mike Lively
 
Automated testing 101
Automated testing 101
Tabitha Chapman
 
Automated Testing & Auto Scaling your Apps with Microsoft & Open Source Techn...
Automated Testing & Auto Scaling your Apps with Microsoft & Open Source Techn...
Pranav Ainavolu
 
Test Data - Food for your Test Automation Framework
Test Data - Food for your Test Automation Framework
Anand Bagmar
 
Testing as a container
Testing as a container
Irfan Ahmad
 
How to level-up your Selenium tests with Visual Testing #SeleniumCamp
How to level-up your Selenium tests with Visual Testing #SeleniumCamp
moshemilman
 
Automated Testing Environment by Bugzilla, Testopia and Jenkins
Automated Testing Environment by Bugzilla, Testopia and Jenkins
walkerchang
 
SeConf2015: Advanced Automated Visual Testing With Selenium
SeConf2015: Advanced Automated Visual Testing With Selenium
adamcarmi
 
Continuous Automated Testing - Cast conference workshop august 2014
Continuous Automated Testing - Cast conference workshop august 2014
Noah Sussman
 
Automated Testing with Agile
Automated Testing with Agile
Ken McCorkell
 
Automated Testing for Embedded Software in C or C++
Automated Testing for Embedded Software in C or C++
Lars Thorup
 
Selenium RC: Automated Testing of Modern Web Applications
Selenium RC: Automated Testing of Modern Web Applications
qooxdoo
 
Automated Regression Testing for Embedded Systems in Action
Automated Regression Testing for Embedded Systems in Action
AANDTech
 
Agile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated Testing
Dimitri Ponomareff
 
Brace yourself from automation death trap
Brace yourself from automation death trap
Prasad Kalgutkar
 
Testing desktop apps with selenium
Testing desktop apps with selenium
Filip Braun
 
Better Page Object Handling with Loadable Component Pattern
Better Page Object Handling with Loadable Component Pattern
Sargis Sargsyan
 
What you can do with WordPress Heartbeat API
What you can do with WordPress Heartbeat API
Tabitha Chapman
 
Page Objects Done Right - selenium conference 2014
Page Objects Done Right - selenium conference 2014
Oren Rubin
 
Automated Unit Testing
Automated Unit Testing
Mike Lively
 
Automated Testing & Auto Scaling your Apps with Microsoft & Open Source Techn...
Automated Testing & Auto Scaling your Apps with Microsoft & Open Source Techn...
Pranav Ainavolu
 
Test Data - Food for your Test Automation Framework
Test Data - Food for your Test Automation Framework
Anand Bagmar
 
Testing as a container
Testing as a container
Irfan Ahmad
 
How to level-up your Selenium tests with Visual Testing #SeleniumCamp
How to level-up your Selenium tests with Visual Testing #SeleniumCamp
moshemilman
 
Automated Testing Environment by Bugzilla, Testopia and Jenkins
Automated Testing Environment by Bugzilla, Testopia and Jenkins
walkerchang
 
SeConf2015: Advanced Automated Visual Testing With Selenium
SeConf2015: Advanced Automated Visual Testing With Selenium
adamcarmi
 
Continuous Automated Testing - Cast conference workshop august 2014
Continuous Automated Testing - Cast conference workshop august 2014
Noah Sussman
 
Automated Testing with Agile
Automated Testing with Agile
Ken McCorkell
 
Automated Testing for Embedded Software in C or C++
Automated Testing for Embedded Software in C or C++
Lars Thorup
 
Selenium RC: Automated Testing of Modern Web Applications
Selenium RC: Automated Testing of Modern Web Applications
qooxdoo
 
Automated Regression Testing for Embedded Systems in Action
Automated Regression Testing for Embedded Systems in Action
AANDTech
 
Agile Testing Framework - The Art of Automated Testing
Agile Testing Framework - The Art of Automated Testing
Dimitri Ponomareff
 
Ad

Similar to Advanced automated visual testing with Selenium (20)

Advanced Visual Test Automation with Selenium
Advanced Visual Test Automation with Selenium
adamcarmi
 
Automated Visual Testing That Doesn't Suck!
Automated Visual Testing That Doesn't Suck!
Applitools
 
Selenium-based Visual Test Automation
Selenium-based Visual Test Automation
Applitools
 
Adam carmi
Adam carmi
CodeFest
 
Introduction to automated visual testing
Introduction to automated visual testing
adamcarmi
 
Colorful world-of-visual-automation-testing-latest
Colorful world-of-visual-automation-testing-latest
Onur Baskirt
 
Advanced Visual Test Automation With Selenium
Advanced Visual Test Automation With Selenium
adamcarmi
 
Advanced Automated Visual Testing for Mobile Apps
Advanced Automated Visual Testing for Mobile Apps
adamcarmi
 
Visual Testing: Quality and Speed -- by GIl Tayar, Applitools
Visual Testing: Quality and Speed -- by GIl Tayar, Applitools
Applitools
 
Performance Measurement and Monitoring for Salesforce Web & Mobile Apps
Performance Measurement and Monitoring for Salesforce Web & Mobile Apps
Salesforce Developers
 
Visual Regression Testing
Visual Regression Testing
VodqaBLR
 
Building Your Own Native App Visual Testing Module with Appium
Building Your Own Native App Visual Testing Module with Appium
Wim Selles
 
Effective Data Visualization - Strata (Feb 2012)
Effective Data Visualization - Strata (Feb 2012)
Hjalmar Gislason
 
Building Your Own Native App Visual Testing Module with Appium by Wim Selles
Building Your Own Native App Visual Testing Module with Appium by Wim Selles
Sauce Labs
 
Building Your Own Native App Visual Testing Module with Appium
Building Your Own Native App Visual Testing Module with Appium
Wim Selles
 
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Applitools
 
Data Visualization: Where (normal) people fall in love with data
Data Visualization: Where (normal) people fall in love with data
Hjalmar Gislason
 
“Building Mobile Optimized Websites,” Nick Bourgeois / Ray Villares
“Building Mobile Optimized Websites,” Nick Bourgeois / Ray Villares
rayvillares
 
Building Mobile Optimized Websites
Building Mobile Optimized Websites
haxorize
 
Web Test Automation Framework - IndicThreads Conference
Web Test Automation Framework - IndicThreads Conference
IndicThreads
 
Advanced Visual Test Automation with Selenium
Advanced Visual Test Automation with Selenium
adamcarmi
 
Automated Visual Testing That Doesn't Suck!
Automated Visual Testing That Doesn't Suck!
Applitools
 
Selenium-based Visual Test Automation
Selenium-based Visual Test Automation
Applitools
 
Adam carmi
Adam carmi
CodeFest
 
Introduction to automated visual testing
Introduction to automated visual testing
adamcarmi
 
Colorful world-of-visual-automation-testing-latest
Colorful world-of-visual-automation-testing-latest
Onur Baskirt
 
Advanced Visual Test Automation With Selenium
Advanced Visual Test Automation With Selenium
adamcarmi
 
Advanced Automated Visual Testing for Mobile Apps
Advanced Automated Visual Testing for Mobile Apps
adamcarmi
 
Visual Testing: Quality and Speed -- by GIl Tayar, Applitools
Visual Testing: Quality and Speed -- by GIl Tayar, Applitools
Applitools
 
Performance Measurement and Monitoring for Salesforce Web & Mobile Apps
Performance Measurement and Monitoring for Salesforce Web & Mobile Apps
Salesforce Developers
 
Visual Regression Testing
Visual Regression Testing
VodqaBLR
 
Building Your Own Native App Visual Testing Module with Appium
Building Your Own Native App Visual Testing Module with Appium
Wim Selles
 
Effective Data Visualization - Strata (Feb 2012)
Effective Data Visualization - Strata (Feb 2012)
Hjalmar Gislason
 
Building Your Own Native App Visual Testing Module with Appium by Wim Selles
Building Your Own Native App Visual Testing Module with Appium by Wim Selles
Sauce Labs
 
Building Your Own Native App Visual Testing Module with Appium
Building Your Own Native App Visual Testing Module with Appium
Wim Selles
 
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Applitools
 
Data Visualization: Where (normal) people fall in love with data
Data Visualization: Where (normal) people fall in love with data
Hjalmar Gislason
 
“Building Mobile Optimized Websites,” Nick Bourgeois / Ray Villares
“Building Mobile Optimized Websites,” Nick Bourgeois / Ray Villares
rayvillares
 
Building Mobile Optimized Websites
Building Mobile Optimized Websites
haxorize
 
Web Test Automation Framework - IndicThreads Conference
Web Test Automation Framework - IndicThreads Conference
IndicThreads
 
Ad

More from adamcarmi (6)

Building Universal Servers (On-prem meets Azure PAAS)
Building Universal Servers (On-prem meets Azure PAAS)
adamcarmi
 
Build mobile application that is easy to automate
Build mobile application that is easy to automate
adamcarmi
 
Advanced Cross-Browser Visual Testing with Applitools Eyes and HP LeanFT
Advanced Cross-Browser Visual Testing with Applitools Eyes and HP LeanFT
adamcarmi
 
Selenium Automation at Incapsula
Selenium Automation at Incapsula
adamcarmi
 
Jenkins & Selenium
Jenkins & Selenium
adamcarmi
 
An Overview of Selenium
An Overview of Selenium
adamcarmi
 
Building Universal Servers (On-prem meets Azure PAAS)
Building Universal Servers (On-prem meets Azure PAAS)
adamcarmi
 
Build mobile application that is easy to automate
Build mobile application that is easy to automate
adamcarmi
 
Advanced Cross-Browser Visual Testing with Applitools Eyes and HP LeanFT
Advanced Cross-Browser Visual Testing with Applitools Eyes and HP LeanFT
adamcarmi
 
Selenium Automation at Incapsula
Selenium Automation at Incapsula
adamcarmi
 
Jenkins & Selenium
Jenkins & Selenium
adamcarmi
 
An Overview of Selenium
An Overview of Selenium
adamcarmi
 

Recently uploaded (20)

Looking for a BIRT Report Alternative Here’s Why Helical Insight Stands Out.pdf
Looking for a BIRT Report Alternative Here’s Why Helical Insight Stands Out.pdf
Varsha Nayak
 
Migrating to Azure Cosmos DB the Right Way
Migrating to Azure Cosmos DB the Right Way
Alexander (Alex) Komyagin
 
Software Testing & it’s types (DevOps)
Software Testing & it’s types (DevOps)
S Pranav (Deepu)
 
OpenTelemetry 101 Cloud Native Barcelona
OpenTelemetry 101 Cloud Native Barcelona
Imma Valls Bernaus
 
What is data visualization and how data visualization tool can help.pdf
What is data visualization and how data visualization tool can help.pdf
Varsha Nayak
 
GDG Douglas - Google AI Agents: Your Next Intern?
GDG Douglas - Google AI Agents: Your Next Intern?
felipeceotto
 
Wondershare PDFelement Pro 11.4.20.3548 Crack Free Download
Wondershare PDFelement Pro 11.4.20.3548 Crack Free Download
Puppy jhon
 
Download Adobe Illustrator Crack free for Windows 2025?
Download Adobe Illustrator Crack free for Windows 2025?
grete1122g
 
Rierino Commerce Platform - CMS Solution
Rierino Commerce Platform - CMS Solution
Rierino
 
Decipher SEO Solutions for your startup needs.
Decipher SEO Solutions for your startup needs.
mathai2
 
Step by step guide to install Flutter and Dart
Step by step guide to install Flutter and Dart
S Pranav (Deepu)
 
Meet You in the Middle: 1000x Performance for Parquet Queries on PB-Scale Dat...
Meet You in the Middle: 1000x Performance for Parquet Queries on PB-Scale Dat...
Alluxio, Inc.
 
Async-ronizing Success at Wix - Patterns for Seamless Microservices - Devoxx ...
Async-ronizing Success at Wix - Patterns for Seamless Microservices - Devoxx ...
Natan Silnitsky
 
Open Source Software Development Methods
Open Source Software Development Methods
VICTOR MAESTRE RAMIREZ
 
MOVIE RECOMMENDATION SYSTEM, UDUMULA GOPI REDDY, Y24MC13085.pptx
MOVIE RECOMMENDATION SYSTEM, UDUMULA GOPI REDDY, Y24MC13085.pptx
Maharshi Mallela
 
Artificial Intelligence Workloads and Data Center Management
Artificial Intelligence Workloads and Data Center Management
SandeepKS52
 
About Certivo | Intelligent Compliance Solutions for Global Regulatory Needs
About Certivo | Intelligent Compliance Solutions for Global Regulatory Needs
certivoai
 
Reimagining Software Development and DevOps with Agentic AI
Reimagining Software Development and DevOps with Agentic AI
Maxim Salnikov
 
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
BradBedford3
 
What is data visualization and how data visualization tool can help.pptx
What is data visualization and how data visualization tool can help.pptx
Varsha Nayak
 
Looking for a BIRT Report Alternative Here’s Why Helical Insight Stands Out.pdf
Looking for a BIRT Report Alternative Here’s Why Helical Insight Stands Out.pdf
Varsha Nayak
 
Software Testing & it’s types (DevOps)
Software Testing & it’s types (DevOps)
S Pranav (Deepu)
 
OpenTelemetry 101 Cloud Native Barcelona
OpenTelemetry 101 Cloud Native Barcelona
Imma Valls Bernaus
 
What is data visualization and how data visualization tool can help.pdf
What is data visualization and how data visualization tool can help.pdf
Varsha Nayak
 
GDG Douglas - Google AI Agents: Your Next Intern?
GDG Douglas - Google AI Agents: Your Next Intern?
felipeceotto
 
Wondershare PDFelement Pro 11.4.20.3548 Crack Free Download
Wondershare PDFelement Pro 11.4.20.3548 Crack Free Download
Puppy jhon
 
Download Adobe Illustrator Crack free for Windows 2025?
Download Adobe Illustrator Crack free for Windows 2025?
grete1122g
 
Rierino Commerce Platform - CMS Solution
Rierino Commerce Platform - CMS Solution
Rierino
 
Decipher SEO Solutions for your startup needs.
Decipher SEO Solutions for your startup needs.
mathai2
 
Step by step guide to install Flutter and Dart
Step by step guide to install Flutter and Dart
S Pranav (Deepu)
 
Meet You in the Middle: 1000x Performance for Parquet Queries on PB-Scale Dat...
Meet You in the Middle: 1000x Performance for Parquet Queries on PB-Scale Dat...
Alluxio, Inc.
 
Async-ronizing Success at Wix - Patterns for Seamless Microservices - Devoxx ...
Async-ronizing Success at Wix - Patterns for Seamless Microservices - Devoxx ...
Natan Silnitsky
 
Open Source Software Development Methods
Open Source Software Development Methods
VICTOR MAESTRE RAMIREZ
 
MOVIE RECOMMENDATION SYSTEM, UDUMULA GOPI REDDY, Y24MC13085.pptx
MOVIE RECOMMENDATION SYSTEM, UDUMULA GOPI REDDY, Y24MC13085.pptx
Maharshi Mallela
 
Artificial Intelligence Workloads and Data Center Management
Artificial Intelligence Workloads and Data Center Management
SandeepKS52
 
About Certivo | Intelligent Compliance Solutions for Global Regulatory Needs
About Certivo | Intelligent Compliance Solutions for Global Regulatory Needs
certivoai
 
Reimagining Software Development and DevOps with Agentic AI
Reimagining Software Development and DevOps with Agentic AI
Maxim Salnikov
 
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
BradBedford3
 
What is data visualization and how data visualization tool can help.pptx
What is data visualization and how data visualization tool can help.pptx
Varsha Nayak
 

Advanced automated visual testing with Selenium

  • 1. Adam Carmi (@carmiadam) Co-Founder and VP R&D at Applitools ADVANCED AUTOMATED VISUAL TESTING WITH SELENIUM
  • 2. YOU CAN AND SHOULD AUTOMATE YOUR VISUAL TESTS!
  • 4. AGENDA  Why automated visual testing?  Tools & Technology  Where does it fit?  Q & A
  • 5. WHAT IS VISUAL TESTING? A quality assurance activity aimed to verify that a Graphical User Interface appears correctly to users
  • 10. WHY AUTOMATE? The test matrix is too big to cover manually  Web browsers  Devices  Operating systems  Screen resolutions  Responsive design  L10n  3rd Party upgrades
  • 11. WHY AUTOMATE? Mobile apps quality is critical  Harder to roll back changes  Can’t push daily  Updates take battery and data  Higher quality bar
  • 12. WHY AUTOMATE? Release cycles keep getting shorter
  • 13. AGENDA  Why automated visual testing?  Tools & Technology  Where does it fit?  Q & A
  • 14. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  • 16. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  • 24. AND MORE…  1 pixel offsets in element positioning  Dynamic content  Moving elements  Images of different size  Performance
  • 27. ImageMagick  A powerful command line tool for image processing.  Fuzzing is used to eliminate slight color differences  An error ratio is usually used to determine a match $ compare –metric AE –fuzz 5% img1.png img2.png diff.png 2246
  • 28. THE JAVASCRIPT ENGINES  Resemble.js Pixel by pixel + error ratio + anti-aliasing http://huddle.github.io/Resemble.js  Blink-Diff Pixel by pixel + error ratio + anti-aliasing + perceptual color distance computation https://github.com/yahoo/blink-diff  Looks-Same Pixel by pixel + perceptual color distance computation + ignore Caret https://github.com/gemini-testing/looks-same
  • 29. APPLITOOLS EYES  Handles anti-aliasing, pixel offsets, color similarity, and image scaling  Dynamic and moving content  Compare images of different sizes  No error ratio configuration required  Validates full UI pages  Fast!  Supports layout / structure matching
  • 31. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  • 32. QUICK FEEDBACK TOOLS MOTIVATION  Get fast feedback on code changes  Run tests locally in the background SETUP  Render screenshots with a headless browser  PhatomJS, SlimerJS  Configuration file driven tests CONS  Partial coverage  Chrome, IE?  Old browser layout engine versions  Limited navigation
  • 33. VISUAL COVERAGE TOOLS MOTIVATION  Verify that the app / site looks right in all real execution environments  Test as many UI states as possible  Execute many tests in parallel SETUP  Render screenshots using real browsers on a variety of operating systems and devices in parallel  WebDriver, Grid  Code driven tests  WebDriver, DSLCONS  Requires test infrastructure  Implement and maintain test code
  • 34. SELENIUM BASED TOOLS CODE / SCRIPT ● Needle (Python WD) ● WebDriverCss (JS WebDriverIO) ● Gemini (JS DSL) ● Selenium Visual Diff (Java WD) ● VisualCeption (PHP CodeCeption) ● Pix-Diff (JS Protractor) ● Shoov (JS WebDriverIO) ● Vizregress (.NET WD) ● Rspec Page Regression (Capibara) ● Applitools Eyes (All WD + Appium) ● Fighting Layout Bugs (Java WebDriver) ● Huxley (Record Playback) CONFIGURATION ● Viff ● CSS Visual Test ● GreenOnion ● Wraith-Selenium
  • 35. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  • 36. REPORT DIFFERENCES As files on the file system WebDriverCSS
  • 37. REPORT DIFFERENCES As a gallery Selenium Visual Diff
  • 38. HOW DOES IT WORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
  • 42. AGENDA  Why automated visual testing?  Tools & Technology  Where does it fit?  Q & A
  • 43. WHERE DOES IT FIT? • Component s • Code review • Developers • Designers • QA Visual testing of frontend components by frontend developers
  • 44. WHERE DOES IT FIT? • Pages • Page sections • Developers • Designers • QA • Others Full or partial validation of application screens A collaboration tool
  • 45. WHERE DOES IT FIT? • Staging vs. Production • Ops • QA Validate your staging deployment using your production deployment as a baseline
  • 46. WHERE DOES IT FIT? • Monitoring • Ops • QA No missing resources in production No corruption due to 3rd party data No corruption due to browser / OS upgrades
  • 47. Adam Carmi (@carmiadam) Co-Founder and VP R&D at Applitools QUESTIONS?