Continuous Quality Assurance using Selenium WebDriver
10 likes9,132 views
- Menta is a Selenium testing framework for PHP that provides components, page objects, and other features to help with test automation using Selenium WebDriver. - It includes components for common tasks like assertions, waiting, taking screenshots. Page objects represent services offered by pages and allow interacting with pages. - Menta manages Selenium sessions, provides configuration management, and supports features like events/observers, Sauce Labs integration, and perceptual diffs for test results. It aims to help with continuous testing.
![Menta for PHPUnit
Text Result PrinterPHPUnit 3.7.20 by Sebastian Bergmann.
Configuration read from /var/www/Tests/Acceptance/conf/devfb.ff.vmhost.xml
> SUITE:
> SUITE:
> SUITE: Acceptance_Tests_Account_Register
> TEST: Acceptance_Tests_Account_Register::register
SUCCESS. (Duration: 8.18 sec)
> TEST: Acceptance_Tests_Account_Register::login
SUCCESS. (Duration: 6.5 sec)
> TEST: Acceptance_Tests_Account_Register::testRegistrationMail
FAILURE: Searching for mail with subject 'Welcome, Test_705b077a962977e1a412696ea42249ae User_705b077a962977e1a412696ea42249ae' timed out (Duration: 22
< Duration: 36 sec
< Duration: 36 sec
> SUITE:
> SUITE: Acceptance_Tests_Checkout_CheckoutMultiAddressTest
> TEST: Acceptance_Tests_Checkout_CheckoutMultiAddressTest::putProductsInCart
FAILURE: Element "//table[@id="shopping-cart-table"]//h3[contains(concat(' ', @class, ' '), ' attentionText ')]" does not contain text "Bulb Vase" Fail
< Duration: 9.87 sec
> SUITE: Acceptance_Tests_Checkout_CheckoutTest
> TEST: Acceptance_Tests_Checkout_CheckoutTest::categoryView
SUCCESS. (Duration: 1.79 sec)
> TEST: Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart
FAILURE: Failed asserting that two strings are equal. (Duration: 3.55 sec)
SKIPPED: This test depends on "Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart" to pass. (Duration: 5.34 sec)](https://image.slidesharecdn.com/2013-05seleniummeetupexport-pdf-130517105954-phpapp02/85/Continuous-Quality-Assurance-using-Selenium-WebDriver-54-320.jpg)
Continuous Quality Assurance using Selenium WebDriver
- 1. Quality Assurance Continuous …using Selenium WebDriver San Francisco Selenium Meetup Group - May 16, 2013
- 3. San Francisco, CA Janine Fiona that’s me It’s a boy!
- 4. E-Commerce: Magento CMS: TYPO3 Portals: ZF, FLOW,… Mobile Searchperience: SOLR 120 people in 7 offices world-wide High Performance /Scale Global Enterprise Projects
- 6. Quality Assurance Continuous Everything sounds better with “continuous”
- 7. “Continuous” ==“Doingstuffoverandoveragain” Builds Deployment Tests Infrastructure Setup Devbox Setup == “Automation”
- 8. timespent task size does it manually does it manually gets annoyed writes script to automate runs script wins loses Geeks vs. Non-Geeks Makes fun of geek’s complicated method
- 9. — Ovidiu Platon We are not shipping your machine! if it works on your machine! I don’t care
- 10. Picture The BIG
- 11. Vagrant Boxes
- 12. Awesome stuff Every single line potentially can break production! Bratwurst (Germans) Donut (Americans) Pizza (Italians)
- 13. Version Control GIT, SVN,… PHP lint
- 14. Version Control Code Reviews “Gerrit”
- 15. Code Reviews Build Files DB Installer Settings Triggered by commit, by time or manually Jenkins minify js/css .tar.gz “virtual package” (revision)
- 17. Version Control Code Reviews Build Files DB Installer Settings
- 19. Static Code Analysis Install on “Integration” First system where code from all developers/teams meets each other unstable system. Might be broken. Will be rebuild without warning
- 21. Unit Tests Integration Tests AOE PMD (for Magento) broken links, HTTP headers, W3C validity,… Test features that rely on other parts (e.g. framework, external services,…) External components (SSL, Redis, Varnish,…) Basically this is also PHPUnit, but addressing other stuff
- 22. Integration Tests Acceptance Tests Here we go: Selenium!
- 23. Acceptance Tests Install on “Staging” Behat (Cucumber) Same setup as production, but different machines Client QA / approval
- 24. Install on “Staging” *http://www.slideshare.net/aoemedia/performance-measurement-and-tuning Stress Tests Jmeter Spin up EC2 instances in the cloud to produce traffic*
- 25. Again: multiple servers or autoscaling setup Stress Tests Install on “Production” A/B Deployments on different scales Plan B: rollback Zero Downtime
- 26. Varnish ArrayEC2 Frontend ArrayBackend ArrayEC2 EC2 EC2 Worker ArrayEC2EC2 Deployment x ELB Load Balancer Route 53 DNS-Service shop.angrybirds.com Deployment
- 27. Varnish Array Frontend Array Worker ArrayEC2 Deployment x ELB Load Balancer VaEC2 FroBackend ArrayEC2 EC2 EC2 EC2 Deployment x+1 Release Y ELB Load Balancer Route 53 DNS-Service shop.angrybirds.com Deployment
- 31. Version Control Code Reviews Build Static Code Analysis Install on “Integration” Unit Tests Integration Tests Acceptance Tests Install on “Staging” Install on “Production” Enable Features
- 32. Pipeline Visualization #284 #283 #282 #281 #280 Build Static Code Analysis Unit Tests Install on “Integration” Integration Tests Acceptance Tests #285 for team dashboards
- 33. Acceptance Tests
- 34. test automation is a developer’s task we need to provide them some good tools… us
- 37. What’s out there? Selenium for PHP PHPUnit_Selenium (Sebastian Bergmann) WebDriver-PHP (Chibimagic) Php-webdriver-bindings (Lukasz Kolczynski) php-webdriver PHPUnit_Selenium (Giorgio Sironi)
- 38. php-webdriver Selenium 2 08/2011 Facebook/php-webdriver (Justin Bishop) Element-34/php-webdriver (Adam Goucher) instaclick/php-webdriver (Anthon Pang)
- 39. php-webdriver “A very thin wrapper of WebDriver”
- 40. So what? A thin layer is good! But not enough… no test logic no higher level components dealing with Selenium specific tasks not compatible to legacy test cases No abstraction / structure
- 41. Architecture HTTP AUT No direct access (DB, classes,…) REST* (over HTTP) *http://code.google.com/p/selenium/wiki/JsonWireProtocol
- 43. Project test cases PHPUnit Project Components Components Library Menta php-webdriver reusable project specific
- 44. Menta Selenium 2 Testing Framework for PHP Who is “Menta”? What is “Menta”?
- 45. Why “Menta”?
- 47. Features MentaThe framework. Not the cat… Components Session Management ?Bonus Features :)Event / ObserverConfiguration Management PHPUnit integration
- 48. Why? ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Data Providers ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Dependent Tests ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Test Infrastructure
- 49. Menta for PHPUnit Abstract test class PHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2ProjectY_Abstract_TestCase Very thin layer on top of PHPUnit ProjectY_TestCase_Registration ProjectY_TestCase_Checkout the “magic” happens in the components
- 50. Legacy Selenium 1 tests PHPUnit_Framework_TestCasePHPUnit_Extensions_SeleniumTestCase ProjectY_TestCase_Registration ProjectY_TestCase_Checkout
- 51. Legacy Selenium 1 tests PHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2 ProjectY_TestCase_Registration ProjectY_TestCase_Checkout Menta_PHPUnit_Testcase_Selenium1 Selenium1Facade Compatible with PHPUnit_Extensions_SeleniumTestCase
- 52. HTML ReportScreenshots (on demand and on failure) test description from phpdoc additional messages error details
- 54. Menta for PHPUnit Text Result PrinterPHPUnit 3.7.20 by Sebastian Bergmann. Configuration read from /var/www/Tests/Acceptance/conf/devfb.ff.vmhost.xml > SUITE: > SUITE: > SUITE: Acceptance_Tests_Account_Register > TEST: Acceptance_Tests_Account_Register::register SUCCESS. (Duration: 8.18 sec) > TEST: Acceptance_Tests_Account_Register::login SUCCESS. (Duration: 6.5 sec) > TEST: Acceptance_Tests_Account_Register::testRegistrationMail FAILURE: Searching for mail with subject 'Welcome, Test_705b077a962977e1a412696ea42249ae User_705b077a962977e1a412696ea42249ae' timed out (Duration: 22 < Duration: 36 sec < Duration: 36 sec > SUITE: > SUITE: Acceptance_Tests_Checkout_CheckoutMultiAddressTest > TEST: Acceptance_Tests_Checkout_CheckoutMultiAddressTest::putProductsInCart FAILURE: Element "//table[@id="shopping-cart-table"]//h3[contains(concat(' ', @class, ' '), ' attentionText ')]" does not contain text "Bulb Vase" Fail < Duration: 9.87 sec > SUITE: Acceptance_Tests_Checkout_CheckoutTest > TEST: Acceptance_Tests_Checkout_CheckoutTest::categoryView SUCCESS. (Duration: 1.79 sec) > TEST: Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart FAILURE: Failed asserting that two strings are equal. (Duration: 3.55 sec) SKIPPED: This test depends on "Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart" to pass. (Duration: 5.34 sec)
- 59. Components Common Assert Wait Screenshot Selenium1Facade Imap WebService W3C Validator Curl … Some might not even require a Selenium session
- 61. Components! PageObjects? PageObjectsComponents { }}⊇{ PageObjects are components
- 62. PageObjects? represents services offered by a page or feature is the only thing with a deep knowledge of the HTML clickCheckoutButton() isInCart($product) getTotals() ✔ ✔ ✔
- 64. Session Management Let Menta manage your sessions auto-close auto-start incl. on exceptions and ctrl+c Reusing sessions Cleaning sessions
- 65. Configuration ManagementFallback mechanism for global settings Create multiple configurations for different browsers, application instances or Selenium Servers Compatible to PHPUnit (xml)
- 66. Event / Observer Menta_Events::dispatchEvent('after_session_create'); Menta_Events::addObserver('after_session_create', function($session) { $session->window('main')->postSize(array('width' => 1024, 'height' => 768)); }
- 67. Bonus Features
- 68. Sauce Labs Support *configuration: http://username:[email protected]:80/wd/hub works fine* update test result (REST)
- 69. Sauce Labs Support Speaking name Test results
- 70. Perceptual diffs Build X Build X-1 Perceptual Diff
- 71. github.com / AOEmedia / Menta github.com / AOEmedia / Menta_SampleProject It’s composer based and you’ can get started in a few minutes!
Editor's Notes
- #59: track of singletons, allows rewrites.