Agile Testing and Test Automation
Download as PPTX, PDF1 like1,414 views
The document is a comprehensive outline of a workshop on agile testing and test automation, focusing on the role and techniques of agile testers. It covers various agile methodologies, principles, and practices, emphasizing collaboration and continuous improvement in software development and testing. Key topics include test-driven development, behavior-driven development, specifications by example, and the importance of a whole team approach in agile environments.
![BDD – Run Output
1 Scenarios ([33m1 undefined[0m)
3 Steps ([33m3 undefined[0m)
0m0.000s
You can implement missing steps with the snippets below:
@Given("^The price of a "([^"]*)" is $(d+)$")
public void The_price_of_a_is_$(String arg1, int arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
@When("^I checkout (d+) "([^"]*)"$")
public void I_checkout(int arg1, String arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
@Then("^the total price should be $(d+)$")
public void the_total_price_should_be_$(int arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException(); }](https://image.slidesharecdn.com/agiletestingandtestautomation-170103090718/85/Agile-Testing-and-Test-Automation-46-320.jpg)
![BDD – Step File
public class TestSteps {
@Given("^The price of a "([^"]*)" is $(d+)$")
public void The_price_of_a_is_$(String arg1, int arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
@When("^I checkout (d+) "([^"]*)"$")
public void I_checkout(int arg1, String arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
@Then("^the total price should be $(d+)$")
public void the_total_price_should_be_$(int arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
} }](https://image.slidesharecdn.com/agiletestingandtestautomation-170103090718/85/Agile-Testing-and-Test-Automation-47-320.jpg)
![BDD – Modified Step File
public class TestSteps extends TestCase {
String product;
int rate, qty;
@Given("^The price of a "([^"]*)" is $(d+)$")
public void The_price_of_a_is_$(String arg1, int arg2) throws Throwable {
product = arg1;
rate = arg2;
}
@When("^I checkout (d+) "([^"]*)"$")
public void I_checkout(int arg1, String arg2) throws Throwable {
qty=arg1;
}
@Then("^the total price should be $(d+)$")
public void the_total_price_should_be_$(int arg1) throws Throwable {
if(product.equals("Banana"))
assertEquals(arg1, rate*qty); }}](https://image.slidesharecdn.com/agiletestingandtestautomation-170103090718/85/Agile-Testing-and-Test-Automation-48-320.jpg)
Agile Testing and Test Automation
- 1. Naveen Kumar Singh Agile Testing and Test Automation +91 9810547500 [email protected] Twitter - @naveenhome Skype – naveen75home
- 2. Capacity Building Workshop by Me… Scrum Developer Workshop Product Discovery and Requirement Analysis Test Driven Development Behavior Driven Development Agile Testing and Test Automation Agile and Scrum Foundation DevOps Foundation Continuous Integration and Delivery Agile Coaching and Facilitation Agile Technical Practices
- 3. Program Overview • This workshop focuses primarily on agile testing techniques and processes in addition to the mindset and role of an agile tester. • Learning outcomes include the ability to distinguish different types of testing on an agile project. • Understanding how business, development, and testing personnel best collaborate on an agile development cadence.
- 4. Learning Objective • What is Agile and available agile frameworks • What is Agile Testing? • Agile mindset and culture for continuous testing • Agile Testing Quadrants and category • Test Automation Pyramid • Pair Programming, Pair Testing and Peer review • Example Driven Development – Spec by Examples, ATDD, BDD • Feature, story, unit and NFRs testing • Roles and Responsibilities of Test Engineers and Managers • Test Strategies and planning • Agile testing metrics • Successful delivery • Test Environment and Infrastructure • Working on Distributed Team
- 5. Risk in Software Development Schedule slips Project canceled System goes sour Defect rate Business misunderstood Business change False feature rich Staff turnover
- 6. Agile Manifesto We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value Individuals and Interactions Processes and Toolsover Working Product Comprehensive Documentation over Customer Collaboration Contract Negotiationover Responding to Change Following a Planover That is, while there is value in the items on the right, we value the items on the left more
- 7. Agile Principles I. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. II. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage III. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale IV. Business people and developers must work together daily throughout the project. V. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. VI. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. VII. Working software is the primary measure of progress. VIII.Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. IX. Continuous attention to technical excellence and good design enhances agility. X. Simplicity--the art of maximizing the amount of work not done--is essential. XI. The best architectures, requirements, and designs emerge from self-organizing teams. XII. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.
- 10. How to select right process framework? Source - Wikipedia
- 11. Agile Testing Building quality into the product rather than trying to test it in later. How agile testing is different from traditional approach if team still struggling to answer what testers will do at beginning of sprint?
- 13. Found a bug during deployment? Manager/ Customer Testers Developer Architect Business Analyst Product Manager
- 14. Agile Testing Let’s understand by a simulation game to understand why agile testing is important? Simulation will cover below:- • Uncovering problem • Whole Team Approach • Agile Testing Principles
- 15. Whole Team Approach • Sub-optimization Vs Whole Team • Care about Cost of Delay • Calculate Cost of Production • Remove waste from software development
- 16. Waste in Software Development Defects Overhead processes Waiting Not-realizing people's potential Task switching Inventory Motion Extra features
- 18. Testing Approach Plan Quality - Test Strategy, Test Data, Infrastructure Built-In Quality - Specification By Examples, API vs GUI Testing, Regression Testing, NFR testing, Tools to test and provide feedback Validate Quality - Acceptance Test Driven Development, UAT, CAT
- 19. Software Testing Pyramid Automated UI Test Integration Test Unit Test Manual Test Anti-Patterns
- 20. Inverted Testing Pyramid Automated UI Test Automated API Test Automated Service Test Automated Component Test Automated Unit Test Exploratory Test 70% 20% 10%
- 21. Brian Marick’s Testing Quadrants Functional Tests Story Tests Prototypes Simulations Exploratory Tests Usability Tests User Acceptance Tests (UAT) Performance Testing Load Testing Security testing NFR Testing Unit Tests Component Tests Integration Tests Automated & Manual Manual Automated Tools SupportingTheTeam CritiqueProduct Business Facing Technology Facing
- 22. Pair Programing • Two developers work together to accomplish a single task. Driver and Navigator • One developer focus on details of task and other focus fitment of task in whole project • Produces code through conversation • Reinforce good programming habits • May be uncomfortable at first • Two brains on one task and two sets of eye produce fewer bugs • Investment in continual training
- 23. Pair Programming
- 24. Pair Programing Team Work High Morale Simple Design Comprehensive Testing Collective Code Ownership Good Development Practices
- 25. Pair Testing • Similar to Pair Programming • Transfer Knowledge • Save Time • Improve Communication • Positive Energy • Promote Negative Testing
- 26. Peer Review • Peer code review • Improve coding practices • Faster feedback • Encourage refactoring • Improve collaboration
- 27. Building right or right product? Business Failure Useless Stuff Business Success Technical Debts Specification By Example Build it Right Build the Right Things
- 28. Specifications by Example Impact Mapping (Vision) Story Mapping (MVP & MMF) User Story (Specification) As A Student I want to Purchase used Books Online So that I can save money
- 29. What are Specifications by Example Thin Slices of System Behavior That Deliver Business Value Described as concrete examples That are potentially automatable To create executable specifications Captured in live documentation
- 30. Refining the Specifications Specifications with examples are acceptance tests. Gojko Adzic • Be Precise and make sure specs are testable • Avoid “Scripts” and “Flows” • Focus on business functionality not design • Avoid UI details • Avoid covering every possible combination
- 32. Specifications By Collaboration • Hold Regularly • Full Team workshop • Three Amigos workshops • Product Owner • Developers • Testers
- 33. Specifications By Collaboration Given_________________ When_________________ Then__________________ Gherkin Given “James” want to upload new prescription When “James” selected “DoctorVisit.jpeg” And Click on “Record” Then Prescription available on search page
- 34. Write Specification Write examples for one specification. In order to maintain health record As a patient I want to enter my doctor visit details
- 35. Discussion of Acceptance Criteria If(user==“new” { object = user } Else error We should encourage employee to update ride Login -> Click on “New” and enter test data and then Result &%$^&
- 36. Collaboration of 3 amigos Technical Feasibility Happy Path Exceptions, Test Data, Boundary Conditions Developer Business Tester
- 37. Collaboration of 3 Amigos In order to maintain health record As a patient I want to enter my doctor visit details • New prescription can be uploaded as Image • Duplicate prescription not allowed • Photo can be captured by application • New prescription can be added manually also
- 38. Collaboration of 3 Amigos In order to maintain health record As a patient I want to enter my doctor visit details • New prescription can be uploaded as Image • Books can be removed from shopping cart • Shopping basket should be empty for new user • User should not be able to add same book twice Given “James” want to upload new prescription When “James” selected “DoctorVisit.jpeg” And Click on “Record” Then “Prescription Uploaded” displayed
- 39. Gherkin Feature – Name of feature Scenario – Behavior to be developed Given – Pre-conditions When – Actions to be performed Then – Expected Result And – Use for multiple Given, When & Then But – Describe exception cases Scenario Outline – Define multiple scenarios Examples – Multiple Scenarios Background – Avoid repeated Given
- 40. Test First Write Test Write Code Test Again Refactor Code Pass Fail Fail
- 41. Behavior Driven Development BDD Given Scenario Then Expected Outcome When Perform Action
- 42. Let’s get started Hands-on Behavior Driven Development • Write Feature to describe scenario in Gherkin • Eclipse, Java, Cucumber, Selenium, Junit • Create Test Runner class file • Generate Steps file • Write code to pass test
- 43. BDD - Characteristics • A testable story (it should be the smallest unit that fits in an iteration) • The title should describe an activity • The narrative should include a role, a feature, and a benefit • The scenario title should say what's different • The scenario should be described in terms of Givens, Events, and Outcomes • The givens should define all of, and no more than, the required context • The event should describe the feature
- 44. BDD – Feature File Feature: Checkout In order to calculate price of groceries As a Store Staff I should be able to calculate price for groceries during checkout Scenario: Checkout a banana Given The price of a “Banana” is 5 When I checkout 1 “Banana” Then the total price should be 5
- 45. BDD – Test Runner Class package test.java; import org.junit.runner.RunWith; import cucumber.api.CucumberOptions; import cucumber.api.junit.Cucumber; @RunWith(Cucumber.class) @CucumberOptions(features = "src/test/resources") public class testrunner { }
- 46. BDD – Run Output 1 Scenarios ([33m1 undefined[0m) 3 Steps ([33m3 undefined[0m) 0m0.000s You can implement missing steps with the snippets below: @Given("^The price of a "([^"]*)" is $(d+)$") public void The_price_of_a_is_$(String arg1, int arg2) throws Throwable { // Express the Regexp above with the code you wish you had throw new PendingException(); } @When("^I checkout (d+) "([^"]*)"$") public void I_checkout(int arg1, String arg2) throws Throwable { // Express the Regexp above with the code you wish you had throw new PendingException(); } @Then("^the total price should be $(d+)$") public void the_total_price_should_be_$(int arg1) throws Throwable { // Express the Regexp above with the code you wish you had throw new PendingException(); }
- 47. BDD – Step File public class TestSteps { @Given("^The price of a "([^"]*)" is $(d+)$") public void The_price_of_a_is_$(String arg1, int arg2) throws Throwable { // Express the Regexp above with the code you wish you had throw new PendingException(); } @When("^I checkout (d+) "([^"]*)"$") public void I_checkout(int arg1, String arg2) throws Throwable { // Express the Regexp above with the code you wish you had throw new PendingException(); } @Then("^the total price should be $(d+)$") public void the_total_price_should_be_$(int arg1) throws Throwable { // Express the Regexp above with the code you wish you had throw new PendingException(); } }
- 48. BDD – Modified Step File public class TestSteps extends TestCase { String product; int rate, qty; @Given("^The price of a "([^"]*)" is $(d+)$") public void The_price_of_a_is_$(String arg1, int arg2) throws Throwable { product = arg1; rate = arg2; } @When("^I checkout (d+) "([^"]*)"$") public void I_checkout(int arg1, String arg2) throws Throwable { qty=arg1; } @Then("^the total price should be $(d+)$") public void the_total_price_should_be_$(int arg1) throws Throwable { if(product.equals("Banana")) assertEquals(arg1, rate*qty); }}
- 49. BDD – Junit Output
- 50. BDD – Lifecycle Your Project Features Scenarios Steps Your System Automation Library Support Code Step Definitions Technology Facing Business Facing
- 51. Gherkin – Background Feature: Feedback when entering invalid credit card details In user testing we've seen a lot of people who made mistakes entering their credit card. We need to be as helpful as possible here to avoid losing users at this crucial stage of the transaction. Background: Given I have chosen some items to buy And I am about to enter my credit card details
- 52. Gherkin – Background Scenario: Credit card number too short When I enter a card number that's only 15 digits long And all the other details are correct And I submit the form Then the form should be redisplayed And I should see a message advising me of the correct number of digits
- 53. Gherkin – Multiple AND Scenario: Expiry date invalid When I enter a card expiry date that's in the past And all the other details are correct And I submit the form Then the form should be redisplayed And I should see a message telling me the expiry date must be wrong
- 54. Gherkin – Comments # This feature covers the account transaction and hardware-driver modules Feature: Withdraw Cash In order to buy beer As an account holder I want to withdraw cash from the ATM # Can't figure out how to integrate with magic wand interface Scenario: Withdraw too much from an account in credit Given I have $50 in my account
- 55. Gherkin – Step Definitions In Cucumber, results are a little more sophisticated than a simple pass or fail. A scenario that’s been executed can end up in any of the following states: • Failed • Pending • Undefined • Skipped • Passed These states are designed to help indicate the progress that you make as you develop your tests.
- 56. Gherkin – Test Runner Setup Element Purpose Default dryRun true (Skip execution of glue code) False strict true (will fail execution if there are undefined or pending steps) False glue where to look for glue code (stepdefs and hooks) {} features the paths to the feature(s) False monochrome whether or not to use monochrome output False format what formatter(s) to use {} tags hat tags in the features should be executed {}
- 57. Gherkin – Test Runner Setup @CucumberOptions( dryRun = false, strict = true, features = "src/test/features/com/sample", glue = "com.sample", tags = { "~@wip", "@executeThis" }, monochrome = true, format = {"pretty", "html:target/cucumber", "json:target_json/cucumber.json", "junit:taget_junit/cucumber.xml“ } )
- 58. Acceptance-TDD • What is A-TDD? • How A-TDD is different than BDD • Inside – out Vs Outside – in • Tools for ATDD • How FitNesse is different than Cucumber
- 59. Let’s build software using A-TDD Let’s try FitNesse • Setup FitNesse • Write tests on WIKI • Create Fixture • Develop system under test • Execute Test from Wiki
- 60. Test Driven Development • Unit test • Why TDD • How TDD is different than A-TDD • What is Right-BICEP • Mocking • Available tools • TDD vs A-TDD vs BDD
- 61. Unit Testing • A unit test is to test one unit of work. Generally one unit means of one requirement for one method. • Unit testing functional testing • Unit testing is not also User Interface testing
- 62. Unit Testing - Characteristic Isolated from the other code Isolated from the other developers Targeted Repeatable Predictable
- 63. Test Driven Development Let’s try to develop a component using TDD focusing Right-BICEP Expected result must be RIGHT B – Boundary Condition I – Inverse Check C – Cross check E – Error Condition P – Performance condition
- 64. Boundary Condition • Totally bogus or inconsistent input values, such as a file name of "!*W:Xn&Gi/w>g/h#WQ@". • Badly formatted data, such as an e-mail address without a top-level domain ("fred@foobar."). • Empty or missing values (such as 0, 0:0, "", or null). • Values far in excess of reasonable expectations, such as a person's age of 10,000 years.
- 65. Boundary Condition • Duplicates in lists that shouldn't have duplicates. • Ordered lists that aren't, and vice-versa. Try handing a pre-sorted list to a sort algorithm, for instance - or even a reverse-sorted list. • Things that arrive out of order, or happen out of expected order, such as trying to print a document before logging in, for instance.
- 66. Boundary Condition - CORRECT Conformance – Does the value conform to an expected format? Ordering – Is the set of values ordered or unordered? Range – Is the value within reasonable range? Reference – Does the code reference anything external that isn’t under direct control of code? Existence – Does the value exist? (not-null etc.) Cardinality – Are there exactly enough values? Time (absolute and relative) – everything happening at right time?
- 67. Test Driven Development TDD was introduced long back in 1999 as part of Extreme Programming (XP) by a group of developers. Basic philosophy is “Before you write code, think about what it will do”. Testing code once will not ensure future changes will not impact to earlier tested code so TDD is something you write once but run once, twice, three times etc. TDD is risk averse programming, investing work in the near term to avoid failures later on.
- 68. Test Driven Development When TDD used as an application design methodology, it works best when the business user is engaged in the process to help the developer define the logic that is being created. Even possible to define a set of input and expected output. TDD ensures quality code from the start. Write only the code needed to pass the test so method is going to have less code. Lesser the code lesser opportunity for defects.
- 69. Test Driven Development TDD ensures a high degree of conformity of the business requirement TDD helps keep unused code out of the system TDD provides built-In regression testing
- 70. Test Driven Development Write a test Compile Fix compilati on error Run Test Write Code Run test Refactor code
- 71. Test Driven Development - Rules First Law You may not write production code until you have written a failing unit test. Second Law You may not write more of a unit test than is sufficient to fail, and not compiling is failing. Third Law You may not write more production code than is sufficient to pass the currently failing test.
- 72. Misconceptions • Writing code 1st then Test is same like TDD • TDD isn’t useful for designing architecture • Takes too much time • Too complex to learn and implement • TDD is only good for small project
- 73. Test Double Test double is just like replacing on-screen Actor with stunt actor during film making. 4 ways to achieve test double • Dummy • Stub • Mock • Fake
- 74. Refactoring • Refactoring is the act of changing the internal implementation of a class or method with the aim of making the code more readable and maintainable. • Refactoring also reduces the code’s overall complexity without changing the external behavior of the class or method. • Refactoring allows you to continuously change and improve your code.
- 75. Code Smells Code smells are simply a collection of commonly known and widely found code-based anti-patterns.
- 77. Technical Debts Also known as design debt or code debt is "a concept in programming that reflects the extra development work that arises when code that is easy to implement in the short run is used instead of applying the best overall solution". Technical debt can be compared to monetary debt. If technical debt is not repaid, it can accumulate 'interest', making it harder to implement changes later on. - Wikipedia
- 78. Reason for Technical Debts
- 79. Common Practices
- 80. Extreme Programming Practices Don’t Repeat Yourself (DRY) Keep It Simple, Stupid (KISS) You Aren’t Gonna Need It (YAGNI)
- 81. Why To Refactor Improve Code Readability Rename Variables/ Methods/Classes Extract Methods Extract Classes Write Tests Collective Code Ownership Improve Code Maintainability Automated unit testing Feature envy code smell Don’t repeat yourself Inappropriate Intimacy code smell Lazy class code smell
- 82. When To Refactor The Rule of Three: The first time you do something, you just do it. The second time you do something similar, you wince at the duplication, but you do the duplicate thing anyway. The third time you do something similar, you refactor. When add new function/method When you need to fix During code review
- 83. Exploratory Testing • How is different than other testing? • Concurrent test design and execution • Balancing it with automated test • Why to perform exploratory testing? • Does exploratory testing add value? • Do we need to plan for exploratory testing?
- 84. Non-Functional Requirement • Need for Non-Functional Requirement Testing • What are the area needs to cover as part of NFR • Isn’t it NFR part of customer requirement? • Different type of NFR testing such performance, security and compliances etc.
- 85. Test Strategies • Strategy is a thought process not a documents • Preventive vs Corrective • Team is responsible not just test engineer • Developer test vs Tester test • Business centric vs Technology centric • Based on – Goal, Risk, Opportunity and Constraints
- 86. Test Planning • Infrastructures • Tools • Strategies • Methodologies • Customer focused • Let’s Do It Yourself
- 87. Test Metrics • Test pass/fail rates • Defects discovery rate • Regression test results • Defects density • Found and fixed cycle time • Code/test coverage • Requirement coverage It’s a trap! Don’t fall for it. Focus on value delivery.
- 88. Environment and Infrastructure Test Environment Configuration Management Automated Build Continuous Integration Continuous Delivery Delivery Pipelines Test Data Management
- 89. Continuous Integration A software development practice where members of the integrate their work frequently, usually each person integrates at least daily- leading to multiple integrations per day. Each integration is verified by an automated build to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly – Martin Fowler
- 91. Continuous Integration • All developers run private builds on their own workstations before committing their code • Developers commit their code to a version control repository at least once a day. • Integration builds occur several times a day on a separate build machine. • 100% of tests must pass for every build.
- 92. Source Code Management This is where developers commit their changes. Purpose of version control repository is to manage changes to source code and other software assets (docs, and database scripts). Popular Version Control – SVN, TFS, CVS, GIT
- 93. CI Server A CI server runs an integration build wherever a change is committed to the version control repository. Usually CI server is configured to check version control repository every few minutes. The CI servers will pull out latest changes and run build scripts to produce new build/product.
- 94. Continuous Build but why? Improve Product Quality Save time and cost Eliminate redundant task Minimize “Bad Builds” Eliminate dependency on individual Need for continuous integration
- 95. Continuous Build • Manual Build Vs Automated Build • Automation using build tools vs Script • Available Build Tools • Server Type • On-demand • Scheduled • Triggered • Continuous Build using Maven/MS Build/Rake/Gradle
- 96. Build Script • Build script is a single script or set of scripts to compile, test, inspect and deploy software. • Build script can be used without CI system. Ant, NAnt, MSBuild and Rake etc are example of build tools. • Team usually build through IDEs. Continuous Integration is much more than just compilation of code.
- 97. Continuous Testing and Inspection Many consider CI without automated test is not useful. This is not true because you can still execute many other types of testing load, component and security etc. Automated code inspections can be used to enhance the quality of the software by enforcing rules. You can use CI system to run these automatically against a code base. Tools like FXCOP and Stylecop is popular for Microsoft platform. SonarQube, CheckStyle, JavaNCSS and CPD are for Java platform.
- 98. Continuous Integration - Practices • Commit code frequently • Don’t Commit broken code • Fix broken build asap • Write automated tests • All tests must pass • Run private build • Avoid getting broken
- 99. Hands-On Exercise Language – Java 7 Build Tool – Maven SCM – GIT & GITHUB TDD – Junit ATDD – Cucumber and Selenium Continuous Integration – Jenkins Code Quality – Cobertura, CheckStyle and PMD
- 100. Role of Test Engineer
- 101. Role of Test Manager See the whole Go See Technical Solutions Improvement Services
- 102. Successful Delivery
- 103. Collaboration - Distributed Do they understand what I am saying? ??
- 104. Distributed - 3 Amigos in Action Given When Then Let’s Write Examples on shared Screen
- 105. Reference Books