BDD using Cucumber JVM
- 1. BDD using Cucumber- JVM Vijay Ramaswamy Test Architect & Consultant
- 2. Introduction to BDD (~5 mins) BDD stands for Behavior Driven Development An agile software development process founded by Dan North Evolved from Test Driven Development (TDD): Often misunderstood as a testing methodology, TDD is actually a software development process BDD focuses on getting the words right: “What is the intended behavior”, NOT “What should I test?” Attempts to prevent the misconceptions that arise while using TDD Can be applied at different levels – the overall system, a specific class, etc. 2
- 3. BDD for unit tests (~10 mins) This is where the BDD movement started from Extends the basic principles of TDD (refer to image on the right) Test names should be sentences, which avoid using the word “test”, preferring words such as “should” instead: E.g.: emptyStackShouldThrowExceptionOnPop() instead of testEmptyStackPop() Test methods usually implement the test logic directly, without delegating to a totally different layer of the test framework Requires the usage of test doubles (mocks, stubs, etc.) to isolate the code under test 3
- 4. BDD for unit tests (~10 mins) Other related terminologies: Specification Driven Development Key benefit: Provides all the benefits of TDD, with an added degree of clarity It is easy to implement BDD at this level using existing tools such as JUnit, Hamcrest matchers, AssertJ, etc. Apart from this, specialized tools are also available. A few Java/Groovy tools in this space: Spock, JDave, EasyB 4
- 5. BDD for acceptance tests (~10 mins) Acceptance criteria are defined using natural, English-like language (Domain Specific Language - DSL): Think from the end user’s point of view Gherkin (Given-When-Then) is one popular language used in this space, although it is not a mandatory choice Acceptance criteria should be directly automatable, using any appropriate tool of choice Acceptance criteria are usually decoupled from their implementation: E.g.: Acceptance criteria using Gherkin, implementation using Java 5
- 6. BDD for acceptance tests (~10 mins) Acceptance criteria should be collaboratively defined by the entire team (not just QA!) -> This is sometimes known as the 3 amigos principle (3 amigos = Dev, BA and QA) Other related terminologies: Acceptance Test Driven Development (ATDD), Specification by Example, Domain Driven Design Key benefit: Promotes a shared understanding of user stories within the team, and improves overall communication It is difficult to implement BDD at this level without specialized tooling. Some of the Java tools in this space: Cucumber-JVM, JBehave, Concordion 6
- 7. Outside-in BDD (~5 mins) It is possible and encouraged to practice BDD at both levels mentioned in the last couple of slides: Start with BDD at the acceptance test layer Proceed to expand the acceptance criteria into lower level specifications as appropriate Focus on getting the specification tests to pass first, followed by getting the acceptance tests to pass This is known as outside-in BDD 7
- 8. Cucumber: An introduction A tool designed to enable BDD for acceptance tests: Originally written in Ruby, then ported to Java The key components of Cucumber are: Feature files: Used to define user stories and corresponding tests Glue code: Used to implement (automate) tests defined within feature files Glue code is further classified into: Step definitions Hooks 8
- 9. Cucumber: Feature files (~15 mins) Text files that use Gherkin to define the user stories (features) and corresponding acceptance criteria (scenarios) Acceptance criteria should be in the form of concrete examples: Include actual test data within the test steps as relevant Feature files promote well defined traceability between requirements and tests, and are meant to act as the “source of truth” for the entire team 9
- 10. Cucumber: Feature files (~15 mins) Gherkin syntax: Given, When, Then, And, But Feature, Scenario, Background, Scenario Outline & Examples Step arguments: Enable passing in more complex test data as part of a test step Options available: Doc strings Data tables Tags: User-defined strings prefixed with “@” -> similar to Java annotations The goal is to enable grouping of features/scenarios as required Comments: Use “#” to specify comments if required 10
- 11. Cucumber: Glue code (~15 mins) Step definitions (stepdefs): Code implementation for various steps used within the scenarios: One Java method corresponding to each unique test step Use annotations (such as @Given, @When, etc.) and regular expressions to match step definition code to the corresponding steps You can use Lambda expressions if using Java 8 or higher Use method arguments to read test data specified within the test steps Use assertions to validate conditions specified within the test steps It is possible to auto-generate stepdef skeleton code from feature files: Simply execute your newly created feature, and Cucumber will automatically print out code stubs corresponding to all unimplemented test steps WebDriver can be used to implement stepdefs for web based applications 11
- 12. Cucumber: Glue code (contd.) Hooks: Scenario hooks: Code that runs before or after each scenario gets executed (similar to @Before and @After in TestNG) Can be used for global variable initialization, test harness setup, etc. Note that hooks are invisible to people reading your feature files; consider using a background instead of a hook if it makes sense Tagged hooks: Hooks that will be executed only for specific tags “Global” hooks: “Global” hooks are those which would run only once per execution (similar to @BeforeClass and @BeforeSuite in TestNG) These are NOT supported in Cucumber! If you do need this, you may need to implement some kind of programmatic work-around 12
- 13. Cucumber: Test execution (~15 mins) Cucumber provides multiple test runners that you can use: JUnit runner TestNG runner Command line runner All test runners have a common set of configuration options. A few prominent options: Path to the feature files: A list of feature files, or a list of directories containing feature files If left unspecified, Cucumber searches within the same package as the test runner 13
- 14. Cucumber: Test execution (~15 mins) All test runners have a common set of configuration options. A few prominent options (contd.): Path to the glue code: A list of packages on the classpath If left unspecified, Cucumber searches within the same package as the test runner A set of tags to be executed Plugins to be used for formatting your test reports (refer next slide for more details) “Strict” execution: Set to true if you want Cucumber to fail scenarios with unimplemented / pending step definitions Default is “false” 14
- 15. Cucumber: Reporting (~10 mins) Cucumber enables generation of test execution reports by specifying a list of report formatting plugins while configuring your test runner Various formatting plugins are available in-built, such as: Pretty JUnit HTML JSON Usage 15
- 16. Cucumber: Reporting (~10 mins) It is easy to create your own formatting plugins as well: This has led to the development of various custom plugins, many of which are even more user-friendly and visually appealing than the built-in ones It is possible to include your own custom content into a report: The scenario.write() API: Embed text into a report The scenario.embed() API: Embed images or even videos into a report 16
- 17. Cucumber: Best practices (~5 mins) Avoid using Cucumber as a standalone automation framework -> remember that it is primarily designed as a BDD tool! Write scenarios at the business logic level -> avoid implementation details: Test implementations may change, but the tests themselves should not! Keep scenarios independent of each other as far as possible Organize features neatly into Epics or Themes as appropriate, so that they are easy to search when required Reuse stepdefs to the maximum extent possible; avoid duplication of steps while writing scenarios: Use Eclipse autocomplete Evolve a common DSL for the entire team 17
- 18. Cucumber: Best practices (~5 mins) Keep stepdef methods unique: It is a common mistake to repeat stepdefs in multiple Java files, leading to ambiguous matches during execution Leverage backgrounds wisely to keep your scenarios more crisp and readable Avoid scenario outlines unless really necessary – most times, a single set of test data should be sufficient to test a given scenario Use tags to organize your scenarios and features into groups as appropriate Use dependency injection to share state between step definitions as required Feature files should be the single source of truth for the entire team: If using a system such as JIRA, explore options to integrate 18
- 19. Reference material Behavior Driven Development references: http://behaviourdriven.org/ https://dannorth.net/introducing-bdd/ Demo websites for practice: http://automationpractice.com http://newtours.demoaut.com/ http://demoqa.com/ http://www.way2automation.com 19
- 20. Recap An overview of BDD: Introduction to BDD BDD for unit tests BDD for acceptance tests Outside-in BDD An overview of Cucumber: Introduction to Cucumber Feature files Glue code Test execution Reporting Best practices 20
- 21. THANK YOU! Vijay Ramaswamy Test Architect & Consultant 21