Agile testingandautomation
- 1. { Agile Testing & Automation: Lessons Learned & Challenges Issued You have to start somewhere to get somewhere
- 2. Things I’ve Learned About Agile Testing & Automation so far…
- 3. { {Output Checking Automatable Assert requirements Verify expected behavior Happy paths & unhappy paths Exploratory Testing Not completely automatable A live thought process Tests beyond the requirements Will this actually provide value? Will the customer want to and be able to use it? 1. Testing is more than checking. Automate as much of the checking as possible. Automation doesn’t replace exploratory testing.
- 4. { 2. Testers are really good at writing automated acceptance/end-to-end tests. • Protractor & Web Driver with Behavior Driven Development (BDD) style • Test the requirements and workflow through the UI • Act as living documentation • Simple, business-user language • Are we making the right software? • Touch as much of the real world as possible • Everyone can learn to write them, and we should write them, but…
- 5. { If you put all of your efforts into only automating end-to-end tests, you will probably end up with a lot of fragile, broken, unmaintainable tests. Tests can be technical debt too. • End-to-end tests validate through the UI so any time the UI changes, they can break • They rely on data points that need to be managed
- 6. { • Confirm the code is doing what we say it will do • Are we making the application right? • Small, light, and less fragile • Unit tests don’t touch the external world and don’t rely on data points • Require more technical skill to write – embrace it! 3. Quality products also need unit and integration tests.
- 7. { 4. Code review improves code quality, reduces bugs, and leads to writing the right tests. • Keep it lightweight, small, frequent, and do it early • Asynchronous reviews allow people to review when it is convenient • Actively share interesting findings or design decisions and champion quality! • Treat code reviews as an active state in your workflow, or they probably won’t happen • Change the mindset to view code review feedback as a learning opportunity for both the reviewer and the reviewee
- 8. { • Leads to clearly thinking about what your code needs to do before you write it • Reduces waste (code and tests) • Sets the goal • It doesn’t just happen. You make it happen. 5. Test Driven Development (TDD) is hard & it takes commitment.
- 9. 6. Automated tests reduce regression testing time, enabling more rapid delivery and a shortened feedback loop.
- 10. Regression testing estimate 2012/2013 = 1 week +++
- 11. Regression testing estimate 2014 = 1 day +/-. Imagine what we could do in 2015!
- 12. { • User stories are the basis for tests • Tests set the goal for code • The relationship between tests and requirements needs to be tight • The communication between testers and programmers needs to be even tighter 7. It takes a whole team. Developers + Testers + Business People = Beautiful Tests
- 13. { Opportunities • A consistent approach to test automation to… • Add value up front and reduce technical debt and maintenance time down the road • Build confidence in our releases • Reducing the feedback loop even more through truly continuous delivery • Leveraging Sauce Labs and other tools to automate cross-browser and mobile testing • Support and empower teams toward consistency, quality, and testing
- 15. { What’s next? • Outreach to teams. What do you need to make testing a part of your development process? • Automation coverage metrics. How are we doing? Where do we need to focus our attention? • Big Team’s Definition of Done (DOD) • Training Roadmap to support DOD • Team representation at monthly QA COE meetings • Working with teams to run Watchfire AppScan and report on security analysis quarterly • Following up with teams to champion resolution of security, eAccessibility, or other defects.
- 16. Challenge Issued! Start changing your mindset right now to champion quality. Push back when you believe quality is at risk! Don’t wait for the perfect idea, perfect steps, perfect plan. Start doing something right now! But what???
- 17. Before you write code, write down what your code should do.
- 18. Then write some simple expectations. What would the user expect at the end of the test?
- 19. Fill in the test steps and run your tests.
- 20. Practice continuous improvement. This is an attitude. I have a set of tests and they all pass. I bet I can find 2 more tests that will make the product even better, or How could I refactor some of my tests to be more readable, more efficient, more valuable? I don’t have all of my tests automated. I can automate 3 new tests each sprint, or I’ve been automating 3 per sprint, I bet I can do 5.
- 21. Pair with pretty much anyone you can an hold one another accountable for adding tests. Pick one story in each sprint and commit as a team to pair or public pair programming for that story. Commit to writing tests first for that story. Make code review a part of your process. Do a pull request. Look at someone else’s pull request. I bet you will find something to improve upon, and maybe even a bug.
- 22. Fix bugs instead of putting them on the backlog. Better yet! When you find a bug, write a test for it, and then fix it or ask someone else to. When the test passes, it’s as if the bug never existed. You just fixed your bug and added to your regression test suite.
- 23. Review your Team Agreement and Definition of Done. Do you have items your team has agreed upon relating to quality and testing? If not, then you’re probably missing some important discussions. Remember, it takes a team!