Reliable application tests for ui5 apps
- 1. Maxim Naidenov, SAP July 20, 2018 Reliable application tests for UI5 applications
- 2. Web Page vs Web App
- 3. How we test Web Apps today Technical level Unit tests for separate components E2E tests that involve backend Organizational level E2E Test are created and operated by a QA team
- 4. What is the problem ? Catches defects late Reporting is hard Local reproduction is hard Retesting is slow Keeping tests consistent is hard Covering new features is hard Covering fixes is unlikely INEFFICIENT
- 5. How should web apps be tested ? Move UI tests in the app development cycle Ask developers to write them Execute frequently Catches regressions very early Reporting is automatic Reproduction is very easy Tests are consistent by design Very easy to cover new features and fixes
- 6. What is preventing us from doing it ? UI tests are written in a different language than than the web app UI tests require external tools and complicated local setup UI tests are hard to debug UI tests are unstable, frequent false positives
- 7. How to fix it ? Write tests in JavaScript Make the UI tests run completely in browser Debug tests in the browser dev tools Make UI tests inherently stable by automatic synchronization with the UI framework
- 8. Application Testing with OPA Testing framework for UI5 based applications UI5 based application Fully browser-based, NO selenium HTTP Server OPA script Application testing Developer centric Mocked backend MockServer
- 9. OPA – launching the test Bootstrap UI5 and QUnit Start the execution Load the test site Plain HTML
- 10. OPA – test suite The test suite loads the test cases Global OPA configuration
- 11. OPA – test case Loading Page Objects We use BDD-line syntax opaTest() is a wrapper for Qunit.test() Opa starts and stops the the application Test looks synchronous Semantical actions and assertions
- 12. OPA – page objects Work with controls, not the DOM OPA helper for creating Page Objects waitFor() is the basic building block Report will contain nice error messages This is the stable ID from the XML template
- 13. OPA – autoWaiter Wait for all XHR requests to complete Wait for all promises to be resolved Wait for all timeouts to be completed Application pollings are not a problem Wait for UI5 rendering to be completed Wait for UI5 NavContainer, OverflowToolbar, etc specific behavior
- 14. One good OPA test
- 15. Limitations Same domain policy means that both test and app should be deployed on the same origin, not suitable for productive systems In-browser operation mean that authentications and browser redirects are not possible OPA is a developer friendly but requires both JavaScript and UI5 proficiency to use effectively
- 16. Thank you. Contact information: Maxim Naidenov [email protected]
Editor's Notes
- #3: Web applications are browser-based javascript applications, frequently built with advanced frameworks like UI5, Angular, React. Web applications bring many benefits to the overall user experience but also provide serious challenges, particularly for automated testing. This reminder is important as most of the automated tools we use today are designed for automating the testing of web pages and not web applications. And particularly, the tools are depending on the interactions being mapped to HTTP communication because that is how the web has initially designed. But the web has changed and this mapping is no longer valid. The de-factor standard tool for automated browser UI testing is Selenium. There are numerous selenium-based frameworks that strive to optimize some aspect of selenium. There are also commercial tools line QTP. All those tools have a basic deficiency – they work on a low level and treat the application as a web page. And this is causing their major issues with reliability of the automated tests. So it is time to evolve the testing tools and start testing the web apps and not the container web page.
- #4: Lets look how we test today Unit tests Rare Qunit for UI5, Jasmin for Angular E2E or integration testing selenium, sikuli, commercial tools QA team owns the tests QA team operates the tests and reports issue How to have a better quality – usuallty boils down to write more automated tests
- #5: Long Feedback Cycle Process is hard - less likely to be followed under the pressure Test quality deuterates over time and UI tests become a burden and not an asset.
- #6: Shorten the feedback cycle use application tests as part of test-driven development - the test is a formal specification write the test together with the code and commit them together
- #7: Those problems are so common that many teams consider automated UI test as impractical and avoid them. So what is left is to live with manual UI regression testing. Manual testing is NOT feasible for regression testing because it is slow and does not scale - could not be executed after each commit. Remove the obstacles and provide value to developer No single open-source UI testing framework so we at SAP UI5 team had to build it ourselves.
- #9: Testing framework for UI5 based applications – intended for UI testing of UI5 applications, no application tweaking is required Fully browser-based, NO selenium – not based on Selenium, does not require any external deployments, only browser and http server to bootstrap the test and the app. Application testing – Application testing in this context means UI testing of single web application with mocked backend. We provide support for ODATA backend mocking with integrated mocking support – MockServer Developer centric – OPA is developer centric because uses inside knowledge of the application to speedup the test creation. Simple test execution and debugging. Reliable by design – Reliability is a challenge, especially for UI tests. Common experience is that UI test are flaky and require special experts or whole teams to write and support them. With OPA, we are challenging the perception. Due to the integration with UI5 core, the OPA test is inherently reliable.
- #10: Opa test is just one html page – open it in the browser and execute it. It will show the progress, you will see the test opening the app and clicking around, at the end you will see the report. Same applies for automation – use any Qunit runner.