![HttpUnit - Sample WebConversation conversation = new WebConversation(); WebRequest request = new GetMethodWebRequest( "http://www.meterware.com/servlet/TopSecret"); WebResponse response = conversation.getResponse(request); WebForm loginForm = response.getForms()[0]; request = loginForm.getRequest(); response = conversation.getResponse( request ); assertTrue( "Login not rejected", response.getText().indexOf("Login failed") != -1 );](https://image.slidesharecdn.com/automated-testing-of-web-applications-using-x-m-l-v2-1192756071730188-2/85/Automated-Testing-Of-Web-Applications-Using-XML-11-320.jpg)
![StrutsTestCase - Sample … public void testFailedLogin() { // setup parameters to pass to the action addRequestParameter("username","deryl"); addRequestParameter("password","express"); // set up the path to identify the action setRequestPathInfo("/login"); // perform action processing actionPerform(); // check that the user has been forwarded to “login” verifyForward("login"); // make sure the error for key ‘error.password.mismatch’ // is present in the request verifyActionErrors(new String[] {"error.password.mismatch"}); // make sure the session state is correct assertNull((String) getSession().getAttribute( "authentication")); } …](https://image.slidesharecdn.com/automated-testing-of-web-applications-using-x-m-l-v2-1192756071730188-2/85/Automated-Testing-Of-Web-Applications-Using-XML-21-320.jpg)
![Contact Here all week Q ‘n’ A sessions [email_address]](https://image.slidesharecdn.com/automated-testing-of-web-applications-using-x-m-l-v2-1192756071730188-2/85/Automated-Testing-Of-Web-Applications-Using-XML-62-320.jpg)
Automated Testing Of Web Applications Using XML
- 1. Automated Testing of Web Applications using XML dIon Gillard, Multitask Consulting Pty Ltd
- 2. Agenda What is a Web Application? Testing Web Apps Code-based testing Data driven testing Tools Automating test case creation Automating tests Comparison to Commercial Tools
- 3. What is a Web Application We’ll be concentrating on J2EE Web Applications A lot of the tools and concepts can be applied to other technologies Web Applications consist of: Server side resources JSPs, Servlets, Java code, configuration Client side resources HTML, images, JavaScript
- 4. Testing Web Applications – ‘Field Experience’ Developers test localised changes have had the desired effect Without automated testing a complete regression test is impractical Application yo-yos from good to bad, working to broken frequently Users have little confidence in the application and wait for a ‘release’ Large effort goes into QA for a release
- 5. Testing Web Applications – Why? There are lots of ways to test, we aren’t going to cover all of them Testing usually has a purpose Does the code work Does the application perform well Does it respect security requirements Can it handle load Does it match business requirements … .
- 6. Testing web applications - How We’re going to be focussing on ‘functional testing’, a.k.a. Black-box testing “ Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution condition” IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries
- 7. Testing Web Applications - Implications Looking at web applications from a user’s perspective - Does it do what I want, how I want it done? What the code does is a side-effect for the user Users expect a web application to be an ‘application’, regardless of how the data is displayed or the functionality accessed User interface is still very important, however, so layout, along with look and feel must still be tested
- 8. Testing Web Applications – What to test Interactions between the user and the application Expected Inputs input fields, hidden fields etc, query string or URL additions Unexpected inputs no password, field lengths, numerical domains etc Error conditions Messages to the user, code ‘failure’ Error 500 Expected processing Processing succeeds or fails Expected output Results of next page, database updates, messages queued etc
- 9. Code based Testing Tools such as JUnit rely on developers creating ‘test cases’ in code to ensure correctness This has all the advantages and disadvantages of developers writing code Several tools have been released that allow this development to be more productive, less tedious and the code more reusable Writing low-level code makes it hard to focus on what the testing is for
- 10. HttpUnit Freeware ‘Browser emulator’ for use in JUnit test cases (and other places) Written in Java, provided as a jar developers would use Takes care of authentication, cookies, page redirection Provides resulting pages as a DOM via NekoHTML HTML pages don’t have to have valid XML syntax to be parsed
- 11. HttpUnit - Sample WebConversation conversation = new WebConversation(); WebRequest request = new GetMethodWebRequest( "http://www.meterware.com/servlet/TopSecret"); WebResponse response = conversation.getResponse(request); WebForm loginForm = response.getForms()[0]; request = loginForm.getRequest(); response = conversation.getResponse( request ); assertTrue( "Login not rejected", response.getText().indexOf("Login failed") != -1 );
- 12. HttpUnit – API Classes are available for creating requests in various ways (WebRequest) Working with the HTTP response (WebResponse) return code, text, headers, cookies etc Working with HTML artifacts getDOM getForms(), getFormWithName(name), getTables() getLinks(), getLinkWith(text) Forms can be ‘submitted’ after retrieval from the response
- 13. HttpUnit – Pros and Cons Code can be repetitive and tedious Working at a low level, you can lose sight of what is being done Each WebRequest is created in isolation in a test Testing a web application is typically a single host and URI, code is needed to unify tests Based on JUnit, is easy to automate Simple clear API is easy to understand
- 14. HtmlUnit Similar in scope to HttpUnit, but with a focus on HTML documents rather than the HTTP protocol details like requests, responses etc Better JavaScript support API is easier to relate to user interaction, e.g. clicking links and pressing buttons Wrapper objects for every html element Browser specific features can be used to fake the browser version and vendor
- 15. HtmlUnit - continued Can track frames, iframes, pop up windows Listener to handle new windows being opened as part of interaction or JavaScript Can easily access the other windows Tab-order can be retrieved and checked Assertion helper methods for unique ids, accessibility Full access to the actual DOM i.e. editable
- 16. HtmlUnit – Sample form submit WebClient client = new WebClient(); URL url = new URL("http://htmlunit.sf.net/"); HtmlPage page = (HtmlPage) client.getPage(url); // find form by id HtmlForm form = page.getFormById(“mainForm”); // ‘type’ text into the userId field HtmlInput input = form.getInput(“userId”); input.setValue(“dion”); // submit – note existing values provided in the page // are ‘automatically’ filled in form.submit();
- 17. ServletUnit Companion tool shipped with HttpUnit For when you need to do less black box and more white box testing Simulates a servlet container by providing Mock Objects Provides API for accessing the ‘Invocation Context’ Other tools such as Cactus provide a fully-featured in-container environment
- 18. ServletUnit – Sample code ServletRunner sr = new ServletRunner(); sr.registerServlet("myServlet", StatefulServlet.class.getName()); // create a client and request ServletUnitClient sc = sr.newClient(); WebRequest request = new PostMethodWebRequest( "http://test.meterware.com/myServlet" ); request.setParameter( "color", "red" ); InvocationContext ic = sc.newInvocation( request ); // call service ic.getServlet().service(request, ic.getResponse()); // test response, session, servlet instance variables etc assertNotNull("Session was not created", ic.getRequest().getSession( false ) );
- 19. StrutsTestCase – What is it? JUnit extension for testing code based on the Apache Struts web app framework Provides Mock Objects for simulating the servlet container Also provides an in-container Cactus extension (CactusStrutsTestCase) Switching approaches means changing parent class to/from StrutsTestCase
- 20. StrutsTestCase – How does it work? Tests are based around Struts Actions There are methods available for hooking into the Struts request lifecycle Extra assert-like methods for checking the Struts environment, like forwards and action errors
- 21. StrutsTestCase - Sample … public void testFailedLogin() { // setup parameters to pass to the action addRequestParameter("username","deryl"); addRequestParameter("password","express"); // set up the path to identify the action setRequestPathInfo("/login"); // perform action processing actionPerform(); // check that the user has been forwarded to “login” verifyForward("login"); // make sure the error for key ‘error.password.mismatch’ // is present in the request verifyActionErrors(new String[] {"error.password.mismatch"}); // make sure the session state is correct assertNull((String) getSession().getAttribute( "authentication")); } …
- 22. Apache’s Cactus A framework for unit testing server side Java code JUnit extension to make it easier to test Servlets, EJBs, Tag Libraries, Servlet Filters etc Comes with ant tasks for automation Uses a ‘proxy’ mechanism to redirect test case requests to a live servlet container
- 23. Apache’s Cactus - API Provides 3 Test classes to subclass from ServletTestCase, JspTestCase and FilterTestCase Code testXXX methods as per usual, except you need to instantiate the object to be tested (e.g. servlet) beginXXX and endXXX methods can be coded to setup or post-process HTTP related parameters such as headers, return code, the response etc
- 24. Apache’s Cactus - Features Provides IDE integration for JBuilder, VAJava EJB unit testing using Servlet ‘proxy’ As it runs in-container, there is good documentation on setting up the proxy Can integrate with HttpUnit and StrutsTestCase
- 25. XML-based Tools These tools rely on tests written in an XML format rather than java code Output varies – can be HTML They can be more easily generated They are more ‘user’ friendly Take away the reliance on technical skills to build tests Work at a higher level and allow more focus on what is being tested and how
- 26. Apache’s Latka Functional testing tool that uses XML to define a series of HTTP(S) requests Validations for the responses are defined as child elements of the requests Has a command prompt interface Can run tests via JUnit Can run tests interactively via a web-app Latka is moving to use Jelly as it’s XML processor
- 27. Apache’s Latka – Document Definition Suite top level element for a set of tests Provides default host, port and a label Session Wrapper for a set of requests that share HTTP state Request A HTTP request to be executed Path, method (“get”), host, port, label Validate Holds all validations for a specific request
- 28. Apache’s Latka - Validations Bytelength – length of the response Cookie – presence of a cookie maxRequestTime Regexp – tests the response matches a regular expression statusCode – e.g. 200, 404 XPath – test the response matches an XPath expression
- 29. Apache’s Latka - Sample <suite defaultHost="jakarta.apache.org" label="Taglibs"> <session> <request path="/taglibs/" label="home page"> <validate> <statusCode /> <regexp pattern="Welcome to Jakarta Taglibs!"/> </validate> </request> <request path=“/logon”> <param><paramName>user</paramName> <paramValue>dion</paramValue></param> <param><paramName>password</paramName> <paramValue>dionspwd</paramValue></param> <validate><statusCode /> </request> </session> </suite>
- 30. Apache’s Latka – Running Tests Command prompt Latka.bat file:./TestSite.xml JUnit public static Test suite() { TestSuite suite = new TestSuite(); String fileName = "tests/samples/TestCommonsWebsite.xml"; suite.addTest( JUnitTestAdapter.createTestFromFile(fileName)); return suite; }
- 31. Apache’s Latka Also provides validators as tags for Jelly Automate via JUnit Ant task Listeners can be attached to pick up test events as they happen XML and Simple (console) reporter provided XSL Style sheet provided to produce text from report
- 32. Apache’s Jelly Java and XML based scripting and processing engine Jelly ‘scripts’ are XML documents which get parsed Scripts are then ‘executed’ to produce XML, HTML, text etc XML elements can be bound to java code similar to JSP custom tags and Ant tasks
- 33. Apache’s Jelly - Taglibs Core: catch, choose, expr, forEach, if, file, import, include, jelly, new, otherwise, set, thread, when, whitespace Xml: attribute, copy, copyOf, element, expr, forEach, if, parse, set Define: attribute, bean, invoke, invokeBody, jellyBean, script, tag, taglib, Sql: dateParam, driver, param, query, setDataSource, transaction, update
- 34. Apache’s Jelly – Taglibs (Continued) JSL: applyTemplates, style, stylesheet, template Ant: ant, filescanner Werkz: attain, attainGoal, goal, postAction, postGoal, preAction, preGoal, project Jeez: target, tagDef Log: debug, error, fatal, info, trace, warn Ojb: broker, store
- 35. Apache’s Jelly – Taglibs (Continued) JMS: connection, destination, mapEntry, mapMessage, message, objectMessage, property, receive, send, textMessage Validate: verifier, validate, assertValid Http: delete, get, header, post, put Interaction: ask Antlr: antlr, grammar Util: tokenize Html: parse
- 36. Apache’s Jelly – Taglibs (Continued) JUnit: assert, assertEquals, case, fail, run, suite Swing: action, component, windowListener Quartz: cronTrigger, job, waitForScheduler Betwixt: introspector, parse
- 37. Apache’s Jelly - Testing Provides a JUnit tag library a HTTP tag library a ‘cron’ like facility an expression language Integration with JMS, SQL etc allows page results to be checked against the database or queue
- 38. Apache’s Jelly - Sample <?xml version="1.0"?> <j:jelly xmlns:j="jelly:core" xmlns:log="jelly:log" xmlns:v="jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary" xmlns="jelly:org.apache.commons.latka.jelly.HttpTagLibrary" trim="false"> <session> <get var="mtc" uri="http://www.multitask.com.au/"/> <j:if test='${mtc.statusCode == 200}'> request ok </j:if> Results for mtc url are: http return code = ${mtc.statusCode} http status text = '${mtc.statusText}' size of result = ${mtc.responseBodyAsString.length()} response time = ${mtc.responseTime} </session> </jelly>
- 39. Apache’s Jelly - Sample <?xml version="1.0"?> <j:jelly xmlns:j="jelly:core“ xmlns:log="jelly:log" xmlns:v="jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary" xmlns="jelly:org.apache.commons.latka.jelly.HttpTagLibrary“> <session> <post var="jdc“ uri="http://developer.java.sun.com/servlet/SessionServlet"> <parameter name="action" value="login" /> <parameter name="url" value="/developer/index.jshtml" /> <parameter name="UserId" value="XXXX" /> <parameter name="Password" value="XXXX" /> </post> jdc login result = ${jdc.statusCode} good login = ${jdc.responseBodyAsString.indexOf("Sorry!") == "-1"} <v:regexp pattern="Sox.y!" var="jdc"> <request-failed var="jdc">bad pattern 1</request-failed> </v:regexp> </session> </jelly>
- 40. XMLUnit JUnit extension providing comparison between XML docs XML can come from lots of places: Strings, DOM, Reader Accepts ‘browser-friendly’ HTML as input Can validate against a DTD
- 41. JXWeb XML Script ‘language’ and extension to JXUnit Uses HttpUnit under the covers No JUnit integration Not as easily extendable as Jelly
- 42. JXWeb – Scripting Language jxw – top level element httpGet, postXml – to retrieve data setRequestParameter, setCookie, setHeader getLink, getTableValue, getFormParameter, getForm, getTable ifEqual, isNull, loop, doLoop, exception, ifNull, save, subst, ifEqualLength, isEqual Echo, set, loadProperties
- 43. JXWeb - Sample <jxw> <set name="req" value="www.testsite.com"/> <httpGet response="response1"/> <save name="respText" file="list.html"/> <getTable name="myTable" tableIndex="0"/> <getTableValue name="tableValue" table="myTable“ row="0" column="0"/> <isEqual name=“tableValue" value="Every Good Beer Deserves Froth“ message=“page is missing froth"/> </jxw>
- 44. AntEater – Ant Functional Testing Testing framework designed around Ant Supplied as a set of Ant tasks and types Integrates easily into an existing build or IDE Comes with a servlet container built-in to allow for ‘call backs’
- 45. Ant Eater - Tasks Action Tasks httpRequest soapRequest Listener Match – groups multiple tests Tests Listening: method, parameter, sendResponse Header, contentEquals, regexp, responseCode, xpath, relaxng
- 46. Ant Eater – Other tags Session Logger – minimal, plain, colour, xml forEach Group – a way of grouping variables, sessions and loggers servletContainer – start internal tomcat Deploy – deploy web apps
- 47. Ant Eater - Sample … <target name="checkonline"> <httpRequest path="/"> <match assign="online"><responseCode value="200"/> </match> </httpRequest> </target> <target name="hitserver" if="online"> <echo>We're online!</echo> <httpRequest path="/"> <match assign="apache"> <header name="Server“ value="Apache/1.3.24 (Unix) Debian GNU/Linux"/> </match> <match assign="unknownserver“/> </httpRequest> </target>
- 48. Limitations XML as a programming language isn’t Not as expressive Tedious syntax for non-techies Usually not as extensible Limits the interaction with external data sources Other than Jelly, most don’t have a mechanism for dealing with java objects outside their framework
- 49. Automating Test Case Creation One of the main reasons developers give for limited testing is limited time Developers are actually happy to test if it isn’t a significant drain on achievements Testing pages or servlets helps developers think of error conditions that could easily crop up
- 50. Automated Creation - JMeter JMeter is often thought of as a load testing tool only It also has a ‘proxy server’ mode where you can record browser activity It can filter out requests for images or other resources using a regexp It can also be used to ‘assert’ text is present on a request
- 51. Automated Creation – ‘Homer’ Microsoft freebie tool in IIS 5.0 resource kit Under windows acts as a proxy and records requests coming in Usually used for load testing Stores data in an Access .mdb file Simply queries can transform this into XML and then into Latka or Jelly etc Merged into VS.NET toolset
- 52. Automated Testing Once the tests are created the next thing is to get them run on a regular basis This means application and developer confidence grows Depending on the tool this can be achieved using the operating system and a build tool
- 53. Apache’s Ant Java based build tool Has JUnit tasks as part of the ‘optional’ tasks Can be scheduled using operating system constructs Can provide email about failures
- 54. CruiseControl Open source tool from ThoughtWorks Provides a way to kick off Ant builds based on source repository changes This allows the automation of the tests to be done when the source code changes Ensures that the application passes all tests Flags immediately when the application breaks
- 55. Commercial Tools LoadRunner from Mercury Interactive Provides access to XML data from the scripting language More geared toward load testing Astra QuickTest from Mercury Interactive Web testing, built-in script creation via browser, cross browser tests from same test case, extract details from browser objects Supports Flash, Real etc
- 56. Commercial Tools Rational RobotJ Eclipse based tool Uses proprietary scripting language Does auto-record, regular expressions Approx 1.3GB space WebLoad Similar features to higher cost tools Multiple server support, higher load focus Supports JavaScript GUI and Command line driven
- 57. Commercial Tools - continued High end Java IDEs such as WebSphere Studio Application Developer 5 now include testing tools Some can not be automated Compete with other vendors in same tool space
- 58. Summary There are simple tools available now that can alleviate the drudgery of functional testing Users can help write the functional tests Automation and repetition are key Choose how far you want to take testing Be consistent Allow for learning curve on new tools and technologies Mike Bowler’s Session on Test Driven Design
- 59. Resources Ant http://jakarta.apache.org/ant/ HttpUnit http://httpunit.sourceforge.net/ JMeter http://jakarta.apache.org/jmeter/ JUnit http://www.junit.org Latka http://jakarta.apache.org/commons/latka
- 60. Resources (Continued) StrutsTestCase http://strutstestcase.sourceforge.net/ JXWeb http://qare.sourceforge.net/web/2001-12/products/jxweb/ Cactus http://jakarta.apache.org/cactus/ XMLUnit http://xmlunit.sourceforge.net
- 61. Resources (Continued) Homer http://webtool.rte.microsoft.com HtmlUnit http://htmlunit.sourceforge.net WebLoad http://www.webload.dk Jelly http://jakarta.apache.org/commons/sandbox/jelly/
- 62. Contact Here all week Q ‘n’ A sessions [email_address]