Evaluating Test Driven Development And Parameterized Unit Testing In Dot Net Presentation

1. Evaluating Test Driven Development [TDD] and Parameterized Unit Test [PUT] in .NET Web PlatformPrepared By: Md. Fazlul Alam Chowdhury, Email: [email protected]: http://fazlulchowdhury.blogspot.comDate: 14.04.20101

2. Topics Covered (i)AbstractIntroductionWhy Test Driven Development?TDD in .NET EnvironmentDatabase Unit Testing - DBUnit Vs. DB Pro

3. Automated Web Testing– Selenium Vs. VS Web Testing

4. Unit Testing - NUnit Vs. MS Test/MVC

5. Load Testing – VS Load Testing 2

6. Topics Covered (ii)MVC .NET Test SuiteParameterized Unit Testing (PUT) and Microsoft PEXStrengths and Limitation of Testing ToolsAgile Influence in Visual Studio Team FoundationScope of Future Research and ImprovementConclusionReferences3

7. Abstract (i)Web technology has become widely accepted application platform this days. It brings the remote and distributed application integration more successful. It brings parties together from distant places and helps making decisions more collaboratively and effectively. So, to have a reliable and well tested portal application Test Driven Design is obvious. 4

8. Abstract (ii)Application Test Driven Development or simply TDD is an evolutionary approach to produce well tested reliable and shippable code and thus improving the quality of the software through Fault avoidance and Fault detection at early stages and Fault recovery with simpler effort. In Test First Design or TFD approach, codes are being written to fail, executing tests and then updating the functional codes to pass. TDD is basically the combination of TFD and Refactoring.5

9. IntroductionThis report is concentrated on Evaluating TDD in.NET environment. Web Application testing comprises of Functionality, Usability, Interface, Compatibility, Performance and Security which is a broad area from testing perspective. Not any of the available toolset performs testing for all the scenarios. So, we tried to narrow down some Web Testing Tools and concepts which contributed to the World Wide acceptance to .NET Applications.6

10. What is Quality?Quality is a term that stands for wide variety of aspects for a particular product or environment. How well the environment is or how good the product is based on some specific criteria could be regarded as Quality. According to Gerald Weinberg: “Quality is a Value to some person”By Dr. Tom DeMarco “Product's quality is a function of how much it changes the world for the better”Reliability, Scalability, Correctness, Completeness, Maintainability, Extensibility, Fault Tolerance etc. are the key selling points of Software Quality http://ase.cpsc.ucalgary.ca/ase/uploads/SENG515615W2010/Lecture4_Agile_Engineering_Practices.pdf7

11. Quality Control Vs. Quality Assurance (i)ISO 9001 Definition ofQuality Control The operational techniques and activities that are used to fulfill requirements for qualityQuality Assurance All those planned and systematic activities implemented to provide adequate confidence that an entity will fulfill requirements for qualityhttp://elsmar.com/pdf_files/QC%20vs%20QA.pdf8

12. Quality Control Vs. Quality Assurance (ii) QC Vs. QAProduct ProcessReactive ProactiveLine function Staff functionFind defects Prevent defectsWalkthrough Quality AuditTesting Defining ProcessInspection Selection of toolsCheckpoint review Traininghttp://elsmar.com/pdf_files/QC%20vs%20QA.pdf9

13. Test Driven DevelopmentThe term TDD has been around since NASA’ s Project, Mercury in 1960s. But, when Kent Beck and Ward Cunningham introduced TDD as a eXtreme Programming (XP) practice, it started gaining popularity. So, What is TDD?10

14. What is Test Driven Development?Ref: A Survey of Evidence for Test Driven Development in Academia, Chetan Desai, David Janzen, Kyle Savage Computer Science California Polytechnic State University San Luis Obispo, California USA {cdesai, djanzen}@calpoly.edu11

15. TDD – Industrial Case Studies@MicrosoftTwo case studies were done at Microsoft to build empirical knowledge on the efficacy of TDD together with performing Cost-Benefit Economic Analysis on the utility of TDD:Cost-benefit tradeoff between the increase in development time and

16. Resulting improvement in software qualityEvaluating the Efficacy of Test-Driven Development: Industrial Case Studies, ThirumaleshBhat & NachiappanNagappan12

17. Windows networking team case studyProject A: Context FactorProject A: Product MeasureProject A: OutcomeEvaluating the Efficacy of Test-Driven Development: Industrial Case Studies, ThirumaleshBhat & NachiappanNagappan13

18. MSN Team case studyProject B: Context FactorProject B: Product MeasureProject B: OutcomeEvaluating the Efficacy of Test-Driven Development: Industrial Case Studies, ThirumaleshBhat & NachiappanNagappan14

19. Can someone write my Test?15

20. TDD Tools Used in this Research (i)Testing Frameworks: Visual Studio Team System Unit Testing Framework

21. MVC .NET Testing Framework

22. Microsoft PEXUnit Testing: NUnit

23. MVC .NET with MOCK

24. MS Test for Visual Studio16

25. TDD Tools Used in this Research (ii)Database Testing: DBUnit .NET

26. DB ProAutomated Web TestingSelenium

27. Visual Studio Automated Web TestLoad/Stress TestingVisual Studio Web Test

28. WCat 6.3 (x86)17

29. Database Testing using DBUnit/DB Pro18

30. 19

31. What is Database Testing and Why? (i)Database testing is a defined process to test database programmability objects. Now, the question comes: what are the database objects that requires testing to produce quality object? A Database consists of the following programmability objects:Stored Procedures

32. DB Functions

33. Triggers

34. Rules

35. DB Assemblies20

36. What is Database Unit Testing and Why?(ii)So, the database unit tests could be created before hand to test the database programmability objects and make sure that they produce the result as expected and existing functionality are workingThe following types of database unit tests could be performed:Schema Test

37. Feature/Functionality Test

38. Security Test

39. Stock data Test

40. Referential Integrity Constraints21

41. What to Test?Refactoring Databases: Evolutionary Database Design, by Scott W. Ambler and Pramod J. Sadalage, Addison Wesley Professional ISBN#: 0-321-29353-3 http://www.ambysoft.com/books/refactoringDatabases.html22

42. Database Testing with DBUnitDBUnit is a Open Source Database Testing Framework which was created by Manuel LaflammeDBUnit is a Database Unit Testing Framework to Test Database Driven Classes, Procedures or Functions. NUnit, MVC .NET/Visual Studio Testing platforms can be used to build a standard test suiteFollowing Operations can be performed using DBUnitUPDATE TRUNCATE INSERT REFRESHDELETE CLEAN_INSERTDELETE_ALL NONEOther OperationsComposite OperationTransaction OperationIdentity Insert Operation23

43. DBUnit Supported DatabasesRef: http://dbunit.wikidot.com/supportedrdbms24

44. Creating a DBUnit Test ClassCreate a XML Dataset or Export Data from Databases25If the DBUnit Test Class Name is DbUnitTestCase then the Data Set name has to be DbUnitTestCaseDataset.xml and has to be in the same folderFlat XML:<?xml version="1.0" encoding="utf-8" ?><dataset> <table_name col1="value" /></dataset>

45. Create a Test Class and Extend DBTestCaseusing System;using System.Data;using System.Data.Sql;using System.Data.SqlClient;using System.Collections.Generic;using System.Reflection;using System.Xml;using DbUnit.Framework;using DbUnit.Core;using DbUnit.Dataset;using NUnit.Framework;using NMock2;namespace MyDBUnitProject{ [TestFixture] public class MyDBUnitTestCase2 : DbUnitTestCase { private SqlConnection connection = null; Mockery mocks; private IDataSet dataset;26

46. Implement SetUp and TearDown [SetUp] public void SetUpTestCase() { mocks = new Mockery(); dataset = new EmbeddedFlatXmlDataSet(this.GetType()); connection = new SqlConnection("Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=BIteDB;Data Source=localhost\\sqlexpress");connection.Open(); } [TearDown] public void TearDownTestCase() {connection.Close(); ; }27

47. Implement Test Method [Test] public void TestQuerySQLMessage() { string _sql = "SELECT AttachmentLink as ReturnValue from Message WHERE CAST(MsgContent AS varchar(4000)) ='My Test'";SqlCommand _cmd = new SqlCommand(_sql, connection);SqlDataReader _reader = _cmd.ExecuteReader(); string _AttachmentLink = ""; while (_reader.Read()) { _AttachmentLink = _reader["ReturnValue"].ToString(); }Assert.AreNotEqual("", _AttachmentLink); }[Test] public void TableNameSetCorrectly() {Assert.AreEqual("table_name", dataset.Tables[0].Name); }28

48. Nunit/DBUnit Test Results29

49. DBUnit BenefitsDBUnit provides a stable framework to create and adopt Database Test cases with minimal effortIt can manage the state of database through the test’s life cycleAt the end of testing the database can be placed back into its pre-test stateA database can be populated with a desired data set before a testDBUnit test can easily automated by controlling major dependenciesDBUnit test can be performed utilizing most types of database objects (Tables, Views, Stored Procedures)DBUnit Tests do not require manual intervention between testsDBUnit Test results doesn’t need manual interpretationCode stability increases in a Team environmentEasy to Learn and ImplementWide range of support for standard databasesDBUnit documentation is pretty well definedDBUnit tests are easily manageableIt is easier to generate the seed data using the dump feature30

50. DBUnit LimitationsPrimary test data is XML but it is also possible to create test cases for standard databases Test Data generation plan is absent and so have to create and refresh the test dataset when Actual schema changes and so the seed and resulted data could be differentPrimary support for XML Dataset is Flat XML and need to have all columns defined in the first rowDBUnit does not support multiple Resultset31

51. DBUnit – Short Demo32

52. Database Testing with MS DB ProThe desire on the part of the database developer to use familiar T-SQL scripts to express the unit-test logic and the desire to take advantage of powerful Microsoft® .NET Framework-targeted languagesEncapsulating the test in T-SQL code and then hosting the T-SQL in C# or Visual Basic .NET code. The unit-test designer generates both, enabling either (or both) to be modified to meet individual needs.Database unit testing with Team System provides or supports the features for entire framework and using proper context [Authentication and Security Aspects etc.]Apply Test-Driven Development to your Database Projects - Jamie Laflenhttp://msdn.microsoft.com/en-us/magazine/cc164243.aspx#S233

53. Two sample joined tables34

54. Stored PreocedureCREATE PROCEDURE [dbo].[uspGetEmployeeManagers] @EmployeeID [int]ASBEGIN SET NOCOUNT ON; -- Use recursive query to list out all Employees required for a particular Manager WITH [EMP_cte]([EmployeeID], [ManagerID], [FirstName], [LastName], [Title], [RecursionLevel]) -- CTE name and columns AS ( SELECT e.[EmployeeID], e.[ManagerID], c.[FirstName], c.[LastName], e.[Title], 0 -- Get the initial Employee FROM [HumanResources].[Employee] e INNER JOIN [Person].[Contact] c ON e.[ContactID] = c.[ContactID] WHERE e.[EmployeeID] = @EmployeeID UNION ALL SELECT e.[EmployeeID], e.[ManagerID], c.[FirstName], c.[LastName], e.[Title], [RecursionLevel] + 1 -- Join recursive member to anchor FROM [HumanResources].[Employee] e INNER JOIN [EMP_cte] ON e.[EmployeeID] = [EMP_cte].[ManagerID] INNER JOIN [Person].[Contact] c ON e.[ContactID] = c.[ContactID] ) -- Join back to Employee to return the manager name SELECT [EMP_cte].[RecursionLevel], [EMP_cte].[EmployeeID], [EMP_cte].[FirstName], [EMP_cte].[LastName], [EMP_cte].[ManagerID], c.[FirstName] AS 'ManagerFirstName', c.[LastName] AS 'ManagerLastName' -- Outer select from the CTE FROM [EMP_cte] INNER JOIN [HumanResources].[Employee] e ON [EMP_cte].[ManagerID] = e.[EmployeeID] INNER JOIN [Person].[Contact] c ON e.[ContactID] = c.[ContactID] ORDER BY [RecursionLevel], [ManagerID], [EmployeeID] OPTION (MAXRECURSION 25) END;35

55. DB Pro Test Results36

56. Adding a Data Generation Plan37

57. Selecting Tables for Test Data38

58. Data Column Configuration39

59. Generate Test Data before Unit Test40

60. DB Pro BenfitsFeature Tests, Schema Tests, Security Tests, Stock-Data Tests can be performed using DB ProAutomatic generation of SQL script stubs of unit tests for stored procedures, functions, and triggersDB Pro Provides a set of built-in test conditions to verify results including the rows returned, scalar values, and execution timeAutomatic Generation of Test Data using Data Generation ToolIt sets the database state through it’s data generation featureSame Test Data can be generated repeatedly based on seed valuePre-Test, Test and Post Test results could be written, executed and evaluated41

61. DB Pro LimitationsOnly supported database is MS SQL ServerNeed Visual Studio Team System - Database Edition to get the DB Pro Test SuiteSupports C#/VB platforms for unit testingData generation tool needed to use to generate test data and so database sandbox concept is not in there so that may have to work on real data and could be rolled back the data after testing42

62. DB Pro – Short Demo43

63. Web UI Testing using Selenium/Web Test44

64. Selenium for Web UI TestingSelenium is an automated web user interface testing tools that provides the environment to write a test case in respective platform [here .NET]. It has two components:Server: A server which automatically launches and kills browsers, and acts as a HTTP proxy for web requests

65. Client: Client libraries to Build a Test Case Using NUnit and Mock45

66. Selenium RC (Remote Control)http://seleniumhq.org/projects/remote-control/46

67. Selenium Feature MatrixRef: http://wiki.openqa.org/pages/viewpage.action?pageId=76347

68. Start with Selenium Project in .NETDownload Selenium from Download and Install NUnit or Any other testing suiteCreate a Visual Studio Test Project Reference the following DLLs: nmock.dll, nunit.core.dll, nunit. framework.dll, ThoughtWorks.Selenium.Core.dll, ThoughtWorks.Selenium. IntegrationTests.dll and ThoughtWorks.Selenium.UnitTests.dllWrite a Selenium Test Case48

69. Selenium Test Classnamespace MySeleniumTest{ [TestFixture] public class MySeleniumTest1 { private ISelenium selenium; private StringBuilderverificationErrors; [SetUp] public void SetupTest() { selenium = new DefaultSelenium("localhost", 2613, "*firefox", "http://localhost:2613");selenium.Start();verificationErrors = new StringBuilder(); } [TearDown] public void TeardownTest() { try {selenium.Stop(); } catch (Exception) { // Ignore errors if unable to close the browser }Assert.AreEqual("", verificationErrors.ToString()); } [Test] public void TheNewTest() {selenium.Open("/Home/");selenium.Click("link=About");selenium.WaitForPageToLoad("30000"); } }}49

70. Selenium BenefitsSelenium could be used for the functional, regression, load testing of the web based applications. The automation tool could be implemented for post release validation with continuous integration tools like Hudson or CruiseControl.C#, Java, Perl, PHP, Python, Ruby are being supported as the programming/testing platformSelenium allows simple and powerful Document Object Level (DOM) level testing It supports traditional waterfall or Agile environmentsSelenium is a pretty good tool for continuous integrationSelenium support AJAX50

71. Selenium LimitationsSetting up Selenium Test Suite is complex and step by step documentation is hard to findOnly supports web application though it has functional, regression, load testing featuresDiagnostics process is cumbersomeIn AJAX implementations, it doesn’t know when AJAX process is completed51

72. Selenium – Short Demo52

73. Visual Studio Web Test for UI TestingThe main purpose of Web UI testing is to uncover functional problems of a Web Application. But, there are some key challenges for Web UI Testing:Different Browser/Client Environment

74. Client Side Scripting/Security Settings/Validation

75. Session Variables and Timeouts53

76. How it works?The test harness system consists of three files (ASP .NET Page, Left.Html and Main.Html)ASP .NET Page is loaded into an HTML frameLeft.Html which contains the test automation JScript code is loaded into another HTML frameMain.Html container for the two frames and holds a global variable called timesAppLoaded which tracks how many times the Web application under test has been loaded into its test frameTest scenario script uses the timesAppLoaded value to determine exactly what actions to take on the Web app and then reloads the Web applicationThis Cycle continues until the system reaches a final state that you specify in the test script, and a pass or fail result is determined.Ref: Lightweight UI Test Automation for ASP.NET Web Apps, James McCaffrey, http://msdn.microsoft.com/en-us/magazine/cc163814.aspx#S154

77. Web Test Container – Main.Html<html> <head> <script language="JScript">vartimesAppLoaded = 0;varscenarioID = "001"; </script> </head> <frameset cols="40%,*"> <frame src="left.html" name="left"> <frame src="color.aspx" name="right" onload="left.updateState();"> </frameset></html>55

78. Visual Studio Web Test56

79. Adding Validation Rule57

80. Visual Studio Web Test Results58

81. Visual Studio Web UI Test BenefitsCreation of Test Cases for Multiple Browser Support and browsers cross-compatibility can be tested using this toolUse different session/data variables for the same test and uncover different feature set according to selected criteria/setting. Let’s say use Admin and Regular user loginsSupport for AJAX/SilverlightMay call and test other test cases(for an example DBUnit/Nunit )Convert Step to Code option allows testers to capture the outline of the test and dig into the code59

82. Visual Studio Web UI Test LimitationsNeed Visual Studio Team System to get the Web UI Testing PlatformSupports C#/VB platforms for unit testingWeb UI Test is designed for .NET platform60

83. MS Web Test – Short Demo61

84. Microsoft MVC .NET Test Suite62

85. Web Application TestingWeb portal applications are being developed using MVC pattern paradigm. MVC stands for Model-View-Controller where :Front End Application logics are being hosted in the “View” component

86. All Business Logics are being hosted in “Model” component

87. And Managed by “Controller” component63

88. MVC Patternhttp://msdn.microsoft.com/en-us/library/ms978748.aspx64

89. Web Portal Testing ConstraintsTesting results may vary from Dev, Test and Production environmentTesting results may vary for users with different security settings or grantsBlackbox testing is challenging as less control on the environment and html parsing is slowApplying TDD is difficult in web applications as the test cases are written to fail and then write the test case to pass followed by refactoring. So, needs a web portal mock up to test small changes time and again 65

90. MVC Project LayersAbstract LayerBusiness LayerData LayerMVC project 66

91. ASP .NET FrameworkModulesGlobalizationCachingControlsMaster PagesPagesRolesProfileMembershipHandlersIntrinsicsEtc.67

92. ASP .NET MVC FrameworkASP.NETDynamic DataPresentationASP.NETWebFormsASP.NETMVCASP.NETCoreRuntimehttp://www.asp.net/(S(d35rmemuuono1wvm1gsp2n45))/mvc/68

93. ASP.NET MVC Features (i)Separation of application tasks (input logic, business logic, and UI logic), testability, and test-driven development (TDD) by defaultAll core contracts in the MVC framework are interface-based and can be tested by using mock objects, which are simulated objects that imitate the behavior of actual objects in the applicationUnit-test applications without having to run the controllers in an ASP.NET process, which makes unit testing fast and flexibleAny unit-testing framework can be utilized that is compatible with the .NET Framework. 69

94. ASP.NET MVC Features (ii)An extensible and pluggable frameworkASP.NET MVC framework components are easily customizableThe ASP.NET MVC framework also supports the use of Dependency Injection (DI) and Inversion of Control (IOC) container models. DI allows you to inject objects into a class, instead of relying on the class to create the object itselfSupport for existing ASP.NET features. ASP.NET MVC lets you use features such as forms authentication and Windows authentication, URL authorization, membership and roles, output and data caching, session and profile state management, health monitoring, the configuration system, and the provider architecture. 70

95. Creating .NET Web App71

96. Creating a Unit Test Project72

97. Model-View-Controller (MVC)73

98. MVC Top Level Directories74

99. Typical MVC .NET Project Structure75

100. Change Password Method [Authorize] [HttpPost] public ActionResultChangePassword(ChangePasswordModel model) { if (ModelState.IsValid) { if (MembershipService.ChangePassword(User.Identity.Name, model.OldPassword, model.NewPassword)) { return RedirectToAction("ChangePasswordSuccess"); } else {ModelState.AddModelError("", "The current password is incorrect or the new password is invalid."); } } // If we got this far, something failed, redisplay form return View(model); }76

101. Change Password Test Method [TestMethod] public void ChangePassword_Post_ReturnsRedirectOnSuccess() { // ArrangeAccountController controller = GetAccountController();ChangePasswordModel model = new ChangePasswordModel() {OldPassword = "goodOldPassword",NewPassword = "goodNewPassword",ConfirmPassword = "goodNewPassword" }; // ActActionResult result = controller.ChangePassword(model); // AssertAssert.IsInstanceOfType(result, typeof(RedirectToRouteResult));RedirectToRouteResultredirectResult = (RedirectToRouteResult)result;Assert.AreEqual("ChangePasswordSuccess", redirectResult.RouteValues["action"]); }77

102. Right Click and Create Test Method78

103. Membership Service Mocking private class MockMembershipService : IMembershipService { public intMinPasswordLength { get { return 10; } } public boolValidateUser(string userName, string password) { return (userName == "someUser" && password == "goodPassword"); } public MembershipCreateStatusCreateUser(string userName, string password, string email) { if (userName == "duplicateUser") { return MembershipCreateStatus.DuplicateUserName; } // verify that values are what we expectedAssert.AreEqual("goodPassword", password);Assert.AreEqual("goodEmail", email); return MembershipCreateStatus.Success; } public boolChangePassword(string userName, string oldPassword, string newPassword) { return (userName == "someUser" && oldPassword == "goodOldPassword" && newPassword == "goodNewPassword"); } }79

104. Unit Test Results in .NET 3.580

105. ASP.NET MVC BenefitsResolving complexity by dividing an application into the model, the view, and the controllerNo view state or server-based forms. So, provides full control over the behavior of an applicationFront Controller pattern processes Web application requests through a single controllerCreating application that supports a rich routing infrastructureProvides better support for test-driven development (TDD)Fits into large teams of Developers and Web designers who need a high degree of control over the application behavior. 81

106. ASP.NET MVC LimitationsDoesn’t use event model to preserve state over HTTPDoesn’t use view state or server-based forms and so hard to manage statesDoesn’t fit well for small teams to implement RADComplex design for application development, because the components (the Page class, controls, and so on) are tightly integrated and usually require more code than the Traditional Web Application model82

107. MVC .Net – Short Demo83

108. Parameterized Unit Testing (PUT) and Microsoft Pex84

109. Unit Testing At a GlanceThree sections of Unit Tests:Data

110. Method Sequence

111. Assertionsvoid Add() {int item = 3;var list = new List();list.Add(item); var count = list.Count;Assert.AreEqual(1, count);}Ref : Parameterized Unit Testing: Principles, Techniques, and Applications in Practice, Nikolai Tillmann, Peli de Halleux, Wolfram Schulte (Microsoft Research) Tao Xie (North Carolina State University)http://research.microsoft.com/en-us/projects/pex/pexpublictutorialslides.pptx85

112. Incomplete Test Coverage with Fixed Datalist.Add(3); Bad data may result in redundant, or worse, incomplete test suites.

113. Fixed data values become invalid when product changes

114. Why choose a value which may not make sense?Ref : Parameterized Unit Testing: Principles, Techniques, and Applications in Practice, Nikolai Tillmann, Peli de Halleux, Wolfram Schulte (Microsoft Research) Tao Xie (North Carolina State University)http://research.microsoft.com/en-us/projects/pex/pexpublictutorialslides.pptx86

115. What is Parameterized Unit Test?Parameterized Unit Test is a Unit Test with ParametersSeparation of concernsData is generated by a toolDeveloper can focus on functional specificationvoid Add(List list, int item) {var count = list.Count;list.Add(item);Assert.AreEqual(count + 1, list.Count);}Ref : Parameterized Unit Testing: Principles, Techniques, and Applications in Practice, Nikolai Tillmann, Peli de Halleux, Wolfram Schulte (Microsoft Research) Tao Xie (North Carolina State University)http://research.microsoft.com/en-us/projects/pex/pexpublictutorialslides.pptx87

116. PUT SampleRef: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA88

117. Parameterized Unit Testing (PUT)Some popular unit testing tools, like Junit, Nunit doesn’t support the automation of creating unit testsIn some cases it even takes more lines of code to create the unit tests than more implementation being testedSome Automatic test generation tools may not even track the divide-by-zero type errors rather than specific error messagePUT or Parameterized Unit Test accepts behavioral parameters/argument sets for a specific test method Ref: Automating Software Testing Using Program Analysis, Patrice Godefroid, Peli de Halleux, Aditya V. Nori, Sriram K. Rajamani, Wolfram Schulte, and Nikolai Tillmann, Microsoft Research Michael Y. Levin, Microsoft Center for Software Excellence89

118. PUT ContributionsThey allow unit tests to play a greater role as specifications of program behavior. In fact, PUTs are axiomatic specifications.They enable automatic case analysis, which avoids writing implementation-specific unit tests.Their generated test cases often result in complete path coverage of the implementation, which amounts to a formal proof of the PUT’s assertions.Ref: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA90

119. PUT FrameworkSymbolic StateConstraintsSymbolic EvaluationAxiomsTest Case generationRef: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA91

120. Symbolic StateIts is basically the state of program execution. It can contain expressions with symbolic variables.Symbolic expressions: E = | o object ids, infinite set of potential object identifiers| v variables, set of symbolic variable identifiers| t types, set of type identifiers| f(E bar) function application, set of function symbols| For All (v bar).E universal quantification, where x to denote lists of items x1, . . . , xn.Function symbols: equals(x, y) denotes whether x and y represent the same value for value Types, type(x) denotes the runtime type of object x, and len(x) the length of array xHeaps:No of Times a method being called in a nested method callSymbolic state: A symbolic state is a 5-tuple S = (O, A, He,Hi, X), where the current set of objects O is a subset of ObjectId, the program stack A is a stack of activation records, He and Hi are expressions denoting the extensional heap and the intentional heap respectively, and finally, X, an object expression, denotes the current exception. Say: O(S) is being written for projection on SRef: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA92

121. Symbolic EvaluationSymbolic Evaluation describes the effect of the current instruction from a given constrained state (S,C).Ref: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA93

122. AxiomsAxiom is basically of summary of external behavior[TestAxiom]public void TestQuickSort(int[] a) {Assume.IsTrue(a != null); QuickSort.Sort(a, 0, a.Length - 1); for (int i = 0; i < a.Length - 1; i++)Assert.IsTrue(a[i] <= a[i + 1]); }Ref: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA94

123. Test Case GenerationEach transition sequence (S0,C0) ! (S1,C1) ! · · · represents a unique execution path of the program. [TestMethod]public void TestQuickSort597(){int[] ints = new int[2];ints[1] = 1;this.TestQuickSort(ints);} [TestMethod]public void TestQuickSort512(){int[] ints = new int[3];ints[0] = -2147475453;ints[1] = 1073750016;ints[2] = 8194;this.TestQuickSort(ints);}Ref: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA95

124. ConstraintsA constraint on a symbolic state is a pair C = (BG, PC), where BG is the static background, which only depends on the program declarations, and PC is the dynamic path condition, which is built up during symbolic evaluationRef: Parameterized Unit Tests, Nikolai Tillmann ([email protected]), Wolfram Schulte ([email protected]), Microsoft ResearchOne Microsoft Way, Redmond WA USA96

125. What is Pex?Pex stands for Program Exploration which is a white box test generation tool developed at Microsoft Research that helps developers to write PUTs in .NET platform and thus generate automated test casesFor each PUT, Pex uses dynamic test-generation techniques to compute a set of input values that exercise all the statements and assertions in the analyzed program which eventually cover all reachable branchesRef: Automating Software Testing Using Program Analysis, Patrice Godefroid, Peli de Halleux, Aditya V. Nori, Sriram K. Rajamani, Wolfram Schulte, and Nikolai Tillmann, Microsoft Research Michael Y. Levin, Microsoft Center for Software Excellence97

126. Pex Sample Project98

127. Sample Web Method to Test [ServicePoints.asmx]99

128. Pex Test Case: Test Axiom [ServicePointsTest.cs]100

129. Pex Test Method 1: Null Validation101

130. Pex Test Method 2: Validation with valid string value102

131. NUnit Test Case [TestFixture] public class UserTest { User _user; [SetUp] public void Setup() { _user = new User(); } [Test] public void EmailTest() { _user.Email = “[email protected]";Assert.AreEqual(“[email protected]", _user.Email); } [Test] public void IsValidTestWhenPasswordsMismatch() { _user.Password = “SENG"; _user.Password = “CPSC";Assert.IsFalse(_user.IsValid()); } [Test] public void IsValidTestWhenPasswordsMatch() { _user.FirstName = “Fazlul"; _user.LastName = “Chowdhury"; _user.Email = "[email protected] "; _user.Password = " SENG "; _user.PasswordConfirmation = " SENG ";Assert.IsTrue(_user.IsValid()); } [Test] public void UserNameTest() { _user.Email = "[email protected] ";Assert.AreEqual(" [email protected] ", _user.UserName); _user.UserName = "[email protected] ";Assert.AreEqual("[email protected] ", _user.Email); } }103

132. Instantiating PUT in PEX [PexMethod] public string ConvertToUpper([PexAssumeUnderTest]MyPexTest1Class target, string Input) { string result = target.ConvertToUpper(Input); return result; } [PexMethod] public void ConvertToUpperAgain(string Input) { MyPexTest1Class MyP = new MyPexTest1Class();var First = MyP.ConvertToUpper(Input);var Second = MyP.ConvertToUpper(First);Assert.AreEqual(First, Second); }104

133. PUT & PEX benefitsPUT specifies the external behavior of the methods for all test arguments and so the tests cover all possible test environment and constraintsPUTs can be instantiated to re-obtain traditional closed unit tests and to re-verify the result of testsPUT enables automatic case analysis, which avoids writing implementation-specific unit testsUsing PUT in PEX, it is fairly simple to auto generate test cases and cover all possible testing scenarios, constraints and values105

134. PUT & PEX limitationsPEX works for deterministic model of test methods and goes in cycle for non deterministic methods until hits exploration boundsPEX doesn’t work in Multi-Thread modelPEX can analyze any .NET language but the code generation only supports C#PEX uses an automatic constraint solver to determine which values are relevant for the test and the code-under-test but which is also limited [Say: floating point arithmetic cannot be reasoned precisely]Lack of Tests for OracleAdvanced Concepts : PUT in PEX, http://research.microsoft.com/en-us/projects/pex/pexconcepts.pdf 106

135. Microsoft Pex – Short Demo107

136. Load Testing108

137. Load Testing on Web ApplicationsTo perform a load test on a web application we need to:Identify the performance-critical scenarios

138. Identify the workload profile for distributing the entire load among the key scenarios

139. Identify the metrics that we want to collect in order to verify them against your performance objectives.

140. Design tests to simulate the load

141. Use tools to implement the load according to the designed tests, and capture the metrics

142. Analyze the metric data captured during the testBy using an iterative testing process, these steps should help us achieve our performance objectivesPerformance Testing Guidance for Web Applications, J.D. Meier, Carlos Farre, PrashantBansode, Scott Barber, and Dennis Reahttp://msdn.microsoft.com/en-us/library/bb924372.aspx109

143. Visual Studio Load Testing110

144. Visual Studio Load Testing Results111

145. VS Load Test – Short Demo112

146. Advantages of TDD centric Web Applications designCode will be separated from the Data layerEasy to understand and code transferIt increases the system performanceEasy to maintain documentation113

147. Conclusion (i)This research was done to investigate numerous testing tools in .NET platform. We found that various testing tools are specialized in specific testing contexts. NUnit/Visual Studio Testing Tools/MVC .NET and Microsoft PEX are built for unit testing where as DBUnit/DB Pro are specialized in Database Testing, Selenium and MS Web test are for Web UI Testing and VS Load Test and Wcat are being used for Load/Stress testing. But, Microsoft Team System Developer edition comes with a full blown test suite of which VS Test is for Unit Testing, DB pro is for database testing, Web Test for web UI testing and VS Load Test is for Load testing. External testing tools like DBUnit and Nunit test cases can also be generated from Visual Studio Testing suite. MVC .NET implements the Model, View and Controller pattern which makes TDD easier for a Web Application. MVC resolves the Server based forms which were difficult to test. MVC framework fits into large teams of Developers and Web designers who need a high degree of control over the application behavior. 114

148. Conclusion (ii)This presentation also highlighted the concept of Parameterized Unit Tests or simply PUT which is basically a generalization of established closed unit tests. Parameterized Unit Test or PUT has been implemented in Microsoft PEX to generate automated Unit tests. Behavioral parameters can be injected into PUT to cover a particular implementation. State Change, Return Values and Exceptional return values states the Method's behavior. Detailed demonstration of PEX covered the Axiomatic approach of Test Methods and it’s test coverage scenario together with automated and regression test generation.115

149. ReferencesEvaluating Current Testing Processes of Web-Portal Applications Link: http://ase.cpsc.ucalgary.ca/ase/uploads/Publications/bajwaMaurerXiong2005a.pdfMultiple Perspectives on Executable Acceptance Test-Driven Development Link: http://ase.cpsc.ucalgary.ca/ase/uploads/Publications/XP2007_Melnik_Maurer.pdfhttp://ase.cpsc.ucalgary.ca/ase/uploads/SENG515615W2010/Lecture4_Agile_Engineering_Practices.pdfhttp://elsmar.com/pdf_files/QC%20vs%20QA.pdfhttp://dotnetslackers.com/articles/aspnet/ASPNETMVCFrameworkPart2.aspxhttp://www.asp.net/learn/mvchttp://www.asp.net/(S(pdfrohu0ajmwt445fanvj2r3))/learn/mvc/tutorial-01-cs.aspxhttp://ase.cpsc.ucalgary.ca/ase/uploads/Publications/XiongMaurerBajwa2005b.pdfBuilding Web Apps without Web Forms, Link: http://msdn.microsoft.com/en-us/magazine/cc337884.aspxMVC architecture in ASP. Net using C# and Microsoft Data Access Application block By MunirShaikh February 28, 2006 , Link: http://www.c-sharpcorner.com/UploadFile/munnamax/mvcarchitecture02262006091052AM/mvcarchitecture.aspxhttp://www.agiledata.org/essays/tdd.htmlhttp://msdn.microsoft.com/en-us/library/ms978748.aspxhttp://onjava.com/pub/a/onjava/2004/01/21/dbunit.htmlhttp://msdn.microsoft.com/en-us/library/bb381703(VS.80).aspxGood and Practical Ideas for Testers ,http://www.tassq.org/quarterly/docs/tassq_magazine-0609.pdfLow-Level Web App UI Test Automation, James McCaffrey http://msdn.microsoft.com/en-us/magazine/cc163723.aspxAdvanced Concepts : PUT in PEX, http://research.microsoft.com/en-us/projects/pex/pexconcepts.pdf 116

150. Thank You117

Evaluating Test Driven Development And Parameterized Unit Testing In Dot Net Presentation

More Related Content

What's hot (16)

Similar to Evaluating Test Driven Development And Parameterized Unit Testing In Dot Net Presentation (20)

Recently uploaded (20)

Evaluating Test Driven Development And Parameterized Unit Testing In Dot Net Presentation