Testing writing
Download as PPTX, PDF6 likes3,796 views
This document discusses issues related to testing writing ability. It emphasizes the importance of ensuring tests are representative, valid, and reliably scored. Representative tasks should sample all possible content domains. Multiple tasks increase validity by providing more opportunities to demonstrate skills. Scoring should be either holistic, considering overall impression, or analytic, separating different skill aspects. Proper scorer training and calibration are needed to reliably rate performance. The goals are to accurately measure writing as an independent skill while maintaining practicality and meaningfulness.
Testing writing
- 1. TESTING WRITING Carolina Salazar Testing and Evaluation
- 2. THE TESTING PROBLEM REPRESENTATIVE VALIDLY SAMPLED VALIDLY AND RELIABLY SCORED REPRESENTATIVE
- 4. SPECIFICATION OF ALL THE POSSIBLE CONTENT Operations Types of text Adressees Length of texts Topics Dialect Style Expressing, directing, describing… Form, letter, fax, note… Boss, friends, family… 1 page, 150 words… Not specialist, relevant… Standard Formal, informal…
- 5. REPRESENTATIVE SAMPLE OF THE SPECIFIED CONTENT Try to include a representative sample of tasks MORE TASKS MORE VALIDITY
- 6. EXAMPLES
- 7. CCSE (CAMBRIDGE CERTIFICATES IN COMMUNICATIVE SKILLS IN ENGLISH) FIRST STAGE: READING TOPIC: Working in a Summer Camp for Children in America
- 8. TASK 1 Write a letter to American Summer Camps. Find out about Dates Hours of work Type of accomodation Ask for the application form
- 9. TASK 2 Fill in the application form.
- 10. TASK 3 You are now working in the ASCFC. You write a postcard to an English-speaking friend On your postcard tell your friend: Where you are Why you are there Two things you like about the summer camp
- 11. TASK 4 You have arranged to go out tonight with two other Helpers at the ASCFC in Florida. You have to change your plans suddenly, and cannot meet them. Leave them a note. Apologize and explain why you cannot meet them Suggest a different day to go out
- 13. THE TESTING PROBLEM REPRESENTATIVE VALIDLY SAMPLED VALIDLY AND RELIABLY SCORED VALIDLY SAMPLED
- 14. VALID SAMPLE OF WRITING ABILITY
- 15. SET AS MANY SEPARATE TASKS AS FEASIBLE People’s performance Consistent Offer them more opportunities to start Desirable Practical
- 16. TEST ONLY WRITING, NOTHING ELSE NO YES
- 17. TEST ONLY WRITING, NOTHING ELSE READING ILLUSTRATIONS SERIES OF PICTURES GRAPHICS – CHARTS NOTES
- 18. THE TESTING PROBLEM REPRESENTATIVE VALIDLY SAMPLED VALIDLY AND RELIABLY SCORED VALIDLY AND RELIABLY SCORED
- 19. VALID AND RELIABLE SCORING
- 20. ENSURE VALID AND RELIABLE SCORING Set tasks which can be reliably scored Set as many tasks as posible Restrict candidates Give no choice of tasks Ensure long enough samples Create appropriate scales for scoring
- 21. HOLISTIC SCORING “Impressionistic” Overall impression Purpose of the test PRO Very rapid CON Scorer reliability? http://www.ets.org/toefl/pbt/scores/writing_score_guide/
- 22. ANALYTIC SCORING http://www.ode.state.or.us/wma/teachlearn/testing/scoring/gui des/2011-12/wriscorguide_eng_no-dates.pdf Separate score – separate aspects PROS Uneven development of subskills Scorers consider details of performance More reliable CONS Time consuming Too specific (The whole is greater than the sum of its parts)
- 23. HOLISTIC OR ANALYTIC? PURPOSE CIRCUMSTANCES ACCURACY RESOURCES DECISIONMULTIPLE SCORING
- 24. MORE CONSIDERATIONS ON SCORING
- 25. SOME CONSIDERATIONS Calibrate the scale Collect samples of performance Cover the range of scales Select and train scorers Native speakers Sensitive to language Experienced Training (3 stages) Follow acceptable scoring procedures Score each task independently Several scorers Identify discrepancies Give feedback Useful – positive Decide the content during callibration
- 26. CONCLUSIONS
- 27. CONCLUSIONS Pay attention to validity and reliability Give importance to writing as an independent skill Always be meaningful and enhance real communication