After you have completed the pilot test do the following:
1. Write into your task 2, i.e. your testing plan:
- When and where the test was executed
- What you have to correct in your testing plan. e.g., your task descriptions were not clear, your time estimation of how long the test takes was not correct, you did not have the right dbriefing questions, you did not get the information that you thought you would from the test.
- Then correct your plan (state what colour you will use for the corrections so that i find them)
- State when and where will the actual testing take place.
Planning Report
• The goal of the test: what do you want to achieve?
Study the following:
- ease of learning
- retention
- efficiency
- reliability
- interaction (between usr and the system)
- user satisfaction
- user convenience
- necessity for proficiency
- importance of accuracy
These criteria are good move them also to the task 1 - thats where they derived from right
MB (19.11.2009) Good you moved them, now you have to select the main ones, you cannot test all of them, there is too much to test. So some of these are more relevant for users than the others.
• Where and when will the test take place?
...
• What should the system/network load and What should the system/network load and response times be? (not too fast or too slow);
• Who will serve as experimenters for the test?
Admin: Wedha
Person who gives tasks: Simone
Timing and note taking: Mikko
• Who are the users going to be and how are we going to get hold of them?
Maybe first year students.
• How many users are needed?More than one
person. Maybe several users together so they woud interact with each other1-3 users. One / test.
• What test tasks will the users be asked to perform?
- Sees the courses (workspacesSubscribe to Usability testing-workspace in Tuubi. (User has the workspace on his/her General-workpsaces)
- Go to workspace Usability testing and read the description about the usability testing from the frontpage of workspace. Can download documents (slides, tasks, materials) and open external links (User finds and can open the description)
- Find and download file named exerciseUsability_document.doc and open it on the test computer. Can submit an assignment (user is reading the content of the file on the computer)
- Open assignment "Usability testing 1" and do as you have been told in the assigment (in the assignment test person will be asked to upload a file from the desktop as submitting an assignment)
- Receive feedback
Can we test this? - ) (user has given his/her answer by submitting a file)
- Open assignment "Usability testing 2" and do as you have been told in the assigment (in the assignment test person will be asked to give feedback about the assignment as a text) (user has given his/her answer by submitting text)
- Read feedback for the assigment (user sees the grade and comment on his/her assignment)See events in the calendar
- Look up the date for the final assignment from calendar (this can be in a file containing explanation of this test-course and as an event in can see the final assignment on the calendar)
- Browse the members of group (like classmates)
Show how to see the people who are taking this same course.described to the same workspace (can see other members of this worspace) - Reply to a message Have a discussion on the discussion board
Send a message to the discussion board. (There is a reply from user to Test-message) - Subscribe to Random-workspace in Tuubi. (User has the workspace on his/her General-workpsaces)This is to test the retention.
You need to open these tasks, i.e., how will you prezsent them to the user. And you need some kind of scenario why the user is performing those taks so that it comes meaningful.
• What criteria will be used to determine when the users have finished each of the test tasks correctly?
Succesfull, fail, succesfull but user needs help, succesfull but addition from user Criteria can be found within the tasks.
Yes, but what is succesful? In login it is getting into the Tuubi and seeing the "front page" of Tuubi, so the sucess criteria is needed here.
• What user aids will be made available to the test users (manuals, online help, etc)?
We will explain that we are not interested in the user manual/documentation.
Help document of Tuubi (in Finnish). Are you sure you want totake this? Is it probabale that the yoru user segment would use the help document? of not do n ot allow it to be used in the test. If you allow you need to take into account the effect of using the manual in the test when you evaluate the results.
• To what extend will the experimenter be allowed to help the users during the test?
Translation help, help to set up the computer.
• What data is going to be collected and how will it be analysed?
Notes with timing.
Its about finished, just some finetuning and you are ready for the pilot test!