After you have completed the pilot test do the following:
1. Write into your task 2, i.e. your testing plan:
- When and where the test was executed
- What you have to correct in your testing plan. e.g., your task descriptions were not clear, your time estimation of how long the test takes was not correct, you did not have the right dbriefing questions, you did not get the information that you thought you would from the test.
- Then correct your plan (state what colour you will use for the corrections so that i find them)
- State when and where will the actual testing take place.
Planning Report
• The goal of the test: what do you want to achieve?We wan't to learn what development needs exists to make using Tuubi more practical and find missing functions. What I try to explain is: The basic functions of Tuubi are probably pretty much fine, but what should be changed or developed
Study the following:
- ease of learning
- retention
- efficiency
- reliability
- interaction (between usr and the system)
- user satisfaction
- user convenience
- necessity for proficiency
- importance of accuracy
These criteria are good move them also to the task 1 - thats where they derived from right
MB (19.11.2009) Good you moved them, now you have to select the main ones, you cannot test all of them, there is too much to test. So some of these are more relevant for users than the others.
• Where and when will the test take place?
Test will be take place at lab-room some time there is no class in session.
• How long is each test session expected to take?
20 60 minutes.?
• What computer support will be needed for the test?
No computer support will be need. If test is conducted on a OS that is not familiar with the test person, a briefing of the common features will be given and necessary programs will be started.
• What software needs to be ready for the test?
Web browser, word processor, text editor, program to open PDF-files.
• What should the state of the system be at the start of the test?User has been logged to Tuubi (with testers account so that we know what to expect)
We will make a workspace in Tuubi to simulate a real course. User will start with Tuubi-login page.
• What should the system/network load and What should the system/network load and response times be? (not too fast or too slow);
• Who will serve as experimenters for the test?
Admin: Wedha
Person who gives tasks: Simone
Timing and note taking: Mikko
• Who are the users going to be and how are we going to get hold of them?
Maybe first year students.
• How many users are needed?
1-3 users. One / test.
• What test tasks will the users be asked to perform?
- Subscribe to Usability testing-workspace in Tuubi. (User has the workspace on his/her General-workpsaces)
- Go to workspace Usability testing and read the description about the usability testing from the frontpage of workspace. (User finds and can open the description)
- Find and download file named Usability_document.doc and open it on the test computer. (user is reading the content of the file on the computer)
- Open assignment "Usability testing 1" and do as you have been told in the assigment (in the assignment test person will be asked to upload a file from the desktop as submitting an assignment) (user has given his/her answer by submitting a file)
- Open assignment "Usability testing 2" and do as you have been told in the assigment (in the assignment test person will be asked to give feedback about the assignment as a text) (user has given his/her answer by submitting text)
- Read feedback for the assigment (user sees the grade and comment on his/her assignment)
- Look up the date for the final assignment from calendar (can see the final assignment on the calendar)
- Show how to see the people who are described to the same workspace (can see other members of this worspace)
- Reply to a message on the discussion board. (There is a reply from user to Test-message)
- Subscribe to Random-workspace in Tuubi. (User has the workspace on his/her General-workpsaces)This is to test the retention.
You need to open these tasks, i.e., how will you prezsent them to the user. And you need some kind of scenario why the user is performing those taks so that it comes meaningful.
• What criteria will be used to determine when the users have finished each of the test tasks correctly?
Criteria can be found within the tasks.
Yes, but what is succesful? In login it is getting into the Tuubi and seeing the "front page" of Tuubi, so the sucess criteria is needed here.
• What user aids will be made available to the test users (manuals, online help, etc)?
We will explain that we are not interested in the user manual/documentation.
Help document of Tuubi (in Finnish). Are you sure you want totake this? Is it probabale that the yoru user segment would use the help document? of not do n ot allow it to be used in the test. If you allow you need to take into account the effect of using the manual in the test when you evaluate the results.
• To what extend will the experimenter be allowed to help the users during the test?
Translation help, help to set up the computer.
• What data is going to be collected and how will it be analysed?
Notes with timing.
Its about finished, just some finetuning and you are ready for the pilot test!