...
Reliability
The tests can be seen as quite realiable. Pilot testing was done with user who has no prior experience about Tuubi but has been using similar software and the first genuine test with unexperienced Tuubi-user showed similar results as in pilot testing.
Most of the problems encountered with piot testing were due to incompatible browsers (IE and Firefox running on Metropolias computers). In the final tests up to date-version of IE was used. Also
Validity
Test users were familiar with IE and Windows. Other had been using Tuubi for few months mostly to read announcements and the other had broader and longer experience about Tuubi knowing most of the functions of Tuubi. Finding user who's know-how of Tuubi would have been somewhere between these two would have perhaps provided more information about the learning curve and which are the biggest pain points in Tuubi.Reliability is the question of whether one would get the same result if the test were to be repeated.
This is hard to acquire in usability tests, but it can be reasoned how significant the findings are. (Example from expert evaluation: This review was made by one reviewer in order to give quick feedback to the development team. To get more reliable results it would have been desirable to use three, or at least two reviewers, as it is often the case that different reviewers look at different things. We do feel, however, that for the purpose of this report, and the essence of quick feedback, one reviewer has given enough feedback to enhance the usability of the system.)
Validity is the question of whether the usability test measured what was thought it would measure, i.e., provide answers to. Typical validity problems involve: using the wrong users, giving them the wrong tasks.
(Example from expert evaluation: "The reviewer is an experienced usability professional that has evaluated systems for many years. We therefore feel that the method used as well as the tasks used give an appropriate view of how ordinary users would behave in the system.")
Summary Appendices
Notes of the pilot test and improvements made to final test
...