...
Reliability
The tests can be seen as quite realiable. Pilot testing was done with user who has no prior experience about Tuubi but has been using similar software and the first genuine test with unexperienced Tuubi-user showed similar results as in pilot testing.
Most of the problems encountered with piot testing were due to incompatible browsers (IE and Firefox running on Metropolias computers). In the final tests up to date-version of IE was used.
Validity
Test users were familiar with IE and Windows. Other had been using Tuubi for few months mostly to read announcements and the other had broader and longer experience about Tuubi knowing most of the functions of Tuubi. Finding user who's know-how of Tuubi would have been somewhere between these two would have perhaps provided more information about the learning curve and which are the biggest pain points in Tuubi.
...