Title Page
Usability Report. | New York Times Reader site |
Due date of Report: | 15.12.2009 |
Actual submission date: | 14 15.12.2009 |
Revised version: | 14 15.12.2009 (final) |
Product name and version: | New York times reader Site |
Organisers of the test: | Name, Name Kumud Sharma |
Date of the test: | dd 10.mm12.yyyy 2009 |
Date of the report: | dd 15.mm12.yyyy 2009 |
Editor: | Name Dinesh Sapkota |
Contact name(s): | Name and e-mail| Kumuds@metropolia.fi |
...
Executive summary
Provide a brief level overview of the test (including purpose of test)
Name the product:Name the product:The Name of the product is New York Times Site. You could have placed the link ibto here.
Purpose/objectives of the test:The main objective of the test is to know the easiness using the navigations in the site.
Method 1:The method is to open the site and work on it.
Number and type of participants:
Tasks (if task are used):
Method 2:The numbers of participants were 10 and the participants were students.
Results in main points
e.g. bullet list (this is needed for being able to get the main results without reading the full report, this is seen important, since the reports serve different purposes and sometimes the need is to get a fast overview) The main result of the test were:
1.The site is complex and includes so many matters.
2.There is not so easy navigation for small matters.If you want to get something like music, the tabs are smaller and its hard to focus when you are using it for the first time.
3.The site includes all the elements of the society.Thus there is some problems in arrangements of items.It can diminish the great outlook of the website.
...
Introduction
Full Product Description
...
1. Describe what parts of the product were evaluated
In the website,we evaluated the navigation system of the website http://www.nytimes.com/.
2.The user population for which the product is intended
The User population cant be estimated correctly because new york times is read by people all around the world.Everyday billions of people use new york times.So the user population is over billions.
3.Brief description of the environment in which it should be used
...
The Website is used in offices,school,houses,in restaurant and all those places which wants the latest world news.
Test Objectives
1.State the objectives for the test and any areas of specific interest
...
The objective for the test was to get the information how the users feels about the website.How the users find it to use.Do they like the color of the site etc.The main object in the website is the navigation.If you click in some links in the site,it takes to another page.This is the navigation and signifies how effective the website is.
2.Reason for focusing on a product subset
The main reason is that navigation of the site gives how the site works.The color of the site gives how attractive is the site.
Method
Participants
The total number of participants tested
...
was 10.The user group was student but with different feelings and choices.
1.Key characteristics and capabilities of user group
...
The key characteristics of this user group is that they study much and all sectors of the website.Students need the information of all sector of the society.So they are creative,investigator and they are regular reader.They can read new york times many times a day. Where do they read?, what are their needs and preferences e.g. easy navogation, long stories, particular lay out preferences etc.
2.How participants were selected; whether they had the essential characteristics
The participants were selected from the library.But we did not select them with some special characteristics.They are just selected because they are reader.
3.Differences between the participant sample and the user population
There are some differences between the participant sample and the user population. The sample is ideal but the user group can't be. The sample is the regular reader but our user population cant be regular reader always. User population gives the rough results of the success of New York Times. They can't be always correct but we take average result from the user population.
You understood the point wrong it means that those people that tested the site are your sample and how they differ from the actual new yor times readers. Since your selection was random, it is likely that the people in the test differ a lot of those that are actual regular New york Times readers. Here you should have stated the difference.
Context of Product Use in the Test
1.Any known differences between the evaluated context and the expected context of use
There is not any known difference between the evaluated context and the expected context of the use because the site using is same process.open the browser and then go through it. But where physically the test was done and how this differs to the physical context of the actual real readers?
2.Tasks
...
The main task of the user is to go through the navigation system of the site.How they find it,do they get it easier or difficult?
3.
...
Explain why these tasks were selected
The task was selected to know more about the site.When the site is known more,then the problems and shortcoming of the site are well known.As per the website of new york times, we asked the users to test with the navigations and how well it goes to another page.When page opens,is it easier to find what is he looking for.If so,what problems did he noticed?These were some criteria for selecting the tasks.
The task cannot be to know more about the site if the users were regular readers as you state, i.e., then they would already know the site. to ask if they find what they are looking for you shouild first know what they were looking for and you did not have in the description of the user characteristics about what they usually would be looking for. And what you describe above does not answer why you chose navigation as the main task.
4.Describe the source of these tasks
The main source of these tasks is New York Times website http://www.nytimes.com/. no no, the selected tasks shopuld be the important ones for the users, you test if what is the important things the users want to do in the site easy etc.
5.Include any task data/information given to the participants
The participants were not given any data.They were only given the website and to go through the task. So you gave them information about the task they had to perform? This is information and you should have explainef which ay this information was given to the users in the test.
6.Completion or performance criteria established for each task
When the participants complete the task,then they say to us.This is the completion criteria.
Test Facility
1.Describe the setting, and type of space in which the evaluation was conducted
The test was performed in the café in the school.There were enough spaces to sit.We also performed test in the smoking area with some students.
2.Detail any relevant features or circumstances, which could affect the results (e.g. There was a brake down of the server, which messed up the test for a while and created unnecessary tension. There was unforeseeable noise that disturbed the test, etc.)
As the testing was outside the room,there were much noise which affected our testing process.Students were in hurry,not enough time was also another circumstance which affect the result.The test was not long so that it was well organised.So it was not boring to any participants.
What you state above has quite a lot of effect on the test, you should state how you take this into concideration when analysing your results.
Participant's Computing Environment
1.Computer configuration, including model, OS version, settings
...
We had five computers for the participants.They first go through the site and then write what they feel about the site in the questionaire we provided.Three of the computers were acer and two of them were macbook pro. System versions?
2.Browser name and version;
The browser we used were safari,mozilla firefox,opera etc. Versions of the browsers?
3.Relevant plug-in names and versions
The plugins were accepted in the browsers.Browsers don't respond if the plugins and pop ups are not accepted.The download helper was installed in the browsers. And these plugins were?
4.
Display Devices(report if relevant, e.g., Paper prototypes are tested or static prototypes are tested on screen)
a.If screen-based, screen size, resolution, and colour setting
...
The prototypes were screen based.The screen size was 13 inches.The resolution was 60 % and the color was Lcd color.
Test Administrator Tools(report if relevant for the particular test)
1.If a questionnaire was used, describe or specify it here (add these to appendix)
The questionaire was used to collect the data.The paper was used and also online ways were used.When the user go through the sites,then they just write what they feel bad or good in the site in the word document.
2.Describe any hardware or software used to control the test or to record data (audio, video)
We did not used any hardware tools.
Experimental Design
1.Define independent variables and control variables
In the experiment, the independent variables were the color of the site and navigation texts.But the control variables which can be controlled during the experiment is the navigation link which the participant can choose or we can control on their decisions. No this means if you had e.g. a control group to be able to compare the tester group and control group. You did not have this kind of setting.
2.Describe the measures for which data were recorded (the scale/scope of the recorded data, if relevant for the particular test, i.e., written notes, think aloud in audio recording, etc.).
The recorded data which we collected from the test can be used in some test again which concerns about the testing of the site based on the navigation.How the website of the news paper should be and what is the criteria which the user likes it is the main outcome of the test.So the data can be safely kept for further purposes and can be used as reference. So you wrote notes? or you used only the questionnaires, about the questionnaires, you should state what the questions concentrated on, what was expected to get as an answer etc.
Procedure
1.Operational definitions of measures
...
The Task is completed which we were working on.The result of the task which we were looking for came as we expected.The participants worked and helped us in the process.We recorded the data in the computer as well as in the papers. Did you really recor the data, was it audio or video?
Policies and procedures for interaction between tester(s) and subjects (e.g., is the test conductor aloud to answer questions of the user, provide help, etc.)
There were many ways of interaction between the testers and the subjects.We provided the guideline using the websites and about filling the questionaire.We provided support with other news site as well. If you provided that much help, it has to be taken into account in the analysing of the results. Previsouly you mentioned that you did not give any information and now you state that you provided a lot of information. very unclear what you actually did.
2.State used: non-disclosure agreements, form completion, warm-ups, pre-task training, and debriefing
For the process,we gave a brief information what the users do in the test.The test was not long so we did not used much time of the participants but we helped them if they need any kind of helps regarding using of the site and then writing down in the computers and papers.You did not have non-disclosure agreement etc. These are papers that the test persons sign that they agree on doing the test and that they undestand that their names and information will not be publicly available.
3.Specific steps followed to execute the test sessions and record data
We did not do all the testing at same time but we did it in different time perioud and in different location in café and one time in smoking area.We used five computers at a time and then we asked them to do as per we had given the information in the initial of the test.
3.Number and roles of people who interacted with the participants during the test session
No other people interacted with the participants during the test session because it can take more time and it can disturb the participants.
4.Specify if other individuals were present in the test environment
No other individuals were present in the test.There were some other people in café but with no concern to us.
5.State whether participants were paid
The participants were not paid for the test.
Participant General Instructions (here or in Appendix)
1.Instructions given to the participants
...
- Metrics for efficiency
- Metrics for satisfaction, etc.
The participants can open other news sites only if needed.He can finish the task as fast as he can.But the participant should fill all the questions.They should go carefully with some links and navigation in the site and note it.
You still ahve not described what the task was? it seems that you had predefined task and not only that the users browse around
2.Task instruction summary
The task is to get the way how the navigation of the site is.In the home page and other pages where the navigations are,they are not magnified enough to notice them at once.As you go inside the pages,it can make it more difficult to find what you want.So the main instruction to the task is to go through the site and watch carefully and note it down.The task includes clicking the navigation link and then exploring inside to find out some links or news.
Result
1.Data Analysis
The result was analysed by all of us sitting together at one place.The data was successfully collected with no losses in the process.
2.Quantitative data analysis
This means how well data was collected.The new york times was liked by 6 of the 10 people using it every day.The participants were regular users. This means did you count percentages, correlations etc quantative manners to process the data. So your description is not about what should be here
3.Qualitative data analysis
The result was in favour of the site.The users liked the site because everything is there in the site.News are updated as soon as they occurs. They liked lots of navigations in the homepage but they did not liked that the texts are smaller .Too much navigation can makes a confusion.
Here you should have stated manners how you qualitatively analysed your data, i.e., you discussed it together, grouped it? or something like that?
4.Presentation of the Results
a.From quantitative data analysis
From the quantitative point of view, the data gives positive result.6 out of 10 likes the navigation and the page structure and contents.They liked background color.2 of them liked the search engine additionally.2 of them pointed on the reliability of news.1 of them did not liked the structure of the text arranged.
b.From
Results
...
qualitative data analysis (descriptive and clarifying presentation of the results)
From the qualitative analysis,the data gives the result that new york times has good navigation and the out look is nice compared to other sites like bbc and cnn.
Reliability and Validity
Reliability is the question of whether one would get the same result if the test were to be repeated.
This is hard to acquire in usability tests, but it can be reasoned how significant the findings are.
(Example from expert evaluation: This review was made by one reviewer in order to give quick feedback to the development team. To get more reliable results it would have been desirable to use three, or at least two reviewers, as it is often the case that different reviewers look at different things. We do feel, however, that for the purpose of this report, and the essence of quick feedback, one reviewer has given enough feedback to enhance the usability of the system.)
Validity is the question of whether the usability test measured what was thought it would measure, i.e., provide answers to. Typical validity problems involve: using the wrong users, giving them the wrong tasks.
(Example from expert evaluation: "The reviewer is an experienced usability professional that has evaluated systems for many years. We therefore feel that the method used as well as the tasks used give an appropriate view of how ordinary users would behave in the system.")
Summary Appendices
The reliability of the test is important because if the data is used in some valuable places and important occasion,the data should not be faked and should come almost true.The user can vary but the nature of collected should be similar in nature.The questionaire should be correct and the results should be stored in the correct manner.The result should be analysed in the correct way.The data when collected should be valid for some span of time because it can be used in some test similar in nature.
Here you shpould write how your tasks were valid, how your analysis methods fitted the wanted achievements etc.
Summary Appendices
Participant General Instructions
The participant general instruction are as followed:
1.The participants are are asked to answer the questionaire in 1 line.
2.They can take maximum 10 minutes for the test.
3.They cant use their own computer because the plugins and speed can be slow in different computers.
4.When the participants finish the task, they can consult us to wrap up the task.
Thank you
where is the questionnaire? and the task descriptions you provided to the users in the test?
...