Design Tutorial Submission Requirements

2019 UXD – Assignment 1 Part 2 Usability testing – write-up guide 03 21.docx Page 1 of 5
User Experience Design Tutorial Submission Requirements –
Assignment 1 part 2 – Usability Test Report
Include a cover page with group member’s names, ID’s, and tute session day / time
Submit an electronic .pdf copy to the Canvas website –
due 11:59pm Saturday 4 May (end week 8).
Carefully follow the instructions for submission posted in Canvas.
Assignment writing guidelines: Emphasise referencing /formatting this time
• Write this assignment up as a stand-alone report.
• It should make sense to someone who did not sit through the tutorial exercises.
• Provide an explanation of the purpose for each section (other than Exec Summary and
Introduction) to show us your understanding of all activities you’ve undertaken.
• Report style requirements:
o table of contents
o page numbers
o clear headings
o references – preferably in Harvard style (in-text and bibliography)
– see library resources on citing and referencing if you are unsure.
o figure numbers and descriptions for any images, graphs and tables you use
Note that the guidelines listed here are intended as a template for writing a usability test report. They
may be a bit over-the-top in terms of detail for a small-scale test of the type you conducted. If
unsure, check with your tutor to see what is appropriate for your particular write-up, but as a rule you
will be rewarded for including extra detail (as long as it is justified to the reader).
The report should ideally contain the following elements:
• Executive summary (10% of Assignment mark)
– max one page –summarising important aspects of the entire report – including significant
results / recommendations – should be a quick reference guide or “spoiler” to the document.
• Introduction (5% of Assignment mark)
o briefly describe the interactive / app you are testing, and (if applicable) the particular
section you are focusing on (feel free to recycle your work from the feature audit to help
describe the interactive).
o indicate the aims/rationale for testing – why it is being done – what you hope to find out.
(You can use license here, feel free to invoke a scenario in which you have been
commissioned by the owner of the interactive to re-design or test it).
o provide a summary of the target audience and outline the characteristics of the key-user
types (there may be more than one category – e.g. end-users and site administrators).
You can re-cycle your persona(e) from part 1 here – feel free to update if you have
learned more. Don’t forget to include explanation of what a persona is – why it is used.
o identify the personnel involved in the testing and their roles
o summarise the methodology / equipment / technology / materials used.
o note the location(s) and date(s) testing was carried out.
2019 UXD – Assignment 1 Part 2 Usability testing – write-up guide 03 21.docx Page 2 of 5
• Test script (5% of Assignment mark)
Explain to the reader why a formal script (read to participants) was required.
You can include the script in the body of the report if it is short, but this would usually be
included in an appendix. There would typically be an introductory section to the script, as well
as specific instructions associated with each task, and a script for any concluding comments.
Task-specific sections of the script can be included with the detailed description of each task
(see below). The script should:
o explain purpose of the test.
o indicate what will be expected of participants.
o alert participants of your intentions to record (screen/audio/video) with their permission.
o put the participants at ease with the process.
o explain how you will deal with privacy of information etc.
• Task design (10% of Assignment mark)
Most tests will need to be broken up into several distinct tasks. These could be anything from
locating a piece of information, completing an online purchase, being able to navigate through
a required sequence etc. In some cases it can be useful to time how long it takes for
participants to achieve these objectives – this can be an unbiased measure that can be used
for comparison after any changes made in response to the testing.
The purpose and objective of each test task should be clearly identified.
o Give an overview of the tasks you have chosen to test, and rationale for why they were
chosen (remember tasks should be chosen to exercise core and typical functionality).
Make sure the reader understands the objective of each task.
o For each task:
State (briefly) the purpose / objective(s) of the task
Describe the scenario you have created for the test participants and any assumptions
you have asked them (or expected them) to make in this scenario
briefly outline the steps required to complete the task, and estimated time required.
Summarise the instructions given to participants and what other info they were given.
Include the section of formal test script read to participants during the task.
Include details of any task cards that were used.
Make sure the purpose of task cards are explained to the reader of the report.
The actual task card details are probably best included in an appendix – but
summarise the intent of each task card in the main body of the report (You don’t need
to submit the physical cards, but the contents and design of the cards should be
clearly set out).
o Comment on the significance of the order of the tasks.
Are you randomising task order? If so explain why.
o Indicate estimated time required to complete each task, and therefore the entire test.
It is important that you have a sound basis for estimating how long participants are going
to take to complete the test. You should have conducted a pilot test to help make this
estimate.
Also estimate the turn-around time between tests so that the time required for multiple
test subjects can be estimated
Make sure what you write is understandable to a lay-person
2019 UXD – Assignment 1 Part 2 Usability testing – write-up guide 03 21.docx Page 3 of 5
• Information about actual test participants: (10% of Assignment mark)
It is common to use questionnaires to determine details such as age, gender, familiarity with
computers / internet, web browsing platform percentages (% mobile / % tablet / % desktop),
social media use/preferences etc.
Include details of questionnaires given to participants in an appendix. These typically include:
o a questionnaire to obtain demographic and other information about participants (ie
degree of familiarity with the web, what sort of sites visited / games played / purchases
made – how often etc). Include a brief rationale for why this information is useful.
o a post-test questionnaire to gather extra info from participants about their experience (Eg:
did they like it? Would they use it in real life? What would they change about it etc).
o any questions asked during tests to generate a better understanding of your participants
or the client app including a participant’s first impression of an app/site/game being tested
before administering a task.
Try to avoid asking participants to classify themselves using subjective criteria (eg “would you
regard your internet skills as beginner, intermediate or advanced”). Instead try to use more
objective, quantitative measures such as “indicate the number of hours per week on average
that you browse the internet” (break this down according to work / personal / mobile / tablet /
desktop). Another useful question is something like: “how regularly do you make purchases /
use services such as this in your typical life (give options)”.
Don’t forget to include consent forms and other instruments you may have used, along with a
scan of completed questionnaires in an appendix.
• Results (10% of Assignment mark)
– summarise the major issues that were discovered.
Organise issues under headings that describe related themes (If you have identified a
number of issues, Affinity Diagraming may help with this process. Include any affinity
diagram(s) in an appendix)
o use annotated screenshots to make clear what you are describing
o point out the impact that these issues have on the usability of the interactive.
Rate these issues by severity. Emphasise any “dealbreaker” issues.
o if applicable, describe how any audio/video evidence corroborates your
observations – especially regarding the impact of these issues on the test participants,
their state of mind when things don’t go as expected etc. You might include a link to a
YouTube video or similar (if you do this make sure videos are private and participants
are de-identified.

o Consider including one or more photographs of the testing process to make it more
tangible to the client.
2019 UXD – Assignment 1 Part 2 Usability testing – write-up guide 03 21.docx Page 4 of 5
• Recommendations (10% of Assignment mark)
–suggest how any usability issues that have been discovered might be rectified. You don’t
need to go into too much written detail, but preferably use annotated screenshots to describe
how the problem might be fixed (eg in a before-after type diagram). The annotated
screenshots (showing your suggested changes) can become the basis of the paper
prototype you will have utilised to test the proposed changes (see below).
Note – if your testing has identified more than one or two issues, it would make sense to
list recommendations immediately after each issue is described under the results section:
– result 1
– recommendation 1
– result 2
– recommendation 2
….
If you do this it is also worthwhile summarising all of the recommendations in a separate
recommendations section.
• Validating your Recommendations (20% of Assignment mark)
You will have created a paper prototype The purpose of the prototype is to represent your
recommendations visually. It could be as simple as a wireframe diagram of the relevant
sections of the site.
By running a short usability test on the prototype you will hopefully have obtained initial
validation of your recommendations.
o Briefly explain how insights gleaned from the usability testing of your client’s
interactive informed your thinking. You should also be able to draw on evidence from
the earlier heuristic evaluation, and card-sort activities.
(ok to express in bullet points).
For example: – changing the information architecture based on card sort results.
– redesign of a screen (eg changing menus, moving controls).
– changing naming/labels.
o The important thing here is that you cite the evidence backing up your change
recommendations, and that the prototype incorporating the changes (hopefully)
demonstrated some improvement when tested.
o If you wish to make any suggestions based on your own observations, you should be
able to draw on the results of your heuristic evaluation (A1 part 1).
o Include a summary of the paper prototyping exercise in an appendix (see next page for
appendix suggested detail). Incorporate one or more photographs of the testing
session to make it more tangible to the client.
• Referencing (10% of Assignment mark)
You are likely to have made use of multiple resources and reference sources in order to
complete this assessment. Make sure you reference them all properly. The mark for each of
the sections above will be influenced by how well you demonstrate your understanding of UX
theory and practice covered in classes (citing references will help).
• Layout / Standard of Doc (10% of Assignment mark)
Looking for professionally laid out document with section numbers, lots of images, with
purpose of each section understandable to a lay-person, and good use of appendices.
Remember – write this assignment in a way that makes sense to someone unfamiliar with the tutorial
material. Define and describe jargon that is not meaningful to lay-persons unfamiliar with UX.
2019 UXD – Assignment 1 Part 2 Usability testing – write-up guide 03 21.docx Page 5 of 5
Appendices for part 2:
Consider including the following in appendices of the report if not included (or if only summarised) in
the main body: (Items highlighted are the ones we expect to see as a minimum).
Don’t just drop items into the appendices – make sure each item is accompanied by some
explanation: – what is it’s purpose – why it is included etc.
• Task design templates. If you used Synder’s template to design your task(s), you can
include it here, along with a line or two explaining it.
• Personae used to identify target audience for test (recycle from A1 part 1) – [if not in body].
• Test script. Include the complete test script in an appendix if not part of main body of report.
• Task card details. Contents and layout of the task cards you used.
• Number of participants planned-for / time budgeted for testing
Estimate how much time to budget for the testing based on the number of test subjects
chosen and the time allowed for each test (including turnaround time ie time between tests).
Have some contingency up your sleeve in case things don’t go fully to plan.
• Location and date for testing. Details of when / where testing was carried out.
• Questionnaires and other instruments. Include details of any “test instruments” used – e.g.
pre and/or post test questionnaires. Include scanned copies of completed questionnaires.
Make sure there is no information identifying participants visible.
• Raw results / observations. Include the list of observations made by your team, and any
techniques you used to process them (ie affinity diagrams to chunk the data).
Include information of where to find links to audiovisual material (eg YouTube link) if
appropriate.
• Consent form for participants. A consent form to ensure that participants understand what
will happen during the session and be fully aware of the nature of their involvement. The form
allows participants to consent to participate and to being recorded (either audio/video/screen
capture or some combination of these – if this is happening. The form should also give them
the option to opt-out of recording. It should also indicate what procedures you have in place
to ensure the privacy of participants. Include scans of signed copies of the forms.
• Observer guidelines. Since testing can often be stressful for participants it is important that
observers behave in a consistent, appropriate fashion and ethical fashion.
Having some guidelines for observers can help ensure this happens.
• Prepare receipts to be given to participants to acknowledge any cash (or chocolates) paid in
return for participation.
• Checklist to use for the test Compile a checklist itemising everything you need to prepare
and do for each test – both before, during and after each test. Eg: clearing the browser
cache, list of materials required, reminder to silence phones, provide water / snacks etc.
• Paper prototyping exercise. Summarise the paper prototyping exercise you conducted:
o Include a paragraph or two explaining the purpose of the exercise.
o Include images of the paper prototyping materials you prepared, list any the applications
you used to create them, and note the group member(s) responsible for creating them.
o Summarise the test methodology (no need to be as formal as the main usability report).
Just explain the process you followed in a few bullet points.
o Summarise results observed and conclusions drawn.
o If you updated task analysis, scripts, task cards, questionnaires and consent form,
include these updated items.