Cognitive walkthrough report

From CIRCA

VTracker
Content deleted. (1 Occurances)
Content inserted. (19 Occurances)
Content structure inserted. (19 Occurances)
Content changed. (4 Occurances)
Content moved. (2 Occurances)
Content NEED DESCRIPTION. (1 Occurances)
Content NEED DESCRIPTION. (1 Occurances)

---

Cognitive Walkthrough Evaluation of GWrit

*Gwrit, the Game of Writing project has developed an online writing environment used in class. This environment allows users to write online and use some analysis tools during their writing. Gamification is also an interesting feature of GWrit, which make tedious writing task as an interesting game. We are going to develop an open version of GWrit, which would be widely spread not only in university. To build the open version of GWrit, we first need to evaluate the usability of the system.

One of the usability evaluation method is cognitive walkthrough.

Cognitive walkthrough (CW) is a method used to evaluate the usability by predicting how a user will understand an interaction in the design stage. We use CW method to focus on how easy the system is to learn. By walking through a list of representative tasks, we identify a number of potential problems of the system.In the cognitive walkthrough evaluation of GWrit, several tasks were identified, most of them could be walked through.

We have four representative task as following.

Task 1: Get an Account and Explore Course

Task 2: Write an Introduction and submit it for review

Task 3: Use analysis tools to analyze writing

Task 4: Post comments

Findings & Recommendations:

In task 1, users may fail to recognize the ???course??? button is for viewing all the courses. So, make ???course??? button more detailed, such as changing to ???View all courses???.

In task 1, after user select a certain course and click join in, user may not able to find the connection between ???My course??? and the result of adding course. So, make an automatically forwarding here. After adding a course, the page will go to ???my course??? page automatically.

In task 2, after user submit writing, they may fail to find the submitted writings. So, we can add a button linked to view submitted writings.

In task 3, when user use word cloud to analyze their writing, they may not know the hovering function for viewing the frequency of a certain word. So, we can add a brief description of this function as a floating text.

In task 3, users may not know what the concordance analysis is. So, we can add a brief description of concordance as a floating text.

In task 4,users may not know the project title is linked to the submitted works. So users may find difficult to follow the right action step by step to find the commenting board. So, we can add a button in the page linked to the submitted writings.

Cognitive Walkthrough Script:

User profile:

Persona 1: Thackary

User profile:Persona 1: ThackaryThackary is new to GWrit. English is not his first language and he is new to Canada. He doesn???t have a lot of experience with computers or the internet and he is worried about having to both learn to write at a university level and use an online system at the same time.Thackary is new to GWrit. English is not his first language and he is new to Canada. He doesn???t have a lot of experience with computers or the internet and he is worried about having to both learn to write at a university level and use an online system at the same time.

Task list:

Scenario 1.1 Get an Account and Explore CourseThe first part of the scenario is his logging in and trying to find the course and the first tasks.

Actions:1. T clicks the ???register for free and startwriting??? button in the login page of Gwrit. Success story: Userscan easily find the button by experience. 2. He fill in some basicinformation, including username, email address, and password. Heclicks submission, then he gets an account for Gwrit.Success story:Users can easily fill in these by experience.3. He clicks ???Signin??? button in the top right corner and goes back to sign in page.Success story: Users can easily find the button by experience.4. Hefills in email and password, then he clicks ???Sign in???. Successstory: Users can easily find the button by experience.5. Afterlogin, he clicks the ???Course??? button in the left column, hegoes to the course panel and see all the courses listedhere.Failure story: Criterion: Will the user associate the correctaction with the effect one is trying to achieve? Problem isdetected. Users may fail to recognize the ???course??? button isfor viewing all courses.6. He clicks course 101, and view all theproject listed in course 101, he clicked ???Join in???.Successstory: Users can easily find the button by experience.7. He clicks???My course??? button, and he can see course 101 have been addedin too ???My course??? panel. Failure story:Criterion: Will theuser associate the correct action with the effect one is trying toachieve?Problem is noticed. Users may not able to find theconnection between ???My course??? and the result of adding course.Recommendations: Action 5. Make ???course??? button more detailed,such as changing to ???View all courses???.Action 7. Make anautomatically forwarding here. After adding a course, the page willgo to ???my course??? page automatically.

Scenario 1.2 Write an Introduction and submit it for review.Thesecond part of the scenario is his completing the first task whichis to write a short introductory biography paragraph of the ???himy name is ?????? type.Actions:1. T clicks ???My course???, then heclicks the first project in course 101, called ???Introduceyourself???, then he clicks the first task called ???One paragraphabout you???, and goes to the writing page.Failure story:Criterion:Will the user associate the correct action with the effect one istrying to achieve?Users may have difficult to find the writing pageas the sequences listed in the action. 2. He writes a paragraph tointroduce himself in the ???Notes??? board. Success story: Userscan easily type in text in the text area (note board) byexperience.3. When he finished, he clicks ???Save??? button.Successstory: Users can easily find the button by experience.4. After awhile, he wants to add something about the introduction. Then heopen the writing page again, and click ???Insert??? button, thenupload a picture about his writing. Then he clicks submit forreview. Success story: Users can easily find the edit button in thetoolbar by experience. The submit button is also obvious.5. Heclicks the project title in the left column to browse the writingsubmitted and waiting for comments from others.Failure story:Criterion: Will the user know that the correct action is available?Problem noticed. There is not button led to the submitted writingshere, and users may not know the title of project is linked to thesubmitted works, so users may fail to find the submittedwritings.Recommendations: Action 5. Add a button linked to viewsubmitted writings.

Scenario 1.3 Use analysis tools to analyze writing Actions:1. Heclicks projects in the homepage, and find out the task called???Essay 101??? he doesn???t finished last time. Success story:Users can find the writing page based on their previous experiencein GWrit. 2. He clicks ???Essay 101???, and goes to the writingpage. Success story: Users can find the writing page based on theirprevious experience in GWrit.3. He add something to the notesboard, and after finished, he clicks the ???Analytics??? button inthe right column.He first clicks ???word cloud??? in the drop list.Then the word cloud shows up, he hovers over a certain word to seethe time it occurs in the writing.Failure story: Criterion: Willthe user know that the correct action is available?Problemnoticed.Users may not know the hovering function for viewing thefrequency of a certain word.4. He clicks ???concordance??? in thedrop list. He types in a word to find and the context length, thenthe concordance of the word shows up.Failure story: Criterion: Willthe user try to achieve the right effect?Problem noticed.Users maynot know what is concordance analysis. 5. He clicks???readability??? in the drop list, and the statistic and theaverage grade of the readability based on different formula showup. Success story: Users can find the button easily and the resultof analysis is intuitive.Recommendations: Action 4. Add a briefdescription of this function as a floating text. Action 5. Add abrief description of concordance as a floating text.

Scenario 1.4 Post commentsActions:1. He login GWrit, and selectsthe course he is in, and then clicks a project title in the leftcolumn, then clicks ???view other submissions??? .Failurestory:Criteria: Will the user associate the correct action with theeffect one is trying to achieve? Problem noticed. Users may notknow the project title is linked to the submitted works. So usersmay find difficult to follow the right action step by step to findthe commenting board. 2. He clicks other user???s name to see theirwriting.Success story: Users can click the link correctly byexperience.3. He click the commenting icon, and leave some messageto this writing.Success story: Users can easily type in comment inthe text area by experience.Recommendations: Action 1. Add anintuitive button linked to the submitted works.

Conclusion:The usability evaluation on GWrit using CW leads to the identification of several problems. The problems include some control of actions are not available, such as ???view submission??? is difficult to find after user submit their writings. Second, the connection between the correct action and the goal is weak. the reason may be the ambiguity of some functions and the definition of buttons. So, based on the evaluation result, we should have some buttons defined clearly and add some interpretive texts for certain functions.

Ps. The four questions to ask in a cognitive walkthrough:Q1: Will the user realistically be trying to do this action?Q2: Is the control for the action visible?Q3: Is there a strong link between the control and the action?Q4: Is feedback appropriate?1. Will the user try to achieve the right effect? 2. Will the user know that the correct action is available? 3. Will the user associate the correct action with the effect one is trying to achieve? 4. If the correct action is performed, will the user see that progress is being made toward solution of one???s task?