CIRCA:Cognitive walkthrough report

From CIRCA

Jump to: navigation, search

Cognitive Walkthrough Evaluation of GWrit

Gwrit, the Game of Writing project has developed an online writing environment used in class. This environment allows users to write online and use some analysis tools during their writing. Gamification is also an interesting feature of GWrit, which make tedious writing task as an interesting game. We are going to develop an open version of GWrit, which would be widely spread not only in university. To build the open version of GWrit, we first need to evaluate the usability of the system.

One of the usability evaluation method is cognitive walkthrough.

Cognitive walkthrough (CW) is a method used to evaluate the usability by predicting how a user will understand an interaction in the design stage. We use CW method to focus on how easy the system is to learn. By walking through a list of representative tasks, we identify a number of potential problems of the system. In the cognitive walkthrough evaluation of GWrit, several tasks were identified, most of them could be walked through.

We have four representative task as following.

Task 1: Get an Account and Explore Course

Task 2: Write an Introduction and submit it for review

Task 3: Use analysis tools to analyze writing

Task 4: Post comments

Findings & Recommendations:

In task 1, users may fail to recognize the “course” button is for viewing all the courses. So, make “course” button more detailed, such as changing to “View all courses”.

In task 1, after user select a certain course and click join in, user may not able to find the connection between “My course” and the result of adding course. So, make an automatically forwarding here. After adding a course, the page will go to “my course” page automatically.

In task 2, after user submit writing, they may fail to find the submitted writings. So, we can add a button linked to view submitted writings.

In task 3, when user use word cloud to analyze their writing, they may not know the hovering function for viewing the frequency of a certain word. So, we can add a brief description of this function as a floating text.

In task 3, users may not know what the concordance analysis is. So, we can add a brief description of concordance as a floating text.

In task 4,users may not know the project title is linked to the submitted works. So users may find difficult to follow the right action step by step to find the commenting board. So, we can add a button in the page linked to the submitted writings.

Cognitive Walkthrough Script:

User profile:

Persona 1: Thackary

Thackary is new to GWrit. English is not his first language and he is new to Canada. He doesn’t have a lot of experience with computers or the internet and he is worried about having to both learn to write at a university level and use an online system at the same time.

Task list:

Scenario 1.1 Get an Account and Explore Course The first part of the scenario is his logging in and trying to find the course and the first tasks.

Actions: 1. T clicks the “register for free and start writing” button in the login page of Gwrit. Success story: Users can easily find the button by experience. 2. He fill in some basic information, including username, email address, and password. He clicks submission, then he gets an account for Gwrit. Success story: Users can easily fill in these by experience. 3. He clicks “Sign in” button in the top right corner and goes back to sign in page. Success story: Users can easily find the button by experience. 4. He fills in email and password, then he clicks “Sign in”. Success story: Users can easily find the button by experience. 5. After login, he clicks the “Course” button in the left column, he goes to the course panel and see all the courses listed here. Failure story: Criterion: Will the user associate the correct action with the effect one is trying to achieve? Problem is detected. Users may fail to recognize the “course” button is for viewing all courses. 6. He clicks course 101, and view all the project listed in course 101, he clicked “Join in”. Success story: Users can easily find the button by experience. 7. He clicks “My course” button, and he can see course 101 have been added in too “My course” panel. Failure story: Criterion: Will the user associate the correct action with the effect one is trying to achieve? Problem is noticed. Users may not able to find the connection between “My course” and the result of adding course. Recommendations: Action 5. Make “course” button more detailed, such as changing to “View all courses”. Action 7. Make an automatically forwarding here. After adding a course, the page will go to “my course” page automatically.

Scenario 1.2 Write an Introduction and submit it for review. The second part of the scenario is his completing the first task which is to write a short introductory biography paragraph of the “hi my name is …” type. Actions: 1. T clicks “My course”, then he clicks the first project in course 101, called “Introduce yourself”, then he clicks the first task called “One paragraph about you”, and goes to the writing page. Failure story: Criterion: Will the user associate the correct action with the effect one is trying to achieve? Users may have difficult to find the writing page as the sequences listed in the action. 2. He writes a paragraph to introduce himself in the “Notes” board. Success story: Users can easily type in text in the text area (note board) by experience. 3. When he finished, he clicks “Save” button. Success story: Users can easily find the button by experience. 4. After a while, he wants to add something about the introduction. Then he open the writing page again, and click “Insert” button, then upload a picture about his writing. Then he clicks submit for review. Success story: Users can easily find the edit button in the toolbar by experience. The submit button is also obvious. 5. He clicks the project title in the left column to browse the writing submitted and waiting for comments from others. Failure story: Criterion: Will the user know that the correct action is available? Problem noticed. There is not button led to the submitted writings here, and users may not know the title of project is linked to the submitted works, so users may fail to find the submitted writings. Recommendations: Action 5. Add a button linked to view submitted writings.


Scenario 1.3 Use analysis tools to analyze writing Actions: 1. He clicks projects in the homepage, and find out the task called “Essay 101” he doesn’t finished last time. Success story: Users can find the writing page based on their previous experience in GWrit. 2. He clicks “Essay 101”, and goes to the writing page. Success story: Users can find the writing page based on their previous experience in GWrit. 3. He add something to the notes board, and after finished, he clicks the “Analytics” button in the right column. He first clicks “word cloud” in the drop list. Then the word cloud shows up, he hovers over a certain word to see the time it occurs in the writing. Failure story: Criterion: Will the user know that the correct action is available? Problem noticed.Users may not know the hovering function for viewing the frequency of a certain word. 4. He clicks “concordance” in the drop list. He types in a word to find and the context length, then the concordance of the word shows up. Failure story: Criterion: Will the user try to achieve the right effect? Problem noticed.Users may not know what is concordance analysis. 5. He clicks “readability” in the drop list, and the statistic and the average grade of the readability based on different formula show up. Success story: Users can find the button easily and the result of analysis is intuitive. Recommendations: Action 4. Add a brief description of this function as a floating text. Action 5. Add a brief description of concordance as a floating text.

Scenario 1.4 Post comments Actions: 1. He login GWrit, and selects the course he is in, and then clicks a project title in the left column, then clicks “view other submissions” . Failure story: Criteria: Will the user associate the correct action with the effect one is trying to achieve? Problem noticed. Users may not know the project title is linked to the submitted works. So users may find difficult to follow the right action step by step to find the commenting board. 2. He clicks other user’s name to see their writing. Success story: Users can click the link correctly by experience. 3. He click the commenting icon, and leave some message to this writing. Success story: Users can easily type in comment in the text area by experience. Recommendations: Action 1. Add an intuitive button linked to the submitted works.

Conclusion: The usability evaluation on GWrit using CW leads to the identification of several problems. The problems include some control of actions are not available, such as “view submission” is difficult to find after user submit their writings. Second, the connection between the correct action and the goal is weak. the reason may be the ambiguity of some functions and the definition of buttons. So, based on the evaluation result, we should have some buttons defined clearly and add some interpretive texts for certain functions.



Ps. The four questions to ask in a cognitive walkthrough: Q1: Will the user realistically be trying to do this action? Q2: Is the control for the action visible? Q3: Is there a strong link between the control and the action? Q4: Is feedback appropriate? 1. Will the user try to achieve the right effect? 2. Will the user know that the correct action is available? 3. Will the user associate the correct action with the effect one is trying to achieve? 4. If the correct action is performed, will the user see that progress is being made toward solution of one‟s task?

Personal tools