CIRCA:Fort Edmonton Park Assessment

From CIRCA

Revision as of 14:44, 4 May 2011 by CalenHenry (Talk | contribs)
(diff) ← Older revision | Current revision (diff) | Newer revision → (diff)
Jump to: navigation, search

ASSESSMENT

Calen Henry – April 2011

Thanks to Erik DeJong for assistance with assessment methods

Contents

Questions Guiding Assessment

In order to assess the Fort Edmonton Park game we first need to consider questions we want to answer

Did participants learn?

fAR-Play has been developed as an academic project with an interest in serious games and games for learning, thus, it is important for us to gauge if participants actually learned through the game.

Did they learn more, differently, or better than they would have without the game?

In order for the game to be successful the participants must learn in a way that differs from the normal park experience. This could be learning a different historical narrative, supplemental learning to what normally takes place in the park or simply an increased engagement with the material. We want to determine in which of these ways, if any, participants learned.

Was the experience fun?

Part of a game (generally speaking) is to have fun, so we want to assess whether or not the experience was fun, as well as conducive to learning

Was it more fun than just a visit to the park?

Did the game make visiting the park more fun than it would have otherwise? The hope is to bring people to the park that may not otherwise visit so an experience on top of the normal visit is important to do that.

How did this process work for the Fort Edmonton Park Staff

Given that fAR-Play is intended for client use it is important to ascertain if the designers and content creators for on the client side had a good experience with the platform and developing the game.

Did participants retain and/or use knowledge after the visit?

This is to assess long term retention to see if anything learned at the park found its way into longer term knowledge or not. Both University of Alberta researchers and Fort Edmonton Park staff are interested in this result.

Did the game interface work?

Since this is the first full scale test of a fAR-Play game it will be important to gather audience feedback regarding the interface and gameplay flow.

Assessment Methods

Broadly speaking there are three methods of assessment to be used

Interaction with participants

In order to determine the success of the game it will be necessary to get information from participants about their experiences. To this end we will need to have two groups of volunteers while the game is running and give each a post-experience questionnaire. One group that does not play the game will be needed as well as a group that does play the game. The questionnaire will be used to gauge how much fun participants in the game had versus those who simply visited the part and did not play our game. The questionnaire will also be used to gauge if they retained more knowledge from playing the game than from simply visiting the park and learning from signs and interpreters. Finally the questionnaire will help us ascertain what parts of the game interface worked and what needs to be improved.

The learning portion will likely have to be in the form of a quiz to test participant’s knowledge. It may work better if they are not aware they are to take a quiz so they do not go through the park and study instead of playing the game for fun.

The fun portion necessitates opinion based responses and to that end question using the Likert scale (1-5, where 1 is strongly disagree and 5 is strongly agree)

The interface portion will likely be a combination of Likert questions and asking fort comments from participants.

The staff at Fort Edmonton Park is also interested in what degree of long term knowledge was gained from the visit so we would also get permission to have participants fill out a second questionnaire one month after their visit, for both control groups. IT will be important to keep track of which participants fill out a short term questionnaire in addition to the long term one, as those having filled out the short term survey may be more likely to retain long term knowledge of the visit.

Interaction with designers and content creators

fAR-Play has been developed with the intention of using it as a platform for game development, therefore, it is important to assess the success of the relationship with the client. We want to interview Fort Edmonton Park staff involved to find out how the experience went for them and to see what could be improved and what aspects they liked.

Metrics/data analysis

Participant interaction can be partly backed up by simple metrics from the game. By recording which questions users answered and how long they stayed on various web pages for the game it will be possible to see to what degree users engaged with the content and if users started the game then gave up or lost interest.

Focused Results

In order to get good analysis out of the data it is important to make sure the assessment if focused, especially as this is the first client-based fAR-Play game. With this in mind it is my recommendation to focus only on fun, learning and client relationship for this process.

Personal tools