CIRCA:OSBA Assessment

From CIRCA

Jump to: navigation, search

Contents

Collecting Feedback from Players of the OSBA SCVNGR based game

(*note - revise for fAR-PLAY version - SL)

In order to determine the success of the game it will be necessary to collect information from participants about their experience. Ideally we would be able to do this in an ongoing manner or several times throughout a period of 30 - 90 days of continuous play. To achieve this goal I propose two possible data collections strategies.

1. A group of volunteer test participants will be asked to play the game and complete an online (or paper) questionnaire about the experience. 2. When the game is released in the wild active participants are emailed a link to voluntarily complete the questionnaire.

In the first case we are working with a self selected group who are playing the game in order to give the research team feedback. In the second case the research team would have access to a self selected group of interested players but may suffer from a low response rate. Ethics for the assessment tools will need to be sought in either case and perhaps it would be best to employ both methods using the same online survey tool and question set.

The goal in the assessment questionnaire will be to gauge how enjoyable the game was to play, if the players learned about the history of the OSBA through playing, and if they felt more engaged with the area and local history through their participation. Finally the questionnaire will help us ascertain what parts of the game interface worked and what needs to be improved.


Demographic Information

General demographic information (age, gender, experience with technology) will be useful to collect to identify who is playing the game. This will help determine which demographics are engaging with the game and how play differs among different groups and/or communities

   * Age: 
       0-12 
       13-17 
       18-35 
       36-55 
       55+ 
   * Did you play the SCVNGR OSBA History Adventure today? 
       yes 
       no 
   * Have you visited the Whyte Ave/OSBA area before? 
       yes (how often on average do you come to this area? - daily, weekly, monthly, yearly)
       no 
   * Have you played SCVNGR before?
       yes 
       no 
   * Have you played other geo-location based games on your smartphone before (e.g. foursquare, Facebook Places, Gowalla, etc?)
       yes 
       no 


Focus

(needs more detail) In contrast to the other geo-location based game being built on the fAR-Play platform for Fort Edmonton Park the focus of the OSBA History game will not be strictly on learning but will aim to engage players to learn about the history of the area while contributing their own knowledge, reflections, and personal histories to the communal narrative of the Whyte Ave and OSBA area in Edmonton. This goal shifts the focus from individual learning to a communal learning and documentation. This will make the assessment of the game more difficult in terms of success or failure (beyond simple participation).

The goal of this game is to show players that the OSBA/Whyte Ave area is not only a commercial district but that it is made up of many layers of local history that go all the way back to the founding of the city in the 19th century. The area is also meaningful for people on a personal level because so many individual have their own history tied to this city and this neighbourhood. Our hope is that this game will make the historical importance of the area more visible and to help people feel connected and an integral part of the cultural community of this place. It is not only about teaching the history of the place, but also engaging players to write a part of that history and to discover multiple subjective narratives and different viewpoints of the same geographical area.

The game play will not necessarily have a set time frame or experiential narrative for players to complete. Some may check in at all locations that are part of an adventure in one day, others may space their play out over a few days, weeks, or even months. Gathering feedback from participants who play over a longer period of time could be an interesting opportunity to compare with those who play in a more focused manner. Ideally an online questionnaire that could reasonably (and usefully) be answered at various play landmarks would be written and made available to all players at all times.


Possible Questions

  * I enjoyed playing this game
       Strongly Agree 
       Agree 
       Neutral 
       Disagree 
       Strongly disagree 
  * Playing the game gave me a different perspective on the OSBA area
       Strongly Agree 
       Agree 
       Neutral 
       Disagree 
       Strongly disagree 
  * I learned something I didn't know about the Whyte Ave/OSBA area while playing the game
       Strongly Agree 
       Agree 
       Neutral 
       Disagree 
       Strongly disagree  
    *If Strongly Agree or Agree, what did you learn? Please describe in detail (text box)
  * I enjoyed contributing my own content to the game (edit: preface with 'Did you upload content to the game?')
       Strongly Agree 
       Agree 
       Neutral 
       Disagree 
       Strongly disagree 
  * What kind of content did you add?
       text (stories, comments, reviews, etc)
       photos
       video
       completed a set challenge (please describe the challenge you completed)
  * What part of the game or locations did you enjoy the most? 
         Find the most interesting?
         Find the most memorable? 
         Find the most unique? 
         Were the most surprising?
  * Are there any places in the OSBA you would like to see be part of the game? (textbox)
  * Would you continue to play or play this game again? (Why or why not)
  * Would you recommend this game to someone else? 
  * Comments


Interface

The interface for the SCVNGR version of the game will be largely dependent on the designers, although we will be able to skin the questions and informational pages to brand the game. When the game is rebuilt in fAR-Play the interface will require redesign.

Some basic questions about the experience of the phone interface would be useful in comparing SCVNGR and fAR-Play later on from an enjoyment, useability, and aesthetic standppint.

Interaction with designers and content creators

fAR-Play has been developed with the intention of using it as a platform for game development, therefore, it is important to assess the success of the relationship with the client. Additionally, the experiences of the students involved in designing and developing the game may provide insight into the overall process of game development.

SCVNGR is a commercial product that we have been given free access to as well as a support person to contact for information.

The OSBA office staff should be interviews as well as some of the business participants in the game if they are willing to see how they experienced the designing and implementation of the game and to see if they have received any valuable feedback from their communities.

Given that the game is intended for client use it is important to ascertain if the designers and content creators for on the client side had a good experience with the platform and developing the game. It will also be important to learn how the presence of players in the OSBA/Whyte Ave community effected business owners (either positively or negatively).


== Questions for Staff ==

   * What were your goals for this game? What did you hope to accomplish through this project?
   * How do you feel this game met or failed to meet these goals?
   * What aspects of the game do you like?
   * What aspects of the game would you like to change and how?
   * Did you enjoy the design and creation phase of the project?
   * How could the creation process have been improved? 
   * Do you think you will continue to use and/or develop the game? (Why or why not? please elaborate)
   * What potential barriers are there for your continued use of the game? (What could we do to help eliminate these barriers?)


Research Team Assessment Techniques

   * Journaling (dropbox folder)
   * Feedback to CS on interface/design
   * Reflections - ask Calen for a reflection?

Comp Sci Students/Faculty

It would be useful to interview the students/developers from Computer Science about their involvement in the process.

Metrics/data analysis

Participant interaction can be partly backed up by simple metrics from the game. By recording which questions users answered and how long they stayed on various web pages for the game it will be possible to see to what degree users engaged with the content and if users started the game then gave up or lost interest. *Game play metrics are available through the SCVNGR and fAR-Play interfaces.


Success rates

   * Most/least attempted question
   * Most/least popular adventure/location/task


Entry points

   * Where did players hear about the game? 
   *(related) are they playing the game solo or with friends?
   * Which business/website/QR code/other location is the most common entry point for new players?
   * Which locations in the game are visited most often?
   * Which locations acrue the most activity? (uploaded content by players)
   * Which locations are visited rarely/not visited by players?


Credits

   * Initial document - based on work by Vicky Varga - June 2011
   * Assistance with assessment methods - Matt Bouchard - June 2011
   * Revision/expansion - Shannon Lucky - June/July 2011

Ideas

  * Compare SCVNGR to fAR-Play
  * Compare authoring (journaling)
  * Study working with community orgs
  * The place of alternative community oral history - how does it work? Is it learning? Can we measure engagement? 
  * How to market and advertise a game and how to sustain
Personal tools