CIRCA:Assessment Framework

From CIRCA

(Difference between revisions)
Jump to: navigation, search
(Undo revision 3912 by SimeonBlimke (Talk))
(8.0 Closure)
 
(18 intermediate revisions not shown)
Line 1: Line 1:
-
=(Tentative) Assessment Framework Questions=
+
The '''GRAND Assessment Framework''' is an iterative assessment framework designed by the University of Alberta GRAND group. The framework starts with the assumption that the group research project involves practical game design, and continues by attempting to ask the most significant over-arching questions.  
-
[[CIRCA:Game Design Taxonomy|Game Design Taxonomy]]is a list of terms that various people have coined for analyzing and classifying games.
+
-
While this document is being composed, the [[CIRCA:initial questions to ask when starting group research projects with a practical game design component -|compact list]] of questions (also under construction) to ask when '''starting''' group research projects with a practical game design component can be found [[CIRCA:initial questions to ask when starting group research projects with a practical game design component -|on its own page. ]]
+
The framework is intended to help game designers and researchers articulate their points (and what those points are in the first place), create a foundation for further work, and to suggest a set of possible methods for that may be of use during that particular part of the design process. Fill in category sub-questions as best you can, and add your own or skip questions as needed.
 +
==1.0 [[CIRCA: Stakeholders and Expectations|Stakeholders and Expectations ]]==
-
Use these question categories in the process of answering your own questions relevant to a specific project.
+
*1.1 What's the point of the project?
-
Fill in category sub-questions as best you can, add your own or skip questions as needed.
+
-
==Affiliations (Stakeholders)==
+
*1.2 Who are the interested parties in this project?
-
*Who are the existing stakeholders?
+
*1.3 What does each stakeholder get out of the project?
-
**administrative
+
-
**ethical / legal
+
-
**research
+
-
**audience / customer
+
-
**others
+
-
*Who are the legacy stakeholders?
+
*1.4 How will you prioritize the needs of stakeholders?
-
**sponsors
+
-
**returning group members
+
-
**community partners
+
-
**audience demographics
+
-
**notion of a corporate "brand" with feedback mechanisms about its reputation
+
-
*Who are new stakeholders, or what are recent changes in makeup/orientation/capacity?
+
==2.0 [[CIRCA: Requirements|Requirements ]]==
-
*Who are potential stakeholders?
+
*2.1 What is the primary purpose of the project?
-
** Is there a market or test group available, already existing or in demand?
+
-
***What are potential inclusion or exclusion factors (Does working with these stakeholders require REB ethics clearance?)
+
-
*Where can more information be found about a particular stakeholder, how can this information be used?
+
-
*How will you rank or sort stakeholders?
+
*2.2 What evidence would indicate success? How will you know it is over?
-
*Which stakeholders will own the work when the project is done?
+
*2.3 How will interested parties discuss the project as it evolves?
-
==Expectations==
+
*2.4 Is there a project charter agreed among stakeholders that makes clear what is expected of everyone?
-
*What is the purpose of the project?
+
==3.0 [[CIRCA: Resources|Resources ]]==
-
*What are the minimum requirements of success?
+
-
*What are the stakeholder groups' main goals?
+
*3.1 What resources are required to do the project?
-
**What is each stakeholder group getting out of the project?
+
-
*What ought to be the shared ambition of this project?
+
*3.2 How will you get the required resources?
-
* What are the stakeholders' responsibilities to the group?
+
*3.3 How will you account to stakeholders for the resources they have provided?
-
* To what extent is the project contract to be formalized or kept informal?
+
*3.4 How will you deal with the loss of resources?
-
==Resources==
+
==4.0 [[CIRCA: Planning|Planning]]==
-
* What are the deliverables?
+
*4.1 Is there a project plan? Does it include the level of detail needed?  
-
** What are our deliverables' timelines
+
-
* What is the budget?
+
*4.2 Is there a method for tracking progress regularly?
-
**Who is responsible for all aspects of the budget?
+
-
*What skills and technology do group members already have?
+
*4.3 How will the plan be adapted if you fall behind or lose resources?
-
* What technology or tools are required to build, maintain, and play the game?
+
==5.0 [[CIRCA: Design|Design]]==
-
* What is the project timeline?  
+
*5.1 What design process will work best for this project?  
-
*What amenities does the group have access to (e.g. a campus for testing)?
+
*5.2 How can you assess the design process?
-
* What is the length of time it would take to build and run the game?
+
*5.3 Which resources are needed to develop the game?
-
** Is there a deadline? Or can this project be delayed?
+
-
*Is there a method for tracking progress regularly to ensure meeting deadlines?
+
*5.4 Does the game meet your goals? Are there unexpected problems?
-
* Is there previous or comparable work, either ours or another group’s, in this area?
+
*5.5 Are there ways to improve the game or the platform based on the results of this attempt?
-
**How does our work compare to other work?
+
-
* How long should the project be able to last?
+
==6.0 [[CIRCA: Delivery|Delivery]]==
-
*Is there a buffer for potential changes?
+
*6.1 How will you deliver the product to your audience?
-
**Time
+
-
**Tools
+
-
**Group
+
-
**Tools
+
-
**Partnerships
+
-
**Capital
+
-
**Ethical conditions
+
-
**Demands from stakeholders
+
-
**Other
+
-
* Are any outside stakeholders responsible for providing content/information/materials/funding/etc.?
+
*6.2 How will they become aware of your project?
-
*Are we using outside intellectual property?
+
*6.3 How will you train the audience to understand and play the game?
-
**Who does it belong to?
+
-
***What sort of license does it use?  
+
-
* Who is responsible for maintaining the project?
+
==7.0 [[CIRCA: Feedback|Feedback]]==
-
==Execution==
+
*7.1 What feedback do you and your stakeholders want and from who?
-
* Does the game work?  
+
*7.2 How does the game work for players? What is their experience?
-
** What doesn't work?
+
-
*** Is the problem technical, conceptual, or both?
+
-
* Are there ways to improve the game or the platform based on the results of this attempt?
+
*7.3 How will reports from the players or other stakeholders be handled?
-
* Should the game be more intuitive or instructional?
+
*7.4 Can we provide feedback to encourage player engagement with the game?
-
** How quickly will the target audience be able to learn the game?
+
-
* If the project is repeatable, will we learn new things or benefit from running the game again?
+
*7.5 How can feedback influence further design?
-
* How will we conclude this project (what will the end-state look like?)
+
==8.0 [[CIRCA: Closure|Closure]]==
-
==Feedback==
+
*8.1 What will the end-state of the project look like?
-
*What feedback are we testing for?
+
*8.2 Does the game meet its goals? How do you know that?
-
* Who did the game work for (or not work for)?  
+
-
** How did the stakeholders react to the game?
+
-
* Who and how do we want to give or receive feedback? (I think this needs to be broken down into several questions - SL)
+
*8.3 How will you know if stakeholders are satisfied?
-
** How will we present findings to our stakeholders?
+
-
** Who will we solicit feedback from?
+
-
*** What type of feedback is required? (qualitative? quantitative? game metrics?)
+
-
* What would be the most efficient method - time and resource wise - of gathering assessment data?
+
-
* Do the chosen assessment techniques require ethics clearance?
+
-
** Do we need new ethics or can this fall under a previous project?
+
-
* What tools will give us the most useful data?
+
-
* What questions should the assessment tools pose?
+
-
** Should there be multiple feedback tools available?
+
-
*What is the criteria for acquiring and updating information about stakeholders?
+
-
*How well does this project address the topic of '_____'?
+
-
*Does the game provide anything different or better from other resources/games?
+
-
(I think this section needs to be much more detailed since this was the original focus of the framework - the assessment process) - SL
+
*8.4 Have you communicated to all interested parties that the project is over?
-
==Misc. categories (fun, education, technology, etc...)==
+
=Related Wiki Resources=
-
 
+
*[[CIRCA:The Expanded GRAND Assessment Framework |The Original (expanded) GRAND Assessment Framework]]
-
* What are measurable variables that can be defined as '____'?
+
*[[CIRCA:Assessment Tools|Recommended Methods]]
-
** Can this game teach something that can be gained through other resources in a way that the targeted audience finds preferable? ( I think this is outside of the scope of our group, we are not really studying how best to teach a topic or curriculum, we are studying games.)''
+
*[[CIRCA:Methods_for_Game_Design]]  
-
* Do we need to advertise?
+
*[[CIRCA:Game Design Taxonomy|Game Design Taxonomy]]
-
* How will we launch the game and attract players?
+
-
** Do we have a captive player group or is the game released in the wild?
+
-
* Can we tweak the game during deployment? Is this a part of the plan?
+
-
 
+
-
==Other things to put in==
+
-
 
+
-
''I've tried incorporating these parts into the framework as much as possible. They've mainly been incorporated into the resources sections-DG
+
-
 
+
-
* '''Intellectual Property:''' What sort of license does it use? Who owns the work? Who published it? What is the intellectual property?
+
-
* '''Time and Money:''' How long did it take to make? How much did it cost to develop? What sort of organization developed it?
+
-
* '''Comparison and Competition:''' What other games are similar? Who is the competition? How does it compare to other games?
+
-
 
+
-
=Other Frameworks=
+
-
==online course==
+
-
''University of Pennsylvania'' offers a complete [https://www.coursera.org/course/gamification online course] on gamification. It is supposedly 4-8 hours a week for it looks like 5-6 weeks. About halfway into the course the topic is "Gamification Design Framework."
+
-
==online course==
+
-
''Stanford' offers this [https://www.coursera.org/course/hci course]on design for human-computer interaction.
+
-
 
+
-
===book===
+
-
*'''Walker, Mark. ''Games that Sell'''''(I have this one out right now)--[[User:SimeonBlimke|SimeonBlimke]] 07:57, 4 November 2012 (UTC)
+
-
 
+
-
Abstract:
+
-
 
+
-
Walker is a journalist, and borrows the term (hot)''topic'' from the journalistic discipline to describe games the gaming public are likely to invest in at a given point in history -- some topics are just hotter than others. This is a way of saying that you need to do broader social analysis, not just formal game design analysis, to understand what compels people to play. See also Montfort and Bogost who in ''Racing the Beam'' take a similar approach to diagnose the rise of the Atari console and subsequent video game crash of 1983.
+
-
 
+
-
Methods used to analyze games:
+
-
*Specific case-studies of games that sold (or should have sold but did not).
+
-
*Industry professionals are interviewed, asked why ''they'' think games sell.
+
-
*Survey of players - rather than using these to compile stats he publishes the complete responses of several individual respondents
+
-
 
+
-
'''Framework:'''
+
-
+
-
*'''Topic''' (franchise reputation, genres, fads)
+
-
 
+
-
"During the real-time strategy craze of the late 90's, publishers could just about guarantee that a solid real-time strategy game would sell 100,000 units. On the other hand, a turn-based strategy game needed to be marketed, promoted, and designed to perfection to crest that magical 100,000 unit mark" (p.2).
+
-
 
+
-
*'''Quality''' (e.g. game is not glitchy, tutorials are well written, appears worth $49)
+
-
 
+
-
*'''Marketing and public relations'''
+
-
*'''Range of appeal''' (this doesn't rule out niche games, they are just exceptional and it's hard to predict when a niche will be popular enough to be viable. What's important is making a game accessible to a wide enough audience ... it could also mean combining popular genres in a way that is inviting to more than one segment of the market)
+
-
 
+
-
*'''Cool factor''' (intangible features that attract players' interest such as a unique game mechanic or a story with a special emotional appeal)
+
-
 
+
-
===book===
+
-
 
+
-
*'''Aldrich, Clark. ''Learning by doing : a comprehensive guide to simulations, computer games, and pedagogy in e-learning and other educational experiences'''''
+
-
 
+
-
Book jacket review/blurb: "Designed for learning professionals and drawing on both game creators and instructional designers, Learning by Doing explains how to select, research, build, sell, deploy, and measure the right type of educational simulation for the right situation. It covers simple approaches that use basic or no technology through projects on the scale of computer games and flight simulators."
+
-
 
+
-
'''Framework:'''
+
-
(I haven't had a chance to borrow this one yet to check quality)--[[User:SimeonBlimke|SimeonBlimke]] 07:36, 4 November 2012 (UTC)
+
-
----
+
-
===book===
+
-
Sketching user experiences : getting the design right and the right design
+
-
===book===
+
-
'''Rollings, Andrew. Andrew Rollings and Ernest Adams on game design'''--[[User:SimeonBlimke|SimeonBlimke]] 08:11, 4 November 2012 (UTC)(U of A has electronic access!)
+
-
 
+
-
===book===
+
-
Rogers, Scott '''Level up! : the guide to great video game design'''--[[User:SimeonBlimke|SimeonBlimke]] 08:11, 4 November 2012 (UTC)
+
-
 
+
-
U of A has electronic access! This is a complete book on game design in a readable style, I've paged through it so far and it looks good.
+
-
 
+
-
===book===
+
-
'''Feiler, Jessie. Get rich with apps'''
+
-
 
+
-
I paged through this book by chance and it actually has some thoughtfully worded advice despite the sensational title. It might be worth going back to.
+
-
 
+
-
'''Framework (bits)'''
+
-
*Use apps - push their capabilities and try new things - consider what's missing. Learn to watch yourself playing.
+
-
*observe other people (watch for when they make mistakes due to the interface, note when they smile with satisfaction)
+
-
*be selective, keep the constraints of a small viewport in mind.
+
-
*As the tendency when everyone has a little bit of say is a product bloated with features, design committees should try to focus on what to remove.
+
-
 
+
-
The contents are fairly specific to strategically assessing and exploiting current platforms like facebook or mobile games.
+
-
 
+
-
===article===
+
-
[http://gamestudies.org/0601/articles/consalvo_dutton Game analysis: Developing a methodological toolkit for the qualitative study of games]by Mia Consalvo and Nathan Dutton.
+
-
 
+
-
'''Framework (tools)'''
+
-
*object inventory
+
-
*interface study
+
-
*interaction map
+
-
*gameplay log
+
-
 
+
-
==External Links==
+
-
 
+
-
* [http://gamification.org/wiki/Game_Design Gamification Game Design Wiki]
+
-
has a long list of items - have a feeling that these are at different levels of importance so we should read through and sort.
+
-
* [http://circa.cs.ualberta.ca/index.php/CIRCA:RockwellGuide Another section of CIRCA wiki in which Geoffrey Rockwell discusses principles of digital project management in general.]
+
-
 
+
-
 
+
-
Bits of advice that could be part of a framework content
+
-
 
+
-
* [http://www.gamasutra.com/view/feature/132483/learning_the_ways_of_the_game_.php?page=2 assessment using pros and cons, example is pros and cons of using a wiki use for collaborative work]  ''(I think this document is supposed to be less a 'how to' guide and more of a check list/framework for doing the theoretical and technical research and implementation. Perhaps we can start a page of solid design 'how to' resources elsewhere on the wiki)''
+
-
 
+
-
* [http://www.jenovachen.com/flowingames/introduction.htm flow in games] ''(This is an awesome paper but please describe how it fits in with the design framework. I'm not sure why this is here)''
+

Current revision as of 21:16, 4 December 2013

The GRAND Assessment Framework is an iterative assessment framework designed by the University of Alberta GRAND group. The framework starts with the assumption that the group research project involves practical game design, and continues by attempting to ask the most significant over-arching questions.

The framework is intended to help game designers and researchers articulate their points (and what those points are in the first place), create a foundation for further work, and to suggest a set of possible methods for that may be of use during that particular part of the design process. Fill in category sub-questions as best you can, and add your own or skip questions as needed.

Contents

1.0 Stakeholders and Expectations

  • 1.1 What's the point of the project?
  • 1.2 Who are the interested parties in this project?
  • 1.3 What does each stakeholder get out of the project?
  • 1.4 How will you prioritize the needs of stakeholders?

2.0 Requirements

  • 2.1 What is the primary purpose of the project?
  • 2.2 What evidence would indicate success? How will you know it is over?
  • 2.3 How will interested parties discuss the project as it evolves?
  • 2.4 Is there a project charter agreed among stakeholders that makes clear what is expected of everyone?

3.0 Resources

  • 3.1 What resources are required to do the project?
  • 3.2 How will you get the required resources?
  • 3.3 How will you account to stakeholders for the resources they have provided?
  • 3.4 How will you deal with the loss of resources?

4.0 Planning

  • 4.1 Is there a project plan? Does it include the level of detail needed?
  • 4.2 Is there a method for tracking progress regularly?
  • 4.3 How will the plan be adapted if you fall behind or lose resources?

5.0 Design

  • 5.1 What design process will work best for this project?
  • 5.2 How can you assess the design process?
  • 5.3 Which resources are needed to develop the game?
  • 5.4 Does the game meet your goals? Are there unexpected problems?
  • 5.5 Are there ways to improve the game or the platform based on the results of this attempt?

6.0 Delivery

  • 6.1 How will you deliver the product to your audience?
  • 6.2 How will they become aware of your project?
  • 6.3 How will you train the audience to understand and play the game?

7.0 Feedback

  • 7.1 What feedback do you and your stakeholders want and from who?
  • 7.2 How does the game work for players? What is their experience?
  • 7.3 How will reports from the players or other stakeholders be handled?
  • 7.4 Can we provide feedback to encourage player engagement with the game?
  • 7.5 How can feedback influence further design?

8.0 Closure

  • 8.1 What will the end-state of the project look like?
  • 8.2 Does the game meet its goals? How do you know that?
  • 8.3 How will you know if stakeholders are satisfied?
  • 8.4 Have you communicated to all interested parties that the project is over?

Related Wiki Resources

Personal tools