CIRCA:Assessment Framework

From CIRCA

(Difference between revisions)
Jump to: navigation, search
((Tentative) Assessment Framework Questions)
(8.0 Closure)
 
(69 intermediate revisions not shown)
Line 1: Line 1:
-
=(Tentative) Assessment Framework Questions=
+
The '''GRAND Assessment Framework''' is an iterative assessment framework designed by the University of Alberta GRAND group. The framework starts with the assumption that the group research project involves practical game design, and continues by attempting to ask the most significant over-arching questions.  
-
While this document is being composed, the [[CIRCA:initial questions to ask when starting group research projects with a practical game design component -|compact list]] of questions to ask when '''starting''' group research projects with a practical game design component can be found [[CIRCA:initial questions to ask when starting group research projects with a practical game design component -|on its own page. ]]
+
 +
The framework is intended to help game designers and researchers articulate their points (and what those points are in the first place), create a foundation for further work, and to suggest a set of possible methods for that may be of use during that particular part of the design process. Fill in category sub-questions as best you can, and add your own or skip questions as needed.
-
Use these question categories in the process of answering your own questions relevant to a specific project.
+
==1.0 [[CIRCA: Stakeholders and Expectations|Stakeholders and Expectations ]]==
-
Fill in category sub-questions as best you can, add your own or skip questions as needed.
+
-
==Affiliations (Stakeholders)==
+
*1.1 What's the point of the project?
-
*How should stakeholders be defined? The sociologist in me thinks like this: everyone has a stake in society and by extension any sub-project of society (like the butterfly effect).
+
*1.2 Who are the interested parties in this project?
-
Obviously for practical purposes you wouldn't want to waste too much time considering affiliations to the project that are intangible and speculative.
+
-
Nonetheless the premise of creative design is that you never know where relevant information might come from, so perhaps affiliations can be categorized by relevance, with placeholder categories at the bottom of the list for affiliations outside the relevant spectrum.
+
-
'''Stakeholders defined'''  (broadly) some entity that requires something from and/or is impinged upon by the project in some way. (specifically) those who invest in the project directly
+
*1.3 What does each stakeholder get out of the project?
-
*Who are the existing stakeholders for this project?
+
*1.4 How will you prioritize the needs of stakeholders?
-
**administrative
+
-
**ethical / legal
+
-
**research
+
-
**audience / customer
+
-
**others
+
-
*Who are the legacy stakeholders for this project or this type of project?
+
==2.0 [[CIRCA: Requirements|Requirements ]]==
-
**sponsors
+
-
**returning group members
+
-
**community partners
+
-
**audience demographics
+
-
**notion of a corporate "brand" with feedback mechanisms about its reputation
+
-
*Who are new stakeholders, or what are recent changes in makeup/orientation/capacity?
+
*2.1 What is the primary purpose of the project?
-
*What if any changes to this category can be expected during the course of this project?
+
-
*Who are potential stakeholders
+
*2.2 What evidence would indicate success? How will you know it is over?
-
** Is there a market or test group available, already existing or in demand?
+
-
**What are potential inclusion or exclusion factors (Does working with these stakeholders require REB ethics clearance?)
+
-
*Where can more information be found about a particular stakeholder, how can this information be used?
+
-
*What is the criteria for acquiring and updating information about stakeholders?
+
-
*How will you rank or sort stakeholders?
+
*2.3 How will interested parties discuss the project as it evolves?
-
**invests money
+
-
**invests time
+
-
**demands time
+
-
**demands money
+
-
**has social capital
+
-
**able to sanction in various ways (e.g. customers sanction a game by giving negative feedback or ignoring it)
+
-
**able to offer critique
+
-
**expected to offer critiques
+
-
**has demands that must be met
+
-
**has preferences that are considered within a project goal
+
-
**is ethically impacted by the project
+
-
**has more than one capacity or role
+
-
**always impacts project
+
*2.4 Is there a project charter agreed among stakeholders that makes clear what is expected of everyone?
-
**impacts constantly over a defined period of time
+
-
**impacts project at key points but not over a long period of time
+
-
**impacts project but we don't know when
+
-
**somewhat likely to impact the project
+
-
**has no relation to the project whatsoever (e.g. unicorns)
+
==3.0 [[CIRCA: Resources|Resources ]]==
-
==Expectations==
+
*3.1 What resources are required to do the project?
-
*What is the purpose of the project?
+
*3.2 How will you get the required resources?
-
* What is the requirement of the project at a minimum?
+
-
* What ought to be the shared ambition of this project? (each stakeholder should describe this)
+
-
**What is the agreed upon shared ambition for this project?
+
-
* What deliverables are due to each stakeholder (what are they getting out of the project)?
+
-
* What are the stakeholders' responsibilities to the group?
+
-
* To what extent is the project contract to be formalized or kept informal?  
+
-
* What is the project timeline?
+
*3.3 How will you account to stakeholders for the resources they have provided?
-
** What are our deliverables' timelines
+
-
* How will we conclude this project (what will the end-state look like?)
+
-
==Resources==
+
*3.4 How will you deal with the loss of resources?
-
* What are the deliverables?
+
==4.0 [[CIRCA: Planning|Planning]]==
-
* What is the budget?
+
*4.1 Is there a project plan? Does it include the level of detail needed?  
-
**Who is responsible for all aspects of the budget?
+
-
* What technology or tools are required to build, maintain, and play the game?
+
*4.2 Is there a method for tracking progress regularly?
-
* What is the length of time it would take to build and run the game?
+
*4.3 How will the plan be adapted if you fall behind or lose resources?
-
** Is there a deadline? Or can this project be delayed?
+
-
* Is there previous work, either ours or another group’s, in this area?
+
==5.0 [[CIRCA: Design|Design]]==
-
* How long should the project be able to last?
+
*5.1 What design process will work best for this project?  
-
* Are any outside stakeholders responsible for providing content/information/materials/funding/etc.?
+
*5.2 How can you assess the design process?
-
* Who is responsible for maintaining the project?
+
*5.3 Which resources are needed to develop the game?
-
* What does the end-state of the project look like?
+
*5.4 Does the game meet your goals? Are there unexpected problems?
-
==Execution==
+
*5.5 Are there ways to improve the game or the platform based on the results of this attempt?
-
* Does the game work?
+
==6.0 [[CIRCA: Delivery|Delivery]]==
-
** What doesn't work?
+
-
*** Is the problem technical, conceptual, or both?
+
-
** Can the game be fixed or is this a future cautionary tale?
+
-
* Who did the game work for (or not work for)?
+
*6.1 How will you deliver the product to your audience?
-
** How did the stakeholders react to the game?
+
-
* Does the game advance our research goals?
+
*6.2 How will they become aware of your project?
-
* Is this a successful research project?
+
-
* Is this a successful game?
+
-
* Are there ways to improve the game or the platform based on the results of this attempt?
+
*6.3 How will you train the audience to understand and play the game?
-
* Should the game be more intuitive or instructional?
+
==7.0 [[CIRCA: Feedback|Feedback]]==
-
** How quickly will the target audience be able to learn the game?
+
-
* If the project is repeatable, will we learn new things or benefit from running the game again?
+
*7.1 What feedback do you and your stakeholders want and from who?
-
==Feedback==
+
*7.2 How does the game work for players? What is their experience?
-
* Who and how do we want to give or receive feedback? (I think this needs to be broken down into several questions - SL)
+
*7.3 How will reports from the players or other stakeholders be handled?
-
** How will we present findings to our stakeholders?
+
-
** Who will we solicit feedback from?
+
-
*** What type of feedback is required? (qualitative? quantitative? game metrics?)
+
-
* What would be the most efficient method - time and resource wise - of gathering assessment data?
+
-
* Do the chosen assessment techniques require ethics clearance?
+
-
** Do we need new ethics or can this fall under a previous project?
+
-
* What tools will give us the most useful data?
+
-
* What questions should the assessment tools pose?
+
-
** Should there be multiple feedback tools available?
+
 +
*7.4 Can we provide feedback to encourage player engagement with the game?
-
(I think this section needs to be much more detailed since this was the original focus of the framework - the assessment process) - SL
+
*7.5 How can feedback influence further design?
-
==Assesment Schemas==
+
==8.0 [[CIRCA: Closure|Closure]]==
-
Since everything is going into a report - it might be useful to try to order the types of activities you might end up having to report into some hierarchies.
+
*8.1 What will the end-state of the project look like?
-
For example, I am trying to make a scale, between nothing to report on the one extreme, and unexpected success (to the extent that reporting has a life outside the project) on the other.
+
*8.2 Does the game meet its goals? How do you know that?
 +
*8.3 How will you know if stakeholders are satisfied?
-
**'''100''' (a new academic area opens up discussing the major impact of this game)
+
*8.4 Have you communicated to all interested parties that the project is over?
-
** (stuff in between)
+
=Related Wiki Resources=
-
 
+
*[[CIRCA:The Expanded GRAND Assessment Framework |The Original (expanded) GRAND Assessment Framework]]
-
*'''12''' best practices with general applicability could be suggested from the project
+
*[[CIRCA:Assessment Tools|Recommended Methods]]
-
*'''11''' Revisions were put into practice and were demonstratively better, revealing best-practices for a specific area
+
*[[CIRCA:Methods_for_Game_Design]]  
-
*'''10''' Comparisons led to good suggestions for revision
+
*[[CIRCA:Game Design Taxonomy|Game Design Taxonomy]]
-
*'''9''' Items implemented were evaluated in some meaningful comparison
+
-
*'''7''' Several things were implemented in a way that achieved goals.
+
-
*'''7''' Attempt to implement something was made that was well justified in terms of a concept/goal
+
-
*'''7''' An attempt to implement with results that have some useful diagnostic value
+
-
*'''7''' Any attempt to implement something was made
+
-
*'''6''' Choices were made about or between concepts in a way that can be demonstrated to align with goals
+
-
*'''5''' several implementation concepts were discussed in terms of goals, etc
+
-
*'''5''' Goals, skills, abilities, values etc were connected to an implementation concept
+
-
*'''4''' associations were categorized in relation to each other (e.g. creative writing for games is sometimes an ability, sometimes a skill that can be developed, and always contingent in determining whether to pursue a project where the gameplay is story-based)
+
-
*'''3''' associations were categorized into general areas (e.g. that's a goal, that's a method, a challenge, an ability, a value)
+
-
*'''2''' associations were categorized as in or out of bounds (e.g. a goal, value, or skill that is definitely or probably not relevant, relevant under some circumstances,or always relevant)
+
-
*'''1''' associations were made between project and ideas that may or may not be applicable to the project
+
-
*'''0''' nothing
+
-
 
+
-
* Thematic Assessments
+
-
**how well and in what ways did this project address the topic of gender?
+
-
 
+
-
**What was the educational value of the project?
+
-
 
+
-
==Misc. categories (fun, education, technology, etc...)==
+
-
 
+
-
* What are measurable variables that can be defined as '____'?
+
-
* Can the game teach anything different than a textbook, class, or other resources? ''(Not all of our games are designed to be serious games - this is a better question for the assessment section for specific games, will not apply to all projects)
+
-
** Can this game teach something that can be gained through other resources in a way that the targeted audience finds preferable? ( I think this is outside of the scope of our group, we are not really studying how best to teach a topic or curriculum, we are studying games.)''
+
-
* Is there technology involved? Or theory? (''This question is very broad, I'm not sure what it means and how it is different from resources section - needs to be defined better or broken up into sub questions)''
+
-
* Do we need to advertise?
+
-
* How will we launch the game and attract players?
+
-
** Do we have a captive player group or is the game released in the wild?
+
-
* Can we tweak the game during deployment? Is this a part of the plan?
+
-
 
+
-
==Other things to put in==
+
-
 
+
-
* '''Intellectual Property:''' What sort of license does it use? Who owns the work? Who published it? What is the intellectual property?
+
-
* '''Time and Money:''' How long did it take to make? How much did it cost to develop? What sort of organization developed it?
+
-
* '''Comparison and Competition:''' What other games are similar? Who is the competition? How does it compare to other games?
+
-
 
+
-
=Other Frameworks=
+
-
 
+
-
==External Links==
+
-
 
+
-
* [http://gamification.org/wiki/Game_Design Gamification Game Design Wiki]
+
-
has a long list of items - have a feeling that these are at different levels of importance so we should read through and sort.
+
-
* [http://circa.cs.ualberta.ca/index.php/CIRCA:RockwellGuide Another section of CIRCA wiki in which Geoffrey Rockwell discusses principles of digital project management in general.]
+
-
 
+
-
 
+
-
Bits of advice that could be part of a framework content
+
-
 
+
-
* [http://www.gamasutra.com/view/feature/132483/learning_the_ways_of_the_game_.php?page=2 assessment using pros and cons, example is pros and cons of using a wiki use for collaborative work] ''(I think this document is supposed to be less a 'how to' guide and more of a check list/framework for doing the theoretical and technical research and implementation. Perhaps we can start a page of solid design 'how to' resources elsewhere on the wiki)''
+
-
 
+
-
* [http://www.jenovachen.com/flowingames/introduction.htm flow in games] ''(This is an awesome paper but please describe how it fits in with the design framework. I'm not sure why this is here)''
+

Current revision as of 21:16, 4 December 2013

The GRAND Assessment Framework is an iterative assessment framework designed by the University of Alberta GRAND group. The framework starts with the assumption that the group research project involves practical game design, and continues by attempting to ask the most significant over-arching questions.

The framework is intended to help game designers and researchers articulate their points (and what those points are in the first place), create a foundation for further work, and to suggest a set of possible methods for that may be of use during that particular part of the design process. Fill in category sub-questions as best you can, and add your own or skip questions as needed.

Contents

1.0 Stakeholders and Expectations

  • 1.1 What's the point of the project?
  • 1.2 Who are the interested parties in this project?
  • 1.3 What does each stakeholder get out of the project?
  • 1.4 How will you prioritize the needs of stakeholders?

2.0 Requirements

  • 2.1 What is the primary purpose of the project?
  • 2.2 What evidence would indicate success? How will you know it is over?
  • 2.3 How will interested parties discuss the project as it evolves?
  • 2.4 Is there a project charter agreed among stakeholders that makes clear what is expected of everyone?

3.0 Resources

  • 3.1 What resources are required to do the project?
  • 3.2 How will you get the required resources?
  • 3.3 How will you account to stakeholders for the resources they have provided?
  • 3.4 How will you deal with the loss of resources?

4.0 Planning

  • 4.1 Is there a project plan? Does it include the level of detail needed?
  • 4.2 Is there a method for tracking progress regularly?
  • 4.3 How will the plan be adapted if you fall behind or lose resources?

5.0 Design

  • 5.1 What design process will work best for this project?
  • 5.2 How can you assess the design process?
  • 5.3 Which resources are needed to develop the game?
  • 5.4 Does the game meet your goals? Are there unexpected problems?
  • 5.5 Are there ways to improve the game or the platform based on the results of this attempt?

6.0 Delivery

  • 6.1 How will you deliver the product to your audience?
  • 6.2 How will they become aware of your project?
  • 6.3 How will you train the audience to understand and play the game?

7.0 Feedback

  • 7.1 What feedback do you and your stakeholders want and from who?
  • 7.2 How does the game work for players? What is their experience?
  • 7.3 How will reports from the players or other stakeholders be handled?
  • 7.4 Can we provide feedback to encourage player engagement with the game?
  • 7.5 How can feedback influence further design?

8.0 Closure

  • 8.1 What will the end-state of the project look like?
  • 8.2 Does the game meet its goals? How do you know that?
  • 8.3 How will you know if stakeholders are satisfied?
  • 8.4 Have you communicated to all interested parties that the project is over?

Related Wiki Resources

Personal tools