CIRCA:Ethics Draft

From CIRCA

Jump to: navigation, search

Contents

Ethics

Research Plan

The Ukrainian Folklore Audio Project goals are to examine how crowds transcribe and translate Ukraine audio clips. Natalie Kononenko is the project leader, with Geoffrey Rockwell in charge of designing the interface. Our objectives in this project are to examine the impact and use of “group-sourcing.” We will look at the motivations behind each volunteer, the usability of the website, the effectiveness of group sourcing, documentation logged from the website and the methodology behind the project. Designing the interface for a project that employs group sourcing allows us to research this type of approach to scholarly works.

Research Goals

  • Environmental Scan, to survey the varied uses of crowd-sourcing in the humanities. Suda Online and Transcribing Bentham are two related projects that rely on volunteers to transcribe and translate humanities material.
  • Developing the Interface, to create an accessible and functioning website for the volunteers (many who do not have an in depth knowledge of technology) to participate in “group-sourcing”.
  • User Study to research how the volunteers used the website and how they approached transcribing and translating Ukrainian folklore from audio clips. To identify any challenges that the volunteers faced in this project through questionnaires and interviews.
  • Evaluation to understand the “group-sourcing” process that volunteers undertook during this project.


Research Objectives

1. Usability of the site – how to make it work for the target audience

  • User-friendliness
  • January a training session/usability on the interface
    • Work with three volunteers to come in and work with the interface and then run a workshop – handout a questionnaire
  • Volunteer interviews
  • Designing for the crowd – what works and what doesn’t?
  • Personas and Scenarios
  • Documenting the usability of the website
    • What is necessary
    • Human response
    • A “group-sourcing” guide
  • It does mention the developing a generalized social network research tool that can handle text and other media for others to adopt.
    • Sharing code.


2. Documentation

  • Log the data about what they do
  • Discussion Forum
    • Discover data through online discussion between the participants.
  • Examining the descriptions being given by the participants
    • Comments on the transcription/translation submit page.
    • The difference between multiple transcriptions of the same text
  • How does this reflect on the use of crowd sourcing?


3. Usage by the volunteers – group-sourcing behaviour and motivation.

  • How do they use the site and participate?
  • Once we have a working system, what do they actually do?
    • Long tail effect
  • Test the hypothesis to transcribe rather than translate (or vice versa)
  • Interview them after the process
  • What do we want to log? Goes to the developer
    • How long do they sign clips out? And sign them back in.
  • Long tail effect
  • Motivation, problems and triumphs
    • Look at the statistics, are the transcriptions/translation accurate? How long did it take them to complete a clip?
    • Look at Transcribing Bentham – the crowd receives points and titles (novice, amateur, etc.)
    • How we motivate the volunteers and how they motivate themselves.
  • Working with people to teach/train them to use the system
    • Volunteer – Designer interaction
    • The elements of Cultural “group-sourcing”


4. General issues around crowd sourcing.

  • “Group-sourcing” is mentioned but as a tool, we may want to examine it as the future of scholarly translations or transcriptions.
  • Quality of submissions – how to control that
    • Editors, and the volunteers are invited or vetted
  • The volunteers are not professionals
  • Motivation
  • Taking care of the group-sourcing community
  • Clarity


Methodology

  • Environmental scan of related projects and other “crowd-sourcing” sites.
  • Personas and Scenarios are created to predict what the volunteers will need.
  • Wireframes, to plan and outline the interfaces as well as limit elements to only what is necessary (information gathered from the personas and scenarios).
  • Research Proposal and Agenda to plan-out how we will approach research in this project.
  • Training Sessions to introduce the volunteers to the website and find out information on the motivations, background and ability of the volunteers.
    • Questionnaires are handed out to gather background information from the participants along with their opinions of the website
    • Researchers will observe the volunteers using the site.
  • Research Protocol will be developed to organize any information gathered during this session
  • Logging website documentation to inform us of how the participants use the site (what do they do? How long does it take them? Etc.).
    • Suggestions, comments, questions and problems are reported through the website. There are buttons along the side of certain WebPages that easily allow volunteers to do this.
  • Final Interviews at the end of the year let us get the volunteer’s opinions on the overall project.
Personal tools