CIRCA:Research Plan


Jump to: navigation, search


Research Plan

The Ukrainian Folklore Audio Project goals are to examine how crowds transcribe and translate Ukraine audio clips. Natalie Kononenko is the project leader, with Geoffrey Rockwell in charge of designing the interface. Our objectives in this project are to examine the impact and use of “group-sourcing.” We will look at the motivations behind each volunteer, the effectiveness of group sourcing, and the methodology behind the project. Designing the interface for a project that employs group sourcing, allows us to research this type of approach to scholarly works.

Usability of the site

As we design, the interface usability is always foremost in our minds (as it should be). How will people use it? Have we made everything easily available? Is the site too busy or have we left anything out? The team created personas and scenarios to ensure that the website incorporates the needs of volunteers in every step. By developing personalities for the users, we can predict their demands and goals. “Group-sourcing,” a take on the common phrase “crowd-sourcing,” is how the volunteers will approach translating and transcribing the audio clips. This is a great opportunity for us to figure out what works and what does not, and to create a “group-sourcing” guide for the scholarly community as a way to track and organize our research. To ensure the sites usability for the volunteers, a workshop will be held in January. We aim to see if the design and all the bugs of the website are worked out. To find out more information about the volunteers and their experience with the Ukrainian Folklore Audio Project’s site, a questionnaire is handed out to the participants. The information we receive from those users will let us change the interface design and understand the volunteer’s perception of “group-sourcing” and their roles in it. Did they enjoy it? What did they do? Was it hard to navigate the site? Why did you volunteer? The interface for the Ukrainian Folklore Audio Project is the foundation for “group-sourcing.”

Usage by the volunteers

The volunteers are considered at every point in this project. Personas and Scenarios are created to predict what the users need. A workshop is held to help them understand and use the website. We will hold interviews with the volunteers around March to learn about their behavior and motivations involved with this project. How do they use the site and participate? What do they actually do? Popular opinion says that there will be a long tail effect, where some people will do most of the work while most people will do some (if any) of the work. Is this true? The interviews with the participants let us find out more than what the website data shows. An additional questionnaire will be handed out at this time to see if there were any changes in attitudes and perceptions of the site. In this, we will ask if anyone helped the volunteer (like a family or community member) in handling the technology, and is that something that we want to address for future users. The motivations, triumphs and problems these people face interest us. We will examine the data of both the volunteers and the administrators – are the majority of the transcriptions or translations correct? How long did it take them to complete a clip? Natalie Kononenko will edit all the submitted works to make sure of the scholarly quality of documents posted on the website. We will make available the transcribed and translated works; this may be behind participant’s motivation. Similar projects like “Transcribe Bentham” provide motivation through tokens and award the most proficient transcribers titles like “expert,” and “amateur.” The Ukrainian Folklore Audio Project does not provide those award incentives, but we are a much smaller group. This is an excellent opportunity to compare the two different approaches. We have looked at motivation from our viewpoint, but another option is to look at is how the volunteers motivate themselves. Will recognition for their contributions be enough? What other factors encourage them? The interviews we plan to do will include questions along these lines. Transcribing and translating the audio is important to our research. Usually the process goes transcribing the audio to translating the transcription. We hope that some of the volunteers who are more comfortable speaking and listening in Ukrainian, but not writing the language will skip the middle step. Translating audio directly from the source to paper is rare and it is a rich source of potential research.


The documentation of our project includes everything people write on their accounts. Participants discuss the Ukrainian Folklore Audio Project in the forum; this allows us to track data found within their conversations. The comments and keyword entries that volunteers add to any of the documents they transcribe/translate let us see how they related to Ukrainian songs and stories. The comments may also provide us information on the participant’s opinions about the site. Data about what each person does in the project is logged to make sure we document everything. Looking at the transcriptions and translations, we will compare them. There may be multiple transcriptions and translations of an audio clip. How do the different texts compare? What are the differences? Seeing how three people tackle a clip may depend on their origins or how they were raised, but the technology may also influence them. People do not write the same way on the computer as on paper, and someone may have helped them.

General Issues around crowd sourcing

Crowd sourcing is not always the ideal way to get things done. Problems found in crowd sourcing are not as serious for the Ukrainian Folklore Audio Project. We limit the number of volunteers, reducing the project to "group-sourcing." Quality is an important issue that we face, but with thorough editing of submissions and controlling who the participants are will limit this. Not everyone can become part of the Ukrainian Folklore Audio Project, criteria must be met. Motivation is often a problem, “Transcribe Bentham” used a reward system but we do not. Is that going to be a problem? Clarity is a particular issue, especially between the interface designers and volunteers. When two languages are involved, there will always be errors in communications, how will we circumvent this? “Group-sourcing” is often looked at as only a tool, but if there are favorable results, this may be the future of scholarly translations and transcriptions.

Personal tools