CIRCA:Viral Analytics Paper


Jump to: navigation, search


In order to determine how analytical badges could be useful to editors of online content we conducted usability interviews with journal and blog editors in the digital humanities. We chose ten humanities journals and blogs and contacted an editor by email from each journal to see if they would take part in an interview. This email gave a brief outline of the project and what we hope to achieve in the interviews.

Once the editor consented, an ethics form was sent and an interview over skype was scheduled. Before the interview we sent the interviewees a pdf of the current eVoyeur tool to look over and try out for themselves. This pdf displayed a screenshot of eVoyeur in an OJS journal and gave a description of the tools that are already in the tool. These tools are Cirrus, Frequency Grid, Summary, Links, and Reader. The last part of the pdf outlined the filters and the option buttons.

At the beginning of the interview the editors were informed that notes would be taken for reference during the interview. In the interview we asked the editors their role in their journal or blog. We then asked them about the pdf they saw and what they thought of the tool; what parts they liked and what parts they disliked. In order to make the tool accessible to many journals we asked what formats their articles are displayed in and if other text formats were available. In order to improve the tool we asked the editors what sorts of tools would be useful to their audience and what kinds of tools would they like to see in their journal or blog? The last set of questions asked if the editor though eVoyeur would be a useful tool for text analysis and if they would like to test out eVoyeur by embedding it in their journal.

After the interview we typed up the notes taken during the interview and emailed them to the interviewee to double check everything was correct.


Since Voyeur is embeddable it is more accessible and useful than other text analysis tools. One person liked that eVoyeur is embeddable and thinks it is an ‘intelligent move’ for Voyeur. However, the current panel on OJS is very small, making it difficult to use. It would be better if eVoyeur pulled up words in context since right now it only pulls up individual words. The tool is small and available and is interesting to make links. Another person felt that eVoyeur works well for a quick visualization but is not smooth and doesn’t feel like part of the journal. It is also hard to manipulate, which takes away from the usefulness of the tool. The automation of eVoyeur is also attractive and the idea of linking terms and visualizations is a nice idea.

Cirrus: 7 out of 10 people in our study liked Cirrus. Overall the respondents felt that Cirrus was the most useful tool. “Cirrus worked the best and is very understandable.” However, one user thought Word Clouds were cute but not powerful.

Frequency Grid: 6 out of 10 people we interviewed liked the Frequency Grid. The Frequency Grid looks useful but would be better if you could see the concordance as well. The frequency charts associated with the tool were cool but were not understandable at first. Users new to this type of chart would need some explanation to understand what it meant, which the small panel does not provide. One user felt that the Frequency Grid was too complicated to effectively work well in the tiny box. In order to use it to its full extent the user would have to export the data to Voyeur tools in order to view the results in the full window.

Summary: 4 people out of 10 people interviewed liked the Summary tool. However, the Summary does not get to the info about the document. Summary was interesting but seems more like a meta tool.

Links: 6 out of 10 people interviewed liked Links. Links would be good if you can use this to navigate between articles in the entire journal. This would make it easier to find other articles related to each other.

Reader: 3 out of 10 people liked the Reader. One user “is looking for [a tool] to help read texts more efficiently”, which the reader does not effectively do since it is difficult to use in such a small window. At the moment only a few words can be displayed per line in the panel, causing documents to become harder to read, as the user has to constantly scroll down or export the data to Voyeur tools in order to view it in the full screen.

Since eVoyeur is in its preliminary stage, the journal/blog editors were asked what additional tools would be useful in an embedded tool. Some of the respondents would like eVoyeur to be used more effectively in an archive. Therefore, better searchability or linkability would be important. A trends list to find trends in a particular field within the journal would be useful for an archive. One user would like the Links tool to see connections between articles in the entire journal so you can see issues throughout the journal easily. Another user “would like a tool that allows you to search or summarize articles in the entire journal.” The tool could be improved if it was “able to pull up words in context, such as a concordance”. One respondant would like the tool to “come up as a stable pre-formed page with many tools displayed at the same time. The search on frequency grid and cirrus should be pre-run and displayed once the page is opened. The tools should then be separated down the page similar to Google Books.” That way the user immediately sees everything the tool can perform and there is no wait for the selected tool to load. Similar to this, a visualization tool to bring together all 5 tools would be useful. In this visualization the user would be able to see the tag cloud with its associated frequency chart and links all in the same visualization. In addition, a TEI wrapper, personography and placeography markup would be a beneficial feature. It would be useful if eVoyeur could look at text in multiple versions and analyze the differences between those versions. Name and entity recognition, which would also be important for analysis, should be incorporated into eVoyeur. One user is interested in discourse, such as “how authors speak of topics and how it changed over time or change between institutions”. It would be useful if there was a tool that could break down authors and themes in articles so you can see similar themes or authors in the journal. The tool should also be capable of processing unstructured data with both XML and HTML.

We asked the editors whether eVoyeur would be useful in their journal or blog. Some of the journals interviewed already use text analysis, although it is not embedded into the journal or blog. One user would like the tool to do more, such as collocates and concordances. While another user thinks eVoyeur is useful if you want to do more quantitative research, but is not entirely useful to most users. It would be better if it were entirely web based and located in the cloud. eVoyeur would be useful but most readers are suspicious of the text analysis route. They see it as bibliographic searching and giving them too many options without asking them if they want the tools may overwhelm the user. eVoyeur should have an API or be integrated in the interface to make it more useful. In addition, the tools should be XML aware in order to be valuable to a journal or blog.

Personal tools