Firstly, thanks to those who took a moment to respond to my one-question poll last week. Opinions were split evenly, with half of the voters preferring different content and half preferring maintaining exactly the same script.
In addition to the poll, I got some great feedback from several readers on experimental design using surveys. The format of the DUG meeting break-out sessions is fairly limiting: we don’t have an opportunity to truly randomize the order in which people view the alternative video styles; it’s difficult to measure whether people have learned from the tutorials; and it’s hard to control for the multiple variables that would be involved when differing the style of videos by more than one effect (zoom and highlighting, say, or sound effects and colorful callouts).
So we’ve decided to use time at the break-out meeting in a more informal way, soliciting feedback and discussion rather than trying to shoe-horn in a rigorous or controlled study. I’ll use the feedback to inform my decisions for the videos I’m recording for the ONEMercury tool. This also means the different videos can focus on different parts of the tool for feedback on content and scripting as well as style. Hopefully we’ll be able to conduct a more rigorously controlled study online, something with sample size and testing conditions fit for statistical analysis.
Next week I’ll continue to record tutorial videos on different parts of the ONEMercury tool. The basic search video is more or less complete, perhaps one more style to be done. I’d like to focus next on the geographic search and move on from there to interpreting search results, so at the DUG meeting I can show how the different styles look in two very different search contexts.