Week 2
June 26, 2012: Closely evaluating the content of the tool, drafting the the user test tool, developing a prototype for the kit
Summary of Today’s Post:
– Evaluated NOAA template/guide content
– Brainstorm user testing tool
– Prototype of ‘kit’
In order to be systematic about trying to improve the content of the DMPtool, I decided to go through each of the templates/guides for the funding groups that pertain to the Earth Sciences community. The following 5 relate to this community:
– NOAA
– NSF- AGS
– NSF-AST
– NSF-BIO
– NSF-EAR
Am I missing others? I didn’t know if chemistry would count?
Evaluating NOAA:
So far I have gone through the NOAA template and noted down areas that I think could be improved, made more clear, where additional resources and links could be added, etc.
Resources (to the right of the template):
– This is a quick fix, yet it should definitely be addressed–the links to the resources are all broken.
1. General Description of data to be managed
– Good
2. Points of contact
– Good
3. Data Stewardship
– I think some explanation about what stewardship is, might be helpful. This thought lead me to my next, which was, I think something to be tested during the user testing could be how much of the explanations and descriptions used in this tool are particularly jargon-y or only understandable to a library community. Do researchers use terms and phrases like stewardship? I am not sure.
– Perhaps a link to a resource that describes what the data lifecycle is, though it may be self-explanatory in one sense, I think when we talk about the lifecycle we mean something very specific and imagine that oft referred to DCC data lifecycle image.
-the language used here “available to the customer”…. is customer the right word? I’m not sure people are selling and buying data, but rather making it available to future researchers, students, ect.
4. Data Documentation
– “which metadata repository will be used to document this data collection?” Is this referring to the metadata standard? This seems like it may be confusing, because when I think of a metadata repository I think of a data registry or something, but not a standard used to document data.
5. Data Sharing
– I think a link/resource to information about OPeNDAP or OGC might be helpful, once again, I wonder if researchers are familiar with data access protocols? Which is why I think a linked resource might be beneficial here.
– “In what catalogs will these services or data be made registered to enable discovery by users and other catalogs.” Strange wording that might confuse the user.
6. Initial Data Storage and Protection
– Good
7. Long-term archiving and preservation
– Good
Was this exercise helpful? It definitely helped me to think about the types of questions I would want to ask users. Unless anyone has any objections, I will continue doing this for all 5 of the templates I’ve identified as being part of the DataONE community.
Brainstorming ideas for the User Testing Tool:
As I am waiting to hear back from Laura Miller at UVa, I began thinking about the types of questions/areas to concentrate on. So far I think I want to focus on 4 main sections of the user test: background information, contents of the tool, the resources provided, and the design. In the parentheses following each section below, I explain exactly why I am concerned with this main section as it relates to the earth sciences community. Then are some ideas for the kinds of questions I can ask researchers. (I have never conducted user testing, so I am not entirely sure what it should entail. Feedback/resources/suggestions requested!)
A) Background information from each of the users (have researchers needed to construct DMPs before? Where have they received help? Was creating a DMP difficult?) :
1. Have you created a data management plan before?
2.What were the most difficult aspects?
3. Have you ever sought help from the library at your university when considering data management or questions about data management, metadata, data storage, etc.?
B.) Contents of the DMPtool: the language used (does the tool use too much library jargon? Does the obscure language deter users? Though earth scientists have jargon of their own, a lot of the language in this tool seems to be library-related jargon and may be confusing to researchers)
1. Are there any particular words used that seem to be particularly confusing or obscure?
2. Are you familiar with the data lifecycle?
3. Are you familiar with data access protocols?
C.) Resources provided by the DMPtool (do extra resources need to be provided to assist researchers in creating their DMPs? Specifically, is metadata the most difficult aspect? Based on my research of the prominent organizations in the earlth sciences community, very little attention is paid to metadata and data management, with exception of some, and so this may be a problem area. Will a tool kit about the most widely used metadata standards encourage more researchers to use the DMPtool?)
1. Are you comfortable with the term metadata?
2. Do you know which metadata standards are most typically used by your community?
3. Does this ‘kit’ help you understand the standards better?
D.) Design of the tool (is the layout and design of the tool user friendly? Not quite sure the impetus for this section specifically as it relates to earth scientists, but this is still an important section to consider improvements to, agreed?)
1.??
Prototype for the DMPtool Kit:
I have also created a snap-shot of the prototype of the ‘kit’. The snapshot is the page about metadata standards. Let me know if the link doesn’t work.
Thanks!
-r
Rachel,
A few comments/questions:
1) overall, this is good and sounds like worthwhile learning for you in the process. I think you have some good ideas and insights and are starting to see it the way we believe researchers do. Continue to work on testing your assumptions by gathering real evidence and feedback. As you think about questions for the researcher, approach it from their point of view…why are they being asked for the DMP? What are they trying to accomplish by preparing the DMP? What do they most want to see in DMP assistance?
2) in trying to coordinate with Laura, I would recommend that you try to arrange a time for a phone call. Talk with her about your main objectives and see what advice and direction she can offer. She has done other tests and can help you develop the appropriate structure.
3) Curious about your progress on one of the week one goals….”Establish metrics/goals for evaluating the success of the marketing effort”. This is important in order to know whether your efforts are successful or not in the end. Thoughts?
Andrew