{"id":2734,"date":"2016-05-20T23:34:12","date_gmt":"2016-05-20T23:34:12","guid":{"rendered":"https:\/\/notebooks.dataone.org\/?p=2734"},"modified":"2016-05-20T23:34:56","modified_gmt":"2016-05-20T23:34:56","slug":"week-1-evaluate-instrument-for-education-literature-review-part-1","status":"publish","type":"post","link":"https:\/\/notebooks.dataone.org\/education-evaluation\/week-1-evaluate-instrument-for-education-literature-review-part-1\/","title":{"rendered":"Week 1: Evaluate Instrument for Education &#8211; Literature Review Part 1"},"content":{"rendered":"<div>On Friday, May 13th, I had my Face-to-Face Meeting with two of my mentors, Amber Budden and\u00a0Heather Soyka (my third mentor is\u00a0Viv Hutchison), to discuss my project,\u00a0Developing a Survey Instrument for Evaluation of Teaching Materials, as well as to develop the project plan for all three parts of the project. The three parts of the project include:\u00a01) a literature review that looks at the best practices for soliciting and receiving user feedback on teaching and educational resources; 2) the development of an assessment tool for evaluating current educational resources from DataONE; and 3) identification of target audiences and recommendations for deployment of the tool.\u00a0As a result of the Face-to-Face Meeting and the project plan, I started\u00a0the first week of my project with the first part of the 2-week literature review phase.<\/div>\n<div>\u00a0\u00a0\u00a0\u00a0 For the first part of the literature review, I focused primarily on looking for information that could help in informing the best practices and actual examples of methods for evaluating education\/training resources. Since my mentors and I had also identified four different education\/training delivery modes that were relevant to DataONE (i.e. in person, online, presentation slides, and text-based documents), I also conducted additional literature reviews that focused on the evaluation of these specific education\/training formats. The following is the list of keywords and the databases that I used to perform the literature review.<\/div>\n<div><\/div>\n<div><\/div>\n<ul>\n<li>Search Engine\/Database &#8211; Google (for information that are published outside of scholarly journal format)\n<ul>\n<li>Searched keywords\/phrases &#8211; (the keywords\/phrases were constructed during the Face-to-Face Meeting)\n<ul>\n<li>training evaluation criteria<\/li>\n<li>develop a method of training evaluation<\/li>\n<li>how to evaluate training effectiveness<\/li>\n<li>education evaluation methods<\/li>\n<li>education effectiveness assessment methods<\/li>\n<li><span style=\"font-size: 14.6667px\">online education elearning evaluation criteria<\/span><\/li>\n<li><span style=\"font-size: 14.6667px\">e learning evaluation survey<\/span><\/li>\n<li><span style=\"font-size: 14.6667px\">e learning evaluation questionnaire<\/span><\/li>\n<li><span style=\"font-size: 14.6667px\">presentation evaluation criteria<\/span><\/li>\n<li><span style=\"font-size: 14.6667px\">best practices when evaluating presentation<\/span><\/li>\n<li>syllabus evaluation criteria<\/li>\n<li>evaluate effectiveness of written communication<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li>Search Engine\/Database &#8211; (for information that are published mainly as peer-reviewed journals)\n<ul>\n<li>Library &amp; Information Science Source from UIUC Library<\/li>\n<li>ERIC (Education Resource Information Center) Education Literature from UIUC Library<\/li>\n<li>Association for Computing Machinery (ACM) Digital Library from UIUC Library\n<ul>\n<li>Searched keywords\/phrases &#8211;\u00a0((training OR workshop) AND effectiveness AND evaluation AND (criteria OR method))\n<ul>\n<li>The same phrase was used for all three databases and was constructed as a result of reviewing the search results from Google.After reviewing more than 750 results, the preliminary literature review result indicates that questionnaire\/survey\/poll might be the most applicable method for soliciting user feedback for DataONE&#8217;s education\/training resources. This is mainly due to the fact that questionnaires\/surveys\/polls can be administered digitally. Also, when designing the questionnaires\/surveys\/polls, evaluation areas such as learning objectives, content structure, reasoning, language\/vocabularies, level of interactivity, usage of visual aides, technical functions, timing\/duration, and return\/retention rate of attendees can all be included. Additionally, specific evaluation areas could\u00a0be emphasized further depending on the category\u00a0of the education\/training delivery methods that is\u00a0being evaluated. However, care should be taken to ensure that the questionnaire\/survey\/poll is easy to use and is not overly time consuming for the users. Further, the frequency of the evaluation (e.g. before vs. during vs. after the education\/training session) and the types of the evaluation questions (i.e.\u00a0Open ended,\u00a0Closed\u00a0Scalar,\u00a0Multi-choice,\u00a0Checklist,\u00a0Ranked\/Ranking) can both be adjusted to support the evaluations more effectively.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<div>After reviewing more than 750 results, the preliminary literature review result indicates that questionnaire\/survey\/poll might be the most applicable method for soliciting user feedback for DataONE&#8217;s education\/training resources. This is mainly due to the fact that questionnaires\/surveys\/polls can be administered digitally. Also, when designing the questionnaires\/surveys\/polls, evaluation areas such as learning objectives, content structure, reasoning, language\/vocabularies, level of interactivity, usage of visual aides, technical functions, timing\/duration, and return\/retention rate of attendees can all be included. Additionally, specific evaluation areas could\u00a0be emphasized further depending on the category\u00a0of the education\/training delivery methods that is\u00a0being evaluated. However, care should be taken to ensure that the questionnaire\/survey\/poll is easy to use and is not overly time consuming for the users. Further, the frequency of the evaluation (e.g. before vs. during vs. after the education\/training session) and the types of the evaluation questions (i.e.\u00a0Open ended,\u00a0Closed\u00a0Scalar,\u00a0Multi-choice,\u00a0Checklist,\u00a0Ranked\/Ranking) can both be adjusted to support the evaluations more effectively.<\/div>\n<div>\u00a0 \u00a0 Continuing the literature review process next week, I will look further into the scholarly publications that are related to perform\u00a0evaluations effectively and the information available regarding the topics of questionnaire\/survey\/poll design and implementation.<\/div>\n","protected":false},"excerpt":{"rendered":"<p>On Friday, May 13th, I had my Face-to-Face Meeting with two of my mentors, Amber Budden and\u00a0Heather Soyka (my third mentor is\u00a0Viv Hutchison), to discuss my project,\u00a0Developing a Survey Instrument for Evaluation of Teaching Materials, as well as to develop the project plan for all three parts of the project. <a class=\"more-link\" href=\"https:\/\/notebooks.dataone.org\/education-evaluation\/week-1-evaluate-instrument-for-education-literature-review-part-1\/\">Continue reading <span class=\"screen-reader-text\">  Week 1: Evaluate Instrument for Education &#8211; Literature Review Part 1<\/span><span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":103,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[369],"tags":[],"_links":{"self":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/2734"}],"collection":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/users\/103"}],"replies":[{"embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/comments?post=2734"}],"version-history":[{"count":2,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/2734\/revisions"}],"predecessor-version":[{"id":2736,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/2734\/revisions\/2736"}],"wp:attachment":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/media?parent=2734"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/categories?post=2734"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/tags?post=2734"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}