{"id":3256,"date":"2018-07-20T19:38:15","date_gmt":"2018-07-20T19:38:15","guid":{"rendered":"https:\/\/notebooks.dataone.org\/?p=3256"},"modified":"2019-05-24T17:47:17","modified_gmt":"2019-05-24T17:47:17","slug":"week-7-dataone-metadata-parser","status":"publish","type":"post","link":"https:\/\/notebooks.dataone.org\/extending-libmagic\/week-7-dataone-metadata-parser\/","title":{"rendered":"Week 7: DataONE Metadata Parser"},"content":{"rendered":"

Hi All,<\/p>\n

This blog is in follow-up with my earlier blogs for the Project 4: Extending Libmagic for Identification of Science Resources<\/a>. In this week, we discussed to add more functionality in our parser and to make it easily configurable for the new file formats to add.<\/p>\n

    \n
  1. Using a configuration file:<\/strong> During our daily meetings, we found that it would be very useful and helpful to have a configuration file which can be used for adding new metadata fields and file format. We created a configFile.xml<\/a> which is an XML containing the metadata fields for the different file format. The file contains the metadata fields as tags or XPath expression. It contains the namespace tags if the file format uses a prefix tag. The application read the config file for detecting file format and then returns the XPath for extracting the metadata field for the file type. If the file is not a dataone file format, it should be handled by a different parser.<\/li>\n
  2. Using XPath for extracting contents: <\/strong>The DataOneXMLParser.java <\/strong>have one method for extracting values using the getelementtags method, and we added one more using the XPath expression. This would help in extracting the specific values using the path if there are multiple values for the same field. We faced some issues while extracting the<\/li>\n
  3. Default Tika parser for default file format: <\/strong>In DataOneMapper.java <\/strong>class we created a method, which returns the List of metadata fields using a default Tika parser if the file is not a dataone file. This handles the boundary case for our application.<\/li>\n
  4. SimpleContex<\/a> class: <\/strong>When we added the XPath expression as a field in the configuration file, it didn’t work as expected in the first run. The XPath for the onedcx file where not resolving successfully. The onedcx file has tags which use prefix as “dc” and “dcterms” and for resolution of the correct path, the namespace needs to be set for these prefixes. The use of namespace and prefix required to create a class SimpleContext<\/a> which sets the prefix and namespace URI in a hashmap list. The XPath object uses the setNamespaceContext method for setting the namespace for the prefix. This worked fine after a small alignment of the method calls in the class.<\/li>\n<\/ol>\n

    In the next week, we will be working on how to use the namespace fields and set them while using the XPath expression. We will be performing some cleanup with the code and testing application for stability.<\/p>\n

    That\u2019s all for now, see you all next week!<\/p>\n

    Have a great weekend!<\/p>\n

    Resource links: Github-file_identification<\/a>, Github-DataONE Parser,<\/a>  Project Plan<\/a><\/p>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"

    Hi All, This blog is in follow-up with my earlier blogs for the Project 4: Extending Libmagic for Identification of Science Resources. In this week, we discussed to add more functionality in our parser and to make it easily configurable for the new file formats to add. Using a configuration file: Continue reading Week 7: DataONE Metadata Parser<\/span>→<\/span><\/a><\/p>\n","protected":false},"author":113,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[389],"tags":[],"_links":{"self":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/3256"}],"collection":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/users\/113"}],"replies":[{"embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/comments?post=3256"}],"version-history":[{"count":3,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/3256\/revisions"}],"predecessor-version":[{"id":3346,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/3256\/revisions\/3346"}],"wp:attachment":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/media?parent=3256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/categories?post=3256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/tags?post=3256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}