{"id":2028,"date":"2014-04-14T21:12:23","date_gmt":"2014-04-14T21:12:23","guid":{"rendered":"https:\/\/notebooks.dataone.org\/?p=2028"},"modified":"2014-04-14T21:12:23","modified_gmt":"2014-04-14T21:12:23","slug":"text-processing-methods-continued-pdf-to-html-conversion","status":"publish","type":"post","link":"https:\/\/notebooks.dataone.org\/data-science\/text-processing-methods-continued-pdf-to-html-conversion\/","title":{"rendered":"Text Processing Methods, Continued (PDF to HTML Conversion)"},"content":{"rendered":"

I am continuing evaluation of some text processing tools that I began in an earlier open notebook post<\/a>\u00a0on the same topic.<\/p>\n

I also had an idea that perhaps I should open my PDF documents in Word, then re-save them as HTML. \u00a0That workflow might standardize the formatting to something less “bizarre” in terms of the underlying formatting.<\/p>\n

iPub PDF to HTML for mac (ipub-pdf-to-html-for-mac.dmg)<\/p>\n

Only 15 files at a time can be converted with this trial ware version, and only 5 pages at a time (which is really not problematic for most LinkedIN profile PDFs, at least among those in my collection).<\/p>\n

I chose Amber Budden, Rebecca Koskela, Mill Michener, and myself to convert. The output location is as follows:<\/p>\n

\/Volumes\/TOSHIBA\/ipubsoft-output<\/pre>\n
As before “TannerJessel.pdf” is available for evaluation on this open notebook’s media management page (See\u00a0TannerJessel<\/a>\u00a0pdf file).<\/p>\n
The output is all one page, which is good. It appears to only go to 2 pages – this may be a limitation of the trialware.<\/p>\n[code language= “HTML”]\n
<DIV style=\"position:absolute;top:912;left:54\"><nobr><span class=\"ft02\">Skills&nbsp;&amp;&nbsp;Expertise<\/span><\/nobr><\/DIV> <DIV style=\"position:absolute;top:946;left:54\"><nobr><span class=\"ft09\"><b>Science<br>Higher&nbsp;Education<br>Teaching<br>Grant&nbsp;Writing<br>Research<br>Lifesciences<br>Data&nbsp;Analysis<\/b><\/span><\/nobr><\/DIV><\/pre>\n[\/code]\n
Two words have spaces, Grant Writing and Data analysis. In this version all non breaking spaces have %nbsp; which might be helpful.<\/p>\n
There is some structure, which is potentially useful.<\/p>\n
In my “converted” LinkedIN profile, my information is presented as follows:<\/p>\n
<DIV style=\"position:absolute;top:57;left:54\"><nobr><span class=\"ft00\"><b>Tanner&nbsp;Jessel<\/b><\/span><\/nobr><\/DIV>\r\n <DIV style=\"position:absolute;top:94;left:54\"><nobr><span class=\"ft01\">Web&nbsp;Content&nbsp;Strategist,&nbsp;Bioinformatics&nbsp;&amp;&nbsp;Internet&nbsp;Services&nbsp;Specialist<\/span><\/nobr><\/DIV>\r\n <DIV style=\"position:absolute;top:125;left:54\"><nobr><span class=\"ft02\">mountainsol[edited out]gmail.com<\/span><\/nobr><\/DIV>\r\n <DIV style=\"position:absolute;top:179;left:54\"><nobr><span class=\"ft03\">Summary<\/span><\/nobr><\/DIV><\/pre>\n
Note I edited out my e-mail address. I’m not sure why LinkedIN gave me this.<\/p>\n
Both Amber and I have an “Experience” section. \u00a0Unfortunately this is rendered differently:<\/p>\n
class=\"ft02\"<\/pre>\n
For both converted HTML documents. (Mine is “ft03” not ft02).<\/p>\n
I surmise the fonts are just assigned based on what’s encountered. \u00a0For example, my e-mail address comes before the section heading. Since all of the saved LinkedIN profiles are different, this might pose a serious problem that will lead me to choose PDF to XLS conversion instead of PDF to HTML.<\/p>\n
The next PDF to HTML software conversion on my list to try [Note: after trying this I DO NOT recommend this] is:\u00a0cbsidlm-sp1_0_150-PDFtoHTML-SEO-100684.dmg. \u00a0This freeware is a bit tricky since it appears to want to install some add-ons, but by selecting a custom install it appears I can circumvent that.<\/p>\n
This appears to be a GUI for the command line version I tried in a previous post.<\/p>\n
“OS X PDFtoHTML” is based on “pdftohtml” from Gueorgui Ovtcharov and Rainer Dorsch which is currently maintained by Mikhail Kruk.<\/p><\/blockquote>\n
It still might be interesting to see if there are some other options I was overlooking.<\/p>\n
There’s a drag and drop interface.<\/p>\n
Dragging the icon for the file to convert results in this path:<\/p>\n
\/Users\/apple\/Documents\/linkedin profiles-WG-Members\/TannerJessel.pdf<\/pre>\n
Clicking “Convert PDFtoHTML” creates a new folder on the desktop (PtH.output.14.04.10_15.36.06) but nothing is in it. And, nothing is in the original folder. \u00a0I’m going to say this app does not work and is probably malware or spyware since it attaches plug-ins and add-ons and messes with your default settings for browsers. \u00a0So, I’ll be un-installing that with AppCleaner.<\/p>\n
Third to test is “Wondershare PDF Editor Pro” 278 MB. This is also trial ware and will leave a watermark on saved documents. The trial ware limit is 5 pages – my own LinkedIN is fairly long and the trialware preserved it. \u00a0However it appears you must convert one at a time, and I don’t think that will work for my purposes. \u00a0Nice app for other purposes though.<\/p>\n
The headings are closer to what I want though. For example, main headings for Experience is as follows:<\/p>\n
<span style=\"font-size:15.942;color:#999999;\">Experience<\/span><\/pre>\n
The same is true for Amber Budden:<\/p>\n
<span style=\"font-size:15.942;color:#999999;\">Experience<\/span><\/pre>\n
And the same is true for “Education:”<\/p>\n
<span style=\"font-size:15.942;color:#999999;\">Education<\/span><\/pre>\n
I also accidentally converted the document to a .doc file, and used the opportunity presented to test converting from .doc to .html from within Word.<\/p>\n
The “Experience” section looks like this:<\/p>\n
<span\u00a0style='font-size:16.0pt;color:#999999'>Experience<o:p><\/o:p><\/span><\/pre>\n
An advantage of doing it this way is that everything is contained within one single HTML file, instead of one HTML file for multiple PDF pages.<\/p>\n
The number of steps required is a bit frustrating though (Convert to Word, Convert to HTML), especially if each one has to be processed individually.<\/p>\n
Just a quick check of the remaining headings:<\/p>\n
<span\r\nstyle='font-size:16.0pt;color:#999999'>Skills &amp; Expertise<o:p><\/o:p><\/span><\/pre>\n
\"education-pdf-editor-pro\"<\/a>
“Education” section as it appears in the .doc copy of the saved PDF version of a LinkedIN profile.<\/figcaption><\/figure>\n
Here is the entire “Education” section (for comparison see the image):<\/p>\n
<p style='margin-left:.5in;line-height:14.4pt;mso-line-height-rule:\r\nexactly;mso-pagination:none;mso-layout-grid-align:none;text-autospace:none'><span\r\nstyle='font-size:16.0pt;color:#999999'>Education<o:p><\/o:p><\/span><\/p><\/pre>\n
<p style='margin-left:.5in;line-height:19.5pt;mso-line-height-rule:\r\nexactly;mso-pagination:none;mso-layout-grid-align:none;text-autospace:none'><b><span\r\nstyle='font-size:12.0pt;color:black'>University of Wales, Bangor<o:p><\/o:p><\/span><\/b><\/p><\/pre>\n
<p style='margin-left:.5in;line-height:14.85pt;mso-line-height-rule:\r\nexactly;mso-pagination:none;mso-layout-grid-align:none;text-autospace:none'><span\r\nstyle='font-size:12.0pt;color:black'>PhD, Behavioral Ecology, 1997 - 2001<o:p><\/o:p><\/span><\/p><\/pre>\n
<p style='line-height:18.1pt;mso-line-height-rule:exactly;\r\nmso-pagination:none;mso-layout-grid-align:none;text-autospace:none'><span\r\nstyle='font-size:12.0pt'><o:p>&nbsp;<\/o:p><\/span><\/p><\/pre>\n
<p style='margin-left:.5in;line-height:10.8pt;mso-line-height-rule:\r\nexactly;mso-pagination:none;mso-layout-grid-align:none;text-autospace:none'><b><span\r\nstyle='font-size:12.0pt;color:black'>University of Bristol<o:p><\/o:p><\/span><\/b><\/p><\/pre>\n
<p style='margin-left:.5in;line-height:14.85pt;mso-line-height-rule:\r\nexactly;mso-pagination:none;mso-layout-grid-align:none;text-autospace:none'><span\r\nstyle='font-size:12.0pt;color:black'>BSc Hons, Zoology \/ Psychology, 1991 -\r\n1994<o:p><\/o:p><\/span><\/p><\/pre>\n
It looks like “line-height” is what distinguishes “place of education” from the date and time.<\/p>\n
Right now this approach (PDF to Word to HTML) appears to have a bit more reliable structure than any of the other unstructured text processing options I’ve looked at so far.<\/p>\n
Still, I’m wondering if the extreme variability in the data (due to different data that LinkedIN has collected through successive iterations of the site) doesn’t make something like NOSql a worthwhile approach. I’ll look into that more later – a quick search turned up <http:\/\/www.computerworld.com\/s\/article\/9247555\/With_MongoDB_2.6_NoSQL_database_gets_a_speed_boost<\/a>>. And if I were to convert the content to Text instead of PDF, then perhaps it could easily be converted to work with MongoDB’s field and value pairs<\/a>. \u00a0This would save me the trouble of having to *design* a database to replicate LinkedIN data.<\/p>\n
So I’ve gone through some Mac applications I wanted to test – just as well since I’m about out of power on my Mac Laptop. \u00a0I’ll let this one charge and switch over to Windows 7 to try out the PC based application I mentioned in my previous open notebook entry<\/a> on this subject..<\/p>\n","protected":false},"excerpt":{"rendered":"
I am continuing evaluation of some text processing tools that I began in an earlier open notebook post\u00a0on the same topic. I also had an idea that perhaps I should open my PDF documents in Word, then re-save them as HTML. \u00a0That workflow might standardize the formatting to something less Continue reading   Text Processing Methods, Continued (PDF to HTML Conversion)<\/span>→<\/span><\/a><\/p>\n","protected":false},"author":35,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[259,262,264,261,260,35,263,255],"_links":{"self":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/2028"}],"collection":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/users\/35"}],"replies":[{"embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/comments?post=2028"}],"version-history":[{"count":9,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/2028\/revisions"}],"predecessor-version":[{"id":2039,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/posts\/2028\/revisions\/2039"}],"wp:attachment":[{"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/media?parent=2028"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/categories?post=2028"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/notebooks.dataone.org\/wp-json\/wp\/v2\/tags?post=2028"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}