Oct. 16, 2006 | One of the biggest stumbling blocks to the success of text mining remains the firewall that surrounds full-text archives. "It is the restricted access to the full text of papers and to citation information, rather than the technology, that is currently the greatest limitation, despite some encouraging open-access initiatives," EMBL bioinformaticist Peer Bork and colleagues wrote in Nature Reviews Genetics earlier this year.

Access to the full text often requires exorbitant subscriptions to publishing powerhouses such as Elsevier, Wiley, and others. The open-access movement, which fueled the launch of the Public Library of Science in 2002, has persuaded some publishers to partially open their archives - but most new papers remain off limits to non-subscribers.

Some of QUOSA's customers have done benchmarking, including one whom MacKenzie says compared full-text versus abstract text mining for protein-protein and protein-drug reactions, and retrieved 25-100 times more useful factoids out of full-text sources. Would more publishers going the open-access route reduce the need for QUOSA? "Open access is just another silo through which you get journal access," says MacKenzie. "The better access there is to full articles, the more people can take advantage of what we do."

Nature Publishing Group (NPG) recently announced an interesting experiment - an effort to enhance machine access to full text literature by proposing a standard content annotation called the Open Text Mining Interface (OTMI), which was first presented by NPG web publishing director Timo Hannay at Bio-IT World's 2006 annual conference.

The XML format of OTMI reorders each paper's sentences alphabetically, rendering the product unreadable to humans while allowing full-text searching of intact sentences. It's what Hannay calls "a potential compromise between business needs and open access." Hannay says he hopes all publishers will adopt OTMI or a similar standard to open up the entire literature for text mining.

One fan is Tim O'Reilly, who says, "It immediately struck me as 'slap your forehead brilliantly obvious...' I love the cleverness of this approach, which lets machines make use of the content in ways that human readers can't. I like it. You might consider it a "copyright hack.' "

Hannay says there's been considerable early interest from other publishers and text-mining researchers. This fall, NPG will roll out OTMI files for much of the Nature archives and encourage people to play with it. NPG is also launching a collaborative website to open up development of the specification and share tools. "Ideally, we want OTMI to become a de facto community standard (and perhaps in time a more formal standard). NPG has no desire to 'own' it," says Hannay. -- K.D.

Return to main article.