By Salvatore Salamone
June 17, 2004 | Saul Kravitz had a big decision to make. The director of software engineering for the J. Craig Venter Science Foundation Joint Technology Center (JTC), he was interested in buying a commercial laboratory information management system (LIMS) to handle the prodigious amount of sequence data produced by the JTC's affiliated institutes. But he encountered a tiny snag.
"When vendors would come in, we'd ask them to do a performance benchmark at our expected lab capacity," Kravitz says. "They would say they'd do the benchmark after [their LIMS was installed]." Not surprisingly, Kravitz was reluctant to install a vendor's LIMS, only to find it didn't have the necessary performance specs. After surveying the field, Kravitz opted to build a LIMS.
When it comes to deploying a LIMS, opinions on whether to build or buy are about as polarized as Americans' support for Bush or Kerry. There is precious little middle ground.
Requirements:
- Capacity to perform about 40 million sequencing reactions per year
- Potential to scale to four times that volume
- Tight integration with fluid-handling robotic systems
Taking into account this list of requirements, Saul Kravitz decided to build, not buy, the Venter Science Foundation Joint Technology Center's LIMS.
|
"Buy" proponents prefer to put more effort into life sciences-related activities and not waste precious internal resources on software development. Ardent "build" supporters, conversely, say LIMS vendors simply don't deliver the goods. In many cases, the choice between build and buy comes down to a few fundamental issues, notably LIMS performance and the ability to integrate various instruments.
The core function of the JTC facility, in Rockville, Md., is to perform DNA sequencing for the organizations affiliated with the J. Craig Venter Science Foundation, using 100 of the latest Applied Biosystems 3730xl automated DNA sequencers. The affiliates include The Institute for Genomic Research, The Center for the Advancement of Genomics, and the Institute for Biological Energy Alternatives. Venter established his eponymous foundation after he left Celera Genomics in early 2002.
"Our development team brought considerable LIMS development experience from Celera, where team members contributed to high-throughput LIMS for both genomics and proteomics," Kravitz explains. "LIMS support at Celera had all been built in-house, and we were tempted to cut our time to deployment by using commercial LIMS products."
| A Toss-Up |
When deciding between building and buying a LIMS, each method has distinct advantages, and managers must take into account several key points that could affect their lab operations.
Read More  |
"Based on our experience at Celera, we knew the challenge of maintaining LIMS performance as data volume grows," Kravitz says. "None of the vendors we approached was willing to provide pre-purchase a relevant performance benchmark for its product at the scale we anticipated." To put the performance into perspective, the JTC currently has the capacity to perform about 40 million sequencing reactions per year and has the potential to scale to four times that volume.
Another factor in Kravitz's decision to build a LIMS was the need for tight integration with fluid-handling robotic systems. "The commercial LIMS packages we looked at could read and write control files for the instruments, but did not support real-time integration with instrument hardware," Kravitz says. "This was not the type of coupling we desired."
Kravitz wanted to be able to read a bar code and stop an experiment immediately if the wrong sample was in a container. Considering all his needs, Kravitz opted to build his organization's LIMS (see "Building a High-Throughput LIMS," page 82).
Buy Not
Many managers say they prefer to buy LIMS and other informatics software rather than develop the applications themselves. But often, they cannot find an appropriate product.
Requirements:
- Flexible enough to handle all types of lab data
- No upfront costs to using it
- Can be used throughout the entire process
- Generates billing reports
Based on his wish list, Allen Hunter chose InterSystems' CACHÉ database to build MPLN's LIMS.
|
"We always try to buy rather than build," says Mark Murcko, vice president and CTO at Vertex Pharmaceuticals. When Murcko joined Vertex in 1990, he didn't want to write software at all. "My preference was — and still is — to write a big check to some software company every year and have it provide the tools we need."
But this approach simply does not work for Murcko. "It is still not possible to get what we need from the commercial vendors, and so we have written quite a lot of tools ourselves," Murcko says.
The Molecular Pathology Laboratory Network (MPLN) took a novel approach to LIMS development. The lab conducts a variety of genomic/DNA testing services for larger labs that subcontract the work. Its workload is expanding, more than doubling the number of monthly tests in the past year.
Furthermore, MPLN has incorporated new testing methods into its repertoire. In addition to using PCR extensively for DNA analysis, it also uses fluorescent in situ hybridization (FISH). That requires that different types of data be incorporated into any LIMS.
With these issues in mind, MPLN selected InterSystems' CACHÉ database to build its LIMS. This choice was based on several factors. First, CACHÉ is not just a database but also has a built-in programming language. It's flexible enough to handle all the data types the lab anticipated using.
And "there are no upfront costs to using it," says Allen Hunter, MPLN's director of software development. "You can download it, develop an application, and you don't pay [any licensing fees] until you deploy the application."
The new LIMS is used throughout the entire process, "from when the specimen comes in the door, through tests, to billing," Hunter says. Use of the system also extends to other operational areas, such as automation of quality-control processes and the generation of billing reports.
"Before we send out reports (Microsoft Word documents), we need two signatures by lab techs," Hunter says. "This is time-consuming." The new system streamlines the process — for example, by using electronic signatures. A process that formerly took several days now takes less than 24 hours.
While building a LIMS has its supporters, in many cases buying a LIMS is the best solution. "Most [commercial] LIMS do well when you run the same experiment over and over," says Seth Pinsky, senior vice president, global research and development, and CTO for MDL Information Systems. "But not so well if doing new [types] of experiments all the time."
One advantage of buying a LIMS is speed of deployment. That was the case with Schwarz Pharma, a specialty pharmaceutical company. The company recently completed the installation of InnaPhase's Newton LIMS, developed specifically to deal with the equipment and processes in a pharmaceutical manufacturing environment. "Rather than spending a year or more customizing and validating a LIMS, we can direct our efforts to using the system to make our laboratories more efficient," says Brad Huddleston, Schwarz's LIMS administrator.
Others see the buy option as a way to simplify compliance testing.
When developing applications from scratch, "those that must be validated to demonstrate and ensure 21 CFR Part 11 compliance take, on average, about 60 percent more work to complete," says Krishnan Rajagopalan, a senior executive in the life science practice of outsourcing company Infosys.
Rajagopalan's group develops clinical LIMS for international life science organizations. He notes that companies that build their own LIMS sometimes outsource the entire project, but some outsource only the validation and compliance tracking part of the project. This additional work required to validate an application translates into a significantly higher cost to develop a compliant program versus a noncompliant one.
Wrapping Up the Equipment
Still others buy a LIMS for help in integrating new lab equipment. "We examined a number of LIMS that would support the data formats from some of the new experiments we anticipated performing in our labs over the next two years," says Charles Reese, lab manager at a New Jersey biotech company. According to Reese, there is a general consensus that the validation process is no trivial task when mixing and matching multiple modules of a packaged LIMS. Reese's solution is to compromise: "We've kept our existing LIMS, and each time we add a new [piece of] lab equipment we deal with its integration into that LIMS."
Much of this integration work requires writing data wrappers to handle the new lab equipment's data formats. However, many LIMS companies try to simplify this process by offering developers toolkits and other aids. For instance, Amersham Biosciences offers registered customers access to more than 70 data wrappers for accessing data in public Web-based databases such as GenBank and Medline, and for importing data from many open-source algorithms such as BLAST and Fasta.
Another reason to purchase a LIMS is to help migration from one system to another. The Pacific Environmental Science Centre (PESC) in Western Canada, which supports the government's Environment Canada, performs more than 500,000 tests annually, covering a wide range of science and technologies. Within the life sciences, the group conducts studies on plant and animal tissues.
In the past 25 years, the center developed 105 applications, adding new applications to a LIMS that ran on a Digital Equipment Corp. PDP-11 computer. When PESC decided to move off the PDP-11 system, it selected a LIMS from STARLIMS Corp. With help from a third-party consulting company, PESC was able to migrate to a new system in about six months.
Third-Party Option
Increasingly, there's a new twist on the build-versus-buy quandary. "Today, it's a choice between build, buy, or download and customize," says Scott Sneddon, senior fellow at Genzyme.
From a developmental standpoint, the "download-and-customize" option shares some of the benefits of both the build and buy options. For instance, like many commercial software packages, most open-source applications are thoroughly vetted and highly reliable, thanks to the attention of the Linux development community. And the flexibility of being able to work with the source code and adapt a program to meet specific company needs brings some of the same benefits of building an application from scratch.
Moreover, the download-and-customize approach allows companies to set their own development schedule and not be constrained to vendors' timetables — often a critical issue. "[We] don't have the luxury to wait six months for a vendor to release a new product," Sneddon says.
Until now, most download-and-customize efforts have been limited to departmental R&D work. For instance, a group doing genetic analysis to identify potential disease targets might rely on BLAST and other open-source analysis routines to conduct its work.
But now, the range of applications available to download and customize is expanding — and extends to LIMS. A good example is caLIMS, an open-source LIMS from the National Cancer Institute. The Web-based system is designed to automate laboratory workflow.
There are other open-source LIMS available. Most are dedicated to specific lab operations, such as the Flow Cytometry Laboratory Information Management System (Flow LIMS), available from the Fox Chase Cancer Center (bioinformatics.fccc.edu).
Besides sharing some of the advantages, the download-and-customize approach to application deployment also shares some of the management challenges of both the build and the buy approaches. Applications must meet 21 CFR Part 11 requirements, but, like commercial software, only some open-source packages are Part 11-compliant. Even if an open-source application were compliant, any modifications would require that the customized software be re-certified for compliance. This means procedures and systems must be in place to document and track any modifications to the downloaded software.
For many life science organizations, the download-and-customize approach to LIMS is still a distant third option to the build-versus-buy debate. Call it the Ralph Nader of the mix.
While no single best LIMS strategy can be universally applied to every organization, the dilemma of whether to build or buy has the same consequence as a presidential vote. Whatever the final decision, your organization has to live with that choice for many years.
Photograph by Randall Scott