BioHDF - Toward Scalable Bioinformatics Infrastructures
- Conference:
- ISMB/ECCB 2009
- Abstract:
- BioHDF will extend HDF5 data structures and library routines with new features to support the high-performance data storage and computation requirements of Next Gen Sequencing. To enable its use within the bioinformatics and research communities, BioHDF will be delivered as an Open Source technology that will include: a data model supporting storage of sequences, their alignments against reference data sources, and annotations such as SNP or splice variation analysis; indexing for fast random-access into this data; analysis tools that target specific applications such as RNA-Seq, gene structure analysis, novel micro-RNA identification, and many others; visualization tools that provide reporting and interactive display of these very large data sets.
- Key findings:
- Initial prototyping of BioHDF has demonstrated clear benefits. Sequences and their alignments against reference data sources, which occupy 10s of Gb stored as highly redundant text-format files, are just a few percent of that size when stored in BioHDF’s compressed, structured binary format. Indexing in BioHDF enables very rapid (typically, few millisecond) random access into these sequence and alignment datasets, essentially independent of the overall HDF5 file size. Additionally, through our prototyping activities we have identified key architectural elements and tools that will form BioHDF.
- Description:
-
The huge volume of data produced by novel sequencing technologies presents significant challenges around data transmission, storage,... » More
Loading comments