Appendix: History of the QIIME platform
15. Appendix: History of the QIIME platform¶
This content is derived from a grant proposal, and needs to be revised and expanded for this book.
QIIME is a widely used microbiome bioinformatics platform and is widely used and highly cited in microbiome research. There have been two major versions of the QIIME platform, QIIME 1 and QIIME 2. QIIME 1 development began around May of 2009 in Rob Knight’s lab at the University of Colorado, building from prior work in the Knight Lab such as the UniFrac metric and web server. I was a postdoctoral scholar in the Knight Lab at the time, and have been centrally involved with the project since it began. QIIME 1, then simply referred to as QIIME, was first officially released on 26 January 2010, as many researchers were transitioning from Sanger sequencing to 454 sequencing for their microbiome studies. An important focus of our development was enabling ourselves and our users to make this transition to much larger datasets, which in many cases required redesign of core algorithms used to compare microbiomes. We also had a strong foundation in open source scientific software development. We felt that transparency and accessibility of bioinformatics methods and software was essential to science. Our success at scaling microbiome analysis to 454 sequencing enabled many new microbiome studies, which of course opened more questions than we had before, highlighting the need for even higher throughput microbiome profiling. This could, for example, support dense temporal or spatial sampling and/or larger cohorts in human microbiome research studies. This drove us to begin exploring the Illumina platform, and our team published the widely used “Earth Microbiome Project (EMP) sequencing protocol”. This represented another massive increase in the quantity of sequencing data available for studying microbiomes, from which followed many studies that were previously impossible. Enabling our software to scale to these data drove us to be early adopters of cloud computing technologies in bioinformatics (we have been providing Amazon Web Services-based distributions of QIIME since 2010), and to redesign core algorithms again.
Throughout the development of QIIME 1, we actively engaged our user community through the QIIME 1 Forum and through QIIME 1 workshops. Between 2010 and 2016 at least 65 QIIME 1 workshops were held in locations around the world, taught by members of the Caporaso Laboratory and/or Knight Laboratory. This community engagement was an excellent way for us to get direct feedback from our users on what they liked and disliked about QIIME. We began to realize that some of the problems that our users experienced, and some of the features they were requesting, were not possible to address with the QIIME 1 system, and in 2015 we began prototyping ideas for a redesign of the QIIME platform. Several prototypes were explored and discarded in 2015, and on January 29, 2016, we committed the first working code that became the QIIME 2 project. We continued to support QIIME 1 through this time, but informed our users that we were beginning work on a system that would replace QIIME 1. We also began releasing alpha versions of QIIME 2 for user testing. We created a QIIME 2 Forum and began teaching combined QIIME 1 and QIIME 2 workshops, for researchers who were interested in the new system. This again was an excellent way for us to collect user feedback on our work, and informed many early design decisions. On July 10, 2017 we informed our users that QIIME 1 would no longer be supported as of 1 January 2018. At present QIIME 2 is the only major version of the platform that we actively support (though users can still access all QIIME 1 releases, virtual machines, educational content, and the QIIME 1 Forum).