Building a Community Cyberinfrastructure to Support Marine Microbial Ecology Metagenomics Invited Talk  2006 Synthetic Biology Symposium Aliso Creek Inn Laguna Beach, CA September 15, 2006 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor,  Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD
Calit2 Brings Computer Scientists and Engineers  Together with Biomedical Researchers Some Areas of Concentration: Metagenomics Genomic Analysis of Organisms Evolution of Genomes Cancer Genomics Human Genomic Variation & Disease Proteomics Mitochondrial Evolution Computational Biology & Bioinformatics Information Theory & Biological Systems UC San Diego UC Irvine 1200 Researchers in Two Buildings www.calit2.net
Most of Evolutionary Time Was in the Microbial World Source: Carl Woese, et al Tree of Life Derived from 16S rRNA Sequences You Are Here
Microbial Genomics Let’s Us Look Back  Nearly 4 Billion Years In the Evolution of Life Falkowski and Vargas  Science  304 (5667) 2004
Moore Microbial Genome Sequencing Project Selected Microbes Throughout the World’s Oceans www.moore.org/microgenome/worldmap.asp Microbes Nominated by Leading Ocean Microbial Biologists
Moore Foundation Funded the Venter Institute to Provide the Full Genome Sequence of 150 Marine Microbes www.moore.org/microgenome/trees_main.asp
Moore Microbial Genome Sequencing Project: Cyanobacteria Being Sequenced by Venter Institute
Full Genome Sequencing is Exploding: Most Sequenced Genomes are Bacterial www.genomesonline.org 55 Metagenomes First Genome 1995  6 Genomes/ Year 2000 Moore 155 In Here Total 422 Completed Genomes Total 1665 Ongoing Genomes
Microbial Metagenomics is  a Rapidly Emerging Field of Research “ Despite their ubiquity, relatively little is known about the majority of environmental microorganisms, largely because of their resistance to culture under standard laboratory conditions.” “ The application of high-throughput shotgun sequencing environmental samples has recently provided global views of those communities not obtainable from 16S rRNA or BAC clone–sequencing surveys .” Comparative Metagenomics of Microbial Communities  Susannah Green Tringe, Christian von Mering, Arthur Kobayashi, Asaf A. Salamov, Kevin Chen, Hwai W. Chang, Mircea Podar, Jay M. Short, Eric J. Mathur, John C. Detter, Peer Bork, Philip Hugenholtz, Edward M. Rubin Science 22 April 2005
The Sargasso Sea Experiment  The Power of Environmental Metagenomics Yielded a Total of  Over 1 billion Base Pairs of Non-Redundant Sequence Displayed the Gene Content, Diversity, & Relative Abundance of the Organisms  Sequences from at Least 1800 Genomic Species, including 148 Previously Unknown Identified over 1.2 Million Unknown Genes MODIS-Aqua satellite image of ocean chlorophyll in the Sargasso Sea grid about the BATS site from 22 February 2003 J. Craig Venter, et al.  Science  2 April 2004: Vol. 304.  pp. 66 - 74
Marine Genome Sequencing Project –  Measuring the Genetic Diversity of Ocean Microbes Sorcerer II Data Will Double Number of Proteins in GenBank!
GOS Sequences are Largely Bacterial Source: Shibu Yooseph, et al. (PLOS Biology in press 2006) ~3 Million Previously Known Sequences ~5.6 Million GOS Sequences
GOS Analysis -- Protein Families in Nature  Have Been Poorly Explored Thus Far Novel Sequence Similarity Clustering Process Predicts Proteins and Groups Related Sequences Into Clusters (Families) GOS Proteins Increase Size / Diversity of Many Protein Families 1,700 Novel GOS-Only Clusters Identified (>20 per Cluster) 10% of 17,000 Clusters Source: Shibu Yooseph, Granger Sutton, --JCVI  NCBI_nr  GOS + NCBI_nr + Ensembl + TIGR Gene Indices + Prokaryotic Genomes
Current Universe of  Medium/ Large Protein Families Source: Shibu Yooseph, et al. (PLOS Biology in press 2006) Protein Families Conserved Across Tree of Life  Protein Families Unique to GOS  17,067 Protein Family Clusters
Metagenomic Data Sets Are Rapidly Being Accumulated “ A majority of the bacterial sequences corresponded to uncultivated species and novel microorganisms.”  “ We discovered significant inter-subject variability.”  “ Characterization of this immensely diverse ecosystem is the first step in elucidating its role in health and disease.” “ Diversity of the Human Intestinal Microbial Flora”  Paul B. Eckburg, et al  Science  (10 June 2005) 395 Phylotypes
Microbes Form  the Base of the Living World White Filamentous Bacteria on 'Pill Bug' Outer Carapace Source:  John Delaney and Research Channel,  U Washington High Definition Still Frame  of Hydrothermal Vent Ecology 2.3 Km Deep  1 cm.
PI Larry Smarr Announced January 17, 2006 $24.5M Over Seven Years
Paul Gilna Has Been Recruited from Los Alamos  to Become Calit2’s Executive Director of CAMERA Formerly Former Director of the Department of Energy’s Joint Genome Institute (JGI) Operations at Los Alamos National Laboratory (LANL) Group Leader of Genomic Science and Computational Biology in LANL’s Bioscience Division JGI  A $70-million-per-Year Collaboration: Lawrence Berkeley,  Lawrence Livermore,  Los Alamos,  Oak Ridge, and  Pacific Northwest  and the Stanford Human Genome Center  Working at The Frontiers of Genome Sequencing and Biosciences
National Lambda Rail (NLR) and TeraGrid Provides  Cyberinfrastructure Backbone for U.S. Researchers San Francisco Pittsburgh Cleveland San Diego Los Angeles Portland Seattle Pensacola Baton Rouge Houston San Antonio Las Cruces / El Paso Phoenix New York City Washington, DC Raleigh Jacksonville Dallas Tulsa Atlanta Kansas City Denver Ogden/ Salt Lake City Boise Albuquerque UC-TeraGrid UIC/NW-Starlight Chicago International  Collaborators NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout NSF’s TeraGrid Has 4 x 10Gb  Lambda Backbone  Links Two Dozen State and Regional Optical Networks DOE, NSF, & NASA Using NLR
Calit2’s Direct Access Core Architecture  Will Create Next Generation Metagenomics Server Traditional User Response Request Source: Phil Papadopoulos, SDSC, Calit2 + Web Services Sargasso Sea Data Sorcerer II Expedition (GOS) JGI Community Sequencing Project Moore Marine  Microbial Project NASA and NOAA  Satellite Data Community Microbial Metagenomics Data Flat File Server Farm W E B  PORTAL Dedicated Compute Farm (100s of CPUs) TeraGrid: Cyberinfrastructure Backplane (scheduled activities, e.g. all by all comparison) (10000s of CPUs)  Web (other service) Local  Cluster Local Environment Direct Access  Lambda Cnxns Data- Base Farm 10 GigE  Fabric
The Future Home of the Moore Foundation Funded Marine Microbial Ecology Metagenomics Complex First Implementation of  the CAMERA Complex Photo Courtesy Joe Keefe, Calit2 Major Buildout of Calit2 Server Room Underway
Analysis Data Sets, Data Services,  Tools, and Workflows Assemblies of Metagenomic Data e.g, GOS, JGI CSP Annotations Genomic and Metagenomic Data “ All-against-all” Alignments of ORFs Updated Periodically Gene Clusters and Associated Data Profiles, Multiple-Sequence Alignments,  HMMs, Phylogenies, Peptide Sequences Data Services ‘ Raw’ and Specialized Analysis Data Rich Query Facilities Tools and Workflows Navigate and Sift Raw and Analysis Data Publish Workflows and Develop New Ones Prioritize Features via Dialogue with Community Source:  Saul Kravitz Director of Software Engineering J. Craig Venter Institute
OptIPortal–Termination Device  for the Dedicated Gigabit/sec Lightpaths Photo Source: David Lee,  Mark Ellisman NCMIR, UCSD Collaborative Analysis of Large Scale Images of Cancer Cells Integration of High Definition Video Streams with Large Scale Image Display Walls
Emerging OptIPortal Sites  on the National LambdaRail Dedicated 10 Gbps CAVEWave Connects  San Diego to Seattle to Chicago to Washington D.C.  NEW! NEW! SunLight CICESE UW JCVI MIT SIO UCSD SDSU UIC EVL UCI OptIPortals
CAMERA Outreach Modes Scientific Advisory Board  Early Adopters – OptIPortal End Points Targeted Workshops  User Forums  User Software Testing Viz Tool Brainstorming Presentations at Scientific Meetings e.g. Demonstration Booth at JCVI Genomes, Medicine, and the Environment Conference October 2006 Partnerships With Metagenomics Projects E.g. DoE’s Joint Genome Institute (JGI) Training and User Services Team
Timeline:  Sprint and Marathon Sprint Release 0.0:  April 2006 Test Cluster for UCSD/JCVI Collaboration Release 1.0:  Late Fall 2006 Initial Data and Core Tools Release  Supports Publication of GOS Papers Marathon Release 2.0:  Fall 2007 Additional/Improved Tools & Better Usability Beyond 2.0 Move Towards Semantic DB Additional Tools Based on Community Feedback

Building a Community Cyberinfrastructure to Support Marine Microbial Ecology Metagenomics

  • 1.
    Building a CommunityCyberinfrastructure to Support Marine Microbial Ecology Metagenomics Invited Talk 2006 Synthetic Biology Symposium Aliso Creek Inn Laguna Beach, CA September 15, 2006 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD
  • 2.
    Calit2 Brings ComputerScientists and Engineers Together with Biomedical Researchers Some Areas of Concentration: Metagenomics Genomic Analysis of Organisms Evolution of Genomes Cancer Genomics Human Genomic Variation & Disease Proteomics Mitochondrial Evolution Computational Biology & Bioinformatics Information Theory & Biological Systems UC San Diego UC Irvine 1200 Researchers in Two Buildings www.calit2.net
  • 3.
    Most of EvolutionaryTime Was in the Microbial World Source: Carl Woese, et al Tree of Life Derived from 16S rRNA Sequences You Are Here
  • 4.
    Microbial Genomics Let’sUs Look Back Nearly 4 Billion Years In the Evolution of Life Falkowski and Vargas Science 304 (5667) 2004
  • 5.
    Moore Microbial GenomeSequencing Project Selected Microbes Throughout the World’s Oceans www.moore.org/microgenome/worldmap.asp Microbes Nominated by Leading Ocean Microbial Biologists
  • 6.
    Moore Foundation Fundedthe Venter Institute to Provide the Full Genome Sequence of 150 Marine Microbes www.moore.org/microgenome/trees_main.asp
  • 7.
    Moore Microbial GenomeSequencing Project: Cyanobacteria Being Sequenced by Venter Institute
  • 8.
    Full Genome Sequencingis Exploding: Most Sequenced Genomes are Bacterial www.genomesonline.org 55 Metagenomes First Genome 1995 6 Genomes/ Year 2000 Moore 155 In Here Total 422 Completed Genomes Total 1665 Ongoing Genomes
  • 9.
    Microbial Metagenomics is a Rapidly Emerging Field of Research “ Despite their ubiquity, relatively little is known about the majority of environmental microorganisms, largely because of their resistance to culture under standard laboratory conditions.” “ The application of high-throughput shotgun sequencing environmental samples has recently provided global views of those communities not obtainable from 16S rRNA or BAC clone–sequencing surveys .” Comparative Metagenomics of Microbial Communities Susannah Green Tringe, Christian von Mering, Arthur Kobayashi, Asaf A. Salamov, Kevin Chen, Hwai W. Chang, Mircea Podar, Jay M. Short, Eric J. Mathur, John C. Detter, Peer Bork, Philip Hugenholtz, Edward M. Rubin Science 22 April 2005
  • 10.
    The Sargasso SeaExperiment The Power of Environmental Metagenomics Yielded a Total of Over 1 billion Base Pairs of Non-Redundant Sequence Displayed the Gene Content, Diversity, & Relative Abundance of the Organisms Sequences from at Least 1800 Genomic Species, including 148 Previously Unknown Identified over 1.2 Million Unknown Genes MODIS-Aqua satellite image of ocean chlorophyll in the Sargasso Sea grid about the BATS site from 22 February 2003 J. Craig Venter, et al. Science 2 April 2004: Vol. 304. pp. 66 - 74
  • 11.
    Marine Genome SequencingProject – Measuring the Genetic Diversity of Ocean Microbes Sorcerer II Data Will Double Number of Proteins in GenBank!
  • 12.
    GOS Sequences areLargely Bacterial Source: Shibu Yooseph, et al. (PLOS Biology in press 2006) ~3 Million Previously Known Sequences ~5.6 Million GOS Sequences
  • 13.
    GOS Analysis --Protein Families in Nature Have Been Poorly Explored Thus Far Novel Sequence Similarity Clustering Process Predicts Proteins and Groups Related Sequences Into Clusters (Families) GOS Proteins Increase Size / Diversity of Many Protein Families 1,700 Novel GOS-Only Clusters Identified (>20 per Cluster) 10% of 17,000 Clusters Source: Shibu Yooseph, Granger Sutton, --JCVI NCBI_nr GOS + NCBI_nr + Ensembl + TIGR Gene Indices + Prokaryotic Genomes
  • 14.
    Current Universe of Medium/ Large Protein Families Source: Shibu Yooseph, et al. (PLOS Biology in press 2006) Protein Families Conserved Across Tree of Life Protein Families Unique to GOS 17,067 Protein Family Clusters
  • 15.
    Metagenomic Data SetsAre Rapidly Being Accumulated “ A majority of the bacterial sequences corresponded to uncultivated species and novel microorganisms.” “ We discovered significant inter-subject variability.” “ Characterization of this immensely diverse ecosystem is the first step in elucidating its role in health and disease.” “ Diversity of the Human Intestinal Microbial Flora” Paul B. Eckburg, et al Science (10 June 2005) 395 Phylotypes
  • 16.
    Microbes Form the Base of the Living World White Filamentous Bacteria on 'Pill Bug' Outer Carapace Source: John Delaney and Research Channel, U Washington High Definition Still Frame of Hydrothermal Vent Ecology 2.3 Km Deep 1 cm.
  • 17.
    PI Larry SmarrAnnounced January 17, 2006 $24.5M Over Seven Years
  • 18.
    Paul Gilna HasBeen Recruited from Los Alamos to Become Calit2’s Executive Director of CAMERA Formerly Former Director of the Department of Energy’s Joint Genome Institute (JGI) Operations at Los Alamos National Laboratory (LANL) Group Leader of Genomic Science and Computational Biology in LANL’s Bioscience Division JGI A $70-million-per-Year Collaboration: Lawrence Berkeley, Lawrence Livermore, Los Alamos, Oak Ridge, and Pacific Northwest and the Stanford Human Genome Center Working at The Frontiers of Genome Sequencing and Biosciences
  • 19.
    National Lambda Rail(NLR) and TeraGrid Provides Cyberinfrastructure Backbone for U.S. Researchers San Francisco Pittsburgh Cleveland San Diego Los Angeles Portland Seattle Pensacola Baton Rouge Houston San Antonio Las Cruces / El Paso Phoenix New York City Washington, DC Raleigh Jacksonville Dallas Tulsa Atlanta Kansas City Denver Ogden/ Salt Lake City Boise Albuquerque UC-TeraGrid UIC/NW-Starlight Chicago International Collaborators NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout NSF’s TeraGrid Has 4 x 10Gb Lambda Backbone Links Two Dozen State and Regional Optical Networks DOE, NSF, & NASA Using NLR
  • 20.
    Calit2’s Direct AccessCore Architecture Will Create Next Generation Metagenomics Server Traditional User Response Request Source: Phil Papadopoulos, SDSC, Calit2 + Web Services Sargasso Sea Data Sorcerer II Expedition (GOS) JGI Community Sequencing Project Moore Marine Microbial Project NASA and NOAA Satellite Data Community Microbial Metagenomics Data Flat File Server Farm W E B PORTAL Dedicated Compute Farm (100s of CPUs) TeraGrid: Cyberinfrastructure Backplane (scheduled activities, e.g. all by all comparison) (10000s of CPUs) Web (other service) Local Cluster Local Environment Direct Access Lambda Cnxns Data- Base Farm 10 GigE Fabric
  • 21.
    The Future Homeof the Moore Foundation Funded Marine Microbial Ecology Metagenomics Complex First Implementation of the CAMERA Complex Photo Courtesy Joe Keefe, Calit2 Major Buildout of Calit2 Server Room Underway
  • 22.
    Analysis Data Sets,Data Services, Tools, and Workflows Assemblies of Metagenomic Data e.g, GOS, JGI CSP Annotations Genomic and Metagenomic Data “ All-against-all” Alignments of ORFs Updated Periodically Gene Clusters and Associated Data Profiles, Multiple-Sequence Alignments, HMMs, Phylogenies, Peptide Sequences Data Services ‘ Raw’ and Specialized Analysis Data Rich Query Facilities Tools and Workflows Navigate and Sift Raw and Analysis Data Publish Workflows and Develop New Ones Prioritize Features via Dialogue with Community Source: Saul Kravitz Director of Software Engineering J. Craig Venter Institute
  • 23.
    OptIPortal–Termination Device for the Dedicated Gigabit/sec Lightpaths Photo Source: David Lee, Mark Ellisman NCMIR, UCSD Collaborative Analysis of Large Scale Images of Cancer Cells Integration of High Definition Video Streams with Large Scale Image Display Walls
  • 24.
    Emerging OptIPortal Sites on the National LambdaRail Dedicated 10 Gbps CAVEWave Connects San Diego to Seattle to Chicago to Washington D.C. NEW! NEW! SunLight CICESE UW JCVI MIT SIO UCSD SDSU UIC EVL UCI OptIPortals
  • 25.
    CAMERA Outreach ModesScientific Advisory Board Early Adopters – OptIPortal End Points Targeted Workshops User Forums User Software Testing Viz Tool Brainstorming Presentations at Scientific Meetings e.g. Demonstration Booth at JCVI Genomes, Medicine, and the Environment Conference October 2006 Partnerships With Metagenomics Projects E.g. DoE’s Joint Genome Institute (JGI) Training and User Services Team
  • 26.
    Timeline: Sprintand Marathon Sprint Release 0.0: April 2006 Test Cluster for UCSD/JCVI Collaboration Release 1.0: Late Fall 2006 Initial Data and Core Tools Release Supports Publication of GOS Papers Marathon Release 2.0: Fall 2007 Additional/Improved Tools & Better Usability Beyond 2.0 Move Towards Semantic DB Additional Tools Based on Community Feedback