Bionimbus - Northwestern CGI Workshop 4-21-2011
Download as PPTX, PDF•2 likes•32,467 views
Bionimbus is a cloud-based infrastructure designed for managing, analyzing, and sharing genomics data, aiming to support the increasing volume of genomic data generated by researchers. It allows users to store and process data efficiently through various pipelines, significantly reducing analysis times. The platform leverages elastic cloud services to provide reliable computing resources that adapt to the needs of the scientific community.
Bionimbus - Northwestern CGI Workshop 4-21-2011
- 1. Bionimbus: A Cloud-Based Infrastructure for Managing, Analyzing and Sharing Genomics Data April 21, 2011Robert GrossmanInstitute for Genomics & Systems Biology (IGSB)Computation InstituteUniversity of ChicagoandOpen Cloud Consortium
- 2. Background
- 3. Growth of Genomic DataSequence everythingAWS HadoopGFSSequence environment200620082003Sequence speciesENCODEHGP20032001197719952005Sanger SequencingMicroarray technology454, Solexa sequencing10^10Genbank10^510^8
- 5. The Challenge is to Support Cubes of High Throughput Sequence DataEach cell in data cube can be ChIP-chip, ChIP-seq, RNA-seq, movie, etc. data set.Different developmental stagesDifferent pathologiesPerturb the environment
- 6. We Have a Problem…vsMore and more of your colleagues produce so much data that they cannot easily manage, move, analyze and share it. Centers and large projects build their own infrastructure.Every else is on their own.
- 7. Part 1. Using Bionimbuswww.bionimbus.org
- 8. Bionimbus is a community cloud for storing, analyzing and sharing genomics and related data.8
- 9. Enabling a broad community to utilize genome researchUser1.3.2.9Bionimbus CloudSequencing Partner or Center
- 10. Step 1. Prepare a Sample
- 11. Step 2. Login to Bionimbus and get a Bionimbus Key.
- 12. Step 3. Fedex your sample to CGI.
- 13. Step 4. Login on to Bionimbus and view your data
- 14. Step 5. Use Bionimbus to perform standard and custom pipelines.Using the ability of Bionimbus to launch multiple virtual machines reduced this analysis from 25 days to 1 day.
- 15. Step 2. Send sample tobe sequenced.Step 1. Get Bionimbus ID (BID), assign project, private/community, public cloud, etc.InternalSequencersBID GeneratorCGIStep 5. Cloud based analysis using IGSB and 3rdparty tools and applications. Step 3a. Return rawreads.Step 3b. Returnvariant calls, CNV, annotation…Bionimbus Private Cloud UCBionimbus Community CloudStep 4. Secure datarouting to appropriatecloud based upon BID.Bionimbus Private Cloud XYAmazondbGaP
- 16. Part 2. Introduction to Clouds
- 17. Clouds provide on-demand computing and storage resources at the scale and with the reliability of a data center.Computer scientists were caught by surprise.17
- 18. What is a Cloud?18Software as a Service (SaaS)
- 19. What Else a Cloud?19Infrastructure as a Service (IaaS)Users get one or more virtual machines “on demand”
- 20. Are There Other Types of Clouds?20ad targeting Hadoop was developed for processing Internet scale data for ad targeting and related applications but is now used for processing genomics data and may other applications.
- 21. What is a new about clouds?21
- 22. 22Scale is New
- 23. Elastic, On-Demand Computing with Usage Based Pricing Is New23costs the same as1 computer in a rack for 120 hours120 computers in three racks for 1 hourData center scale computing often leverages virtualization technologies.
- 24. Part 3. Some BionimbusCases
- 25. Case Study: Public Datasets in Bionimbus
- 27. Case Study: ModENCODEBionimbus is used to process the modENCODE data from the White lab (over 1000 experiments).BionimbusVMs were used for some of the integrative analysis.Bionimbus is used as a backup for the modENCODE DCC
- 28. 28>300 ChIP datasetsChromatin/RNA timecourse
- 29. CBP
- 30. PolII
- 31. Pho/silencers
- 32. HDACs
- 33. Insulators
- 34. TFsPredictions537 silencers2,307 new promoters12,285 enhancers14,145 insulatorswww.modencode.orgwww.cistrack.orgNegre et al. Nature 2011
- 35. Case Study: IGSBAll samples processed by the Institute for Genomics & Systems Biology High-Throughput Genome Analysis Core (HGAC) at the University of Chicago use Bionimbus.
- 36. Bionimbus Virtual Machine Releases 30
- 37. Part 431Data Centers for Science
- 39. Open Science Data CloudAstronomical dataBiological data (Bionimbus)NSF-PIRE OSDC Data ChallengeEarth science data (& disaster relief)
- 40. The goal is to build a data center in Chicago for biological, scientific, medical and health care data in 4 to 5 years.
- 41. Part 5. More About Bionimbus
- 42. GWT-based Front EndElastic Cloud ServicesDatabase ServicesAnalysis Pipelines & Re-analysis ServicesIntercloud ServicesLarge Data Cloud ServicesData Ingestion Services
- 43. (Eucalyptus,OpenStack)GWT-based Front EndElastic Cloud Services(PostgreSQL)Database ServicesAnalysis Pipelines & Re-analysis ServicesIntercloud Services(IDs, etc.)Large Data Cloud Services(UDT, replication)Data Ingestion Services(Hadoop,Sector/Sphere)
- 44. Bionimbus Deployment OptionsBionimbus Community Cloudwww.bionimbus.orgBionimbusAMIs & Amazon hosted applicationsBionimbus Private Clouds
- 45. A successful cloud will…3. High performance ingestion and transport of data.2. Provide Compute services at the scale of a data center.1. Provide long term persistent storage services at the scale of a data center.
- 46. A successful cloud will…6. Peer with private genomics clouds.5. Peer with public clouds.4. Support the liberation of data.
- 47. Bionimbus satisfies each of these six requirements.
- 48. Bionimbus Road MapOver the next 3 to 4 months, we will:Launch Bionimbus (we are in a pre-launch)Add Galaxy-based workflow to BionimbusAdd secure routing of genomesAdd more public datasetsAdd more pipelines