#+title: Connecting Galaxy to tools with alternative storage and compute models
#+author: Brad Chapman \\ Bioinformatics Core, Harvard School of Public Health \\ https://github.com/chapmanb/bcbio-nextgen \\ http://j.mp/bcbiolinks
#+date: 1 July 2014

#+OPTIONS: toc:nil H:2

#+startup: beamer
#+LaTeX_CLASS: beamer
#+latex_header: \usepackage{url}
#+latex_header: \usepackage{hyperref}
#+latex_header: \hypersetup{colorlinks=true}
#+BEAMER_THEME: default
#+BEAMER_COLOR_THEME: seahorse
#+BEAMER_INNER_THEME: rectangles

* Tradeoffs
** Tradeoff: Assessing code

\huge
Community > Implementation
\normalsize

\vspace{3cm}

\footnotesize
Galaxy \\
Biopython: http://biopython.org \\
OpenBio: http://www.open-bio.org
\normalsize

** Tradeoff: Reproducibility

\huge
Validation > Replication
\normalsize

\vspace{3cm}
\footnotesize
Genome in a Bottle: [[http://www.genomeinabottle.org/]] \\
ICGC-TCGA DREAM: https://www.synapse.org/#!Synapse:syn312572 \\
SMaSH: http://smash.cs.berkeley.edu/
\normalsize

** Tradeoff: Development focus

\huge
Scaling > Configurability
\normalsize

\vspace{3cm}

\footnotesize
bcbio scaling: http://j.mp/bcbioscale
\normalsize

* bcbio overview

** Overview

#+ATTR_LATEX: :width 1.0\textwidth
[[./images3/bcbio_nextgen_highlevel.png]]

\vspace{1cm}
https://github.com/chapmanb/bcbio-nextgen

** Uses

\Large
- Aligners: bwa-mem, novoalign, bowtie2
- Variantion: FreeBayes, GATK, MuTecT, SnpEff, VEP, GEMINI, Lumpy, Delly
- RNA-seq: Tophat, STAR, cufflinks, HTSeq
- Quality control: fastqc, bamtools, RNA-SeQC
- Manipulation: bedtools, bcftools, biobambam, sambamba, samblaster, samtools,
  vcflib
\normalsize

** Provides

\Large
- Community -- collected set of expertise
- Tool integration
- Validation -- outputs + automated evaluation
- Installation of tools and data
- Scaling
\normalsize

* Challenges

** Scaling: avoid intermediates

#+begin_src python :exports code
("{bwa} mem -M -t {num_cores} -R '{rg_info}' -v 1 "
 "  {ref_file} {fastq_file} {pair_file} "
 "| {samblaster} "
 "| {samtools} view -S -u /dev/stdin "
 "| {sambamba} sort -t {cores} -m {mem} --tmpdir {tmpdir}"
 "   -o {tx_out_file} /dev/stdin")
#+end_src

** Parallelization: split and batched jobs

[[./images2/parallel-genome.png]]

** Storage: extract outputs

- Intermediates -- 6x final
#+begin_src sh
$ du -sh *
353G final
2.2T work
#+end_src

- 1500 whole genome scale -- 110Tb

#+begin_src sh
$ du -sh alz-p3f_2-g5/final
3.4T  alz-p3f_2-g5/final
$ ls -lhd *alz* | wc -l
31
#+end_src

* Galaxy

** bcbio as a Galaxy tool

[[./images2/bcbio_tool.png]]

\scriptsize
https://github.com/chapmanb/bcbio-nextgen/tree/master/config/galaxy
\normalsize

** Tool shed install: Docker

#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.6\textwidth
[[./images/homepage-docker-logo.png]]
#+END_CENTER

\small
https://github.com/chapmanb/cloudbiolinux \\
https://github.com/chapmanb/bcbio-nextgen-vm
\normalsize

* Summary

** Summary

\Large
- Focus: Community, Validation, Scaling
- bcbio-nextgen \small https://github.com/chapmanb/bcbio-nextgen \Large
- Challenges: parallelization, scaling and storage
- Galaxy integration: Simple tool with Docker installation
\normalsize
