%\input{common/conf_top.tex}
\input{common/conf_top_print.tex}  %settings for printed booklets - comment out by default, uncomment for print and comment out line above. don't save this change! "conf_top" should be default

\input{common/conf_titles.tex}

\usepackage{url}
\def\UrlBreaks{\do\/\do-\do_}

\input{generated_buildinfo.tex}

\lstdefinestyle{pythoncode}{
  language=python,
  frame=single,
  breaklines,
  basicstyle=\ttfamily\scriptsize,
  commentstyle=\textsl,% comment style
  keywordstyle=\ttfamily\scriptsize,
  numbers=left,% display line numbers on the left side
  numberstyle=\scriptsize,% use small line numbers
  numbersep=10pt,% space between line numbers and code
  backgroundcolor=\color{white},
  showstringspaces=false %don't show spaces as weird char.
}

\begin{document}

\input{common/conf_listings.tex} %see note for `conf_top_print.tex` above
%\input{common/conf_listings_colorized.tex}  % Work in progress....  Use this for online/pdf version


\thispagestyle{empty} %removes page number
\newgeometry{bmargin=0cm, hmargin=0cm}


\begin{center}
\textsc{\Large\bf{Machine Learning with Python and H2O}}

\bigskip
\line(1,0){250}  %inserts  horizontal line 
\\
\bigskip
\small

\textsc{Pasha Stetsenko}

\textsc{Edited by: Angela Bartz}

\normalsize

\line(1,0){250}  %inserts  horizontal line

{\url{http://h2o.ai/resources/}}

\bigskip

\monthname \hspace{1pt}  \the\year: Fifth Edition

\bigskip
\end{center}

% commenting out lines image due to print issues, but leaving in for later
%\null\vfill
%\begin{figure}[!b]
%\noindent\makebox[\textwidth]{%
%\centerline{\includegraphics[width=\paperwidth]{waves.png}}}
%\end{figure}

\newpage
\restoregeometry

\null\vfill %move next text block to lower left of new page

\thispagestyle{empty}%remove pg#

{\raggedright 

Machine Learning with Python and H2O\\
  by Pasha Stetsenko \\
  with assistance from Spencer Aiello, \\
  Cliff Click, Hank Roark, \&\ Ludi Rehak \\
Edited by: Angela Bartz
\bigskip

Published by H2O.ai, Inc. \\
2307 Leghorn St. \\
Mountain View, CA 94043\\
\bigskip
\textcopyright 2016-\the\year \hspace{1pt} H2O.ai, Inc. All Rights Reserved. 
\bigskip

\monthname \hspace{1pt}  \the\year: Fifth Edition
\bigskip

Photos by \textcopyright H2O.ai, Inc.
\bigskip

All copyrights belong to their respective owners.\\
While every precaution has been taken in the\\
preparation of this book, the publisher and\\
authors assume no responsibility for errors or\\
omissions, or for damages resulting from the\\
use of the information contained herein.\\
\bigskip
Printed in the United States of America. 
}


\newpage
\thispagestyle{empty}%remove pg#

\tableofcontents
\thispagestyle{empty}%remove pg#

\newpage

\section{Introduction}

This documentation describes how to use H2O from Python. More information on H2O's system and algorithms
(as well as complete Python user documentation) is available at the H2O website at {\url{http://docs.h2o.ai}}.

H2O Python uses a REST API to connect to H2O. To use H2O in Python or launch H2O from Python, specify the IP address and port number of the H2O instance in the Python environment. Datasets are not directly transmitted
through the REST API. Instead, commands (for example, importing a dataset at specified HDFS location) are sent either through the browser or the REST API to perform the specified task.

The dataset is then assigned an identifier that is used as a reference in  commands to the web server.
After one prepares the dataset for modeling by defining significant data and removing insignificant data,
H2O is used to create a model representing the results of the data analysis.
These models are assigned IDs that are used as references in commands.

Depending on the size of your data,  H2O can run on your desktop or scale using multiple nodes with Hadoop,
an EC2 cluster, or Spark.  Hadoop is a scalable open-source file
system that uses clusters for distributed storage and dataset processing. H2O nodes run as JVM invocations on Hadoop
nodes. For performance reasons, we recommend that you do not run an H2O node on the same hardware as the Hadoop
NameNode.

H2O helps Python users make the leap from single machine based processing to large-scale distributed environments.
Hadoop lets H2O users scale their data processing capabilities based on their current needs.
Using H2O, Python, and Hadoop, you can create a complete end-to-end data analysis solution.

This document describes the four steps of data analysis with H2O:
\begin{enumerate}

\item installing H2O
\item preparing your data for modeling
\item creating a model using simple but powerful machine learning algorithms
\item scoring your models

\end{enumerate}

\newpage
\input{common/what_is_h2o.tex}
%Need to put into something about where H2O fits into the PyData / SciKit ecosystem.

\subsection{Example Code}

Python code for the examples in this document is located here:

\url{https://github.com/h2oai/h2o-3/tree/master/h2o-docs/src/booklets/v2_2015/source/Python_Vignette_code_examples}

\subsection{Citation}

To cite this booklet, use the following: 

Aiello, S., Cliff, C., Roark, H., Rehak, L., Stetsenko, P., and Bartz, A. (\shortmonthname\ \the\year). {\textit{Machine Learning with Python and H2O}. {\url{http://h2o.ai/resources/}}.


\section{Installation} 
\Urlmuskip=0mu plus 1mu\relax %needed to make long URLs break nicely

H2O requires Java; if you do not already have Java installed, install it from {\url{https://java.com/en/download/}} before installing H2O. 

The easiest way to directly install H2O is  via a Python package.

\subsection{Installation in Python}

To load a recent H2O package from PyPI, run:

{\texttt{pip install h2o}}

To download the
latest stable H2O-3 build from the H2O download page:

\begin{enumerate}
\item Go to {\url{http://h2o.ai/download}}.
\item Choose the latest stable H2O-3 build.
\item Click the ``Install in Python'' tab.
\item Copy and paste the commands into your Python session.
\end{enumerate}

After H2O is installed, verify the installation:

\begin{lstlisting}[style=pythoncode]
import h2o

# Start H2O on your local machine
h2o.init()

# Get help
help(h2o.estimators.glm.H2OGeneralizedLinearEstimator)
help(h2o.estimators.gbm.H2OGradientBoostingEstimator)

# Show a demo
h2o.demo("glm")
h2o.demo("gbm")
\end{lstlisting}


\section{Data Preparation}
The next sections of the booklet demonstrate the Python interface using examples, which include  short snippets of code and the
resulting output.  

In H2O, these operations all occur distributed and in
parallel and can be used on very large datasets.  More information about the
Python interface to H2O can be found at {\texttt{docs.h2o.ai}}.

Typically, we import and start H2O on the same machine as the running Python process:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_initializeh2o_example.py}

To connect to an established H2O cluster (in a multi-node Hadoop environment, for example):
\lstinputlisting[style=pythoncode, firstline=1, lastline=2]{python/ipython_dataprep_output_fixed.txt}

To create an H2OFrame object from a Python tuple:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_create_frame_from_python_tuple.py}

To create an H2OFrame object from a Python list:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_create_frame_from_python_list.py}

To create an H2OFrame object from \texttt{collections.OrderedDict} or a Python dict:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_create_frame_from_ordered_dict.py}

To create an H2OFrame object from a Python dict and specify the column types:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_create_frame_from_python_dict.py}

To display the column types:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_display_column_types.py}

\subsection{Viewing Data}
To display the top and bottom of an H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_view_top_and_bottom_of_frame.py}

\newpage
To display the column names:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_display_column_names.py}

To display compression information, distribution (in multi-machine clusters), and summary statistics of your data:

\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_display_compressiondistributionsummary.py}

\subsection{Selection}
To select a single column by name, resulting in an H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_select_column_name.py}

To select a single column by index, resulting in an H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_select_column_index.py}

To select multiple columns by name, resulting in an H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_select_multiple_column_names.py}

To select multiple columns by index, resulting in an H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_select_multiple_columns_by_index.py}

\newpage
To select multiple rows by slicing, resulting in an H2OFrame: 

\textbf{Note} By default, H2OFrame selection is for columns, so to slice by rows
and get all columns, be explicit about selecting all columns:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_select_rows_by_slicing.py}


To select rows based on specific criteria, use Boolean masking:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_select_rows_boolean.py}


\subsection{Missing Data}
The H2O parser can handle many different representations of missing data types, including `' (blank),
`NA',  and None (Python).  They are all displayed as \texttt{nan} in Python.

To create an H2OFrame from Python with missing elements:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_create_frame_with_missing_elements.py}

To determine which rows are missing data for a given column (`1' indicates missing):
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_find_rows_missing_data_for_a_column.py}

To change all missing values in a column to a different value:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_change_missing_values_to_new_value.py}

To determine the location of all missing data in an H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_find_missing_data_in_frame.py}

\subsection{Operations}
When performing a descriptive statistic on an entire H2OFrame, missing data is generally excluded
and the operation is only performed on the columns of the appropriate data type:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_descriptive_stats_entire_frame.py}

When performing a descriptive statistic on a single column of an H2OFrame, missing data is generally 
\textit{not} excluded:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_descriptive_stats_single_column.py}

In both examples,  a native Python object is returned (list and float respectively
in these examples).

\newpage

When applying functions to each column of the data, an H2OFrame containing the means of each column is returned:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_apply_funct_to_columns.py}

When applying functions to each row of the data, an H2OFrame containing the sum of all columns is returned:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_apply_funct_to_rows.py}

H2O provides many methods for histogramming and discretizing data.
Here is an example using the {\texttt{hist}} method on a single data frame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_histogramming_data.py}

H2O includes a set of string processing methods in the H2OFrame class
that make it easy to operate on each element in an H2OFrame.  

To determine the number of times a string is contained in each element:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_determine_string_count_in_element.py}

To replace the first occurrence of `l' (lower case letter) with `x' and return a new H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_replace_first_l_with_x.py}

For global substitution, use {\texttt{gsub}}.  Both {\texttt{sub}} and {\texttt{gsub}} support regular expressions. 

To split strings based on a regular expression:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_split_using_regex.py}

\newpage
\subsection{Merging}
To combine two H2OFrames together by appending one as rows and return a new H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_combine_frames_append_one_as_rows.py}

For successful row binding, the column names and column types between the two H2OFrames must match. To combine two H2O frames together by appending one as columns and return a new H2OFrame:

\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_combine_frames_append_one_as_columns.py}

\newpage
H2O also supports merging two frames together by matching column names:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_merge_frames_by_column_name.py}

\subsection{Grouping}

"Grouping" refers to the following process:

\begin{itemize}
\item splitting the data into groups based on some criteria 
\item applying a function to each group independently
\item combining the results into an H2OFrame
\end{itemize}

To group and then apply a function to the results:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_group_and_apply_function.py}

To group by multiple columns and then apply a function:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_group_by_multiple_columns.py}

Use {\texttt{merge}} to join the results into the original H2OFrame:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_join_results.py}

\subsection{Using Date and Time Data} 
H2O has powerful features for ingesting and feature engineering using time data.  Internally, H2O
stores time information as an integer of the number of milliseconds since the epoch.

To ingest time data natively, use one of the supported time input formats:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_injest_time_natively.py}

To display the day of the month:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_display_day_of_month.py}

To display the day of the week:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_display_day_of_week.py}

\subsection{Categoricals}
H2O handles categorical (also known as enumerated or factor) values in an H2OFrame.  This is significant because categorical
columns have specific treatments in each of the machine learning algorithms.

Using `df12' from above, H2O imports columns A and B as categorical/enumerated/factor types:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_show_column_types.py}

To determine if any column is a categorical/enumerated/factor type:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_column_is_anyfactor.py}

To view the categorical levels in a single column:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_view_categorical_levels_in_column.py}

To create categorical interaction features:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_create_categorical_interaction_features.py}

To retain the most common categories and set the remaining categories to a common `Other' category
 and create an interaction of a categorical column with itself:

\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_retain_common_categories.py}

These can then be added as a new column on the original dataframe:
\lstinputlisting[style=pythoncode]{Python_Vignette_code_examples/python_add_common_column_to_orig_dataframe.py}

\subsection{Loading and Saving Data}
In addition to loading data from Python objects, H2O can load data directly from:
\begin{itemize}
\item disk
\item network file systems (NFS, S3)
\item distributed file systems (HDFS)
\item HTTP addresses
\end{itemize}

 H2O currently supports the following file types:

\begin{frame}%no line table for list of 6+ items

\begin{tabular}{p{5.5cm}p{5.5cm}}

\begin{itemize}
\item CSV (delimited) files
\item ORC
\item SVMLite
\item Parquet
\end{itemize} &
\begin{itemize}
\item ARFF
\item XLS
\item XLSX 
\item AVRO
\end{itemize}
\end{tabular}
\end{frame}

To load data from the same machine running H2O:
\lstinputlisting[style=pythoncode, firstline=6, lastline=7]{python/ipython_dataprep_output_fixed.txt}

To load data from the machine(s) running H2O to the machine running Python:
\lstinputlisting[style=pythoncode, firstline=8, lastline=9]{python/ipython_dataprep_output_fixed.txt}

To save an H2OFrame on the machine running H2O:
\lstinputlisting[style=pythoncode, firstline=10, lastline=11]{python/ipython_dataprep_output_fixed.txt}

To save an H2OFrame on the machine running Python:
\lstinputlisting[style=pythoncode, firstline=12, lastline=13]{python/ipython_dataprep_output_fixed.txt}

\section{Machine Learning}

The following sections describe some common model types and features. 

\subsection{Modeling}
The following section describes the features and functions of some common models available in H2O.  For more information about running these models in Python using H2O, refer to the documentation on
the H2O.ai website or to the booklets on specific models.


H2O supports the following models:  

\begin{frame}%no line table for list of 6+ items

\begin{tabular}{p{5.0cm}p{6.0cm}}

\begin{itemize}
  \item Deep Learning
  \item Na\"{i}ve Bayes
  \item Principal Components Analysis (PCA)
  \item K-means
  \item Stacked Ensembles
  \item XGBoost
\end{itemize} &

\begin{itemize}
  \item Generalized Linear Models (GLM) 
  \item Gradient Boosting Machine (GBM)
  \item Generalized Low Rank Model (GLRM)
  \item Distributed Random Forest (DRF)
  \item Word2vec
\end{itemize}

\end{tabular}

\end{frame}

The list continues to grow, so check \url{www.h2o.ai} to see the latest additions. 

\subsubsection{Supervised Learning}


{\textbf{Generalized Linear Models (GLM)}}: Provides flexible generalization of ordinary linear regression for response variables with error distribution models other than a Gaussian (normal) distribution. GLM unifies various other statistical models, including Poisson, linear, logistic, and others when using $\ell_1$ and $\ell_2$ regularization.

{\textbf{Distributed Random Forest}}: Averages multiple decision trees, each created on different random samples of rows and columns. It is easy to use, non-linear, and provides feedback on the importance of each predictor in the model, making it one of the most robust algorithms for noisy data.

{\textbf{Gradient Boosting Machine (GBM)}}: Produces a prediction model in the form of an ensemble of weak prediction models. It builds the model in a stage-wise fashion and is generalized by allowing an arbitrary differentiable loss function. It is one of the most powerful methods available today.

{\textbf{Deep Learning}}: Models high-level abstractions in data by using non-linear transformations in a layer-by-layer method. Deep learning is an example of supervised learning, which can use unlabeled data that other algorithms cannot.

{\textbf{Na\"{i}ve Bayes}}: Generates a probabilistic classifier that assumes the value of a particular feature is unrelated to the presence or absence of any other feature, given the class variable. It is often used in text categorization.

{\textbf{Stacked Ensembles}}: Using multiple models built from different algorithms, Stacked Ensembles finds the optimal combination of a collection of prediction algorithms using a process known as "stacking."

{\textbf{XGBoost}}: XGBoost is an optimized gradient boosting library that implements machine learning algorithms under the Gradient Boosting Machine (GBM) framework. For many problems, XGBoost is the one of the best GBM frameworks today.  In other cases, the H2O GBM algorithm comes out on top.  Both implementations are available on the H2O platform.

\subsubsection{Unsupervised Learning}
{\textbf{K-Means}}: Reveals groups or clusters of data points for segmentation. It clusters observations into $k$-number of points with the nearest mean.

{\textbf{Principal Component Analysis (PCA)}}: The algorithm is carried out on a set of possibly collinear features and performs a transformation to produce a new set of uncorrelated features.

{\textbf{Generalized Low Rank Model (GLRM)}}: The method reconstructs missing values and identifies important features in heterogeneous data. It also recognizes a number of  interpretations of low rank factors, which allows clustering of examples or of features.

{\textbf{Anomaly Detection}}: Identifies the outliers in your data by invoking the deep learning autoencoder, a powerful pattern recognition model.

\subsubsection{Miscellaneous}
{\textbf{Word2vec}}: Takes a text corpus as an input and produces the word vectors as output. The result is an H2O Word2vec model that can be exported as a binary model or as a MOJO.

\subsection{Running Models}
This section describes how to run the following model types:

\begin{itemize}
\item Gradient Boosting Machine (GBM)
\item Generalized Linear Models (GLM)
\item K-means
\item Principal Components Analysis (PCA)

\end{itemize}
This section also shows how to generate predictions.

\newpage
\subsubsection{Gradient Boosting Machine (GBM)}
To generate gradient boosting machine models for creating forward-learning ensembles,
use {\texttt{H2OGradientBoostingEstimator}}.  

The construction of the estimator
defines the parameters of the estimator and the call to
{\texttt{H2OGradientBoostingEstimator.train}}  trains the estimator on
the specified data.  This pattern is common for each of the H2O algorithms.

\lstinputlisting[style=pythoncode]{python/python_train_gbm_model.py}

To generate a classification model that uses labels, \\ use 
{\texttt{distribution="multinomial"}}:

\lstinputlisting[style=pythoncode]{python/python_train_gbm_multinomial_model.py}

\subsubsection{Generalized Linear Models (GLM)}
Generalized linear models (GLM) are some of the most commonly-used
models for many types of data analysis use cases. While some data
can be analyzed using linear models, linear models
may not be as accurate if the variables are more complex.
For example, if the dependent variable has a non-continuous
distribution or if the effect of the predictors is not linear,
generalized linear models will produce more accurate results than linear models.

Generalized Linear Models (GLM) estimate regression models for
outcomes following exponential distributions in general. In
addition to the Gaussian (i.e. normal) distribution, these include
Poisson, binomial, gamma and Tweedie distributions. Each serves
a different purpose and, depending on distribution and link
function choice, it can be used either for prediction or
classification.

H2O's GLM algorithm fits the generalized linear model with
elastic net penalties. The model fitting computation is distributed,
extremely fast,and scales extremely well for models with a limited
number ($\sim$ low thousands) of predictors with non-zero
coefficients. 

The algorithm can compute models for a single value of a penalty argument or the full regularization path, similar to glmnet. It can compute Gaussian (linear), logistic, Poisson, and gamma regression models.
To generate a generalized linear model for developing
linear models for exponential distributions, use
{\texttt{H2OGeneralizedLinearEstimator}}. You can apply
regularization to the model by adjusting the lambda and alpha
parameters.

\lstinputlisting[style=pythoncode]{python/python_train_glm_model.py}

\subsubsection{K-means}
To generate a K-means model for data characterization, use
{\texttt{h2o.kmeans()}}. This algorithm does not require a
dependent variable.

\lstinputlisting[style=pythoncode]{python/python_train_kmeans_model.py}

\subsubsection{Principal Components Analysis (PCA)}
To map a set of variables onto a subspace using linear
transformations, use {\texttt{H2OPrincipalComponentAnalysisEstimator}}.
This is the first step in Principal Components Regression.
\lstinputlisting[style=pythoncode]{python/python_train_pca_model.py}

\subsection{Grid Search}
H2O supports grid search across hyperparameters:

\lstinputlisting[style=pythoncode]{python/python_grid_search.py}

\subsection{Integration with scikit-learn}
The H2O Python client can be used within scikit-learn pipelines and cross-validation searches.  This extends the capabilities of both H2O and scikit-learn. Note that the sklearn and scipy packages are required to use the H2O Python client with scikit-learn. 

\subsubsection{Pipelines}

To create a scikit-learn style pipeline using H2O transformers and estimators:

\lstinputlisting[style=pythoncode]{python/python_scikit_learn_pipeline.py}

\subsubsection{Randomized Grid Search}
To create a scikit-learn style hyperparameter grid search using k-fold cross validation:
\lstinputlisting[style=pythoncode]{python/python_randomized_grid_search.py}

\section{Acknowledgments}
We would like to acknowledge the following individuals for their contributions to this booklet: Spencer Aiello, Cliff Click, Hank Roark, Ludi Rehak, and Jessica Lanford.

\section{References}
\bibliographystyle{plainnat}  %alphadin}
\nobibliography{bibliography.bib} %hides entire bibliography so just \bibentry items are included
%use \bibentry{bibname} (where bibname is the entry name in the bibliography) to include entries from bibliography.bib; double brackets {{ are required in .bib file to preserve capitalization

\bibentry{h2osite}

\bibentry{h2odocs}


\bibentry{Pydocs}

\bibentry{h2ogithubrepo}

\newpage

\bibentry{h2odatasets}

\bibentry{h2ojira}

\bibentry{stream}


\enddocument
