![Conventions
In this book, you will find a number of text styles that distinguish between different
kinds of information. Here are some examples of these styles and an explanation of
their meaning.
Code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles are shown as follows:
"Using the out_file=None setting, we directly assigned the dot data to a dot_data
variable, instead of writing an intermediate tree.dot file to disk."
A block of code is set as follows:
>>> from sklearn.neighbors import KNeighborsClassifier
>>> knn = KNeighborsClassifier(n_neighbors=5, p=2,
... metric='minkowski')
>>> knn.fit(X_train_std, y_train)
>>> plot_decision_regions(X_combined_std, y_combined,
... classifier=knn, test_idx=range(105,150))
>>> plt.xlabel('petal length [standardized]')
>>> plt.ylabel('petal width [standardized]')
>>> plt.show()
Any command-line input or output is written as follows:
pip3 install graphviz
New terms and important words are shown in bold. Words that you see on the
screen, for example, in menus or dialog boxes, appear in the text like this: "After we
click on the Dashboard button in the top-right corner, we have access to the control
panel shown at the top of the page."
Note
Warnings or important notes appear in a box like this.
Tip
Tips and tricks appear like this.](https://image.slidesharecdn.com/3364622-250529202844-a5f2ffed/85/Python-Machine-Learning-Sebastian-Raschka-Vahid-Mirjalili-34-320.jpg)
Python Machine Learning Sebastian Raschka Vahid Mirjalili
- 1. Python Machine Learning Sebastian Raschka Vahid Mirjalili download https://ebookbell.com/product/python-machine-learning-sebastian- raschka-vahid-mirjalili-6729244 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Python Machine Learning Projects Learn How To Build Machine Learning Projects From Scratch Deepali R Vora https://ebookbell.com/product/python-machine-learning-projects-learn- how-to-build-machine-learning-projects-from-scratch-deepali-r- vora-49422988 Python Machine Learning Projects Learn How To Build Machine Learning Projects From Scratch Dr Deepali R Vora https://ebookbell.com/product/python-machine-learning-projects-learn- how-to-build-machine-learning-projects-from-scratch-dr-deepali-r- vora-49763682 Python Machine Learning Projects 1st Edition Lisa Tagliaferri https://ebookbell.com/product/python-machine-learning-projects-1st- edition-lisa-tagliaferri-53726322 Python Machine Learning 1st Edition Weimeng Lee https://ebookbell.com/product/python-machine-learning-1st-edition- weimeng-lee-57017492
- 3. Python Machine Learning Second Edition 2nd Edition Sebastian Raschka Vahid Mirjalili https://ebookbell.com/product/python-machine-learning-second- edition-2nd-edition-sebastian-raschka-vahid-mirjalili-20632882 Python Machine Learning By Example 3rd Edition Yuxi Hayden Liu https://ebookbell.com/product/python-machine-learning-by-example-3rd- edition-yuxi-hayden-liu-22069088 Python Machine Learning Raschka Sebastian https://ebookbell.com/product/python-machine-learning-raschka- sebastian-22122802 Python Machine Learning Machine Learning And Deep Learning With Python Scikitlearn And Tensorflow 2 3rd Edition 3rd Edition Raschka https://ebookbell.com/product/python-machine-learning-machine- learning-and-deep-learning-with-python-scikitlearn-and- tensorflow-2-3rd-edition-3rd-edition-raschka-22122808 Python Machine Learning Blueprints Intuitive Data Projects You Can Relate To 1st Edition Combs https://ebookbell.com/product/python-machine-learning-blueprints- intuitive-data-projects-you-can-relate-to-1st-edition-combs-23300286
- 4. Python Machine Learning Case Studies Five Case Studies For The Data Scientist 1st Edition Haroon https://ebookbell.com/product/python-machine-learning-case-studies- five-case-studies-for-the-data-scientist-1st-edition-haroon-32704098
- 6. Python Machine Learning Second Edition
- 7. Table of Contents Python Machine Learning Second Edition Credits About the Authors About the Reviewers www.PacktPub.com eBooks, discount offers, and more Why subscribe? Customer Feedback Preface What this book covers What you need for this book Who this book is for Conventions Reader feedback Customer support Downloading the example code Downloading the color images of this book Errata Piracy Questions 1. Giving Computers the Ability to Learn from Data Building intelligent machines to transform data into knowledge The three different types of machine learning Making predictions about the future with supervised learning Classification for predicting class labels Regression for predicting continuous outcomes Solving interactive problems with reinforcement learning Discovering hidden structures with unsupervised learning Finding subgroups with clustering Dimensionality reduction for data compression Introduction to the basic terminology and notations A roadmap for building machine learning systems Preprocessing – getting data into shape Training and selecting a predictive model
- 8. Evaluating models and predicting unseen data instances Using Python for machine learning Installing Python and packages from the Python Package Index Using the Anaconda Python distribution and package manager Packages for scientific computing, data science, and machine learning Summary 2. Training Simple Machine Learning Algorithms for Classification Artificial neurons – a brief glimpse into the early history of machine learning The formal definition of an artificial neuron The perceptron learning rule Implementing a perceptron learning algorithm in Python An object-oriented perceptron API Training a perceptron model on the Iris dataset Adaptive linear neurons and the convergence of learning Minimizing cost functions with gradient descent Implementing Adaline in Python Improving gradient descent through feature scaling Large-scale machine learning and stochastic gradient descent Summary 3. A Tour of Machine Learning Classifiers Using scikit-learn Choosing a classification algorithm First steps with scikit-learn – training a perceptron Modeling class probabilities via logistic regression Logistic regression intuition and conditional probabilities Learning the weights of the logistic cost function Converting an Adaline implementation into an algorithm for logistic regression Training a logistic regression model with scikit-learn Tackling overfitting via regularization Maximum margin classification with support vector machines Maximum margin intuition Dealing with a nonlinearly separable case using slack variables Alternative implementations in scikit-learn Solving nonlinear problems using a kernel SVM Kernel methods for linearly inseparable data Using the kernel trick to find separating hyperplanes in high-dimensional space Decision tree learning Maximizing information gain – getting the most bang for your buck Building a decision tree
- 9. Combining multiple decision trees via random forests K-nearest neighbors – a lazy learning algorithm Summary 4. Building Good Training Sets – Data Preprocessing Dealing with missing data Identifying missing values in tabular data Eliminating samples or features with missing values Imputing missing values Understanding the scikit-learn estimator API Handling categorical data Nominal and ordinal features Creating an example dataset Mapping ordinal features Encoding class labels Performing one-hot encoding on nominal features Partitioning a dataset into separate training and test sets Bringing features onto the same scale Selecting meaningful features L1 and L2 regularization as penalties against model complexity A geometric interpretation of L2 regularization Sparse solutions with L1 regularization Sequential feature selection algorithms Assessing feature importance with random forests Summary 5. Compressing Data via Dimensionality Reduction Unsupervised dimensionality reduction via principal component analysis The main steps behind principal component analysis Extracting the principal components step by step Total and explained variance Feature transformation Principal component analysis in scikit-learn Supervised data compression via linear discriminant analysis Principal component analysis versus linear discriminant analysis The inner workings of linear discriminant analysis Computing the scatter matrices Selecting linear discriminants for the new feature subspace Projecting samples onto the new feature space LDA via scikit-learn
- 10. Using kernel principal component analysis for nonlinear mappings Kernel functions and the kernel trick Implementing a kernel principal component analysis in Python Example 1 – separating half-moon shapes Example 2 – separating concentric circles Projecting new data points Kernel principal component analysis in scikit-learn Summary 6. Learning Best Practices for Model Evaluation and Hyperparameter Tuning Streamlining workflows with pipelines Loading the Breast Cancer Wisconsin dataset Combining transformers and estimators in a pipeline Using k-fold cross-validation to assess model performance The holdout method K-fold cross-validation Debugging algorithms with learning and validation curves Diagnosing bias and variance problems with learning curves Addressing over- and underfitting with validation curves Fine-tuning machine learning models via grid search Tuning hyperparameters via grid search Algorithm selection with nested cross-validation Looking at different performance evaluation metrics Reading a confusion matrix Optimizing the precision and recall of a classification model Plotting a receiver operating characteristic Scoring metrics for multiclass classification Dealing with class imbalance Summary 7. Combining Different Models for Ensemble Learning Learning with ensembles Combining classifiers via majority vote Implementing a simple majority vote classifier Using the majority voting principle to make predictions Evaluating and tuning the ensemble classifier Bagging – building an ensemble of classifiers from bootstrap samples Bagging in a nutshell Applying bagging to classify samples in the Wine dataset Leveraging weak learners via adaptive boosting
- 11. How boosting works Applying AdaBoost using scikit-learn Summary 8. Applying Machine Learning to Sentiment Analysis Preparing the IMDb movie review data for text processing Obtaining the movie review dataset Preprocessing the movie dataset into more convenient format Introducing the bag-of-words model Transforming words into feature vectors Assessing word relevancy via term frequency-inverse document frequency Cleaning text data Processing documents into tokens Training a logistic regression model for document classification Working with bigger data – online algorithms and out-of-core learning Topic modeling with Latent Dirichlet Allocation Decomposing text documents with LDA LDA with scikit-learn Summary 9. Embedding a Machine Learning Model into a Web Application Serializing fitted scikit-learn estimators Setting up an SQLite database for data storage Developing a web application with Flask Our first Flask web application Form validation and rendering Setting up the directory structure Implementing a macro using the Jinja2 templating engine Adding style via CSS Creating the result page Turning the movie review classifier into a web application Files and folders – looking at the directory tree Implementing the main application as app.py Setting up the review form Creating a results page template Deploying the web application to a public server Creating a PythonAnywhere account Uploading the movie classifier application Updating the movie classifier Summary
- 12. 10. Predicting Continuous Target Variables with Regression Analysis Introducing linear regression Simple linear regression Multiple linear regression Exploring the Housing dataset Loading the Housing dataset into a data frame Visualizing the important characteristics of a dataset Looking at relationships using a correlation matrix Implementing an ordinary least squares linear regression model Solving regression for regression parameters with gradient descent Estimating coefficient of a regression model via scikit-learn Fitting a robust regression model using RANSAC Evaluating the performance of linear regression models Using regularized methods for regression Turning a linear regression model into a curve – polynomial regression Adding polynomial terms using scikit-learn Modeling nonlinear relationships in the Housing dataset Dealing with nonlinear relationships using random forests Decision tree regression Random forest regression Summary 11. Working with Unlabeled Data – Clustering Analysis Grouping objects by similarity using k-means K-means clustering using scikit-learn A smarter way of placing the initial cluster centroids using k-means++ Hard versus soft clustering Using the elbow method to find the optimal number of clusters Quantifying the quality of clustering via silhouette plots Organizing clusters as a hierarchical tree Grouping clusters in bottom-up fashion Performing hierarchical clustering on a distance matrix Attaching dendrograms to a heat map Applying agglomerative clustering via scikit-learn Locating regions of high density via DBSCAN Summary 12. Implementing a Multilayer Artificial Neural Network from Scratch Modeling complex functions with artificial neural networks Single-layer neural network recap
- 13. Introducing the multilayer neural network architecture Activating a neural network via forward propagation Classifying handwritten digits Obtaining the MNIST dataset Implementing a multilayer perceptron Training an artificial neural network Computing the logistic cost function Developing your intuition for backpropagation Training neural networks via backpropagation About the convergence in neural networks A few last words about the neural network implementation Summary 13. Parallelizing Neural Network Training with TensorFlow TensorFlow and training performance What is TensorFlow? How we will learn TensorFlow First steps with TensorFlow Working with array structures Developing a simple model with the low-level TensorFlow API Training neural networks efficiently with high-level TensorFlow APIs Building multilayer neural networks using TensorFlow's Layers API Developing a multilayer neural network with Keras Choosing activation functions for multilayer networks Logistic function recap Estimating class probabilities in multiclass classification via the softmax function Broadening the output spectrum using a hyperbolic tangent Rectified linear unit activation Summary 14. Going Deeper – The Mechanics of TensorFlow Key features of TensorFlow TensorFlow ranks and tensors How to get the rank and shape of a tensor Understanding TensorFlow's computation graphs Placeholders in TensorFlow Defining placeholders Feeding placeholders with data Defining placeholders for data arrays with varying batchsizes
- 14. Variables in TensorFlow Defining variables Initializing variables Variable scope Reusing variables Building a regression model Executing objects in a TensorFlow graph using their names Saving and restoring a model in TensorFlow Transforming Tensors as multidimensional data arrays Utilizing control flow mechanics in building graphs Visualizing the graph with TensorBoard Extending your TensorBoard experience Summary 15. Classifying Images with Deep Convolutional Neural Networks Building blocks of convolutional neural networks Understanding CNNs and learning feature hierarchies Performing discrete convolutions Performing a discrete convolution in one dimension The effect of zero-padding in a convolution Determining the size of the convolution output Performing a discrete convolution in 2D Subsampling Putting everything together to build a CNN Working with multiple input or color channels Regularizing a neural network with dropout Implementing a deep convolutional neural network using TensorFlow The multilayer CNN architecture Loading and preprocessing the data Implementing a CNN in the TensorFlow low-level API Implementing a CNN in the TensorFlow Layers API Summary 16. Modeling Sequential Data Using Recurrent Neural Networks Introducing sequential data Modeling sequential data – order matters Representing sequences The different categories of sequence modeling RNNs for modeling sequences Understanding the structure and flow of an RNN
- 15. Computing activations in an RNN The challenges of learning long-range interactions LSTM units Implementing a multilayer RNN for sequence modeling in TensorFlow Project one – performing sentiment analysis of IMDb movie reviews using multilayer RNNs Preparing the data Embedding Building an RNN model The SentimentRNN class constructor The build method Step 1 – defining multilayer RNN cells Step 2 – defining the initial states for the RNN cells Step 3 – creating the RNN using the RNN cells and their states The train method The predict method Instantiating the SentimentRNN class Training and optimizing the sentiment analysis RNN model Project two – implementing an RNN for character-level language modeling in TensorFlow Preparing the data Building a character-level RNN model The constructor The build method The train method The sample method Creating and training the CharRNN Model The CharRNN model in the sampling mode Chapter and book summary Index
- 16. Python Machine Learning Second Edition
- 17. Python Machine Learning Second Edition Copyright © 2017 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: September 2015 Second edition: September 2017 Production reference: 1120917 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78712-593-3 www.packtpub.com
- 18. Credits Authors Sebastian Raschka Vahid Mirjalili Reviewers Jared Huffman Huai-En, Sun (Ryan Sun) Acquisition Editor Frank Pohlmann Content Development Editor Chris Nelson Project Editor Monika Sangwan Technical Editors Bhagyashree Rai Nidhisha Shetty Copy Editor Safis Editing Project Coordinator Suzanne Coutinho
- 19. Proofreader Safis Editing Indexer Tejal Daruwale Soni Graphics Kirk D'Penha Production Coordinator Arvindkumar Gupta
- 20. About the Authors Sebastian Raschka, the author of the bestselling book, Python Machine Learning, has many years of experience with coding in Python, and he has given several seminars on the practical applications of data science, machine learning, and deep learning including a machine learning tutorial at SciPy—the leading conference for scientific computing in Python. While Sebastian's academic research projects are mainly centered around problem- solving in computational biology, he loves to write and talk about data science, machine learning, and Python in general, and he is motivated to help people develop data-driven solutions without necessarily requiring a machine learning background. His work and contributions have recently been recognized by the departmental outstanding graduate student award 2016-2017 as well as the ACM Computing Reviews’ Best of 2016 award. In his free time, Sebastian loves to contribute to open source projects, and the methods that he has implemented are now successfully used in machine learning competitions, such as Kaggle. I would like to take this opportunity to thank the great Python community and developers of open source packages who helped me create the perfect environment for scientific research and data science. Also, I want to thank my parents who always encouraged and supported me in pursuing the path and career that I was so passionate about. Special thanks to the core developers of scikit-learn. As a contributor to this project, I had the pleasure to work with great people who are not only very knowledgeable when it comes to machine learning but are also excellent programmers. Lastly, I'd like to thank Elie Kawerk, who volunteered to review the book and provided valuable feedback on the new chapters. Vahid Mirjalili obtained his PhD in mechanical engineering working on novel methods for large-scale, computational simulations of molecular structures. Currently, he is focusing his research efforts on applications of machine learning in various computer vision projects at the department of computer science and engineering at Michigan State University.
- 21. Vahid picked Python as his number-one choice of programming language, and throughout his academic and research career he has gained tremendous experience with coding in Python. He taught Python programming to the engineering class at Michigan State University, which gave him a chance to help students understand different data structures and develop efficient code in Python. While Vahid's broad research interests focus on deep learning and computer vision applications, he is especially interested in leveraging deep learning techniques to extend privacy in biometric data such as face images so that information is not revealed beyond what users intend to reveal. Furthermore, he also collaborates with a team of engineers working on self-driving cars, where he designs neural network models for the fusion of multispectral images for pedestrian detection. I would like to thank my PhD advisor, Dr. Arun Ross, for giving me the opportunity to work on novel problems in his research lab. I also like to thank Dr. Vishnu Boddeti for inspiring my interests in deep learning and demystifying its core concepts.
- 22. About the Reviewers Jared Huffman is an entrepreneur, gamer, storyteller, machine learning fanatic, and database aficionado. He has dedicated the past 10 years to developing software and analyzing data. His previous work has spanned a variety of topics, including network security, financial systems, and business intelligence, as well as web services, developer tools, and business strategy. Most recently, he was the founder of the data science team at Minecraft, with a focus on big data and machine learning. When not working, you can typically find him gaming or enjoying the beautiful Pacific Northwest with friends and family. I'd like to thank Packt for giving me the opportunity to work on such a great book, my wife for the constant encouragement, and my daughter for sleeping through most of the late nights while I was reviewing and debugging code. Huai-En, Sun (Ryan Sun) holds a master's degree in statistics from the National Chiao Tung University. He is currently working as a data scientist for analyzing the production line at PEGATRON. Machine learning and deep learning are his main areas of research.
- 23. www.PacktPub.com
- 24. eBooks, discount offers, and more Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks. https://www.packtpub.com/mapt Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
- 25. Why subscribe? Fully searchable across every book published by Packt Copy and paste, print, and bookmark content On demand and accessible via a web browser
- 26. Customer Feedback Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1787125939. If you'd like to join our team of regular reviewers, you can email us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
- 27. Preface Through exposure to the news and social media, you are probably aware of the fact that machine learning has become one of the most exciting technologies of our time and age. Large companies, such as Google, Facebook, Apple, Amazon, and IBM, heavily invest in machine learning research and applications for good reasons. While it may seem that machine learning has become the buzzword of our time and age, it is certainly not a fad. This exciting field opens the way to new possibilities and has become indispensable to our daily lives. This is evident in talking to the voice assistant on our smartphones, recommending the right product for our customers, preventing credit card fraud, filtering out spam from our email inboxes, detecting and diagnosing medical diseases, the list goes on and on. If you want to become a machine learning practitioner, a better problem solver, or maybe even consider a career in machine learning research, then this book is for you. However, for a novice, the theoretical concepts behind machine learning can be quite overwhelming. Many practical books have been published in recent years that will help you get started in machine learning by implementing powerful learning algorithms. Getting exposed to practical code examples and working through example applications of machine learning are a great way to dive into this field. Concrete examples help illustrate the broader concepts by putting the learned material directly into action. However, remember that with great power comes great responsibility! In addition to offering a hands-on experience with machine learning using the Python programming languages and Python-based machine learning libraries, this book introduces the mathematical concepts behind machine learning algorithms, which is essential for using machine learning successfully. Thus, this book is different from a purely practical book; it is a book that discusses the necessary details regarding machine learning concepts and offers intuitive yet informative explanations of how machine learning algorithms work, how to use them, and most importantly, how to avoid the most common pitfalls. Currently, if you type "machine learning" as a search term in Google Scholar, it returns an overwhelmingly large number of publications—1,800,000. Of course, we cannot discuss the nitty-gritty of all the different algorithms and applications that have emerged in the last 60 years. However, in this book, we will embark on an exciting journey that covers all the essential topics and concepts to give you a head
- 28. start in this field. If you find that your thirst for knowledge is not satisfied, this book references many useful resources that can be used to follow up on the essential breakthroughs in this field. If you have already studied machine learning theory in detail, this book will show you how to put your knowledge into practice. If you have used machine learning techniques before and want to gain more insight into how machine learning actually works, this book is for you. Don't worry if you are completely new to the machine learning field; you have even more reason to be excited. Here is a promise that machine learning will change the way you think about the problems you want to solve and will show you how to tackle them by unlocking the power of data. Before we dive deeper into the machine learning field, let's answer your most important question, "Why Python?" The answer is simple: it is powerful yet very accessible. Python has become the most popular programming language for data science because it allows us to forget about the tedious parts of programming and offers us an environment where we can quickly jot down our ideas and put concepts directly into action. We, the authors, can truly say that the study of machine learning has made us better scientists, thinkers, and problem solvers. In this book, we want to share this knowledge with you. Knowledge is gained by learning. The key is our enthusiasm, and the real mastery of skills can only be achieved by practice. The road ahead may be bumpy on occasions and some topics may be more challenging than others, but we hope that you will embrace this opportunity and focus on the reward. Remember that we are on this journey together, and throughout this book, we will add many powerful techniques to your arsenal that will help us solve even the toughest problems the data-driven way.
- 29. What this book covers Chapter 1 , Giving Computers the Ability to Learn from Data, introduces you to the main subareas of machine learning in order to tackle various problem tasks. In addition, it discusses the essential steps for creating a typical machine learning model by building a pipeline that will guide us through the following chapters. Chapter 2 , Training Simple Machine Learning Algorithms for Classification, goes back to the origins of machine learning and introduces binary perceptron classifiers and adaptive linear neurons. This chapter is a gentle introduction to the fundamentals of pattern classification and focuses on the interplay of optimization algorithms and machine learning. Chapter 3 , A Tour of Machine Learning Classifiers Using scikit-learn, describes the essential machine learning algorithms for classification and provides practical examples using one of the most popular and comprehensive open source machine learning libraries: scikit-learn. Chapter 4 , Building Good Training Sets – Data Preprocessing, discusses how to deal with the most common problems in unprocessed datasets, such as missing data. It also discusses several approaches to identify the most informative features in datasets and teaches you how to prepare variables of different types as proper input for machine learning algorithms. Chapter 5 , Compressing Data via Dimensionality Reduction, describes the essential techniques to reduce the number of features in a dataset to smaller sets while retaining most of their useful and discriminatory information. It discusses the standard approach to dimensionality reduction via principal component analysis and compares it to supervised and nonlinear transformation techniques. Chapter 6 , Learning Best Practices for Model Evaluation and Hyperparameter Tuning, discusses the dos and don'ts for estimating the performances of predictive models. Moreover, it discusses different metrics for measuring the performance of our models and techniques to fine-tune machine learning algorithms. Chapter 7 , Combining Different Models for Ensemble Learning, introduces you to the different concepts of combining multiple learning algorithms effectively. It teaches you how to build ensembles of experts to overcome the weaknesses of
- 30. individual learners, resulting in more accurate and reliable predictions. Chapter 8 , Applying Machine Learning to Sentiment Analysis, discusses the essential steps to transform textual data into meaningful representations for machine learning algorithms to predict the opinions of people based on their writing. Chapter 9 , Embedding a Machine Learning Model into a Web Application, continues with the predictive model from the previous chapter and walks you through the essential steps of developing web applications with embedded machine learning models. Chapter 10 , Predicting Continuous Target Variables with Regression Analysis, discusses the essential techniques for modeling linear relationships between target and response variables to make predictions on a continuous scale. After introducing different linear models, it also talks about polynomial regression and tree-based approaches. Chapter 11 , Working with Unlabeled Data – Clustering Analysis, shifts the focus to a different subarea of machine learning, unsupervised learning. We apply algorithms from three fundamental families of clustering algorithms to find groups of objects that share a certain degree of similarity. Chapter 12 , Implementing a Multilayer Artificial Neural Network from Scratch, extends the concept of gradient-based optimization, which we first introduced in Chapter 2, Training Simple Machine Learning Algorithms for Classification, to build powerful, multilayer neural networks based on the popular backpropagation algorithm in Python. Chapter 13 , Parallelizing Neural Network Training with TensorFlow, builds upon the knowledge from the previous chapter to provide you with a practical guide for training neural networks more efficiently. The focus of this chapter is on TensorFlow, an open source Python library that allows us to utilize multiple cores of modern GPUs. Chapter 14, Going Deeper – The Mechanics of TensorFlow, covers TensorFlow in greater detail explaining its core concepts of computational graphs and sessions. In addition, this chapter covers topics such as saving and visualizing neural network graphs, which will come in very handy during the remaining chapters of this book.
- 31. Chapter 15 , Classifying Images with Deep Convolutional Neural Networks, discusses deep neural network architectures that have become the new standard in computer vision and image recognition fields—convolutional neural networks. This chapter will discuss the main concepts between convolutional layers as a feature extractor and apply convolutional neural network architectures to an image classification task to achieve almost perfect classification accuracy. Chapter 16 , Modeling Sequential Data Using Recurrent Neural Networks, introduces another popular neural network architecture for deep learning that is especially well suited for working with sequential data and time series data. In this chapter, we will apply different recurrent neural network architectures to text data. We will start with a sentiment analysis task as a warm-up exercise and will learn how to generate entirely new text.
- 32. What you need for this book The execution of the code examples provided in this book requires an installation of Python 3.6.0 or newer on macOS, Linux, or Microsoft Windows. We will make frequent use of Python's essential libraries for scientific computing throughout this book, including SciPy, NumPy, scikit-learn, Matplotlib, and pandas. The first chapter will provide you with instructions and useful tips to set up your Python environment and these core libraries. We will add additional libraries to our repertoire; moreover, installation instructions are provided in the respective chapters: the NLTK library for natural language processing (Chapter 8, Applying Machine Learning to Sentiment Analysis), the Flask web framework (Chapter 9, Embedding a Machine Learning Algorithm into a Web Application), the Seaborn library for statistical data visualization (Chapter 10, Predicting Continuous Target Variables with Regression Analysis), and TensorFlow for efficient neural network training on graphical processing units (Chapters 13 to 16).
- 33. Who this book is for If you want to find out how to use Python to start answering critical questions of your data, pick up Python Machine Learning, Second Edition—whether you want to start from scratch or extend your data science knowledge, this is an essential and unmissable resource.
- 34. Conventions In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning. Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "Using the out_file=None setting, we directly assigned the dot data to a dot_data variable, instead of writing an intermediate tree.dot file to disk." A block of code is set as follows: >>> from sklearn.neighbors import KNeighborsClassifier >>> knn = KNeighborsClassifier(n_neighbors=5, p=2, ... metric='minkowski') >>> knn.fit(X_train_std, y_train) >>> plot_decision_regions(X_combined_std, y_combined, ... classifier=knn, test_idx=range(105,150)) >>> plt.xlabel('petal length [standardized]') >>> plt.ylabel('petal width [standardized]') >>> plt.show() Any command-line input or output is written as follows: pip3 install graphviz New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "After we click on the Dashboard button in the top-right corner, we have access to the control panel shown at the top of the page." Note Warnings or important notes appear in a box like this. Tip Tips and tricks appear like this.
- 35. Reader feedback Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply email <[email protected]>, and mention the book's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
- 36. Customer support Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
- 37. Downloading the example code You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files emailed directly to you. You can download the code files by following these steps: 1. Log in or register to our website using your email address and password. 2. Hover the mouse pointer on the SUPPORT tab at the top. 3. Click on Code Downloads & Errata. 4. Enter the name of the book in the Search box. 5. Select the book for which you're looking to download the code files. 6. Choose from the drop-down menu where you purchased this book from. 7. Click on Code Download. You can also download the code files by clicking on the Code Files button on the book's web page at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account. Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of: WinRAR / 7-Zip for Windows Zipeg / iZip / UnRarX for Mac 7-Zip / PeaZip for Linux The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Python-Machine-Learning-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
- 38. Downloading the color images of this book We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from http://www.packtpub.com/sites/default/files/downloads/PythonMachineLearningSecondEditio In addition, lower resolution color images are embedded in the code notebooks of this book that come bundled with the example code files.
- 39. Errata Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title. To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
- 40. Piracy Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy. Please contact us at <[email protected]> with a link to the suspected pirated material. We appreciate your help in protecting our authors and our ability to bring you valuable content.
- 41. Questions If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.
- 42. Chapter 1. Giving Computers the Ability to Learn from Data In my opinion, machine learning, the application and science of algorithms that make sense of data, is the most exciting field of all the computer sciences! We are living in an age where data comes in abundance; using self-learning algorithms from the field of machine learning, we can turn this data into knowledge. Thanks to the many powerful open source libraries that have been developed in recent years, there has probably never been a better time to break into the machine learning field and learn how to utilize powerful algorithms to spot patterns in data and make predictions about future events. In this chapter, you will learn about the main concepts and different types of machine learning. Together with a basic introduction to the relevant terminology, we will lay the groundwork for successfully using machine learning techniques for practical problem solving. In this chapter, we will cover the following topics: The general concepts of machine learning The three types of learning and basic terminology The building blocks for successfully designing machine learning systems Installing and setting up Python for data analysis and machine learning
- 43. Building intelligent machines to transform data into knowledge In this age of modern technology, there is one resource that we have in abundance: a large amount of structured and unstructured data. In the second half of the twentieth century, machine learning evolved as a subfield of Artificial Intelligence (AI) that involved self-learning algorithms that derived knowledge from data in order to make predictions. Instead of requiring humans to manually derive rules and build models from analyzing large amounts of data, machine learning offers a more efficient alternative for capturing the knowledge in data to gradually improve the performance of predictive models and make data-driven decisions. Not only is machine learning becoming increasingly important in computer science research, but it also plays an ever greater role in our everyday lives. Thanks to machine learning, we enjoy robust email spam filters, convenient text and voice recognition software, reliable web search engines, challenging chess-playing programs, and, hopefully soon, safe and efficient self-driving cars.
- 44. The three different types of machine learning In this section, we will take a look at the three types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. We will learn about the fundamental differences between the three different learning types and, using conceptual examples, we will develop an intuition for the practical problem domains where these can be applied:
- 45. Making predictions about the future with supervised learning The main goal in supervised learning is to learn a model from labeled training data that allows us to make predictions about unseen or future data. Here, the term supervised refers to a set of samples where the desired output signals (labels) are already known. Considering the example of email spam filtering, we can train a model using a supervised machine learning algorithm on a corpus of labeled emails, emails that are correctly marked as spam or not-spam, to predict whether a new email belongs to either of the two categories. A supervised learning task with discrete class labels, such as in the previous email spam filtering example, is also called a classification task. Another subcategory of supervised learning is regression, where the outcome signal is a continuous value: Classification for predicting class labels
- 46. Classification is a subcategory of supervised learning where the goal is to predict the categorical class labels of new instances, based on past observations. Those class labels are discrete, unordered values that can be understood as the group memberships of the instances. The previously mentioned example of email spam detection represents a typical example of a binary classification task, where the machine learning algorithm learns a set of rules in order to distinguish between two possible classes: spam and non-spam emails. However, the set of class labels does not have to be of a binary nature. The predictive model learned by a supervised learning algorithm can assign any class label that was presented in the training dataset to a new, unlabeled instance. A typical example of a multiclass classification task is handwritten character recognition. Here, we could collect a training dataset that consists of multiple handwritten examples of each letter in the alphabet. Now, if a user provides a new handwritten character via an input device, our predictive model will be able to predict the correct letter in the alphabet with certain accuracy. However, our machine learning system would be unable to correctly recognize any of the digits zero to nine, for example, if they were not part of our training dataset. The following figure illustrates the concept of a binary classification task given 30 training samples; 15 training samples are labeled as negative class (minus signs) and 15 training samples are labeled as positive class (plus signs). In this scenario, our dataset is two-dimensional, which means that each sample has two values associated with it: and . Now, we can use a supervised machine learning algorithm to learn a rule—the decision boundary represented as a dashed line—that can separate those two classes and classify new data into each of those two categories given its and values:
- 47. Regression for predicting continuous outcomes We learned in the previous section that the task of classification is to assign categorical, unordered labels to instances. A second type of supervised learning is the prediction of continuous outcomes, which is also called regression analysis. In regression analysis, we are given a number of predictor (explanatory) variables and a continuous response variable (outcome or target), and we try to find a relationship between those variables that allows us to predict an outcome.
- 48. For example, let's assume that we are interested in predicting the math SAT scores of our students. If there is a relationship between the time spent studying for the test and the final scores, we could use it as training data to learn a model that uses the study time to predict the test scores of future students who are planning to take this test. Note The term regression was devised by Francis Galton in his article Regression towards Mediocrity in Hereditary Stature in 1886. Galton described the biological phenomenon that the variance of height in a population does not increase over time. He observed that the height of parents is not passed on to their children, but instead the children's height is regressing towards the population mean. The following figure illustrates the concept of linear regression. Given a predictor variable x and a response variable y, we fit a straight line to this data that minimizes the distance—most commonly the average squared distance—between the sample points and the fitted line. We can now use the intercept and slope learned from this data to predict the outcome variable of new data:
- 50. Solving interactive problems with reinforcement learning Another type of machine learning is reinforcement learning. In reinforcement learning, the goal is to develop a system (agent) that improves its performance based on interactions with the environment. Since the information about the current state of the environment typically also includes a so-called reward signal, we can think of reinforcement learning as a field related to supervised learning. However, in reinforcement learning this feedback is not the correct ground truth label or value, but a measure of how well the action was measured by a reward function. Through its interaction with the environment, an agent can then use reinforcement learning to learn a series of actions that maximizes this reward via an exploratory trial-and-error approach or deliberative planning. A popular example of reinforcement learning is a chess engine. Here, the agent decides upon a series of moves depending on the state of the board (the environment), and the reward can be defined as win or lose at the end of the game: There are many different subtypes of reinforcement learning. However, a general scheme is that the agent in reinforcement learning tries to maximize the reward by a series of interactions with the environment. Each state can be associated with a
- 51. positive or negative reward, and a reward can be defined as accomplishing an overall goal, such as winning or losing a game of chess. For instance, in chess the outcome of each move can be thought of as a different state of the environment. To explore the chess example further, let's think of visiting certain locations on the chess board as being associated with a positive event—for instance, removing an opponent's chess piece from the board or threatening the queen. Other positions, however, are associated with a negative event, such as losing a chess piece to the opponent in the following turn. Now, not every turn results in the removal of a chess piece, and reinforcement learning is concerned with learning the series of steps by maximizing a reward based on immediate and delayed feedback. While this section provides a basic overview of reinforcement learning, please note that applications of reinforcement learning are beyond the scope of this book, which primarily focusses on classification, regression analysis, and clustering.
- 52. Discovering hidden structures with unsupervised learning In supervised learning, we know the right answer beforehand when we train our model, and in reinforcement learning, we define a measure of reward for particular actions by the agent. In unsupervised learning, however, we are dealing with unlabeled data or data of unknown structure. Using unsupervised learning techniques, we are able to explore the structure of our data to extract meaningful information without the guidance of a known outcome variable or reward function. Finding subgroups with clustering Clustering is an exploratory data analysis technique that allows us to organize a pile of information into meaningful subgroups (clusters) without having any prior knowledge of their group memberships. Each cluster that arises during the analysis defines a group of objects that share a certain degree of similarity but are more dissimilar to objects in other clusters, which is why clustering is also sometimes called unsupervised classification. Clustering is a great technique for structuring information and deriving meaningful relationships from data. For example, it allows marketers to discover customer groups based on their interests, in order to develop distinct marketing programs. The following figure illustrates how clustering can be applied to organizing unlabeled data into three distinct groups based on the similarity of their features and :
- 53. Dimensionality reduction for data compression Another subfield of unsupervised learning is dimensionality reduction. Often we are working with data of high dimensionality—each observation comes with a high number of measurements—that can present a challenge for limited storage space and the computational performance of machine learning algorithms. Unsupervised dimensionality reduction is a commonly used approach in feature preprocessing to remove noise from data, which can also degrade the predictive performance of
- 54. certain algorithms, and compress the data onto a smaller dimensional subspace while retaining most of the relevant information. Sometimes, dimensionality reduction can also be useful for visualizing data, for example, a high-dimensional feature set can be projected onto one-, two-, or three- dimensional feature spaces in order to visualize it via 3D or 2D scatterplots or histograms. The following figure shows an example where nonlinear dimensionality reduction was applied to compress a 3D Swiss Roll onto a new 2D feature subspace:
- 55. Exploring the Variety of Random Documents with Different Content
- 56. "Oh, you can laugh," said Peter, indignantly; "but if I was in love with a girl, I would teach her some better words than about the weather, and how do you do!" "I have done so," replied Jack, quietly; "but those words are for private use." At this moment Dolores, laughing behind her fan, was speaking to Doña Serafina, who thereupon advanced towards Peter. "I can speak to the Americano," she announced to the company; then, fixing Peter with her eye, said, with a tremendous effort, "Darling!" "Oh!" said the modest Peter, taken aback, "she said, 'darling'!" "Darling!" repeated Serafina, who was evidently quite ignorant of the meaning. "That's one of the words for private use, eh, Jack?" laughed Philip, quite exhausted with merriment. "A very good word. I must teach it to Doña Eulalia." "It's too bad of you, Doña Dolores," said Jack, reproachfully; whereat Dolores laughed again at the success of her jest. "Did the Señor have good sport with Cocom," asked Don Miguel, somewhat bewildered at all this laughter, the cause of which, ignorant as he was of English, he could not understand. "Did you have a good time, Peter," translated Tim, fluently, "with the beetles." "Oh, splendid! tell him splendid. I captured some Papilionidae! and a beautiful little glow-worm. One of the Elateridae species, and——" "I can't translate all that jargon, you fat little humming-bird! He had good sport, Señor," he added, suddenly turning to Don Miguel.
- 57. "Bueno!" replied the Spaniard, gravely, "it is well." It was no use trying to carry on a common conversation, as the party invariably split up into pairs. Dolores and Eulalia were already chatting confidentially to their admirers. Doña Serafina began to make more signs to Peter, with the further addition of a parrot-cry of "Darling," and Tim found himself once more alone with Don Miguel. "I have written out my interview with the President," he said slowly; "and it goes to England to-morrow. Would you like to see it first, Señor?" "If it so pleases you, Señor Correspoñsal." "Good! then I shall bring it with me to-morrow morning. Has that steamer gone to Acauhtzin yet?" "This afternoon it departed, Señor. It will return in two days with the fleet." "I hope so, Don Miguel, but I am not very certain," replied Tim, significantly. "His Excellency Gomez does not seem very sure of the fleet's fidelity either." "There are many rumours in Tlatonac," said Maraquando, impatiently. "All lies spread by the Opposidores—by Xuarez and his gang. I fear the people are becoming alarmed. The army, too, talk of war. Therefore, to set all these matters at rest, to-morrow evening his Excellency the President will address the Tlatonacians at the alameda." "Why at the alameda?" "Because most of them will be assembled there at the twilight hour, Señor. It is to be a public speech to inspire our people with confidence in the Government, else would the meeting be held in the great hall of the Palacio Nacional."
- 58. "I would like to hear Don Franciso Gomez speak, so I and my friends will be at the alameda." "You will come with me, Señor Correspoñsal," said Miguel, politely; "my daughter, niece, and sister are also coming." "The more the merrier! It will be quite a party, Señor." "It is a serious position we are in," said Maraquando, gravely; "and I trust the word of his Excellency will show the Tlatonacians that there is nothing to be feared from Don Hypolito." At this moment Doña Serafina, who had swooped down on her charges, appeared to say good night. Both Dolores and Eulalia were unwilling to retire so early, but their aunt was adamant, and they knew that nothing could change her resolution, particularly as she had grown weary of fraternising with Peter. "Bueno noche tenga, Vm," said Doña Serafina, politely, and her salutation was echoed by the young ladies in her wake. "Con dios va usted, Señora," replied Tim, kissing the old lady's extended hand, after which they withdrew. Dolores managed to flash a tender glance at Jack as they descended into the patio, and Philip, leaning over the balustrade of the azotea caught a significant wave of Eulalia's fan, which meant a good deal. Cassim knew all those minute but eloquent signs of love. Shortly afterwards they also took their leave after refusing Maraquando's hospitable offer of pulque. "No, sir," said Tim, as they went off to their own mansion; "not while there is good whisky to be had." "But pulque isn't bad," protested Jack, more for the sake of saying something than because he thought so. "Well, drink it yourself, Jack, and leave us the crather!"
- 59. "Talking about 'crathers,'" said Philip, mimicking Tim's brogue, "what do you think of Doña Serafina, Peter?" "A nice old lady, but not beautiful. I would rather be with Doña Eulalia." "Would you, indeed?" retorted Cassim, indignantly. "As if she would understand those idiotic signs you make." "They are quite intelligible to——" "Be quiet, boys!" said Tim, as they stopped at the door of Jack's house, "you'll get plenty of fighting without starting it now. There's going to be a Home Rule meeting to-morrow." "Where, Tim?" "In the alameda, no less. His Excellency the Lord Lieutenant is to speak to the crowd." "He'll tell a lot of lies, I expect," said Jack, sagely. "Well, he can say what he jolly well pleases. I'll lay any odds that before the week's out war will be proclaimed." He was a truer prophet than he thought.
- 60. CHAPTER VIII. VIVA EL REPUBLICA. No king have we with golden crown, To tread the sovereign people down; All men are equal in our sight— The ruler ranks but with the clown. Our symbol is the opal bright, Which darts its rays of rainbow light, All men are equal in our sight— Prophetic of all coming things, Of blessing, war, disaster, blight. Red glow abroad the opal flings, To us the curse of war it brings; All men are equal in our sight— And evil days there soon shall be, Beneath the war-god's dreaded wings. Yet knowing what we soon shall see, We'll boldly face this misery, All men are equal in our sight— And fight, though dark our fortunes frown, For life, and home, and liberty. Padre Ignatius always said that his flock were true and devout Catholics, who believed in what they ought to believe. Strictly speaking, the flock of Padre Ignatius was limited to the congregation of a little adobe church on the outskirts of the town, but his large heart included the whole population of Tlatonac in that ecclesiastical appellation. Everyone knew the Padre and everyone loved him,
- 61. Jesuit though he was. For fifty years had he laboured in the vineyard of Tlatonac, but when his fellow-labourers were banished, the Government had not the heart to bid him go. So he stayed on, the only representative of his order in all Cholacaca, and prayed and preached and did charitable works, as had been his custom these many years past. With his thin, worn face, rusty cassock, slouch hat, and kindly smile, Padre Ignatius, wonderfully straight considering his seventy years, attended to the spiritual wants of his people, and said they were devout Catholics. He always over-estimated human nature, did the Padre. So far as the Padre saw, this might have been the case, and nobody having the heart to undeceive him, he grew to believe that these half-civilised savages were Christians to the bone; but there was no doubt that nine out of every ten in his flock were very black sheep indeed. They would kneel before the gaudy shrine of the adobe chapel, and say an Ave for every bead of the rosary, but at one time or another every worshipper was missing, each in his or her turn. They had been to the forest for this thing, for that thing; they had been working on the railway fifty miles inland, or fishing some distance up the coast. Such were the excuses they gave, and Padre Ignatius, simple-hearted soul, believed them, never dreaming that they had been assisting in the worship of the Chalchuih Tlatonac in the hidden temple of Huitzilopochtli. The belief in the devil stone was universal throughout Cholacaca. Not only did the immediate flock of Padre Ignatius revere it as a symbol of the war-god, but every person in the Republic who had Indian blood in his or her veins firmly believed that the shining precious stone exercised a power over the lives and fortunes of all. Nor was such veneration to be wondered at, considering how closely the history of the great gem was interwoven with that of the country. The shrine of the opal had stood where now arose the cathedral; the Indian appellation of the jewel had given its name to the town; and the picture representation of the gem itself was
- 62. displayed on the yellow standard of the Republic. Hardly any event since the foundation of the city could be mentioned with which the harlequin opal was not connected in some way. It was still adored in the forest temple by thousands of worshippers, and, unknown as it was to the padres, there were few peons, leperos, or mestizos who had not seen the gem flash on the altar of the god. Cholacacans of pure Spanish blood, alone refrained from actual worship of the devil stone, and even these were more or less tinctured with the superstition. It is impossible to escape the influence of an all- prevailing idea, particularly in a country not quite veneered by civilisation. On this special evening, when President Gomez was to address the populace, and assure them that there would be no war, the alameda presented an unusually lively appearance. It had been duly notified that His Excellency would make a speech on the forthcoming crisis, hence the alameda was crowded with people anxious to hear the official opinion of the affair. The worst of it was, had Gomez but known it, that the public mind was already made up. There was to be war, and that speedily, for a rumour had gone forth from the sanctuary of the opal that the gem was burning redly as a beacon fire. Everyone believed that this foreboded war, and Gomez, hoping to assure the Tlatonacians of peace, might as well have held his tongue. They would not believe him as the opal stone had prophesied a contrary opinion. But beyond an idle whisper or so, Gomez did not know this thing, therefore he came to the alameda and spoke encouragingly to the people. From all quarters of the town came the inhabitants to the alameda, and the vast promenade presented a singularly gay appearance. The national costumes of Spanish America were wonderfully picturesque, and what with the background of green trees, sparkling fountains, brilliant flower-beds, and, over all, the violet tints of the twilight, Philip found the scene sufficiently charming. He was walking beside Jack, in default of Eulalia, who, in company with Dolores, marched
- 63. demurely beside Doña Serafina. This was a public place, the eyes of Tlatonac gossips were sharp, their tongues were bitter, so it behoved discreet young ladies, as these, to keep their admirers at a distance. In the patio it was quite different. Tim had gone off with Don Miguel, to attach himself to the personal staff of the President, and take shorthand notes of the speech. It had been the intention of Peter to follow his Irish friend, but, unfortunately, he lost him in the crowd, and therefore returned to the side of Philip, who caught sight of him at once. "Where's Tim?" asked the baronet, quickly; "gone off with Don Miguel?" "Yes; to the Palacio Nacional." "I thought you were going?" "I lost sight of them." "An excuse, Peter," interposed Jack, with a twinkle in his eye. "You remained behind to look at the Señoritas." Peter indignantly repudiated the idea. "His heart is true to his Poll," said Philip, soothingly; "thereby meaning Doña Serafina. Darling!" Philip mimicked the old lady's pronunciation of the word, and Jack laughed; not so Peter. "How you do go on about Doña Serafina?" he said fretfully. "After all, she is not so very ugly, though she may not have the thirty points of perfection." "Eh, Peter, I didn't know you were learned in such gallantries; and what are the thirty points of perfection?"
- 64. The doctor was about to reply, when Cocom, wrapped in his zarape, passed slowly by, and took off his sombrero to the party. "A dios, Señores," said Cocom, gravely. "Our Indian friend," remarked Jack, with a smile. "Ven aca Cocom! Have you come to hear the assurance of peace." "There will be no peace, Señor Juan. I am old—very old, and I can see into the future. It is war I see—the war of Acauhtzin." "Ah! Is that your own prophecy or that of the Chalchuih Tlatonac." "I know nothing of the Chalchuih Tlatonac, Don Juan," replied Cocom, who always assumed the role of a devout Catholic; "but I hear many things. Ah, yes, I hear that the Chalchuih Tlatonac is glowing as a red star." "And that means war!" "It means war, Señor, and war there will be. The Chalchuih Tlatonac never deceives. Con dios va usted Señor." "Humph!" said Jack, thoughtfully, as Cocom walked slowly away; "so that is the temper of the people, is it? The opal says war. In that case it is no use Gomez saying peace, for they will not believe him." During this conversation with the Indian, Philip had gone on with Peter, so as to keep the ladies in sight. Jack pushed his way through the crowd and found them seated near the bandstand, from whence the President was to deliver his speech. As yet, His Excellency had not arrived, and the band were playing music of a lively description, principally national airs, as Gomez wished to arouse the patriotism of the Tlatonacians. The throng of people round the bandstand was increasing every moment. It was composed of all sorts and conditions of men and women, from delicate señoritas, draped in lace mantillas, to brown-
- 65. faced Indian women, with fat babies on their backs; gay young hidalgos, in silver-buttoned buckskin breeches, white ruffled shirts, and short jackets, and smart military men in the picturesque green uniform of the Republic. All the men had cigarettes, all the women fans, and there was an incessant chatter of voices as both sexes engaged in animated conversation on the burning subject of the hour. Here and there moved the neveros with their stock of ice- creams, grateful to thirsty people on that sultry night, the serenos keeping order among the Indians with their short staves, and many water-carriers with their leather clothes and crocks. Above the murmur of conversation arose the cries of these perambulating traders. "Tortillas de cuajuda," "Bocadillo de Coco," and all the thousand and one calls announcing the quality of their goods. Many of the ladies were driving in carriages, and beside them rode caballeros, mounted on spirited horses, exchanging glances with those whom they loved. The air of the alameda was full of intrigue and subtle understandings. The wave of a fan, the glance of a dark eye, the dropping of a handkerchief, the removal of a sombrero, all the mute signs which pass between lovers who dare not speak, and everywhere the jealous watching of husbands, the keen eyes of vigilant duennas. "It is very like the Puerta del Sol in Madrid," said Philip in a low whisper, as he stood beside Eulalia; "the same crowd, the same brilliance, the same hot night and tropic sky. Upon my word, there is but little difference between the Old Spain and the New." "Ah!" sighed Eulalia, adjusting her mantilla; "how delightful it must be in Madrid!" "Not more delightful than here, Señorita. At least, I think so—now." Eulalia cast an anxious glance at her duenna, and made a covert sign behind her fan for him to be silent.
- 66. "Speak to my aunt, Don Felipe!" "I would rather speak to you," hinted Philip, with a grimace. "Can young ladies speak to whom they please in your country?" "I should rather think so. In my country the ladies are quite as independent as the gentlemen, if not more so." "Oh, oh! El viento que corre es algo fresquito." "The wind which blows is a little fresh," translated Philip to himself; "I suppose that is the Spanish for 'I don't believe you.' But it is true, Señorita," he added quickly, in her own tongue; "you will see it for yourself some day." "I fear not. There is no chance of my leaving Tlatonac." "Who knows?" replied Philip, with a meaning glance. Eulalia cast down her eyes in pretty confusion. Decidedly this Americano was delightful, and remarkably handsome; but then he said such dreadful things. If Doña Serafina heard them—Eulalia turned cold at the idea of what that vigorous lady would say. "Bueno!" chattered the duenna at this moment; "they are playing the 'Fandango of the Opal!'" This was a local piece of music much in favour with the Tlatonacians, and was supposed to represent the Indian sacred dance before the shrine of the gem. As the first note struck their ears, the crowd applauded loudly; for it was, so to speak, the National Anthem of Cholacaca. Before the band-stand was a clear space of ground, and, inspired by the music, two Mestizos, man and woman, sprang into the open, and began to dance the fandango. The onlookers were delighted, and applauded vehemently.
- 67. They were both handsome young people, dressed in the national costume, the girl looking especially picturesque with her amber- coloured short skirt, her gracefully draped mantilla, and enormous black fan. The young fellow had castanets, which clicked sharply to the rhythm of the music, as they whirled round one another like Bacchantes. The adoration of the opal, the reading of the omen, the foretelling of successful love, all were represented marvellously in wonderful pantomime. Then the dancers flung themselves wildly about, with waving arms and mad gestures, wrought up to a frenzy by the inspiriting music. Indeed, the audience caught the contagion, and began to sing the words of the opal song— Breathe not a word while the future divining, True speaks the stone as the star seers above, Green as the ocean the opal is shining, Green is prophetic of hope and of love. Kneel at the shrine while the future discerning, See how the crimson ray strengthens and glows; Red as the sunset the opal is burning, Red is prophetic of death to our foes. At this moment, the carriage of the President, escorted by a troop of cavalry, arrived at the band-stand. The soldiers, in light green uniforms, with high buff boots, scarlet waistbands, and brown sombreros, looked particularly picturesque, but the short figure of the President, arrayed in plain evening dress, appeared rather out of place amid all this military finery. The only token of his Excellency's rank was a broad yellow silk ribbon, embroidered with the opal, which he wore across his breast. Miguel Maraquando and Tim were in the carriage with the President, and the Irishman recognised his friends with a wave of his hand. "Tim is in high society," said Peter, with a grin. "We will have to call him Don Tim after this."
- 68. "We'll call you 'Donkey' after this, if you make such idiotic remarks," replied Jack, severely. "Be quiet, doctor, and listen to the speechifying." The President was received with acclamation by those in the alameda, which showed that Tlatonac was well disposed towards the established Government. It is true that one or two friends of Xuarez attempted to get up a counter demonstration; but the moment they began hissing and shouting for Don Hypolito, the serenos pounced down and marched them off in disgrace. His Excellency, attended by Don Miguel and several other members of the Junta, came forward, hat in hand, to the front of the band-stand, and, after the musicians had stopped playing the "Fandango," began to speak. Gomez was a fat little man, of no very striking looks; but when he commenced speaking, his face glowed with enthusiasm, and his rich, powerful voice reached everyone clearly. The man was a born orator, and, as the noble tongue of Castille rolled sonorously from his mouth, he held his mixed audience spell-bound. The listeners did not believe in his assurances, but they were fascinated by his oratory. It was a sight not easily forgotten. The warm twilight, the brilliant equatorial vegetation, the equally brilliant and picturesque crowd, swaying restlessly to and fro; far beyond, through a gap in the trees, in the violet atmosphere, the snow-clad summit of Xicotencatl, the largest of Cholacacan volcanoes, and everywhere the vague languor of the tropics. Gomez, a black figure against the glittering background of uniforms, spoke long and eloquently. He assured them that there would be no war. Don Hypolito Xuarez had no supporters; the Junta was about to banish him from the country; the prosperity of Cholacaca was fully assured; it was to be a great nation; he said many other pleasant things, which flattered, but deceived not the Tlatonacians. "Yes, señores," thundered the President, smiting his breast, "I, who stand here—even, I, Francisco Gomez, the representative of the
- 69. Republic of Cholacaca—tell you that our land still rests, and shall rest under the olive tree of Peace. We banish Don Hypolito Xuarez—we banish all traitors who would crush the sovereign people. The rulers of Cholacaca, elected by the nation, are strong and wise. They have foreseen this tempest, and by them it will be averted. Believe not, my fellow-countrymen, the lying rumours of the streets! I tell you the future is fair. There will be no war!" At this moment he paused to wipe his brow, and then, as if to give the lie to his assertion, in the dead silence which followed, was heard the distant boom of a cannon. Astonished at the unfamiliar sound, the Tlatonacians looked at one another in horror. Gomez paused, handkerchief in hand, with a look of wonderment on his face. No one spoke, no one moved, it was as though the whole of that assemblage had been stricken into stone by some powerful spell. In the distance sounded a second boom, dull and menacing, there was a faint roar far away as of many voices. It came nearer and nearer, and those in the alameda began to add their voices to the din. Was the city being shelled by the revolting war-ships; had Don Hypolito surprised the inland walls with an army of Indians. Terror was on the faces of all—the clamour in the distance came nearer, waxed louder. A cloud of dust at the bend of the avenue, and down the central walk, spurring his horse to its full speed, dashed a dishevelled rider. The horse stopped dead in front of the band-stand, scattering the people hither and thither like wind-driven chaff; a young man in naval uniform flung himself to the ground, and ran up to the astonished President. "Your Excellency, the fleet have revolted to Don Hypolito Xuarez! He is entrenched in the rebel town of Acauhtzin. I alone have escaped, and bring you news that he has proclaimed war against the Republic!" A roar of rage went up to the sky.
- 70. "The opal! The prophecy of the Chalchuih Tlatonac!" cried the multitude. "Viva el Republica! Death to the traitor Xuarez!" Gomez was listening to the messenger, who talked volubly. Then the President turned towards the people, and, by a gesture of his hand, enjoined silence. The roar at once sank to a low murmur. "What Don Rafael Maraquando says is true," said Gomez, loudly. "This traitor, Xuarez, has seduced the allegiance of the fleet—of Acauhtzin. The Republic must prepare for war——" He could speak no further, for his voice was drowned in the savage roaring of the multitude. Everyone seemed to have gone mad. The crowd of people heaved round the band-stand like a stormy sea. A thousand voices cursed the traitor Xuarez, lauded the Republic, and repeated the prophecy of the harlequin opal. The whole throng was demoralised by the news. "War! War! To Acauhtzin!" roared the throats of the mob. "Death to Xuarez! Viva el Republica! Viva libertad!" Gomez made a sign to the band, which at once burst out into the Fandango of the Opal. A thousand voices began singing the words, a thousand people began to dance wildly. Ladies waved their handkerchiefs, men shouted and embraced one another, and amid the roar of the mob and the blare of the band, Don Francisco Gomez entered his carriage and drove away escorted by the cavalry. Tim fought his way through the crowd down from the band-stand, and reached the Maraquando part, where he found the three ladies, more excited than frightened, standing for safety in the circle formed by the five men. Two of the men were embracing—Don Miguel and his son. "It's a great day for Cholacaca," cried Tim, excitedly. "I wouldn't have missed it for a fortune. Viva el Republica! Ah, Peter, my boy, this is better than the butterflies."
- 71. "My son! my son, how did you escape?" said Don Miguel, throwing his arms round Rafael's neck. "I will tell you all at the house, my father," replied the young man. "Let us go now with the ladies to our home. Señores," he added, turning to the Englishmen, "you will come, too, I trust?" It was no easy matter to get through the crowd, but ultimately the five men managed to push a path to a caleza for the ladies, placed them therein, and when it drove off, hastened themselves to the Casa Maraquando. The whole city was in commotion. In the Plaza de los Hombres Ilustres a crowd had collected to salute the great yellow standard of the Republic, which streamed from the tower of the Palacio Nacional. "The opal! the opal! The prophecy of the Tlatonac Chalchuih," roared the crowd, stamping and yelling. "They will believe in that stone more than ever now," whispered Philip to Jack, as they entered the zaguan of Maraquando's house. "What do you think of it, Jack?" "Oh, it's easy to prophesy when you know," retorted Jack, scornfully. "Of course, Xuarez told the Indians he was going to revolt, and the priests of the temple have used the information to advertise the stone. Of course it grew red, and prophesied war under the circumstances. That is all the magic about the affair." In the patio the ladies were waiting for them in a state of great excitement, and welcomed Don Rafael as one returned from the dead. He embraced his sister, cousin, and aunt; which privilege was rather envied by the four friends, as regards the first two, and was then formally introduced to the Englishmen. His eye flashed as he saluted Tim and heard his vocation.
- 72. "You will have plenty to write about, Señor Correspoñsal," he said, fiercely; "there will be a war, and a bitter war too. I have barely escaped with my life from Acauhtzin." "Tell me all about it, Señor," said Tim, taking out his pocket-book; "and the news will go off to London to-night." "A thousand regrets, Señor Correspoñsal, that I cannot give you a detailed account at present, but I am worn out. I have not slept for days!" "Pobrecito," cried the ladies, in a commiserating tone. "I will, at all events, tell you shortly," resumed Rafael, without taking any notice of the interruption. "I commanded The Pizarro, and went up to Acauhtzin to arrest Xuarez, according to the order of the Government. As he refused to surrender, and as the town had declared in his favour, I thought we would have to bombard it. But think, Señores, think. When I came back to my ship, I was arrested by my own crew, by my own officers. Seduced by the oily tongue of Xuarez, they had revolted. In vain I implored! I entreated! I threatened! I commanded! They refused to obey any other than the traitor Xuarez. The other ships behaved in the same way. All the officers who, like myself, were known to be true to the Government, were arrested and thrown into prison, I among the number." "Ay de mi," cried Serafina, in tears, "what an indignity!" Don Rafael was choking with rage, and forgot his manners. "Carambo!" he swore roundly, "behold me, gentlemen. Look at my uniform! Thus was it insulted by the rebels of Acauhtzin, whose houses, I hope, with the blessing of God, to burn over their heads. I swear it!" He wrenched a crucifix from his breast, and kissed it passionately. It was a striking scene: the dim light, the worn-out young fellow in the
- 73. ragged uniform, and his figure black against the lights in the patio, passionately kissing the symbol of his faith. "How did you escape, my son," said Maraquando, whose eyes were flashing with hatred and wrath. "There was a man—one of my sailors, to whom I had shown favour —he was made one of the prison guards, and, out of kindness, assisted me to escape; but he was too fearful to help any of the others. In the darkness of night, I cut through my prison bars with a file he had given me. I climbed down the wall by a rope, and, when on the ground, found him, waiting me. He hurried me down to the water's edge, and placed me in a boat with food for a few days. I rowed out in the darkness, past the ships, and luckily managed to escape their vigilance. Then I hoisted the sail, and, as there was a fair wind, by dawn I was far down the coast. I need not tell you all my adventures, how I suffered, how I starved, how I thirsted— cursed, cursed, Xuarez!" He stamped with rage up and down the patio while the ladies exclaimed indignantly at the treatment to which he had been subjected. Then he resumed his story hurriedly, evidently wishing to get it over— "This morning, I fortunately fell in with the steamer sent up by the Government, which picked me up. I told the captain all, and he returned at once with the news, arriving at Tlatonac some time ago. I ordered him to fire those guns announcing my arrival, and hearing his Excellency was addressing a meeting at the alameda, jumped on a horse and rode here. The rest you know." "Good!" said Tim, who had been busily taking notes, "I'm off to the telegraph-office, Señores. Good night." Tim went off, and the others were not long in following his example. Overcome by fatigue, Don Rafael had fallen, half-fainting, in a chair,
- 74. and the ladies were attending to him; so, seeing they were rather in the way, Jack and his friends, saying good night, left the house. The city was still heaving with excitement. Bands of men went past dancing and singing. The bells clashed loudly from every tower, and every now and then a rocket scattered crimson fire in the sky. War was proclaimed! the whole of Tlatonac was in a state of frenzy, and there would be no sleep for anyone that night. "We're in for it now," said Jack, jubilantly, "hear the war-song!" A band of young men with torches tramped steadily towards the Square, singing the National Anthem of Tlatonac. Philip caught the last two lines roared triumphantly as they disappeared in the distance: Red as the sunset the opal is burning, Red is prophetic of death to our foes.
- 75. CHAPTER IX. THE CALL TO ARMS. Ta ra ra! Ta ra ra! The trumpets are blowing, And thrice hath their brazen notes pealed. To battle! to battle the soldiers are going, To conquer or die on the field. On, soldiers! brave soldiers, who venture your lives You fight for your country and sweethearts and wives. Ta ra ra! Ta ra ra! The drums roll like thunder, And women's tears falling like rain. For lovers! for lovers are parted asunder, Till victory crowns the campaign. On, soldiers! brave soldiers go forth to the fray, And close with the foe in their battle array. Ta ra ra! Ta ra ra! The banners are flying, And horses prance proudly along, For women! for women are bitterly crying, As passes the red-coated throng. On, soldiers! brave soldiers! soon homeward you'll ride, Encircled with bay leaves and greeted with pride. At this eventful moment of its history, Cholacaca woke from its slumber of years, as did the Sleeping Beauty from her century sleep. No more the lethargic life, the indolent enjoyments, the languorous dreamings in an enchanted city. A sharp breath of war from the north swept away the sedative atmosphere; the thunder of the
- 76. cannon roused Tlatonac to unexampled excitement. Rebellion and preparation for invasion at Acauhtzin, indignation and preparation for defence, for punishment in the capital of the Republic. In these days of alarm and danger, the city resembled one vast camp, and the descendants of the Conquistadores, the posterity of the Mayas, proved themselves to be not unworthy of their glorious traditions, both Spanish and Indian. It was a turning-point in the history of the Republic. The two persons most desirous for the speedy commencement of this fratricidal war were Tim and Don Rafael: the former as he wished information for his journal, the latter because he was burning to revenge the insults and indignities to which he had been subjected by the rebels at Acauhtzin. Jack was rather dismayed at the near prospect of hostilities, fearing lest harm should result therefrom to Dolores at the hands of Don Hypolito, or those of the Forest Indians. For their part, Philip and Peter assumed a neutral position, the one from indolence, the other because he was entomologically engaged. What was the hunting of men compared with the hunting of butterflies, the capture of rebels with the capture of rare beetles? No, Peter preferred science to war. The loss of the fleet was a great blow to the strength of the Government, as it, comparatively speaking, placed the capital at the mercy of the rebel, Xuarez. Communication between the two places was only possible by water, owing to the roughness and savagery of the interior, so the Government were unable to march their troops to Acauhtzin, and nip the rebellion in the bud. On the other hand, as soon as Xuarez had completed his plans, he would doubtless come south with his ships and bombard Tlatonac from the sea. Most of the city being built on the hill, topped by the vast fabric of the cathedral, offered considerable advantages to the besiegers, and as their vessels would keep well out of the range of the forts, it would be difficult to silence their guns.
- 77. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com