Time series analysis : Refresher and Innovations
1 like306 views
This document provides an overview of a presentation on time series analysis using the QuSandbox platform. The presentation was given by Sri Krishnamurthy, founder and CEO of QuantUniversity, at a QuantUniversity meetup in Boston on November 29, 2018. It covered topics including machine learning techniques for time series analysis, case studies analyzing temperature and swap rate data, and a demonstration of modeling time series data with neural networks.
Time series analysis : Refresher and Innovations
- 1. Time Series Analysis with the QuSandbox 2018 Copyright QuantUniversity LLC. Presented By: Sri Krishnamurthy, CFA, CAP [email protected] www.quantuniversity.com 11/29/2018 QuantUniversity Meetup Boston
- 2. 2 About us: • Data Science, Quant Finance and Model Governance Advisory • Technologies using MATLAB, Python and R • Programs ▫ Analytics Certificate Program ▫ Fintech programs • Platform
- 4. • Founder of QuantUniversity LLC. and www.analyticscertificate.com • Advisory and Consultancy for Financial Analytics • Prior Experience at MathWorks, Citigroup and Endeca and 25+ financial services and energy customers. • Regular Columnist for the Wilmott Magazine • Author of forthcoming book “Financial Modeling: A case study approach” published by Wiley • Charted Financial Analyst and Certified Analytics Professional • Teaches Analytics in the Babson College MBA program and at Northeastern University, Boston Sri Krishnamurthy Founder and CEO 4
- 5. 5 • 2-day Advances in Time series Workshop ▫ December/Jan - TBD ▫ Stay tuned for details at www.analyticscertificate.com • 2-day Machine Learning in Finance Workshop ▫ Jan 23rd & 24th ▫ Register at www.analyticscertificate.com/MachineLearning Upcoming events
- 6. 6 Slides will be available at: https://researchhub.qusandbox.com
- 7. 7 You too can do data science!
- 8. 8 • Brief intro to Machine Learning • Time Series Analysis – Example • Time Series Analysis – Case study • Demo Agenda
- 9. 9
- 10. Dataset, variable and Observations Dataset: A rectangular array with Rows as observations and columns as variables Variable: A characteristic of members of a population ( Age, State etc.) Observation: List of Variable values for a member of the population
- 11. Variables — A variable could be: —Categorical — Yes/No flags — AAA,BB ratings for bonds —Numerical — 35 mpg — $170K salary
- 12. Datasets • Longitudinal ▫ Observations are dependent ▫ Temporal-continuity is required • Cross-sectional ▫ Observations are independent
- 14. 14
- 15. 15 • Given a dataset, build a model that captures the similarities in different observations and assigns them to different buckets. • Given a set of variables, predict the value of another variable in a given data set ▫ Predict Salaries given work experience, education etc. ▫ Predict whether a loan would be approved given fico score, current loans, employment status etc. Predictive Analytics : Cross sectional datasets
- 16. 16 • Supervised Algorithms ▫ Given a set of variables !", predict the value of another variable # in a given data set such that ▫ If y is numeric => Prediction ▫ If y is categorical => Classification Machine Learning x1,x2,x3… Model F(X) y
- 17. 17 • Given a time series dataset, build a model that can be used to forecast values in the future Predictive Analytics : Time series datasets
- 18. 18 • Goal is to extract and model the various components Time series datasets
- 19. 19 • https://youtu.be/nIKwXzGieK8?t=63 • Extracting components from time series datasets is like squeezing juice out of sugarcane. Sugarcane analogy
- 20. 20 • http://faculty.chicagobooth.edu/ruey.tsay/teaching/introTS/ch3case 2.pdf Case 1: Analysis of Temperature Anomalies
- 21. 21 Case 2 : Analyzing Swap rates Ref: Statistical Models and Methods for Financial Markets Tze Leung Lai and Haipeng Xing. New York, NY: Springer, 2008. ISBN: 978-0-387-77826-6.
- 23. Supervised learning Cross-sectional ▫ Observations are independent ▫ Given X1----Xi, predict Y ▫ CNNs
- 24. Supervised learning Sequential ▫ Sequentially ordered ▫ Given O1---OT, predict OT+1 1 Normal 2 Normal 3 Abnormal 4 Normal 5 Abnormal
- 25. 25 • Given : X1,X2,X3----XN • Convert the Univariate time series dataset to a cross sectional Dataset Time series modeling in Keras using MLPs X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X Y X1 X2 X2 X3 X3 X4 X4 X5 X5 X6 X6 X7 X7 X8 X8 X9 X9 X10 X10 X11 X11 X12 X12 X13 X13 X14 X14 X15
- 26. 26 • Monthly data • Computational Intelligence in Forecasting • Source: http://irafm.osu.cz/cif/main.php?c=Static&page=download Sample data 0 200 400 600 800 1000 1200 1400 1600 1800 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100 103 106
- 27. 27 • Use 72 for training and 36 for testing • Lookback 1, 10 • Longer the lookback, larger the network Multi-Layer Perceptron Size 8 Size 1
- 28. 28 Demo Train Score: 1972.20 MSE (44.41 RMSE) Test Score: 3001.77 MSE (54.79 RMSE) Train Score: 2631.49 MSE (51.30 RMSE) Test Score: 4166.64 MSE (64.55 RMSE) Lookback = 1 Lookback = 10
- 29. 29 • Has 3 types of parameters ▫ W – Hidden weights ▫ U – Hidden to Hidden weights ▫ V – Hidden to Label weights • All W,U,V are shared Recurrent Neural Networks1 1. http://ai.stanford.edu/~quocle/tutorial2.pdf
- 30. 30 Where can Recurrent Neural Networks be used?1 1. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ 1. Vanilla mode of processing without RNN, from fixed-sized input to fixed-sized output (e.g. image classification). 2. Sequence output (e.g. image captioning takes an image and outputs a sentence of words). 3. Sequence input (e.g. sentiment analysis where a given sentence is classified as expressing positive or negative sentiment). 4. Sequence input and sequence output (e.g. Machine Translation: an RNN reads a sentence in English and then outputs a sentence in French). 5. Synced sequence input and output (e.g. video classification where we wish to label each frame of the video).
- 31. 31 Recurrent Neural Networks • A recurrent neural network can be thought of as multiple copies of the same network, each passing a message to a successor. 1 • Backpropagation(computing gradient wrt all parameters of the network) which is process used to propagate errors and weights needs to be modified for RNNs due to the existence of loops http://colah.github.io/posts/2015-08-Understanding-LSTMs/
- 32. 32 • BPTT begins by unfolding a recurrent neural network through time as shown in the figure. • Training then proceeds in a manner similar to training a feed- forward neural network with backpropagation, except that the training patterns are visited in sequential order. Back Propagation through time (BPTT)1 1. https://en.wikipedia.org/wiki/Backpropagation_through_time
- 33. 33 • Backpropagation through time (BPTT) for RNNs is difficult due to a problem known as vanishing/exploding gradient . i.e, the gradient becomes extremely small or large towards the first and end of the network. • This is addressed by LSTM RNNs. Instead of neurons, LSTMs use memory cells 1 Addressing the problem of Vanishing/Exploding gradient http://deeplearning.net/tutorial/lstm.html
- 34. 34 www.QuSandbox.com Model Analytics Studio QuResearchHub QuSandbox Prototype, Iterate and tune Standardize workflows Productionize and share
- 35. Sri Krishnamurthy, CFA, CAP Founder and Chief Data Scientist [email protected] srikrishnamurthy www.QuantUniversity.com www.analyticscertificate.com www.qusandbox.com Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not be distributed or used in any other publication without the prior written consent of QuantUniversity LLC. 35