Houston Energy Data Science Meet up_TIBCO Slides
Download as PPTX, PDF•3 likes•2,470 views
The document details Tibco's advanced analytics capabilities, emphasizing the importance of data science in enhancing business productivity and decision-making. It outlines the functionalities of Tibco's analytics stack, including real-time and predictive analytics, self-service dashboards, and data access methods, aimed at providing immediate and long-term value to organizations. Additionally, it highlights specific applications in the energy sector, such as production forecasting, maintenance optimization, and equipment management, underscoring the platform's role in driving innovation and operational efficiency.
![Free Scripts - GeoCluster [kmeans(x,y)]](https://image.slidesharecdn.com/houstondatasciencejuly2015distn-150824130220-lva1-app6891/85/Houston-Energy-Data-Science-Meet-up_TIBCO-Slides-31-320.jpg)
![Free Scripts - Contours [contourLines(x,y,z)]](https://image.slidesharecdn.com/houstondatasciencejuly2015distn-150824130220-lva1-app6891/85/Houston-Energy-Data-Science-Meet-up_TIBCO-Slides-32-320.jpg)
Houston Energy Data Science Meet up_TIBCO Slides
- 1. TIBCO Advanced Analytics Houston Energy Data Science Meetup Michael O’Connell Chief Data Scientist [email protected] @moc_tib August 2015
- 2. • Data Science Process • Data Analysis Pipeline • Understand – Anticipate – Act • Advanced Analytics • TIBCO’s R engine • GeoLocation Analytics • Real-Time Analytics • Remote Monitoring – the Digital Nervous System • Software & APIs • Wrap-Up / Questions Increase Productivity Grow Revenue Value Reduce Risk ROI TIBCO Analytics – Insight to Action © Copyright 2000-2015 TIBCO Software Inc.
- 3. “Data Science” Engineer/Marketeer “Address the business issue” Statistician “Build the best model” IT / Developer “Manage my infrastructure” Engineer/Marketeer: Knows the business problem but doesn’t know how to prepare data or build models. Statistician: Knows how to develop appropriate models to address business problems but is in short supply and can’t deploy IT or business systems IT / Developer: Knows databases, application provisioning and development tools but isn’t familiar with data meaning or analytical workflow purpose What is a Data Scientist © Copyright 2000-2015 TIBCO Software Inc.
- 4. Data Access & Prep Exploratory Data Analysis Features Visual Dashboard Model & Predict Deploy Champion Model Test & Learn Channel Social Loyalty Campaign Filter Map Merge Shape Propensity Affinity ImproveGuided -------- Deploy -------- In-LineExplore Data Aggregate Prepare DataBusiness Case Increase Productivity Grow Revenue Ensemble Forest Regression Additive Models Segment Visualize Pricing Promotion Challenger Models At Rest In Motion Value Theses Reduce Risk ROI Value Dashboard Updates Data a Insight a Action © Copyright 2000-2015 TIBCO Software Inc.
- 6. Custom GUI-driven data access via SDK Enterprise Data Access Siebel eBusiness Local data sources AccessExcel STDF Drag-and-drop MySQL SQL Server Oracle Information Services (join, transform, reusable, parameterized, dynamic query for in-memory use) Databases JDBC/ODBC Hadoop SFDC PostgreSQL Teradata Netezza Etc.XML RDBMS Flat Files Spread- sheets Web Services Oracle E-Business RDBMS RDBMS RDBMS SAP BWSAP R/3 D A T A F A B R I C Salesforce ODBC OLE DB SqlClient Direct connection Oracle TeradataAsterMS SSAS Teradata Direct Query (dynamically query and retrieve data for visualization and analysis) Databases MySQL Etc. OBIEE Netezza Hadoop © Copyright 2000-2015 TIBCO Software Inc.
- 7. Immediate Long-Term Competitive AdvantageValue to the Organization TIBCO is the only analytics platform that provides business value across the Analytics Spectrum Self-service Dashboards Event Processing Predictive and Prescriptive Analytics Measure Diagnose Predict Optimize Operationalize Automate Analytics Maturity Analytics Spectrum
- 8. Immediate Long-Term Competitive AdvantageValue to the Organization TIBCO is the only analytics platform that provides business value across the Analytics Spectrum Self-service Dashboards Measure Diagnose Predict Optimize Operationalize Automate Analytics Maturity Analytics Spectrum Predictive and Prescriptive Analytics Event Processing
- 9. Immediate Long-Term Competitive AdvantageValue to the Organization TIBCO is the only analytics platform that provides business value across the Analytics Spectrum Self-service Dashboards Predictive and Prescriptive Analytics Measure Diagnose Predict Optimize Operationalize Automate Analytics Maturity Analytics Spectrum Event Processing
- 10. © Copyright 2000-2015 TIBCO Software Inc. 10 Visual Analytics – Spotfire
- 11. Visual Analytics – Spotfire 3D rotate SurfacePolar Contour Network Funnel
- 12. Spotfire Extensions – d3 and JS © Copyright 2000-2015 TIBCO Software Inc. Sankey Venn ChordDonut Dials Gantt
- 13. Visual Analytics – Dashboards
- 14. Visual Analytics – Dashboards
- 15. Visual Analytics – Dashboards
- 16. Visual Analytics – Dashboards
- 17. Visual Analytics – Dashboards
- 21. Jaspersoft Pixel-Perfect Embedded Reports © Copyright 2000-2015 TIBCO Software Inc.
- 22. Analytic Workspaces & Analytic Fabric APIs Search,Sharingetc. Business Analysts Report Developers Analytic Workspaces Analytic Fabric Data Discovery Analytics Dashboards Reports © Copyright 2000-2015 TIBCO Software Inc.
- 23. Spotfire is Super Simple to Use US Homeless Analysis Step-by-Step YouTube Playlist • Dashboards • Predictive • GeoLocation © Copyright 2000-2015 TIBCO Software Inc.
- 24. Immediate Long-Term Competitive AdvantageValue to the Organization TIBCO is the only analytics platform that provides business value across the Analytics Spectrum Self-service Dashboards Measure Diagnose Predict Optimize Operationalize Automate Analytics Maturity Analytics Spectrum Predictive and Prescriptive Analytics Event Processing
- 25. Advanced Analytics Ecosystem © Copyright 2000-2015 TIBCO Software Inc.
- 26. TIBCO Enterprise Runtime for R (TERR) © Copyright 2000-2015 TIBCO Software Inc. • TIBCO has rewritten R as a Commercial Compute Engine • Latest statistics scripting engine: S a S-PLUS® a R a TERR • Runs R code including CRAN packages • Engine internals rebuilt from scratch at low-level • Redesigned data objects, memory management • High performance + Big Data • TERR is licensed from TIBCO • TERR Installs (free) with Spotfire Analyst / Desktop and other TIBCO products (CEP, Stats) • Spotfire Server can manage all TERR / R scripts, artifacts for reuse • Standalone Developer Edition: www.TIBCOmmunity.com • Supported by TIBCO
- 27. Model Fitting: 5 Million Rows Model Scoring: 20 Million Rows TERR 7X faster 84X TERR Performance © Copyright 2000-2015 TIBCO Software Inc.
- 28. Spotfire and TERR local TERR on server Spotfire-TERR – Local and Server • Build models on data using local TERR engine embedded in Spotfire • Build models on big data directly in TERR on server and display results in Spotfire • Run TERR as parallel sessions on Hadoop cluster, controlled and visualized in Spotfire Data Source TERR TSSS Spotfire Results ODBC JDBC SDC File Data Function Larger Data Modeling Spotfire Local TERR ODBC JDBC SDC File Data Data Source Both Spotfire and TERR can load data from any ODBC or JDBC compliant source or from Spotfire Data Connections (SDC) or Spotfire Information Links stored in the Spotfire library. © Copyright 2000-2015 TIBCO Software Inc.
- 29. © Copyright 2000-2015 TIBCO Software Inc. Simple Predictive Analytics – Forecasting & Modeling Contextual Analytics - Forecasting Contextual Analytics - Machine Learning
- 30. Extensible Predictive Analytics – Analysis Workflows Interactive Spotfire Analytics with R - Data Function - Robust Cluster Analysis - Any Analysis in R / CRAN Variables driving segments - Random Forest Revenue by product - Color by segment
- 31. Free Scripts - GeoCluster [kmeans(x,y)]
- 32. Free Scripts - Contours [contourLines(x,y,z)]
- 33. Spotfire-TERR : Data Types, Analyses Spotfire data functions support any type of data as input and output parameters to and from TERR. TERR data functions used for data prep, integration, predictive & prescriptive analytics, … TERR data functions can output content metadata to Spotfire • formatting of fields • handling of binary data including images and geospatial objects. Rows Columns Values Tables Metadata Blobs Geometries Images Spotfire TERR Data Function © Copyright 2000-2015 TIBCO Software Inc.
- 34. Trade Areas
- 35. Smart Routing
- 36. Smart Routing
- 37. Smart Routing
- 38. Production Forecasting Forecast Production – Set Expected Production for Wells• Resource Play • Repeatable distribution for EUR • Offset not reliable predictor • Continuous hydrocarbon system • Free hydrocarbon not held in place by hydrodynamics • Geologic Subset • Analogous Wells • Geology, completion, spacing, vintage • Analysis and Data • Production forecasting (EUR) • Probability of production • Proven (P90), Probable (P50), Possible (P10) • Cluster and Regression Analysis © Copyright 2000-2015 TIBCO Software Inc.
- 39. Proven, Probable and Possible Production• Resource Play • Repeatable distribution for EUR • Offset not reliable predictor • Continuous hydrocarbon system • Free hydrocarbon not held in place by hydrodynamics • Geologic Subset • Analogous Wells • Geology, completion, spacing, vintage • Analysis and Data • Production forecasting (EUR) • Probability of production • Proven (P90), Probable (P50), Possible (P10) • Cluster and Regression Analysis Probability: Proven & Probable Production © Copyright 2000-2015 TIBCO Software Inc.
- 40. Completions Optimization • Business Opportunities • Completions optimization by well • Production prediction for new wells • Identify factors driving production vs expected production e.g. operator • Analysis and Data • Subsurface (e.g. Spectra) • Location • Completions • Production • Value and Financial Impact • Optimal completions • Operations management • Asset valuation & “where to drill” Optimize Completions – Location, Subsurface © Copyright 2000-2015 TIBCO Software Inc.
- 41. 41 © Copyright 2000-2014 TIBCO Software Inc. • Business Opportunities • Maintenance optimization • Analysis and Data • Failure times and locations • Maintenance and failure costs • Root cause analysis • Value and Financial Impact • Visibility into maintenance expenses and root causes • Optimal maintenance scheduling Maintenance Optimization Equipment Reliability - Refining
- 42. Winner of 2014 Strata Cloudera Award For Best Advanced Analytics Application Big Data Analytics with Spotfire and TERR © Copyright 2000-2015 TIBCO Software Inc.
- 43. Big Data Analytics with TERR TERR on the nodes of Hadoop Cluster TERR in Action • Hadoop cluster compute • TIBCO Cloud Compute Grid • TIBCO Streambase • TIBCO Business Events • KNIME • Lavastorm • Rstudio • Teradata • TIBCO Statistics Services • TIBCO Spotfire © Copyright 2000-2015 TIBCO Software Inc.
- 44. © Copyright 2000-2015 TIBCO Software Inc. Predictive & Collaborative Analytics Library of Data Functions – everyone Shares • Analysts use functions – no code • Coders develop new functions – R Data Function Samples • Ship with Spotfire Server • Geospatial • Computations with polygons on a map • Computing optimal routes in logistics • Machine Learning • Fitting models and making predictions • Applications • Customers, Finance, Machines, … IT View - GovernanceUser View - Functions
- 45. Immediate Long-Term Competitive AdvantageValue to the Organization TIBCO is the only analytics platform that provides business value across the Analytics Spectrum Self-service Dashboards Predictive and Prescriptive Analytics Measure Diagnose Predict Optimize Operationalize Automate Analytics Maturity Analytics Spectrum Event Processing
- 46. BIG DATA AT REST FAST DATA IN MOTION Insight to Action © Copyright 2000-2015 TIBCO Software Inc.
- 47. Analyze And Act On “Critical Business Moments” Optimize pricing Check for fraud Make offer to customer Restock inventory Reroute transport Give customer service Proactively maintain machines © Copyright 2000-2015 TIBCO Software Inc.
- 48. Big Data – Analysis of production – Analysis of contracts and product inventory Fast Data – Location data from ships and trains, weather and tides – Manage product supply – Optimize fuel use Benefits – Optimize product contracts – Maximize product shipped – Minimize logistics cost Managing Supply Chain
- 50. Managing Industrial Equipment Big Data – Analysis of production – Failure analytics Fast Data – Real-time sensor data – Leading indicator for shutdowns – Drilling: kick detection – Flow monitoring Benefits – Reduced NPT: Big $$s – System reliability – Efficient drilling
- 51. Data Monitoring • Motor temperature • Motor vibration • Current • Intake pressure • Intake temperature Flow Electrical power cable Pump Intake Protector ESP motor Pump monitoring unit Pump Components Equipment Monitoring & Management Video: https://youtu.be/vIVepQRl5SY
- 52. • Business Opportunities • Pump health & performance surveillance • Condition-based maintenance • Analysis and Data • Effects of operating conditions on performance • Effects of suppliers on reliability • Component faults and failure analysis • Value and Financial Impact • Prioritization of engineering and retrofit • Supplier involvement in system reliability • ID systems for Engineering focus • Warranty cost recovery Equipment Monitoring & Management Video: https://youtu.be/vIVepQRl5SY
- 53. Equipment Monitoring & Management Video: https://youtu.be/vIVepQRl5SY
- 54. Trend Analysis Combination of Rules CUSUM Analysis Statistical Analysis Statistical Process Control Machine Learning Location Change – Variable moves up or down Slope Change – Variable changes trend Variance Change – Variable becomes more/less volatile Process Threshold – Shewhart control chart Failure Model y (0/1) = f (X, b) + e; f = logistic regression, trees, svm, nnet, ... Sensor Analytics
- 55. 1. Analytics models 2. Data streams 3. Calculations on live data 4. Analysis notifications Fast Data Analytics Video: https://youtu.be/vIVepQRl5SY
- 57. Alerting In The Field
- 59. Industrial Equipment Management Improves Operations
- 60. IT & Governance © Copyright 2000-2015 TIBCO Software Inc.© Copyright 2000-2014 TIBCO Software Inc. • Library Services • Centralized management of Spotfire analysis files, metadata, information links, TERR scripts, … • User Services • User authentication, role-based authorization • Audit Services • Content access, modification, deletion • User authentication, data access, library operations • Usage Log Analytics • Sessions, Users, Admin, Local Files • Library, Information Links, Admin, Detailed Logs • Analysis Profiler • Automate every analysis file during upgrade / migration
- 61. © Copyright 2000-2015 TIBCO Software Inc. Tibco’s Fast Data Platform Architecture
- 62. Learn how some of the major players in the energy industry are using Spotfire to revolutionize their business: • How to minimize risks by better understanding exposure to asset integrity issues • Using analytics to control margins and conduct customer profiling • Leveraging forensics to reduce NPT and monitor production • Production optimization techniques http://energyforum.tibco.com/ Energy Forum September 1st – 2nd | Norris Conference Center | Houston, TX
- 64. Monthly Knowledge Share Hosted by Quintus Linked In hosted by Syntelli LinkedIn
- 65. Webcasts Insight and Action - Analyzing Your OSIsoft PI System Data Tuesday, July 7, 2015 1 PM EST Presenter: Michael O'Connell & Dave Leigh Predictive Analytics in the Energy Sector: Asset Valuation Tuesday, July 28, 2015 1PM EST Presenter: Michael O'Connell & Peter Shaw with Haas Engineering and R Lacy Seeing Stars: the Gartner BI Bakeoff Recording, May 27, 2015 Presenter: Anna Nowakowska & Michael O'Connell Events spotfire.tibco.com/about-us/events
- 66. 66 © Copyright 2000-2014 TIBCO Software Inc. Spotfire Ecosystem
- 67. Thank you! Michael O’Connell, PhD Chief Data Scientist TIBCO [email protected] @moc_tib http://about.me/moconnell +1-919-7401560 First to Insight, First to Action © Copyright 2000-2015 TIBCO Software Inc.
Editor's Notes
- #11: Visual Analytics For exploratory analysis And publication reporting
- #48: Finally, one of the most valuable initiatives, which builds on the previous one, is the ability to sense, respond and influence business moments. Business moments are situations of interest, opportunities for the business to marry insights from big data with the understanding of the context in real-time, to take an action. Example: predictive maintenance. Machine is close to maintenance period but not there yet. The production forecast is low right now but will become intense. Propose to operations team to execute maintenance operations ASAP as it’s the scenario with least impact on the forecast.
- #49: Managing ships, trains, vehicles Taking into account: Weather Business metrics Pit to Port Long train tracks, has the longest trains in the world Ships waiting to be loaded Need to manage tide while complying with SLAs Big Data provides
- #51: Michael O’Connell
- #55: Thresholds can include a change in location, slope or variance e.g. motor temperature jumping 20 degrees in an hour; anomalies exceeding process control limits; or an empirical machine-learning model.
- #56: The Event Server calculates the models on live data and provides notifications – including emails to engineers and/or logging to operational data stores or BPM systems.