Security, ETL, BI & Analytics, and Software Integration

Editor's Notes

• #3: Two-thirds of P&C insurers surveyed currently use predictive models for underwriting and risk selection, an increase of over 10 percentage points compared to the 2015 survey. The reasons behind such an increase are clear. There is unanimous agreement from personal lines insurers about the fundamental importance of using more sophisticated predictive techniques to drive success in today’s market. Equally, many commercial lines carriers are recognizing that the traditional barrier of the relative paucity of homogenous risk data in commercial portfolios can be overcome, enabling models to contribute significantly in more unique underwriting environments. Eighty-six percent of small- to mid-market carriers rate more sophisticated risk selection as essential or very important to future success. Over half (56%) of large account or specialty lines carriers share that view.
• #4: Two-thirds of P&C insurers surveyed currently use predictive models for underwriting and risk selection, an increase of over 10 percentage points compared to the 2015 survey. The reasons behind such an increase are clear. There is unanimous agreement from personal lines insurers about the fundamental importance of using more sophisticated predictive techniques to drive success in today’s market. Equally, many commercial lines carriers are recognizing that the traditional barrier of the relative paucity of homogenous risk data in commercial portfolios can be overcome, enabling models to contribute significantly in more unique underwriting environments. Eighty-six percent of small- to mid-market carriers rate more sophisticated risk selection as essential or very important to future success. Over half (56%) of large account or specialty lines carriers share that view.
• #5: https://dataworkssummit.com/san-jose-2017/sessions/from-big-data-to-data-discovery-one-small-footprint-one-giant-leap-to-understanding/
• #7: Most people don’t know that Liberty Mutual has over 4,000 technical employees who create our solutions. In order to keep up with the demands of our customers, we are changing the way our company works. We are moving to a faster paced, customer centric model. We want to offer innovative products and services in order to provide best in class experiences for our customers. We are basically operating like a startup backed by the strength of a Fortune 100 company.
• #9: Our group is involved in the entire lifecycle of analytics from Get to Study. We think about “Analytics” in 3 phases: Get/Land/Study All the way from obtaining the data orginally to landing it somewhere and then studying it
• #12: The necessary tools are often not to scale or not available Majority of people don’t have the training or understanding of how to use the tools In some areas we are relying on 3rd party vendors to solve our problems rather than build expertise – is this really an issue outside of Hadoop? But, data scientists want better performance from R and Python, want the freedom to use downloaded data science libraries, want to use Spark, Tensorflow, H2O, etc. want to be able to pull data directly from Liberty databases, want to be able to deploy models without IT involvement, want to be able to work with large datasets We could certainly create opportunities for people to expand their skills with R, Python, and increase out knowledge and level of support on the IT side. But you need to pick a tool Security and analytics seems to be opposing forces There is a bureaucratic and or autocratic view controlling data and it’s flow Data scientists generally don’t need NPPI – they want to analyze inputs and predict outcomes No understanding of the risk or lack of risk associated with using business data for analytics Unclear how to traverse governance and approval processes No resources available to assist with data requests or scrub data to prepare for analytics Need a place to persist prepared data, refresh as new data becomes available, make scrubbed data available to multiple projects
• #13: This works great for operational reporting… but not data analytics Some of those frustrations came from environments like this. Way to many data sources… very complex… wall between IT and Business Why does it take that long? For one… the data is everywhere… There are operational reasons for these EDW’s and Data Marts. I’m not saying there “not useful”, however as an analysts/ds they don’t alone met the need. What did the original data look like? Who do I talk to? Is there more data I don’t see? What about using R or Spark? Can I use open source?
• #14: Cleansing and cleaning is now shifting more towards the business side… Moving away from hard wall UPDATE: IM -> Data Analytics Excellent! I’m not bound by data storage or PC capacity. I can access/see all the data available to me I can “fail” and try again quickly!
• #15: How we work… we work together as one team with our business partners. We have Data Scientists and Engineers on our team, along with the software developers Next Yiqing will talk about how this team tackled various “big data” problems and how we used our lake in practice.
• #18: Remember our frustrations: Security and access for our users If you don’t setup security you have a lake that nobody can use!
• #20: This is an example of our Data Lake in action. GET:Were taking usage/billing data from various cloud providers LAND: and landing it in the LAKE. STUDY: Were leveraging PowerBI to surface that data to our end users Remember the OLD WAY: everyone talks about it forever in meetings, agrees on a schema, then an ETL developer starts the work.
• #21: Another Example: We leveraged the SAME LAKE to LAND that large amount of Experian data to HDFS. Then we used SPARK to preform ETL (Convert data) and write text documents to ELASTICSEARCH. In this example we used the same lake, but extended our capabilities with Elastic Search. REMEMBER THE OLD WAY: We whould have loaded into standard RDBMS, slow performance, and will have to write your own Queries and fuzzy matching. Large table scans. Would only look at a subset of the data because of the size. LONG time from idea to UNDERSTANDING!!!
• #23: Another example of how we use the SAME LAKE: Streaming Analytics for Security and Operational logs – Splunk cost containment
• #25: Get started Check back to North stars… Be mindful of transitions SPEAKER: MAKE SURE YOU HAVE TRANSTION STATEMENTS - Add more….