SUSE, Hadoop and Big Data Update. Stephen Mogg, SUSE UK
3 likes2,560 views
The document outlines SUSE's offerings and partnerships related to Hadoop and big data, highlighting its certified compatibility with leading Hadoop platforms. It emphasizes features such as reliability, scalability, security, and management capabilities of SUSE Linux Enterprise Server, making it a suitable foundation for Hadoop deployments. Additionally, it provides resources for best practices in deploying and managing Hadoop on SUSE systems.
SUSE, Hadoop and Big Data Update. Stephen Mogg, SUSE UK
- 1. SUSE for Hadoop & Big Data Stephen Mogg SUSE UK October 2014
- 2. 2 About SUSE • Established 1992 • Original Provider of Enterprise Linux About Me • SUSE Employee 4 years • Systems Engineer
- 3. 3 If you want to know more about SUSE • New Certifications • New Resources • New Lab
- 5. 5 Big Data Reference Architecture Operating System OS / Cloud Platform Source: Hortonworks Modern Data Architecture - http://hortonworks.com/partner/suse/
- 6. 6 SUSE Big Data Reference Architecture Source: Hortonworks Modern Data Architecture - http://hortonworks.com/partner/suse/
- 7. 7 SUSE Big Data Partners Hadoop Data Systems Applications Services
- 8. 8 Certified for Leading Hadoop Platforms Additional level of testing and quality assurance to make sure SUSE Linux Enterprise Server integrates with partner software, saving our customers time while providing them with an assurance of interoperability. We hereby declare that SUSE Linux Enterprise Server is officially certified for: Cloudera CDH 5 Hortonworks HDP2
- 10. 10 SUSE in High Performance “Teradata's extensive financial, technical, and management resources can create a unique, high-performance Hadoop appliance that few other vendors can match.” – Forrester Feb 2014 High Performance Computing ‒ Half of the world's largest super computer clusters run SUSE Linux Enterprise Server Mainframe Computing ‒ Over 80% of all Linux running on mainframe computers is SUSE Linux SAP Hana ‒ SUSE Linux Enterprise Server is the recommended OS for the market leading analytics appliance, SAP HANA. Teradata ‒ SUSE Linux Enterprise Server is the OS foundation for Hadoop in the Aster Big Analytics Appliance IBM Watson ‒ Power artificial intelligence computer runs SUSE Linux and Hadoop
- 11. 11 What Makes an Optimal Foundation for Hadoop? SLAs and Business Continuity Resource Utilization and Efficiency Security and Compliance Affordable, No Vendor Lock-in
- 12. 12 Power, Scalability Reliability, Availability,Serviceability: Swap-over NFS Built-in open source multi-path IO CPU/Memory hot-plugging Horizontal/Vertical Scalability: Large capacity and faster system interconnect (OFED, Infiniband) A rock-solid, certified foundation for deploying Hadoop clusters. Huge Data, Massive Compute: 4096 logical CPU 64 TiB RAM Supports latest Intel CPUs: Ivy Bridge v2 Haswell SUSE Linux Enterprise Server
- 13. 13 Flexibility, Agility Massively Scalable Private Cloud Implementations Deploy pre-configured Hadoop clusters on KVM, Xen, Hyper-V, ESXi Spin up fully configured and optimized Hadoop Cluster in minutes for dev/test Scale-out Hadoop cluster Infrastructure easily API for Cloud-aware Applications SUSE Cloud Hadoop in the Cloud: OpenStack based enterprise ready IaaS Cloud Platform.
- 14. 14 Improve Resource Utilization and Efficiency Batch Command Speeds Up Cluster Implementation Centralized Server Infrastructure Management Software and Patch Management for Linux and Hadoop Batch-deploy config files to entire Hadoop cluster Asset Management and Reporting Application and Infrastructure Monitoring SUSE Manager A perfect complement to the monitoring and management capabilities provided in the Hadoop cluster management software.
- 15. 15 Security and Certifications 90% of companies cite data access and data protection as either extremely or very important security capabilities. - IDG Big Data Survey 2014 Security Features SUSE Linux Enterprise Server System Hardening YaST2 Security Center Application Confinement AppArmor System Confinement SE Linux (stack support) Intrusion Detection (file system) AIDE Fine-grained Access Rights File system POSIX capabilities Encryption Capabilities Three ways: Full disk, Volume, Filesystem (eCryptFS) Certifications Carrier Grade Linux (CGL) 4.0 IPv6 (refresh) Measure and Monitor System Integrity During Trusted Platform Modules (TPM)—Trusted Reboot Computing System Requirements for Cryptographic Modules FIPS 140-2 Validation for OpenSSL Common Criteria for IT Security Evaluation Common Criteria Certification for SP2 (x86 64 with KVM; IBM System z)
- 16. 16 Summary: Key Features and Benefits Key Features Benefits Reliability, Availability, Serviceability, Scalability Swap over NFS Cut cost with less expensive diskless servers Kernel 3.0 Enhanced RAS capabilities Intel Ivy-Bridge 2 and Haswell Support Harness the latest CPU technologies and provides excellent 4096 Logical CPU, 64TiB RAM Support vertical scalability InfiniBand, iSCSI Target (LIO) and OFED Faster connectivity with networking and storage equipment Dual Hypervisor Support: Xen and KVM Cross-platform Maximum choice both as a host and as a guest Virtualization Optimized for vSphere, Hyper-V, Open Source Hypervisors Linux Containers Light weight OS level virtualization UEFI Secure Boot Less malicious attach risk in boot Security and Compliance FIPS 140-2 Validation and Common Criteria Certification Security standard compliance AppArmor Protects from external/internal threats and zero-day attacks Integrated System Management Snapper and BTRFS Snapshot and rollback for easy management YaST, AutoYaST and Zypp Integrated single system management and fast update tools Interop with Other Platforms SAMBA 3.6 Compatible with Windows IPv6 Compliance Networking with IPv6 equipment
- 17. SUSE Big Data Resources
- 18. 18 Hadoop on SLES Best Practices White Paper: • Deployment scenarios • Proposed Architecture using SLES • Infrastructure considerations • Basic optimization of the Linux OS • Installation and configuration of Hadoop on SLES
- 19. 19 SUSE Manager and Hadoop Step-by-step guide for using SUSE Manager to deploy Cloudera on SLES: • Automate OS provisioning • Deploy new servers with identical characteristics • Auto-deployment of RPM-based applications • Centralize management of configuration files • Connect to SUSE Customer Center for updates • Create / manage multiple organizations from a single remote console. • Create customized repositories • Maintain the security of enterprise systems • Leverage the SUSE Manager API to create custom scripts to manage tasks or integrate third-party applications and management tools
- 20. 20 Hadoop / HP Reference Architecture HP Reference Architechture: • Written by SUSE, HP & Hortonworks, • Proposed Architecture using SLES • HP Recommends SLES
- 21. 21 SUSE Big Data Lab Big Data Cluster in USA for: • Benchmarking • Software certification • Integration / test • Reference architectures
- 22. Learn About: Register: 22 SUSE Linux Expert Days • SUSE and Big Data • Towards Zero Uptime with SUSE Tecnology • SUSE Linux Enterprise Server https://www.suse.com/events/slef-2014/#Liste
- 23. 23 Learn More Visit our web site www.suse.com/solutions/platform.html#big_data Read our whitepapers Deploying Hadoop on SLES Deploy and Manage Hadoop with SUSE Manager HP Reference Architecture. Contact us [email protected]
- 24. 24
- 25. Unpublished Work of SUSE LLC. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE LLC. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.