MQ Guide France - IBM MQ and Containers
- 1. IBM MQ certified containers Rob Parker Security focal, Software Engineer IBM MQ [email protected]
- 2. Important disclaimers Hybrid Cloud / November 2019 / © 2019 IBM Corporation IBM’s statements regarding its plans, directions and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
- 3. Important disclaimers • IBM Confidential. Unless specifically advised otherwise, you should assume that all the information in this presentation (whether given in writing or orally) is IBM Confidential and restrict access to this information in accordance with the confidentiality terms in place between your organization and IBM. • Content Authority. The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. • Performance. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. • Customer Examples. Any customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. • Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Hybrid Cloud / November 2019 / © 2019 IBM Corporation
- 4. Trademarks • IBM, IBM Cloud etc are trademarks of International Business Machines Corporation, registered in many jurisdictions • Amazon Web Services, AWS and related names may be trademarks owns by AWS as described at https://aws.amazon.com/trademark- guidelines/ • Other company, product and service names may be trademarks, registered marks or service marks of their respective owners • A current list of IBM trademarks is available on the web at "Copyright and trademark information" ibm.com/legal/copytrade.shtml Hybrid Cloud / November 2019 / © 2019 IBM Corporation
- 6. © 2018 IBM Corporation Characteristics of a cloud environment • Self-service • Empowers users to provision resources without requiring human intervention, most likely using a web-based portal or an API. • Elastic scaling • Enables scaling up and down on demand, driving the need for high levels of automation. • Shared resources • Offers economies of scale through the use of shared infrastructure and software, securely separating the resources at a logical level. • Metered usage • Allows pay-as-you-go billing through monitoring, measurement and reporting of usage. Private cloud IBM Cloud Private Public Cloud
- 7. Integration Technical Conference 2019 7 Containers Introduction Containers • Containers provide a similar environment to a VM but lighter in weight Ø A virtual machine provides an abstraction of the physical hardware Ø A container abstracts the OS level, typically at the user level • Linux containers Ø Containers all share the same OS kernel Ø Images are constructed from layered filesystems Ø Containers isolate applications from each other and the underlying infrastructure Container Bins / libs App App Container App App Container App App Operating system Bins / libs
- 8. Benefits of a container strategy Scalability and Infrastructure Optimization Improved utilization of system resources when compared to virtualized isolation runtimes Build Agility Container build process reduces the effort for maintaining multiple runtime environments Operational Consistency Homogeneous administration of heterogeneous components, reducing the range of skillsets required to operate the environments Fine-grained Resilience Containers are disposable. Failing containers are removed and replaced with new instances, remove the need to nurture Component Portability Containers reduce the barrier to moving components between environments (on-premise, public & private clouds) Team Productivity Accelerated development, improved consistency across environments, empowering autonomous teams improving productivity and quality
- 9. Separate storage from compute Physical or Virtual Machine /usr/… /etc/… /opt/mqm /var/mqm Cloud Virtual Machine or Container /usr/… /etc/… /opt/mqm /var/mqm Contents of /var/mqm is populated at MQ installation time Contents of /var/mqm is populated by running crtmqdir at runtime.
- 10. What is Kubernetes ? 10Think 2019 / © 2019 IBM Corporation Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. • Can be used to deploy containers and their resources. • e.g. Persistent volumes, load balancers, secrets. • Containers are deployed as “pods” which can contain multiple containers • Pods can be placed on any “node” and be moved between nodes • Multiple pods can be deployed either singularly or via “Sets” • E.g. StatefulSet or ReplicaSet • Network access is provided by “Services”
- 11. 11 IBM Cloud Private 1111 Enterprise Content Catalog Open Source and IBM Middleware, Data, Analytics, and AI Software Core Operational Services Log Management, Monitoring, Metering, Security, Alerting Kubernetes Container Orchestration Platform Strategic Value: Reduced infrastructure, license, and maintenance costs Self-service catalog Agility, scalability, and elasticity Self-healing Enterprise security No vendor lock-in Choose your infrastructure: IBM Z © 2019 IBM Corporation VMWare Intel Microsoft Azure Amazon Web Services Google Cloud OpenStack Kubernetes Docker
- 12. IBM Cloud Pak for Integration 12 Taking IBM’s market leading integration capabilities and adding value to become one simple, fast, and secure integration experience IBM Cloud Private foundation IBM Cloud Pak for Integration API Lifecycle Secure Access Application Integration Messaging & Events High Speed Transfer 12 – Most powerful integration platform on the market NEW offering incorporating traditional and modern integration including APIs, App Integration, Message queuing, Event streams and Fast file transfer – Deploy wherever needed Supports deployment on-premises or in any cloud – Enterprise grade Secure, scalable modern architecture © 2019 IBM Corporation
- 13. Replatform establishing the container orchestration platform, services and capabilities to succeed, and move to a runtime topology that is native to the platform Containerize MQ queue managers, with applications connected as clients MQ Modernization Containerization facilitates the modernization of MQ deployments. (These pattern also apply outside of containers) Repackage break down the existing artefacts so that they are bounded along line of business and development teams to improve the agility of the organization Queue managers are dedicated to an application Refactor re-work the artefacts that are hard to maintain or prevent the organization from realising the full benefits of their modernization journey Deploy MQ patterns that provide horizontal scaling and continuous availability13© 2019 IBM Corporation QM QM QM QM QMQM QM QM QMQM IBM Cloud Transformation Advisor Analyses your queue managers and JEE applications for suitability for moving to IBM containers
- 14. IBM MQ in a container
- 15. MQ in Containers MQ has been supporting Docker containers since 2015 with images on Docker Hub and Docker Store and sample setups on Github github.com/ ibm-messaging/ mq-container MQ Advanced is available as fully supported IBM certified containers with IBM Cloud Private and the IBM Kubernetes Service on IBM Cloud Deploy fully supported IBM certified software containers into an IBM provided Kubernetes platform or an existing Red Hat OpenShift environment IBM has introduced the ability to purchase an entitlement based on the container size in Virtual Processor Cores and the number of hours that MQ was deployed in each container 15 Traditional licensing Hourly licensing Red Hat Enterprise Linux Red Hat OpenShift IBM Middleware IBM Cloud Private IBM Confidential © 2019 IBM Corporation
- 16. MQ is supported in containers – MQ V8.0.0.4 onwards is supported in Docker V1.6+ – MQ V9.1.0.0 onwards is supported in Docker V1.12+ – IBM recommends using either MQ V9.1 LTS, or MQ V9 Continuous Delivery releases • Adds web console • Adds REST APIs • Easier storage management (crtmqdir) • Quicker to receive new features – IBM will support MQ issues, agnostic to the orchestration environment – The orchestration vendor will need to support and provide assistance for orchestration issues
- 17. MQ container orchestration support Standalone container IBM Cloud Kubernetes Service IBM Cloud Private IBM Cloud Private on Red Hat OpenShift Red Hat OpenShift Microsoft Azure Kubernete s Service Amazon Elastic Kubernetes Service Compone nt/arch * x86_64 x86_64 ppc64le (POWER) s390x (z/Linux) x86_64 x86_64 x86_64 x86_64 MQ Advanced Server ● ✭ ✭ ✭ ✭ ✭ ✭ (9.1.4) ▲ ▲ MQ Server ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ MQ Clients ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ MFT Agent ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ Salesforc e Bridge ● ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ Blockchai n Bridge ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ RDQM ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ Internet Pass-Thru ● ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ✭ IBM MQ certified container Supported image and Helm chart available MQ supported with no sample Supported, and you need to build your own image. ▲ ● MQ supported with sample Supported, and you need to build your own image (samples/blog available) Not supported in containers✖
- 18. MQ Advanced certified container for Cloud Pak for Integration MQ Advanced for Developers MQ Advanced certified container Self-build Hybrid Cloud / November 2019 / © 2019 IBM Corporation
- 19. MQ Advanced certified container for Cloud Pak for Integration MQ Advanced for Developers MQ Advanced certified container Self-build Hybrid Cloud / November 2019 / © 2019 IBM Corporation UBI/RHEL/SLES/ Ubuntu IBM MQ or IBM MQ Advanced Container-specific code (sample available) Red Hat Universal Base Image IBM MQ Advanced for Developers Container-specific code Default developer config Red Hat Universal Base Image IBM MQ Advanced Container-specific code Red Hat Universal Base Image IBM MQ Advanced Container-specific code Cloud Pak extras ibm-mqadvanced- server-dev Helm chart + ibm-mqadvanced- server-prod Helm chart ibm-mqadvanced- server-integration-prod Helm chart + +
- 21. © 2019 IBM Corporation22 High availability with Kubernetes The RDQM solution does not apply to container environments High availability of the MQ data requires highly available replicated storage Container orchestrators such as Kubernetes handle much of the monitoring and restart responsibilities… Node 1 Node 3Node 2 HA network storage Pod 1 StatefulSet, Replicas=1 Kubernetes
- 22. © 2019 IBM Corporation23 High availability with Kubernetes The RDQM solution does not apply to container environments High availability of the MQ data requires highly available replicated storage Container orchestrators such as Kubernetes handle much of the monitoring and restart responsibilities… …but not all. StatefulSets such as MQ are not automatically restarted following a Kubernetes node failure Node 3Node 2 HA network storage StatefulSet, Replicas=1 Kubernetes
- 23. © 2019 IBM Corporation24 High availability with Kubernetes The RDQM solution does not apply to container environments High availability of the MQ data requires highly available replicated storage Container orchestrators such as Kubernetes handle much of the monitoring and restart responsibilities… …but not all. StatefulSets such as MQ are not automatically restarted following a Kubernetes node failure The MQ container image and Certified Container now supports a two-replica multi-instance queue manager deployment pattern to handle Kubernetes node failures Node 1 Node 3Node 2 HA network storage Pod 1 multi-instance active Pod 2 multi-instance standby StatefulSet, Replicas=2 Kubernetes IBM MQ 9.1.3 CD
- 24. Storage providers Hybrid Cloud / November 2019 / © 2019 IBM Corporation IBM Cloud File Storage – Backed by NFS V4 – Not replicated between zones AWS Elastic File System (EFS) – Backed by NFS V4 – Replicated between zones – Limited to 256 locks — ensure this meets your scalability needs IBM Spectrum Scale – Older versions (GPFS V4.0) have been successfully tested with MQ GlusterFS – V3, V4 and V5 do not meet POSIX standards around locking (e.g. canceling a thread does not release locks)
- 25. Connection Routing Hybrid Cloud / March 2018 / © 2018 IBM Corporation Static Routing Client Connection Definition Table Load Balancer Client embeds endpoints Performance impact when primary unavailable Brittle configuration No load balancing Client references endpoints Enhanced Workload Management Strategies Central configuration management Client references endpoints Enhanced Workload Management Strategies Central configuration management Not recommended for JMS Endpoint List Application MQ MQ CCDT Application MQ MQ IP Gateway Application MQ MQ
- 26. Recommended Routing MQMQMQ Application Application IP Gateway Messaging Layer Application Network Connection Container(s) Connection Routing Application ApplicationApplication IP Gateway Application IP Gateway Application IP Gateway Application MQ PUT MQ PUT MQ PUT MQ PUT MQ PUT MQ GET MQ GET MQ GET CCDT CCDT CCDT CCDT CCDT IP Gateway IP Gateway
- 27. Features of IBM MQ Certified Containers
- 28. Catalog • Provides quick deployment • Hundreds of IBM and non- IBM products available • Can import custom products for own use Logging • Central location for all log output • Searchable • Uses Kibana technology • Queue Manager logs mirrored to this service. Monitoring • Central location for monitoring metrics • Searchable • Custom graphs • Uses Grafana technology • IBM MQ Monitoring metrics sent to service. Metering • Monitors up time & CPU cost of deployments • Can be used for licensing (hourly license model) • Searchable on deployment and version. 29Think 2019 / © 2019 IBM Corporation Integration with ICP4I features
- 29. Sample Kibana dashboard Hybrid Cloud / November 2017 / © 2017 IBM Corporation
- 30. Sample Grafana dashboard Hybrid Cloud / November 2017 / © 2017 IBM Corporation
- 31. Non-Root 33Think 2019 / © 2019 IBM Corporation • 9.1.3: Container now runs as the “mqm” user rather than root, with a fixed UID and GID • Allows for running under a tighter security policy • Currently able to run under an ‘anyuid’ style Pod Security Policy… More on this later!
- 32. Custom Kubernetes Labels 34Think 2019 / © 2019 IBM Corporation • Kubernetes uses key/value labels on resources to specify attributes that are meaningful/relevant to users • This release brings functionality to add custom labels to Queue Manager resources (pod, stateful set, PVC, etc) • Necessary for connection routing as part of scaling • Useful for creating custom resource queries
- 33. TLS 35Think 2019 / © 2019 IBM Corporation • You can now supply PKCS#8 & PKCS#1 unencrypted PEM files to add certificates to your Console & queue manager. • Multiple certificates and keys can be provided with labels to use. • First alphabetically will be set as default certificate. • Uses Kubernetes Secrets to stored Certificate and keys. QM PodK8s Secret Certificate.crt Certificate.key
- 34. IBM MQ in containers with App Connect Enterprise
- 35. Why does ACE use MQ? © 2018 IBM Corporation 1. As an asynchronous messaging provider 2. As a co-coordinator for global (two phase commit) transactions But it needs to connect to a local MQ Server
- 36. Running MQ and ACE in containers 38© 2018 IBM Corporation For cases where ACE can use Client bindings connections then you can run MQ and ACE in separate containers. For cases where ACE requires local (server) bindings connections then you either: MQ IIB Container Embedded in same container Separate containers with shared PID Namespaces* PID /var/mqm MQ Container ACE Container https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.con.doc/q114488_.html *Docker 1.12 Kubernetes 1.10 Alpha
- 37. Thank you
- 38. ®