Deep Dive OpenShitt on Azure & .NET Core on OpenShift

Editor's Notes

• #2: Hello, my name is Takayoshi Tanaka. Today I talk about Deep Dive OpenShift on Azure and .NET Core on OpenShift.
• #4: Let’s me give my background. I work for Red Hat K K in Japan. My position is a Software Maintenance Engineer. My focus is OpenShift, Red Hat solutions on Azure and .NET Core on RHEL. In personal, I’m a Microsoft MVP for Visual Studio and Development Technologies. I’m interested in C# language and .NET Core on Linux. I write blogs in Red Hat Developers and Personal Blog “Silver light and Blue Sky”.
• #5: My goal for the audience is, leaning about OpenShift on Azure Reference Architecture. Being able to know how to integrate Azure Features with OpenShift, and .NET Core 2.0. integrating OpenShift features with Asp.NET Core.
• #6: The first thing is deploying OpenShift on Azure.
• #7: Red Hat released the reference architecture document　for deploying OpenShift on Azure.
• #8: Azure has similar services with OpenShift. You can run your applications on PaaS and CaaS. CaaS means Container as a Service. OpenShift is a CaaS. And Azure Container services and Service Fabric are also CaaS. And recently Microsoft released new CaaS “Azure Container Instances”. OpenShift based on RHEL OS Virtual Machines. We support both on Azure Stack and Azure Public Cloud.
• #9: You can install this reference architecture from this repository. This repository is an ARM template. When you click the deploy button, you’ll see this form. After you fill in all parameters, the install process begins.
• #10: There are three steps in installation. At first, ARM Template creates Azure resources such as Virtual Machines, Load Balancers, Networks and so on. Second, Custom Script Extension will generate configuration files and execute ansible playbook. At last, this ansible installer installs OpenShift.
• #11: However, there are known issues. This ARM template only supports marketplace VM. It will be duplicated billing. Supports custom VHD is on roadmap. This template is unofficial support, so you should troubleshoot by yourself. Then it’s fixed VM configuration and you can’t change it. Three masters with etcd at the same hosts, three infra nodes, three and more nodes and one bastion server.
• #12: Here is the architecture diagram. Let’s explain each component.
• #13: Here is a bastion server. This is the only VM accessible from the Internet. You will operate everything on this VM.
• #14: There are two external Azure Load Balancers. One is for OpenShift API and web portal. The backend is master servers with Availability Sets. Etcd is located at the same host.
• #15: The other external Azure Load Balancer is public endpoint for accessing your applications. The backend is infra nodes with availability set. Router and docker-registry run on these nodes.
• #16: The last component is a node server with availability set. Your application pods run on these nodes.
• #17: So you have to launch at least ten VMs. Also, you may have to make support request to Microsoft in order to increase cpu core limit. This limitation is not due to OpenShift itself, but due to design of ARM template. For example, you can install all-in-one OpenShift on one host.
• #18: This reference architecture uses Azure Features to integrate with OpenShift. Availability Set and Load Balancer are told before. Another one is Azure VHD for Persistent Volume, PV.
• #19: It’s a little complicated how Azure VHD for PV works. At first, this feature is depends on kubernetes Azure Volume plugin. This plugin works as follows. Node service receives Volume Mount request. Then load azure.conf file and create an empty VHD if dynamic provisioning is enabled. Next, mount VHD to Azure VM where node is running. If needed, create filesystem. Finally, mount filesystem to the container.
• #20: Sometimes, you may have to create azure.conf file manually. It’s not difficult. You can follow three steps with Azure CLI 2.0.
• #21: When you use Azure VHD for Persistent Volume, you may pay attention somethings. At first, managed disk is unavailable. kubernetes plugin doesn’t support Managed Disk now. And your Azure VM name must be same as hostname. Then, it’s not mandatory but I recommend use Azure internal DNS, because Virtual Machines must be able to communicate with their VM name. Azure internal DNS can do it without any configuration. If you don’t use Azure internal DNS for example when you use VNET Peering, you must configure DNS by yourself.
• #22: There are more Azure feature which are not used in the reference architecture. I’ll explain some of them. If you use Azure Active Directory, you can use it for authenticating master API with Open ID connect.Also, if you prefer LDAP authentication, Azure Active Directory Domain Services is available. Or you can use on-premised Active Directory. OpenShift has an internal docker registry. Azure Blob Storage is available for this registry’s storage. Generally speaking, object storage such an Azure Blob Storage is suitable for docker registry storage. Azure File Storage is also available for Persistent Volume. However, it depends on Linux kernel CIFS module, which is still experimental in SMB 3 protocol.
• #23: To set up an OpenID, it’s the easiest way to create an AAD app in Azure Portal page. You can see the endpoints here. Also, you should set up Reply URLs and Keys.
• #24: If you prefer LDAP authentication, you have two options. When you want to use AAD, you can use optionA. AAD itself doesn’t have LDAP mechanism, but you can do this with AAD Domain Services. However, AAD DS is only available for Classic Vnet at this point. So you should connect classic VNET and ARM Vnet. You can connect them by VNet peering or VNET-to-VNET VPN.
• #25: Here is an example configuration for LDAP authentication with AAD DS. Organization Unit should be “AADDC User”, which is default value on AAD. usually you can’t change this Organization Unit as far as you use only AAD. Also, userPrincipalName will be the value of email.
• #26: When you prefer to use your on-premised AD, you should connect by VPN to use LDAP authentication.
• #27: There are several Azure services you can use for storage in OpenShift. Azure Blob Storage is only available for internal docker registry’s storage. However, this object storage is suitable for docker registry. This feature depends on docker registry driver provided by docker. Other two features both depend on kubernetes plugin. The one is Azure VHD, which I told before. The other is Azure File Storage. However it’s experimental. Also, you can use external NFS Server in and out of Azure. You can construct and maintain your own NFS Server or use NFS as a Service from third party vendor.
• #32: To collecting and analyzing logs, OpenShift provides EFK stack. Also, Azure provides OMS. You can see OpenShift logs with OMS Agent. To use OMS agent, after creating an OMS workspace, you have only to connect VM from Azure portal page.
• #33: You can also see the container logs in OMS workspace. You have only to add container solution in OMS portal.
• #34: We’re sometime asked the future of integrating with Windows container. However, the answer is “we have no roadmap for Windows container”. As kubernetes is working for Windows container, we can show a capability to run Windows Container on OpenShift. However, please note it’s not a road map at this point.
• #35: Welcome back to my talk. The second part is about .NET Core on OpenShift.
• #37: ABI: Application Binary Interface
• #38: What is the reason for using .NET Core on RHEL? One is Red Hat software collections. .NET Core on RHEL ships in this software collections. Usually it’s included in RHEL subscription, so you can use .NET Core without extra cost. Also, we’re tracking Bugs not only for upstream GitHub but also Red Hat Bugzilla.
• #39: Red Hat provides and supports .NET Core SDK for RHEL. RHEL Server subscription includes .NET Core. Not only Red Hat provides .NET Core, but also OpenShift supports .NET Core. S2i means Source code in the git repository to docker image. s2i can run out of OpenShift. S2i requires the base docker image to run build process. Red Hat provides an official s2i image for .NET Core. It can be customized to add parameters or add scripts. OpenShift provides a template project for .NET Core and ASP.NET Core, so you can start .NET Core and ASP.NET Core on OpenShift with few clicks.
• #40: This is a figure how s2i works. At first, a builder pod will clone source code and build it. After generating a dotnet binary, builder pod creates a application docker image with this binary and pushes this image to internal registry. Once build is finished, deployer pod will start. Deployer pod will create an application container from pushed docker image, and run the dotnet application.
• #41: .NET Core will be coming soon, maybe this autumn. Red Hat .NET Core will support csproj style build tools at .NET Core 2.0. s2i also supports .NET Core 2.0 and ASP.NET Core 2.0. More new feature are coming.
• #42: From now, I’ll explain four use case examples. I’m now working for another examples.
• #43: When you want to execute .NET Core application as a scheduled job, you can use cron job in OpenShift. Web portal hasn’t supported cronjob yet, so you should create resource from CLI.
• #44: Here are two points to pay attention. One is specify the dotnet path as an absolute path. The other is specifying docker image as a full URL. You will be able to specify with Image Stream Tag at OCP 3.6.
• #45: Sometime you want to treat different environments with one code. For example, connecting to different database on development and production. Using Redis as a cache only in production. Using different OpenID account for development and production. In this case, you can use environment feature in ASP.NET Core and it can be integrated with OpenShift.
• #46: This example shows how you check the current environment in the code. Startup class is a key calss for ASP.NET Core and it has three important member: a constructor, configure method and configureservice method. IHostingEnvironment can be injected into constructor and Configure methods. So you can check EnvironmentName property.
• #47: However, as ConfigureService method can’t be injected IHostingEnvironment, you can define different methods for each environment.
• #48: Configure method can also be defined with environment specific methods.
• #49: Also, when you want to pass some values from Environment Variable, you can use “AddEnvironmentVariables” method. To set environment variables, you can use Visual Studio or Visual Studio Code during developing on your local machine. In the OpenShift you can set on the Portal page or CLI.
• #50: Sometimes you don’t want to store configuration in the repository, you can use OpenShift Secret feature. The application can read secret values from external file or environment variable. This example shows secret values are injected to environment variable and application read these value s from environment variable.
• #51: Next is Razor page example. This is a new feature at ASP.NET Core 2.0. Also, C# 7.1 will be available at .NET Core 2.0. However, C# 7.1 is not available in Razor page due to a bug. It’ll be fixed at RTM.
• #52: Usually you will run your application with multiple pods. When you use HTTP session by default configuration, session is sticky by default router configuration. Http session is stored in the memory and encrypted by pod specific key. It means, when a pod has died, a user session will be lost. In this case, you may want to store session data external. But you should pay attention.
• #53: There are two key points in storing HTTP session. You must configure both of them. IDistibutedCache provides distribution cache to store session across multiple servers. IDataProtection provides key management for encryption. Redis supports both services.
• #54: When you only configure Idistribution cache to use Redis, HTTP session can’t be shared across pods. This is because the encrypted key is generated and stored in each pod. These keys are different.
• #55: You must configure DataProtection to use Redis. After doing so, the encryption key is stored in Redis and all pods use this key.
• #56: Here is an example to configure Redis. In this example, I use Azure Redis service. Off course, you can run Redison OpenShift and you will get redis ip and port from environment variables when you run redis on OpenShift.
• #59: Here is a inside of a debugger. On Windows, Visual Studio contains a VS Debugger and also works as a debugger frontend. On Linux, Visual Studio Code works similarly. Visual Studio Code contains debugger called vsdbg. It is constructed as an Open Source Project MIDebug Engine and depends on GDB and LLDB. Even more remarkable, debugger has a same interface on Windows and Linux. So Visual Studio can remote debug to .NET Core running on Linux and Visual Studio Code on Linux can remote debug to .NET Core running on Windows.
• #70: In summary, Thank you for listening today. Have a great day.