Using high-availability virtual IP addresses (HaVips) and Keepalived for primary/secondary failover - Virtual Private Cloud

You can use high-availability virtual IP addresses (HaVips) to ensure that service IP addresses remain unchanged during server failover within the same zone in the cloud.

If Keepalived itself can support high availability of virtual IP addresses, why use it with HaVips?

In traditional data centers, when Keepalived performs primary/secondary failover, it determines the new primary server based on the VRRP protocol. The new primary server can directly bind the virtual IP address to its own network interface and actively send Gratuitous ARP broadcasts to announce that it has taken over the virtual IP address. After devices in the local area network receive this ARP broadcast, they update their local ARP cache to point the virtual IP address to the MAC address of the new primary server.

However, most cloud providers use SDN architecture and virtualization technology to build network environments, where virtual server IP addresses are allocated and managed by the underlying virtualization stack of the cloud platform. Your applications cannot modify host IP addresses as in traditional environments. Additionally, the entire virtual network is based on Layer 3 tunneling technology, where ARP is terminated at the sender, preventing hosts from announcing IP addresses. To address this issue, Alibaba Cloud has introduced the HaVip feature.

An HaVip is a private IP address resource that can be independently created and released. You can set the virtual IP in the Keepalived configuration file to the HaVip address and associate the HaVip with multiple servers. When Keepalived elects a new primary server, the system updates the mapping relationship between the HaVip and the primary server, achieving an effect similar to Gratuitous ARP, thus ensuring that the service IP address remains unchanged during primary/secondary failover.

How it works

The following figure shows the architecture of a high-availability primary/secondary cluster implemented with one HaVip and two ECS instances. The system works as follows:

Keepalived configuration: The HaVip is associated with ECS1 and ECS2, both of which have Keepalived installed. In the Keepalived configuration file, the virtual_ipaddress (virtual IP) is set to the HaVip address for both instances. You also need to set the priority in the configuration file. The higher the value, the higher the priority of the server to become the primary server.
Primary server election: Based on the VRRP protocol, Keepalived automatically elects ECS1 as the primary server by comparing the priority values of ECS1 and ECS2. The system automatically updates the mapping relationship between the HaVip and the primary server. All traffic to the HaVip is forwarded to ECS1.
Primary/secondary failover: The primary server ECS1 periodically sends heartbeat messages to the secondary server ECS2 (the heartbeat interval is determined by advert_int in the configuration file). If ECS2 does not receive heartbeat messages within the specified time, Keepalived automatically switches the primary server to ECS2. After detecting the change in the primary server, the system automatically updates the mapping relationship between the HaVip and the new primary server. All traffic to the HaVip is forwarded to ECS2, ensuring that the service IP address remains unchanged during the failover process.

If Internet access is required, you can associate an elastic IP address (EIP) with the HaVip to provide high-availability services over the Internet.

Use HaVip for primary/secondary failover

An HaVip can be associated with ECS instances or elastic network interfaces (ENIs) in the same vSwitch. When used with software such as Keepalived, an HaVip ensures that the service IP address remains unchanged during a primary/secondary failover.

1. An HaVip can only be associated with resources of the same type at a time. To associate a different resource type, you must first disassociate the currently associated resources.

2. When associating an HaVip with an ENI, ensure that the ENI is attached to an ECS instance.

3. If an associated ECS instance or ENI is deleted, the system automatically disassociates the HaVip from the corresponding ECS instance or ENI.

4. If you detach a secondary ENI that is associated with an HaVip from an ECS instance, the association between the HaVip and the secondary ENI is not affected.

Console

Create an HaVip and associate it with primary and secondary instances

Go to VPC console - HaVip, select the region where the ECS instances are located at the top of the page, and click Create HaVip.
Select the VPC and vSwitch to which the ECS instances belong. You can either automatically assign a private IP address from the selected vSwitch CIDR block or manually specify an unallocated IP address.

Install Keepalived on the primary and secondary ECS instances.

Keepalived installation example

This example demonstrates how to install Keepalived on ECS instances running CentOS in a primary/secondary configuration. We recommend that you use version 1.2.15 and later.

If you have multiple secondary ECS instances, you need to declare the IP addresses of all peer instances in unicast_peer of each ECS instance.

You can visit Keepalived GitHub to learn more.

Primary server configuration

Log on to the instance by referring to ECS remote connection methods.
Run yum install keepalived to install Keepalived.

Run vim /etc/keepalived/keepalived.conf to edit the keepalived.conf file.

This example shows only the parts that need to be modified. Modify the keepalived.conf file based on your specific instances. Do not directly copy this example over your existing keepalived.conf file.

! Configuration File for keepalived
vrrp_instance VI_1 {
    state MASTER            # Set as the primary instance
    interface eth0          # Network interface for binding the VIP, configured as eth0 in this example  
    virtual_router_id 51    # virtual_router_id for the primary/secondary cluster; different primary/secondary clusters in the same VPC need different virtual_router_ids
    nopreempt               # Set non-preemption mode
    priority 100            # Set priority, higher number means higher priority; in this example, priority is set to 100 to make this instance the primary
    advert_int 1            # Heartbeat packet sending interval in seconds. If set too small, it may be affected by network jitter, causing frequent failovers and temporary dual-primary situations (split-brain). If set too large, it may lead to longer failover time when the primary instance fails.
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    unicast_src_ip 192.168.0.25     # Private IP address of this instance, configured as 192.168.0.25 in this example
    unicast_peer {
        192.168.0.26          # Private IP address of the peer instance, configured as 192.168.0.26 in this example; if there are multiple secondary ECS instances, you need to declare the IPs of all peer instances, each address on a separate line without commas or other separators.
    }
    virtual_ipaddress {
        192.168.0.24          # Virtual IP address, configured as the HaVip address, which is 192.168.0.24 in this example
    }   
    garp_master_delay 1       # Delay in seconds before updating ARP cache after becoming the primary instance
    garp_master_refresh 5     # Interval in seconds for sending ARP packets

    track_interface {
        eth0                  # Network interface for binding the VIP, configured as eth0 in this example
    }
}

Run systemctl start keepalived to start Keepalived.

Secondary server configuration

Log on to the instance by referring to ECS remote connection methods.
Run yum install keepalived to install Keepalived.

Run vim /etc/keepalived/keepalived.conf to edit the keepalived.conf file.

This example shows only the parts that need to be modified. Modify the keepalived.conf file based on your specific instances. Do not directly copy this example over your existing keepalived.conf file.

! Configuration File for keepalived
vrrp_instance VI_1 {
    state BACKUP            # Set as the secondary instance
    interface eth0          # Network interface for binding the VIP, configured as eth0 in this example  
    virtual_router_id 51    # virtual_router_id for the primary/secondary cluster; different primary/secondary clusters in the same VPC need different virtual_router_ids
    nopreempt               # Set non-preemption mode
    priority 10             # Set priority, higher number means higher priority; in this example, priority is set to 10 to make this instance the secondary
    advert_int 1            # Heartbeat packet sending interval in seconds. If set too small, it may be affected by network jitter, causing frequent failovers and temporary dual-primary situations (split-brain). If set too large, it may lead to longer failover time when the primary instance fails.
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    unicast_src_ip 192.168.0.26   # Private IP address of this instance, configured as 192.168.0.26 in this example
    unicast_peer {
        192.168.0.25          # Private IP address of the peer instance, configured as 192.168.0.25 in this example. You need to declare the IPs of all peer instances, each address on a separate line without commas or other separators.
    }
    virtual_ipaddress {
        192.168.0.24          # Virtual IP address, configured as the HaVip address, which is 192.168.0.24 in this example
    }    
    garp_master_delay 1       # Delay in seconds before updating ARP cache after becoming the primary instance
    garp_master_refresh 5     # Interval in seconds for sending ARP packets

    track_interface {
        eth0                  # Network interface for binding the VIP, configured as eth0 in this example
    }
}

Run systemctl start keepalived to start Keepalived.

Click the target HaVip ID, in the Resources section, click ECS Instances and then click Associate, and select the ECS instance or ENI that you want to associate.
After the association is complete, you can view the current primary/secondary relationship in the Associated Instances column of the target HaVip or in the Resources section of the details page.

Verification

You can run the following commands on the primary and secondary instances to create a web test service that returns different results.
You can check port usage with netstat -an | grep 8000. If port 8000 is in use, you need to choose another port.
Primary instance:
```
echo "ECS 1" > index.html  # Primary instance returns "ECS 1"
python3 -m http.server 8000
```
Secondary instance:
```
echo "ECS 2" > index.html  # Secondary instance returns "ECS 2"
python3 -m http.server 8000
```
On another ECS instance in the same VPC, run curl <havip_private_ip>:8000, which will return ECS 1. When the primary server is stopped, it will return ECS 2.
Make sure that the security groups of the primary and secondary instances allow HTTP traffic from within the same VPC to access port 8000.

Disassociate resources

Click the target HaVip ID, in the Resources section, find the target ECS instance or ENI under Associated ECS Instances or Associated ENIs, and click Disassociate.

Delete an HaVip

First ensure that the HaVip is not associated with any ECS instance, ENI, or EIP, then click Delete in the Actions column or on the details page of the target HaVip.

API

Call CreateHaVip to create an HaVip.
Call AssociateHaVip to associate an HaVip with an ECS instance or an ENI.
Call UnassociateHaVip to disassociate an HaVip from an ECS instance or an ENI.
Call DeleteHaVip to delete an HaVip.

Terraform

After running the Terraform script, two preemptible ECS instances and one HaVip will be created for you. The specific cost depends on how long you keep the ECS instances. We recommend that you clean up resources promptly to prevent continued billing.

Resource: alicloud_havip, alicloud_havip_attachment, alicloud_instance, alicloud_security_group, alicloud_security_group_rule

# Specify the region for creating the HaVip
provider "alicloud" {
  region = "cn-hangzhou"
}

# Specify the VPC ID
variable "vpc_id" {
  default = "vpc-bp1k******" # Modify to the actual VPC ID
}

# Specify the vSwitch ID
variable "vswitch_id" {
  default = "vsw-bp1y******" # Modify to the actual vSwitch ID
}

# Specify the instance type
variable "instance_type" {
  default = "ecs.e-c1m1.large"
}

# Specify the image ID
variable "image_id" {
  default = "aliyun_3_x64_20G_alibase_20221102.vhd"
}

# Create an HaVip
resource "alicloud_havip" "test_havip" {
  ha_vip_name = "test_havip_name"
  vswitch_id  = var.vswitch_id
  ip_address  = "192.168.0.24" # Specify the IP address for the HaVip from within the vSwitch CIDR block; if not specified, the system will allocate one
}

# Create a security group
resource "alicloud_security_group" "test_security_group" {
  security_group_name = "test_security_group_name"
  vpc_id              = var.vpc_id
}

# Create security group rules
resource "alicloud_security_group_rule" "allow_vpc_tcp" {
  type              = "ingress"
  ip_protocol       = "tcp"
  nic_type          = "intranet"
  policy            = "accept"
  port_range        = "1/65535"
  priority          = 1
  security_group_id = alicloud_security_group.test_security_group.id
  cidr_ip           = "0.0.0.0/0"
}

# Create the primary server 
resource "alicloud_instance" "test_master_instance" {
  instance_name        = "test_master_instance_name"
  vswitch_id           = var.vswitch_id
  instance_type        = var.instance_type
  image_id             = var.image_id
  system_disk_category = "cloud_essd"
  security_groups      = [alicloud_security_group.test_security_group.id]
  user_data = base64encode(<<-EOT
    #!/bin/sh
    yum install keepalived -y

    printf '! Configuration File for keepalived
    vrrp_instance VI_1 {
        state MASTER            # Set as the primary instance
        interface eth0          # Network interface for binding the VIP, configured as eth0 in this example  
        virtual_router_id 51    # virtual_router_id for the primary/secondary cluster; different primary/secondary clusters in the same VPC need different virtual_router_ids
        nopreempt               # Set non-preemption mode
        priority 100            # Set priority, higher number means higher priority; in this example, priority is set to 100 to make this instance the primary
        advert_int 1            # Heartbeat packet sending interval in seconds. If set too small, it may be affected by network jitter, causing frequent failovers and temporary dual-primary situations (split-brain). If set too large, it may lead to longer failover time when the primary instance fails.
        authentication {
            auth_type PASS
            auth_pass 1111
        }
        unicast_src_ip 192.168.0.25     # Private IP address of this instance, configured as 192.168.0.25 in this example
        unicast_peer {
            192.168.0.26                # Private IP address of the peer instance, configured as 192.168.0.26 in this example; if there are multiple secondary ECS instances, you need to declare the IPs of all peer instances. Each address on a separate line, without commas or other separators.
        }
        virtual_ipaddress {
            192.168.0.24                # Virtual IP address, configured as the HaVip address, which is 192.168.0.24 in this example
        }   
        garp_master_delay 1             # Delay in seconds before updating ARP cache after becoming the primary instance 
        garp_master_refresh 5           # Interval in seconds for sending ARP packets 

        track_interface {
            eth0                        # Network interface for binding the VIP, configured as eth0 in this example
        }
    }' > /etc/keepalived/keepalived.conf
    systemctl start keepalived
  EOT
  )                                           # Specify the initialization script for the primary server to install keepalived
  private_ip           = "192.168.0.25"       # Specify the private IP for the primary server
  instance_charge_type = "PostPaid"           # Specify the payment type as pay-as-you-go
  spot_strategy        = "SpotWithPriceLimit" # Set as a preemptible instance with price limit
}

# Create the secondary server 
resource "alicloud_instance" "test_backup_instance" {
  instance_name        = "test_backup_instance_name"
  vswitch_id           = var.vswitch_id
  instance_type        = var.instance_type
  image_id             = var.image_id
  system_disk_category = "cloud_essd"
  security_groups      = [alicloud_security_group.test_security_group.id]
  user_data = base64encode(<<-EOT
    #!/bin/sh
    yum install keepalived -y

    printf '! Configuration File for keepalived
    vrrp_instance VI_1 {
        state BACKUP            # Set as the secondary instance
        interface eth0          # Network interface for binding the VIP, configured as eth0 in this example  
        virtual_router_id 51    # virtual_router_id for the primary/secondary cluster; different primary/secondary clusters in the same VPC need different virtual_router_ids
        nopreempt               # Set non-preemption mode
        priority 10             # Set priority, higher number means higher priority; in this example, priority is set to 10 to make this instance the secondary
        advert_int 1            # Heartbeat packet sending interval in seconds. If set too small, it may be affected by network jitter, causing frequent failovers and temporary dual-primary situations (split-brain). If set too large, it may lead to longer failover time when the primary instance fails
        authentication {
            auth_type PASS
            auth_pass 1111
        }
        unicast_src_ip 192.168.0.26   # Private IP address of this instance, configured as 192.168.0.26 in this example
        unicast_peer {
            192.168.0.25          # Private IP address of the peer instance, configured as 192.168.0.25 in this example. You need to declare the IPs of all peer instances. Each address on a separate line, without commas or other separators.
        }
        virtual_ipaddress {
            192.168.0.24          # Virtual IP address, configured as the HaVip address, which is 192.168.0.24 in this example 
        }    
        garp_master_delay 1       # Delay in seconds before updating ARP cache after becoming the primary instance 
        garp_master_refresh 5     # Interval in seconds for sending ARP packets 

        track_interface {
            eth0                  # Network interface for binding the VIP, configured as eth0 in this example
        }
    }' > /etc/keepalived/keepalived.conf
    systemctl start keepalived
  EOT
  )                                           # Specify the initialization script for the secondary server to install keepalived
  private_ip           = "192.168.0.26"       # Specify the private IP for the secondary server 
  instance_charge_type = "PostPaid"           # Specify the payment type as pay-as-you-go
  spot_strategy        = "SpotWithPriceLimit" # Set as a preemptible instance with price limit
}

# Associate the primary server
resource "alicloud_havip_attachment" "test_havip_attachment" {
  ha_vip_id   = alicloud_havip.test_havip.id
  instance_id = alicloud_instance.test_master_instance.id # Specify the instance ID to associate with the HaVip
}

# Associate the secondary server
resource "alicloud_havip_attachment" "test_havip_attachment_new" {
  ha_vip_id   = alicloud_havip.test_havip.id
  instance_id = alicloud_instance.test_backup_instance.id # Specify the instance ID to associate with the HaVip
}

Verification

You can run the following commands on the primary and secondary instances to create a web test service that returns different results.
You can check port usage with netstat -an | grep 8000. If port 8000 is in use, you need to choose another port.
Primary instance:
```
echo "ECS 1" > index.html  # Primary instance returns "ECS 1"
python3 -m http.server 8000
```
Secondary instance:
```
echo "ECS 2" > index.html  # Secondary instance returns "ECS 2"
python3 -m http.server 8000
```
On another ECS instance in the same VPC, run curl <havip_private_ip>:8000, which will return ECS 1. When the primary server is stopped, it will return ECS 2.
Make sure that the security groups of the primary and secondary instances allow HTTP traffic from within the same VPC to access port 8000.

Associate an EIP for Internet access

HaVip is a private IP resource within a vSwitch. If Internet access is required, you can associate the HaVip with an elastic IP address (EIP). The use of EIPs will incur charges.

1. The EIP must be in the same region as the HaVip and in the Available state.

2. When an ECS instance accesses the Internet through an HaVip-associated EIP, the source IP address of the packets received by the Internet client is the HaVip address.

Console

Associate/disassociate an EIP

Before associating an EIP, make sure you have created one. You can create an EIP through the EIP console, or click Create EIP on the association page.

In the Actions column of the target HaVip, click Associate EIP or Disassociate EIP to complete the corresponding operation.

API

Before associating an EIP, make sure you have called AllocateEipAddress to create an EIP.

Call AssociateEipAddress to associate an HaVip with an EIP.
Call UnassociateEipAddress to disassociate an HaVip from an EIP.

Terraform

Resource: alicloud_eip_address, alicloud_eip_association

# Specify the region where the HaVip is located 
provider "alicloud" {
  region = "cn-hangzhou"
}

# Specify the HaVip ID
variable "havip_id" {
  default = "havip-8vb0******"  # Modify to the actual HaVip ID
}

# Create an EIP
resource "alicloud_eip_address" "test_eip" {
  address_name = "test_eip_name"
  isp          = "BGP"
  netmode      = "public"
  bandwidth    = "1"
  payment_type = "PayAsYouGo"
}

# Associate the EIP
resource "alicloud_eip_association" "test_eip_havip_association" {
  allocation_id = alicloud_eip_address.test_eip.id
  instance_type = "HAVIP"
  instance_id   = var.havip_id # Specify the HaVip ID
}

More information

Billing

The HaVip feature is in beta testing and can be used free of charge, but does not guarantee any service-level agreement (SLA) terms.

Supported regions

Area	Regions
Asia Pacific - China	China (Hangzhou), China (Shanghai), China (Nanjing - Local Region), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Chengdu), China (Hong Kong), China (Wuhan - Local Region), and China (Fuzhou - Local Region)
Asia Pacific - Others	Japan (Tokyo), South Korea (Seoul), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Philippines (Manila), and Thailand (Bangkok)
Europe & Americas	Germany (Frankfurt), UK (London), US (Silicon Valley), US (Virginia), and Mexico
Middle East	UAE (Dubai) and SAU (Riyadh - Partner Region) Important The SAU (Riyadh - Partner Region) region is operated by a partner.

Quotas

The HaVip feature is in beta testing. You can submit an application in the Alibaba Cloud Quota Center console.

Name/ID	Description	Default value	Adjustable
N/A	Network types that support HaVips	VPC	N/A
	Maximum number of HaVips that can be associated with each ECS instance	5
	Maximum number of EIPs that can be associated with each HaVip	1
	Maximum number of ECS instances or ENIs that can be associated with each HaVip	10 1. Each HaVip can be associated with up to 10 ECS instances or 10 ENIs. However, you cannot associate an HaVip with an ECS instance and an ENI at the same time. 2. HaVips can be associated with ECS instances or ENIs in the same vSwitch.
	Whether HaVips support broadcasting or multicasting	N/A HaVips only support unicast. If using third-party software like Keepalived for high availability, change the communication mode to unicast in the configuration file.
	Maximum number of HaVips that can be created with each Alibaba Cloud account	50
	Maximum number of HaVips that can be created in each route table	50
vpc_quota_havip_custom_route_entry	Maximum number of route entries that point to an HaVip in each VPC	5	You can increase the quota by performing the following operations: Go to the Quota Management page and submit a quota increase request. For more details, see Manage VPC quotas. Go to the Quota Center page and request an increase in quota. For more information, see Request a quota increase.