Introduction

The default load balancing tool on Engine Yard Cloud is HAProxy running on the Application Master instance. If advanced load balancing is required Amazon Load Balancers can be deployed, at additional cost.

Deploying an Amazon Load Balancer into your Engine Yard environment(s) allows you to make use of Amazon's Elastic Load Balancing service with your AWS environments.

Application Load Balancers are Amazon's next generation load balancers, offering improved functionality and configurability over the former Classic Load Balancers (previously referred to as Elastic Load Balancers). For a comparison of Application and Classic Load Balancers, please see this blog post.

Application Load Balancing at its most basic provides the same functionality as Classic Load Balancing, in that it distributes requests to multiple instances (servers) across multiple AWS availability zones (AZs) in order to distribute traffic evenly across all instances, keeping the instances as evenly loaded as possible. Application Load Balancers exist in all AZs where attached instances reside, so if an entire AZ goes offline the ALB continues to function, routing traffic to instances in the remaining AZs. The health of instances registered with the load balancer is also monitored and requests are sent only to the healthy instances. If an instance becomes unhealthy, the ALB stops sending traffic to that instance and spreads the traffic across the remaining healthy instances.

Where Application Load Balancing expands upon Classic Load Balancing is with the functionality for request URL path or host routing, allowing you to direct requests hitting the ALB to specific instances, either in the same EY Environment or a different Environment.

Requirements

To make use of an ALB:

The AWS region must have a VPC available, either a default-vpc (the case for most newly created accounts and regions) or a custom VPC (or a Network in EY parlance).
The instances that are to have traffic routed to by the ALB must also be attached to the same VPC/Network.
Instances must be VPC-native, traffic cannot be routed to ClassicLinked instances.

This document covers the following topics:

Deploying an ALB
Testing an ALB
Setting an ALB live
Verifying an ALB
Monitoring an ALB
Moving an ALB

Deploying an ALB

Target Groups

A Target Group is a set of instances to which an ALB can route all, or specific, traffic to. Before an ALB can be configured, Target Groups must be created. Within the Engine Yard Platform the instances in a Target Group must reside within the same environment and that environment be assigned to the same VPC as the ALB sending the traffic. However, a single ALB can route traffic to multiple Target Groups in different environments.

The Environment page of the dashboard now offers a Target Groups page under More Options, from where your Target Groups can be created.

Note: Target Groups do not support ClassicLinked EC2-Classic instances, therefore the Target Group creation functionality is disabled on Environments linked to Networks with ClassicLink enabled. If you see the functionality disabled please contact Support in order for them to review the ClassicLink status.

Once a Target Group has been created it is then available to be set as a Service for an ALB.

If the environment runs a single application then only a single Target Group is required (which is recommended to be a Dynamic Spread group), whilst if an environment runs multiple applications, or has specific path routing requirements, then multiple Target Groups can be used to achieve the required traffic routing.

Note: When using multiple Target Groups for a single ALB, custom Services must be configured on the ALB to route traffic as required, please see the Custom Routing section of this document for further details.

Instance Association Strategy

Target Groups can have one of two Instance Association Strategies, which define how the environment instances are associated with the Target Group(s).

Note: It is not possible to change the Instance Association Strategy of a Target Group after it has been created.

Dynamic Spread - with a dynamic group the platform will automatically associate instances in the environment with the Target Group in order to maintain the group size. The size is configured via two settings, the Target Size and the Minimum Size, both of which can be configured as either a precise number of instances, or as a percentage of all instances in the environment.
- In the case that a single application is running in an environment and all requests should hit all instances, then just a single Target Group is required and both Target Size and Minimum Size values can be set to 100%.
- In the case of multiple applications under a single environment then the multiple Target Groups can be created and the Size values scaled. Given the example of two applications running under a single environment that requires the two applications' traffic to be routed at a ratio of 3:1, setting the Minimum Size of 3 instances and a Target Size of 75% for one Target Group and a Minimum Size of 1 instance and a Target Size of 25% would achieve this. Instances may overlap and be used for multiple applications should the minimum or target sizes require it.
- Another case would be that of an application where specific requests (e.g. long running ones) are directed to distinct instances. Again, the Target Size and Minimum Size values would be set to determine the amount of instances in each Target Group.
- When using Dynamic Groups any newly booted instances will be automatically assigned to the Target Group(s) in line with the rules.
Static - with a static group the member instances are selected directly, thus allowing to you specify exactly which instances in an environment belong to each Target Group.
- With Static Groups any newly booted instances will require manually assigning to the Target Group(s) by editing the Target Group after the instance has been created, otherwise they will receive no traffic.

Other Target Group Settings

There are several other configuration options for Target Groups, and each are described on the Target Group creation page, but the most important are:

Internal Protocol - this is the protocol used to pass traffic from the ALB to the instances. Any HTTPS ALB listener will handle SSL termination at the ALB layer, requests can then sent from the ALB to the instances as either plain HTTP (still inherently secure due to the security of the VPC) or HTTPS should your own security or compliance require it. To use the HTTPS Internal Protocol requires an SSL to be assigned to the environment through the Cloud Dashboard, such that Nginx on the instances is configured to handle HTTPS requests.
Internal Port - this is the port running Nginx on the instances and is dependent on the EY Stack and Internal Protocol in use:
- Solo environment, Stable-v4 and HTTP: Port 80
- Solo environment, Stable-v4 and HTTPS: Port 443
- Multi-instance environment, Stable-v4 and HTTP: Port 81
- Multi-instance environment, Stable-v4 and HTTPS: Port 444
- Multi-instance or Solo, Stable-v5 and HTTP: Port 8081
- Multi-instance or Solo, Stable-v5 and HTTPS: Port 8082
- Note: It is not possible to change the Internal Protocol or Port of a Target Group after it has been created.
Health Check Path - an application path to check the health of the instances. Any instances failing the health check will be removed from the ALB pool until the check passes again. This path defaults to / but is best advised to be updated to a path that utilises all application components (e.g. an application page which makes a database call, rather than a plain html page) such that it is a holistic check. It should be noted that unlike with Classic Load Balancers, should all instances fail the health check then they will all be added to the pool. In such a case the application may be served correctly, but any failing instances would not be detected by the health check and not removed from the pool. Please check all Health Check values and request EY Support confirm instances' health check status if you are unsure.
Health Check Response Code - by default the health check requires a 200 response to be successful. Should that not be possible, due to application redirects for example, additional codes can be set as valid. This is advised against though, as requiring a 200 response gives the best guarantee of application issues being detected by the health check.
Health Check Protocol and Port - this is the protocol used to check the Health Check Path. This defaults to HTTP and the same port as set for the Internal Port. However due to the default response code requiring to be 200, any application with built-in HTTPS redirection will find health check requests failing due to the redirection, unless the health check path is specifically excluded from the redirection. It is therefore required to either add the redirection code (301, 302) to the allowed response codes, or preferably change the Health Check Protocol to HTTPS and Port to 8082 to avoid redirection (as with the HTTPS Internal Protocol this requires an SSL to be assigned at the Environment level).

Editing Target Groups

Editing Dynamic Spread Target Groups allows for the alteration of the group sizings, whilst editing Static Target Groups allows for the addition and removal of instances from the group. Both groups can have their health check settings updated, but it is not possible to change the Instance Association Strategy or the Internal Protocol or Port of an existing Target Group and a new one must be created.

Deleting Target Groups

Target Groups can only be deleted if they are not associated with any ALB Services. If this is the case the Delete button will be greyed out.

Application Load Balancer Creation

To deploy an Application Load Balancer:

Select Tools from the main title bar menu, then App Load Balancers.
Click the + Add Application Load Balancer button.
Enter a unique name for the ALB to identify it.
Select the AWS Region the ALB should be deployed in.
Select the Network the ALB should be deployed in - this Network must match the VPC/Network attached to the environment(s) in which the desired Target Group(s) exist.
Select the External Protocol and Port for the ALB Endpoint to listen on. If you select HTTPS then a valid SSL must be assigned (created via the Tools -> SSL Certificates page). The Security Policy can also be changed should you require to customise the available TLS versions and Ciphers in use. If you require multiple certificates for the Endpoint (SNI) please contact Support at this time.
Enter a name for the Default Service or accept the standard one.
Select the Target Group that this Default Endpoint will route all traffic to.
Click Create Application Load Balancer.
Refresh the page to see the creation progress.
During creation only a single Endpoint and Service can be created, so if you require a second Endpoint please wait for the ALB to provision, then click the name to expand the details and click + Add Endpoint and + Add Service to add the additional ones.

Endpoints and Services

An Endpoint is a listener on the ALB and accepts traffic on the set protocol and port. Each Endpoint must then have a Default Service, which is the Target Group that all traffic reaching that Endpoint is routed to, unless additional Services are added to route traffic to other Target Groups.

A Service is a rule acting upon this Endpoint that routes the traffic to the required Target Group. As above, each Endpoint has a Default Service, which routes all traffic to a particular Target Group, then further Services can be added to route specific traffic to other Target Groups, dependent on the hostname and path used in making the requests.

Adding Additional Endpoints & Services

Note: At this time only one Endpoint per external port can be created and the Engine Yard Platform does not yet reflect Amazon's support of SNI for multiple SSLs per listener, so if you require multiple SSLs then multiple ALBs are required (which can be pointed at the same Target Groups as required). A recommended solution would be to make use of a wildcard or SAN SSL to cover all domains with a single SSL and thus single ALB.

To add a new Endpoint expand the ALB by clicking on it in the ALB list, click + Add Endpoint and add the Endpoint as required. Refresh the page to see your changes.

To add a new Service expand the ALB by clicking on it in the ALB list and click + Add Service. If there is no existing Service using the Endpoint you can create a new Default Service by setting Host Pattern to *, otherwise a Custom Routing rule must be used.

Custom Routing

As previously stated, the Endpoints will send all traffic to the Target Group selected as Default when creating the ALB and/or Endpoint. To route traffic to other Target Groups requires the creation of additional custom Services, which route based on hostname or path of the requests.

In the case that there are multiple applications' traffic directed to the ALB, then to route each application to the correct instances (either within a single environment or across multiple environments on the same network), then (once the Target Groups have been created) Host pattern based routing can be used, for example:

Two applications, responding on cloud.engineyard.com and api.engineyard.com are hosted in either a single environment, or two separate environments on the same Network.
There are two Target Groups, called cloud and api, with the required instances assigned.
The ALB is configured with an Endpoint whose Default Service points to the cloud Target Group.
A custom Service is then created, making use of the same Endpoint, but with the Host pattern set to api.engineyard.com and Target Group set to api.
Requests to api.engineyard.com are then routed to the instances in the api Target Group, whilst all other requests are routed to the instances in the cloud Target Group.
Further Services can then be added for other domains as required.
The Service priority decides the order in which the rules are evaluated, so should be incremented/adjusted as required to ensure any overlapping rules are handled as required.

In the case that any application requires certain requests to be routed to different instances, then (once the Target Groups have been created) Path pattern based routing can be used, for example:

An application requires all uploads to be handled via dedicated instances.
This environment has two Target Groups, called general and uploads.
The ALB is configured with an Endpoint whose Default Service points to the general Target Group.
A custom Service is then created, making use of the same Endpoint, but with the Path pattern set to /uploads/* and Target Group set to uploads.
Requests to /uploads are then routed to the instances in the uploads Target Group, whilst all other requests are routed to the instances in the general Target Group.
The Path pattern can be combined with Host pattern if required for multiple application ALBs.
Further Services can then be added for other paths as required.
The Service priority decides the order in which the rules are evaluated, so should be incremented/adjusted as required to ensure any overlapping rules are handled as required.

Editing Existing Endpoints & Services

To edit the Protocol, Port, SSL Certificate, Security Policy, Name or Default Target Group for an existing Endpoint expand the ALB by clicking on it in the ALB list and click Edit for the relevant Endpoint, make the change and click Update Load Balancer Endpoint. Refresh the page to see your changes.

To edit the Host Pattern, Path Pattern or Target Group for a custom Service expand the ALB by clicking on it in the ALB list and click Edit for the relevant Service, make the change and click Update Load Balancer Service. Refresh the page to see your changes.

Editing ALB Settings

Clicked the Edit button next to an ELB allows you to the idle timeout value for the ALB and to toggle Deletion Protection.

Deleting ALBs

Before attempting to delete an ALB please ensure that no DNS records remain pointing to it and that Deletion Protection is off via the Edit menu. Target Groups do not need to be dissociated or deleted. Then use the Delete button next to the ALB to delete it.

Testing an ALB

Before directing production traffic to an ALB it is recommended to test it:

Find the relevant ALB on the App Load Balancer page and expand it.
Copy the Hostname shown.
Paste this URL into your browser.
If your site redirects to HTTPS you will see a certificate mismatch error, but accept this to view and test your application.

As previously mentioned, if all the instances in a Target Group fail the Health Check then traffic is still directed to them, rather than all being pulled from the pool and the ALB returning an error. That being the case, now is a good time to ensure the validity of your Health Check. This can be done via the AWS Console as per the Monitoring an ALB section.

Setting an ALB live

To set an ALB live you need to modify the DNS records for your domain(s). By default DNS for an application would be configured as an A record pointing to the EIP attached to the Application Master instance. Any A records pointing to the EIP should be replaced with CNAME records pointing to the Hostname of the ALB, which can be found by expanding out the relevant ALB on the App Load Balancers page.

Important: It should be noted that CNAME records are only allowed to be configured against FQDNs, e.g. www.engineyard.com. If you make use of the root domain, e.g engineyard.com, then you must either keep this record as an A record pointing to the EIP (and therefore have the traffic bypass the ALB) or preferably setup an ALIAS record to point the root domain to the ALB's hostname. Not all DNS providers support ALIAS records, so please check with your DNS provider if you have any questions about setting this up.

Verifying an ALB

To verify that after being updated the DNS correctly points to the ALB you can do the following:

Open a UNIX shell.
For each record you pointed to the ALB type:
```
dig your.customdomainname.com +short
```
Depending on your DNS configuration, it may return the hostname for the load balancer. For example:
```
EYalb-569421495.us-east-1.elb.amazonaws.com
```
Or it may return a list of IP addresses. In this case run a dig on the load balancer hostname:
```
dig EYalb-569421495.us-east-1.elb.amazonaws.com +short
```
This will also return a list of IP addresses. The two lists should contain the same IPs.
If the address is still the Environment's EIP you might need to wait for DNS propagation; try again after a few minutes or contact your domain name provider.

Monitoring an ALB

You can monitor the state of your Application Load Balancer via the AWS Console. On the App Load Balancers page, expand out the relevant load balancer and click the View this load balancer in the AWS console link. This will provide limited access to the AWS Console, allowing you to review the status of the ALB and Target Groups. Note: No changes can be made here and all configuration updates must be made via the EY Dashboard.

There are two relevant sections of the AWS Console and you can switch between them via the Load Balancing section in the left-hand pane menu. Within these sections the relevant information can be found under:

Load Balancers:
- Monitoring tab - This shows you graphical representations of the CloudWatch statistics for the load balancer, giving you insights into the traffic levels and errors.
Target Groups:
- Targets tab - This will list the instances and availability zones assigned to the Target Group. Each instance has a Status field, with any unhealthy instances stating the reason why. If all instances are unhealthy then the load balancer will revert to routing requests to all instances in the Target Group, so it will appear to work, but the Health Check functionality will be void and so requests still routed to any failing instances resulting in application errors being returned to visitors. You should therefore ensure your instances pass the Health Check. In most cases any Health Check failure is caused by a misconfigured Health Check, so please consult Support should you require any assistance.
- Monitoring tab - This shows you graphical representations of the CloudWatch statistics for the Target Group, giving you insights into the traffic levels, errors and instance failures.

Moving an ALB

If you wish to 'move' an ALB, such that it sends traffic to (a) different Environment(s), then the relevant Endpoints should be updated to new Target Groups associated with the other Environment(s). To do this:

Follow the instructions to create Target Groups for the other Environment.
Follow the instructions to edit the existing Endpoints and update the selected Target Group to the new ones.

Using Amazon Application Load Balancers