Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

Fastly GSLB: Scaling
your microservice and
multi-cloud
environments
Chris Buckley
Sales Engineer | Fastly

• Sales Engineer @ Fastly, Media & Publishing
• Previously Director of DevOps @ Business Insider
A little about me...

• Introduction to Load Balancing
• Workshop 1: Multi-Cloud Load Balancing
• Workshop 2: SOA/Microservice Routing
• Workshop 3: Geographic Load Balancing
Load Balancing Workshop Program

Introduction to Load
Balancing

As the name implies, load balancing is designed to distribute traffic between
multiple resources.
From Wikipedia:
“Load balancing aims to optimize resource use, maximize throughput,
minimize response time, and avoid overload of any single resource.”
Load Balancers, How Do They Work?

• Global Server Load Balancing (GSLB) is a method of
directing traffic across multiple networks, be it on the
same site, or on the other side of the world.
• GSLB can be used to load balance across multiple
networks, and primarily uses DNS.
Traditional GSLBs (Global Server Load Balancers)

Fastly GSLBs
● Fastly allows
customers to make
unlimited number of
custom rules
● Fastly software load
balances between
nodes in POP
● Fastly uses DNS to
load balance
between POPs
Browser
Recursive
Resolver
Authoritative
Name Server
Reverse HTTP
Proxy
Application
DNSL4L7
Fastly executes rules
TCP Handshake

Multi-Cloud/Multi-Datacenter Load Balancing
88
Infrastructure agnostic global and
local server load balancing
Unified approach to microservice
architecture

Infrastructure-Agnostic Distribution
● Manage traffic across multiple IaaS,
datacenters, and hybrid-clouds
● Use as GSLB and/or LSLB
● Instant convergence and failover
Content-Aware Routing
● Load balance requests to servers using an
any number of custom rules
● Support microservices architecture
● Choose from various distribution algorithms
to manage load across servers
Immediate Control
● Programmatically add/delete/modify your
servers without having to version your VCL
● Automatic server health checks
● Changes to custom rules update routing
configuration globally within seconds
Traffic Scalability
● Instantly scale to multiple terabits per
second (Tbps) in a manner that is both
cost-effective and transparent
● Mitigates thundering herd problem
Why Fastly Load Balancing?

• To ease usage of the API, set environment variables for API key and service
ID.
• Copy API key from the Github README.md, and your service ID from the
sheet given to you at the beginning of this workshop
• Assuming a Bash shell, run the following:
Initial Setup
export API_KEY=<api_key>
export SERVICE_ID=<service_id>

• We will be working with an API that returns JSON as its response.
• Make sure you are taking note of the responses!
• JQ can be used for making responses readable (link in README.md).
• If at any time you get version drift in this workshop, you can find out your
current active version and work from that using the following command:
General Workshop Tips
curl sv -H "Fastly-Key: ${API_KEY}"
https://api.fastly.com/service/${SERVICE_ID}/details | jq

You will receive a JSON response listing all versions. You will want to find the one
where "active": true:
General Workshop Tips cont...
{
"testing": false,
"locked": true,
"number": 10,
"active": true,
"service_id": "3SewbytL2TOibn8tO3OFrM",
"staging": false,
"created_at": "2017-06-16T01:07:30+00:00",
"deleted_at": null,
"comment": "",
"updated_at": "2017-06-16T01:14:14+00:00",
"deployed": false
},
You can then clone from your current active version and
work from there.

Workshop 1: Multi-cloud
Load Balancing

• Get familiar with the Fastly API
• Learn about Dynamic Server Pools
• Use a pool to add servers dynamically
• Create a multi-cloud load balancing configuration
Goals of Workshop 1: Multi-cloud Load Balancing

Goals of Workshop 1: Multi-cloud Load Balancing

• In order to begin adding Dynamic Server Pools, we will first need to clone
your service and create a new pool.
• In this case we will be cloning version 1 of your service to version 2:
Step 1: Cloning a new version
curl -sv -H "Fastly-Key: ${API_KEY}"
https://api.fastly.com/service/${SERVICE_ID}/version/1/clone
| jq

JSON response:
Step 1: Cloning a new version cont..
{
"testing": false,
"locked": false,
"number": 2,
"active": false,
"service_id": "1OPpKYvOlWVx37twfGVsq0",
"staging": false,
"created_at": "2017-06-16T21:10:09+00:00",
"deleted_at": null,
"comment": "",
"updated_at": "2017-06-16T21:10:11+00:00",
"deployed": false
}

In the repo there is included a main.vcl to upload to our service. Two things to
note:
1. There is a clearly defined space in vcl_recv where we will be doing our
custom VCL for this workshop.
2. Below you will see return(pass). For this workshop we are passing all traffic
to the backends so that we can see immediate responses (and not cached
responses).
Step 2: Upload boilerplate VCL for service

To upload VCL we must URL encode the file so that we can send it via Curl:
Step 2: Upload boilerplate VCL for service cont...
curl -vs -H "Fastly-Key: ${API_KEY}" -X POST -H
"Content-Type: application/x-www-form-urlencoded"
--data "name=main&main=true"
--data-urlencode "content@main.vcl"
https://api.fastly.com/service/${SERVICE_ID}/version/2/vcl | jq

JSON response:
Step 2: Upload boilerplate VCL for service cont...
{
"name": "main",
"main": true,
"content": "<lots of VCL>,
"version": 2,
"deleted_at": null,
"created_at": "2017-06-20T20:23:52+00:00",
"updated_at": "2017-06-20T20:23:52+00:00"
}

Next, we will need to create a Dynamic Server Pool to add our servers to (note we
are now working with version 2):
Step 3: Create Dynamic Server Pool
curl -sv -H "Fastly-Key: ${API_KEY}" -X POST
https://api.fastly.com/service/${SERVICE_ID}/version/2/pool -d
'name=cloudpool&comment=cloudpool' | jq

JSON response:
Step 3: Create Dynamic Server Pool cont...
{
"name": "cloudpool",
"comment": "cloudpool",
"service_id": "5wFmyFopB74kQoYtrxY3tw",
"version": "2",
"max_tls_version": null,
"shield": null,
"tls_ca_cert": null,
"request_condition": null,
"first_byte_timeout": "15000",
"tls_ciphers": null,
"tls_sni_hostname": null,
"updated_at": "2017-06-21T20:37:51+00:00",
"id": "3qoucl7vkyLGf2KXmJ1XIK",
…
}

Let’s add the pool ID as an environment variable...
export POOL_ID_1=3qoucl7vkyLGf2KXmJ1XIK

We can now begin to start adding servers to the pool (use the IPs listed above to
add the servers)
You will run this command twice with the different IP addresses:
Step 4: Add servers to the pool
curl -vs -H "Fastly-Key: ${API_KEY}" -X POST
https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_1}/server -d
'address=X.X.X.X' | jq

JSON response:
Step 4: Add servers to the pool cont...
{
"address": "13.58.97.100",
"pool_id": "49Y2yUtPUukwoGu9KDxFM",
"deleted_at": null,
"port": "80",
"max_conn": 0,
"created_at": "2017-06-20T20:55:33+00:00",
"comment": "",
"weight": "100",
"updated_at": "2017-06-20T20:55:33+00:00",
"id": "6G190tLMQZ9QDE2jRcLkwA"
}

Our last configuration step. Now we have added our pool, activate the version.
Once activated, dynamic servers are not tied to a version and can be added
and removed dynamically:
Step 5: Activate new version
curl -vs -H "Fastly-Key: ${API_KEY}" -X PUT
https://api.fastly.com/service/${SERVICE_ID}/version/2/activate |
jq

JSON response:
Step 5: Activate new version cont...
{
"testing": false,
"locked": true,
"number": 2,
"active": true,
"staging": false,
"created_at": "2017-06-20T20:09:40+00:00",
"deleted_at": null,
"comment": "",
"updated_at": "2017-06-20T21:03:44+00:00",
"deployed": false,
"msg": "WARNING: Unused backend 'dummy'"
}
You can safely ignore the dummy WARNING, it is an unused pool to activate your initial service for
this workshop.

Open your browser and navigate to your supplied domain:
<num>.lbworkshop.tech
You should now be able to see something like this:
Step 6: Browse the new load balanced pool
GCE serving the request EC2 serving the request

Workshop 2: SOA /
Microservice Routing

• Get familiar with Dynamic Server Pools & TLS
• Define a routing condition for sending to your
microservice Server Pool.
• Create two different Server Pools for routing to
different independently scalable load balancers.
Goals of Workshop 2: SOA/Microservice Routing

Goals of Workshop 2: SOA/Microservice Routing

• Clone the active service to a new development version (this time cloning
version 2 to version 3):
Step 1: Create Dynamic Server Pool
curl -sv -H "Fastly-Key: ${API_KEY}" -X PUT
https://api.fastly.com/service/${SERVICE_ID}/version/2/clone | jq
{
"testing": false,
"locked": false,
"number": 3,
....
}
● JSON response:

• Add an environment variable for the blog hostname (for TLS)::
export BLOG_HOST=blog.lbworkshop.tech
● Create our Server Pool (with TLS data):
"name=blog&comment=blog&use_tls=1&tls_cert_hostname=${BLOG_HOST}|
jq

• JSON response:
{
"name": "wordpress",
"comment": "wordpress",
"use_tls": 1,
"tls_cert_hostname": "blog.lbworkshop.tech",
"version": "3",
"id": "5sufVWoTHlOAOYVRm5jg2G",
…
}
● Export the new pool ID (for use later):
export POOL_ID_BLOG=5sufVWoTHlOAOYVRm5jg2G

• In your repo you will find a file wordpress.vcl.
• This contains VCL to send any requests on the blog route (in this case /blog) to the new created
created Wordpress load balancer.
• Let's add this to main.vcl underneath our workshop 1 set req.backend = cloudpool;. It should
look something more like this, ready to upload:
Step 2: Add conditional routing for blog pool
main.vcl before:
###########################################
# LB WORKSHOP - DO STUFF HERE
###########################################
# Workshop 1 default backend.
set req.backend = cloudpool;
###########################################
# END LB WORKSHOP
###########################################

main.vcl after:
Step 2: Add conditional routing for blog pool cont...
# Adding conditional routing to our Wordpress service
if (req.url == "/blog") {
set req.backend = wordpress;
# We will also need to change the host header and modify the request url
# so that we hit the right endpoint.
set req.http.Host = "blog.lbworkshop.tech";
# Remove the /blog from the request URL so that we hit the root of the service
# using regular expressions
set req.url = regsub(req.url, "^/blog", "/");
}

• Upload this as main-blog so to use the new configuration and backend for
the blog:
curl -vs -H "Fastly-Key: ${API_KEY}" -X POST -H "Content-Type:
application/x-www-form-urlencoded" --data
"name=main-blog&main=true" --data-urlencode "content@main.vcl"
https://api.fastly.com/service/${SERVICE_ID}/version/3/vcl | jq

• JSON response:
{
"name": "main-blog",
"main": true,
"content": <lots of vcl>,
"version": 3,
"deleted_at": null,
"created_at": "2017-06-21T05:04:54+00:00",
"updated_at": "2017-06-21T05:04:54+00:00"
}

• Add the Wordpress TLS endpoint to the blog pool (using the previously
added POOL_ID_BLOG environment variable):
Step 3: Add TLS backend to Server Pool
https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_BLOG}/server -d
"address=${BLOG_HOST}&comment=wp&port=443" | jq

• JSON response:
Step 3: Add TLS backend to Server Pool cont...
{
"address": "blog.lbworkshop.tech",
"comment": "wp",
"pool_id": "5sufVWoTHlOAOYVRm5jg2G",
"deleted_at": null,
"port": "443",
"max_conn": 0,
"created_at": "2017-06-21T05:11:18+00:00",
"weight": "100",
"updated_at": "2017-06-21T05:11:18+00:00",
"id": "RzWDZMrMIV1FiHYUY4k5k"
}

• Activate new version (ensuring we’re working with version 3):
Step 3: Add TLS backend to Server Pool cont...
https://api.fastly.com/service/${SERVICE_ID}/version/3/activate | jq

• Navigate to the new endpoint and see the blog in all its glory (replace the
boilerplate URL with your custom service URL):
http://<num>.lbworkshop.tech/blog
Step 4: Check out your new blog!

Workshop 3: Geographic
Dynamic Server Pools

• Use geographic conditions to route to the closest origin
load balancing pool.
• Learn about failover logic in the case of origins being
unavailable.
• Define a fault tolerant configuration built for speed and
redundancy.
Goals of Workshop 3: Geographic Dynamic Server
Pools

• As we will be introducing code to fail over to a secondary origin (in the case
of the primary origin failing or being unavailable), we will need to add health
checks to our service.
• Set a simple check to test the index of the site, with default health check
settings.
• First, clone current version (this time to version 4):
Step 1: Create origin health check
curl -sv -H "Fastly-Key: ${API_KEY}" -X PUT
https://api.fastly.com/service/${SERVICE_ID}/version/3/clone | jq

• Add health check to the new version. As the health check will be done over
HTTP/1.1 we will also add a host header in the check:
Step 1: Create origin health check cont...
https://api.fastly.com/service/${SERVICE_ID}/version/4/healthcheck -d
"name=geo-healthcheck&host=lbworkshop.tech&path=/" | jq

JSON response:
Step 1: Create origin health check cont...
{
"name": "geo-healthcheck",
"path": "/",
"version": 4,
"threshold": 3,
"window": 5,
"http_version": "1.1",
"timeout": 500,
"method": "HEAD",
"updated_at": "2017-06-16T19:05:02+00:00",
"expected_response": 200,
"deleted_at": null,
"host": "lbworkshop.tech",
…
}

• Create 2 server pools. One called "west" and one "east".
• Attach the health check created previously to each pool.
• East first:
Step 2: Create Geographic Dynamic Server Pools
'name=east&comment=east&healthcheck=geo-healthcheck' | jq
● Add the pool ID as an environment variable:
export POOL_ID_EAST=<pool_id>

• And then west:
cont...
'name=west&comment=west&healthcheck=geo-healthcheck' | jq
● Add environment variable:
export POOL_ID_WEST=<pool_id>

• Add two instances from workshop 1 into separate pools; the GCE instance
into west, and the EC2 instance into east (run twice, with the different pool
IDs and the instance for each pool):
cont...
https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_EAST}/server -d
'address=13.58.97.100&comment=ec2' | jq`
https://api.fastly.com/service/${SERVICE_ID}/pool/${POOL_ID_WEST}/server -d
'address=104.196.253.201&comment=gcs' | jq

• In your repo you will find a file geo.vcl. This contains the logic for backend
selection based on geography, with failover to the secondary origin in case
the original pool is unavailable (based on the health checks).
• Edit the main.vcl file, and replace the following code:
Step 3: Adding VCL for backend selection

• Replace it with the new geo.vcl code:
Step 3: Adding VCL for backend selection cont...
# APAC, Asia, and US West all go to US West backend.
if (server.region ~ "^(US-West|APAC|Asia)") {
set req.backend = west;
# All other regions default to US East
} else {
set req.backend = east;
}
# West failover to East
# from unhealthy backend or from restart because of backend status code
if(req.backend == west && (!req.backend.healthy || req.restarts > 0)) {
set req.backend = east;
# East failover to West
} else if(req.backend == east && (!req.backend.healthy || req.restarts > 0)) {
set req.backend = west;
}

• Upload main.vcl as an update to development version (we have given the
new VCL a new name to distinguish from the old, so that we can set this as
the new "main":
curl -vs -H "Fastly-Key: ${API_KEY}" -X POST -H
"Content-Type: application/x-www-form-urlencoded"
https://api.fastly.com//service/${SERVICE_ID}/version/4/vcl --data
"name=main-blog-geo&main=true" --data-urlencode "content@main.vcl"

• Activate our new version and see the new Geo Load Balancing in effect:
https://api.fastly.com/service/${SERVICE_ID}/version/4/activate
● As this workshop is being held in San Francisco, you should be see the GCE
instance. After Apache is shut down on the GCE us-west1 instance, you
should see failover to our EC2 instance in us-east2.

• Workshop GitHub Repository: http://bit.ly/lbworkshop
• API documentation: https://docs.fastly.com/api/
• Dynamic servers documentation: https://docs.fastly.com/guides/dynamic-servers/
• Working with services: https://docs.fastly.com/api/config#service
• Working with service versions: https://docs.fastly.com/api/config#version
• Conditions documentation: https://docs.fastly.com/guides/conditions/
• Geo related features: https://docs.fastly.com/guides/vcl/geoip-related-vcl-features
• Creating health checks via the API: https://docs.fastly.com/api/config#healthcheck
Further Reading

Personal Contact Information
• Name: Chris Buckley
• Email: cbuckley@fastly.com
• Twitter: @ChrisBuckleySA
Contact Information
Fastly Contact Information
• Email: support@fastly.com
• IRC: #fastly on Freenode
• Community: https://community.fastly.com/

Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments

More Related Content

What's hot (20)

Similar to Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments (20)

More from Fastly (20)

Recently uploaded (20)

Altitude SF 2017: Fastly GSLB: Scaling your microservice and multi-cloud environments