Building ‘Bootiful’ microservices cloud

Building ‘Bootiful’
microservices cloud
Powered by Spring Cloud Netflix libraries
Idan Fridman, Tikal

Agenda
● Microservices - Quick overview
● Implementing microservices - Build your endpoints with Spring-Boot
● Spring Cloud and Netflix
● Service Discovery: Eureka instances
● Circuit Breaker: Hystrix
● Client Side Load Balancer: Ribbon
● Router and Filter: Automatic registration of Zuul
● RxJava: Aggregation of multiple micro-service requests
● Declarative REST Client: Feign

● There is a strong trend in distributed systems with lightweight
architectures
● People have started to call them "microservices"

What are microservices?
(In short, I promise;P)

● Smaller units of a larger system
small enough to be owned by a small agile development team
re-writable within one or two agile sprints ( typically two to four
weeks)
The complexity does not require to refactoring or require further
divide into another microservice.
● Not monolithic
● Runs in its own process
Running independently of the application If fail app
won't fail and deliver workarounds
Life continues as if nothing has happened.
● Lightweight communication protocols
Rely on HTTP(Rest, etc..) or a lightweight
message bus
Establish a boundary between components
Ensures loose coupling.
● Single responsibility principle(SRP)
Should perform a single business function
Must be functionally complete
Also knowns as bounded context from
Domain-Driven Design
● Independent data source

Soa vs Microservices
- SOA and microservices don’t conflict or replace each other -> they
complement
- SOA is about how to reuse existing functionality as services
- Microservices is about how to make functionality to scale better with high
resilience and short release cycles...

Challenges with Microservices
● Distributed/versioned configuration
○ Auto configurations and refresh on runtime
● New services can auto-register at startup
○ Service registration and discovery
● Routing and load balancing
○ Clients can detect new instances as they are started up
● Centralized log management
○ Collects and visualize log events from distributed processes
● Circuit Breaker
○ Prevent problems with chain of failures
● Security

Agenda
● Router and Filter: automatic registration of Zuul

Build your endpoints with Spring-
Boot
It can get pretty much small:
@RestController
class ThisWillActuallyRun {
@RequestMapping("/")
String home() {
"Hello World!"
}
}
Pretty much micro?

Spring Boot
● A tool for getting started very quickly with Spring
● Common non-functional requirements for a "real" application
● Exposes a lot of useful features by default
● Gets out of the way quickly if you want to change defaults
● Available for web, batch, integration, data, amqp, aop, jdbc, ...

Spring Boot Modules
● Spring Boot - main library supporting the other parts of Spring Boot
● Spring Boot Autoconfigure - single @EnableAutoConfiguration annotation
creates a whole Spring context
● Spring Boot Starters - a set of convenient dependency descriptors that you
can include in your application.

Spring Boot Modules
● Spring Boot Actuator - common non-functional features that make an app
instantly deployable and supportable in production
● Spring Boot Tools - for building and executing self-contained JAR and
WAR archives
● Spring Boot Samples - a wide range of sample apps

Spring cloud - Netflix
● Spring Cloud Netflix provides Netflix OSS integration
● With simple annotations enable and configure can build large distributed
systems with battle-tested Netflix components.
● The patterns provided include Service Discovery (Eureka), Circuit Breaker
(Hystrix), Intelligent Routing (Zuul) and Client Side Load Balancing
(Ribbon)..

Service discovery Eureka
Discover

Motivation
● When you’re running your software across a large number of replaceable pieces of hardware, it’
s inevitable that one of them will fail at some point.
● In a cloud environment you are guaranteed failure, and it is absolutely
● Critical to have a service discovery system that can survive failures.

Eureka features
● Eureka has a built-in concept of service heartbeats to prevent stale data - if a service doesn’t phone home often enough,
then Eureka will remove the entry from the service registry
● HeartBeat - New services can register, but “dead” ones will be kept, just in case a client might still be able to contact them
● Cache - Eureka caches on the client side. So even if every last Eureka server goes down, or there is a partition where a
client can’t talk to any of the Eureka servers, then the service registry still won’t be lost.
● HA - The server can be configured and deployed to be highly available, with each server replicating state about the
registered services to the others.

Building ‘Bootiful’ microservices cloud

Agenda
● Netflix Turbine

What is the circuit breaker pattern?
● Communication with microservices(or any other services)
○ Remote calls
○ Different machines across a network
○ API calls and dependencies
● Unresponsive supplier
● Hang without a response
● Critical resources leading to cascading failures across systems
Failure risks

How it works?
Hystrix uses the bulkhead pattern to isolate dependencies and limit concurrent access.

The idea behind circuit breaker
1. Monitor For failures
2. Failure occur -> Circuit is open -> Protected call wont be made -> Do something else + raise alert
3. Need external intervention to reset it when things are well again
Not all errors should trip the circuit, some should reflect normal failures and be dealt regularly
Remote function
call
Circuit breaker object
wrapper

Circuit breaker monitoring
● Any change in breaker state should be logged
● Breakers should reveal details of their state for deeper monitoring

What’s next after the break?
● You need to consider what to do in case of failure.
○ perhaps business decision?
○ Workarounds?
○ Show partials results?
Examples:
1. A bank action authorization could be put on a queue to deal with later.
2. Show partial content (better than nothing right?)

What’s Hystrix?
● Hystrix evolved out of resilience engineering work that the Netflix API team began in 2011
● Isolating points of access between the services
● Handling fallbacks and protect API calls by stopping them(defining thresholds, etc..)
● Provide temporary fallback exchanges
● Monitoring dashboard to watch our our critical access points
Taken from spring.io:

HystrixCommand
● Wrap calls to dependencies in a HystrixCommand object
● HystrixCommand follows the command pattern and typically executes within a separate thread
● Hystrix maintains a thread-pool (or semaphore) for each dependency and rejects requests
(instead of queuing requests)
● if the thread-pool becomes exhausted -> It provides circuit-breaker functionality that can stop all
requests to a dependency.

Spring cloud for the rescue
● Using spring-cloud-netflix library we can implement circuit breaker logic
easily:
○ Hystrix Synchronous
○ Hystrix Asynchronous
○ Hystrix Observable

Hystrix synchronous
fallbackMethod
@HystrixCommand(fallbackMethod = "fallbackInvocation")
private void generateService(ObjectInput input) {
//invoke your service
}
public void fallbackInvocation(ObjectInput input) {
//keepDefaultvalues
//raise alert
//getNextServer from Eureka and invoke the method
//return data from a local cache
}

Hystrix asynchronous
Using future
@HystrixCommand(fallbackMethod = "stubbedBro")
public Future<String> runFutureBro(final String name) {
return new AsyncResult<String>() {
public String invoke() {
return(executeRemoteService(name));
}
};}
//and somewhere else:
this.runFutureBro(name).get();

Using subscriptions
public Observable<String> executeObservableBro(String name) {
return new ObservableResult<String>() {
@Override
return executeRemoteService(name);
}
};
}

Using subscriptions
broConsumerService.executeObservableBro(input).subscribe(new Observer<String>() {
@Override
public void onCompleted() {
System.out.println("completed");
}
@Override
public void onError(Throwable e) {
System.out.printf(e.getMessage());
}
@Override
public void onNext(String s) {
System.out.println("on next.." + s);
}
});

Command properties
@HystrixCommand(commandProperties = {
//execute on local thread or separate thread(Thread,Semaphore)
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "500")
//sets the time in milliseconds after which the caller will observe a timeout and walk away from the command
//execution (default 1000)
},
threadPoolProperties = {
@HystrixProperty(name = "coreSize", value = "30"),
//maximum number of HystrixCommands that can execute concurrently
@HystrixProperty(name="hystrix.command.default.circuitBreaker.requestVolumeThreshold",
value = "30"),
//sets the minimum number of requests in a rolling window that will trip the circuit.
//maximum queue size
@HystrixProperty(name = "metrics.rollingStats.numBuckets", value = "12"),
@HystrixProperty(name = "metrics.rollingStats.timeInMilliseconds", value = "1440")
})

Drawbacks
● Each command execution involves:
○ Queueing
○ Scheduling
○ Context switching involved in running a command on a separate thread.
● Netflix’s experiences proved it minor enough to not have major cost or performance impact.

Demo - 2
Protected with Hystrix

Hystrix Dashboard
● You can Monitor your access points(wrapped points) via ui dashboard and grab metrics

Enabling HystrixDashboards
Add @EnableHystrixDashboard

Agenda
● Netflix Turbine
● RxJava for aggregation of micro-service requests

Netflix Turbine
● Aggregating metrics streams from all cloud instances into one stream
● Use case: Use Hystrix real-time dashboard that uses Turbine to aggregate
from many machines
Demo

Rabbitmq
● Turbine can aggregate all streams by receiving all clients metrics via amqp
● Hystrix commands push metrics to Turbine and Spring Cloud enables that
with AMQP messaging
● Hysteric use RabbitMQ to communicate between circuit breakers and
dashboards

Annotate your turbine server with:
@EnableTurbineAmqp
Have all your Hystrix clients importing:
compile("org.springframework.cloud:spring-cloud-starter-hystrix:1.0.0.RELEASE")
compile("org.springframework.cloud:spring-cloud-starter-bus-amqp:1.0.0.RELEASE")
compile("org.springframework.cloud:spring-cloud-netflix-hystrix-amqp:1.0.0.RELEASE")
That library has everything you need in order to find running rabbitMQ broker and start pushing
messages.
In your Hystrix dashboard add the turbine’s amqp listening port: http://host:8989 (set via application.
yaml)

Demo4
1. Start rabbitmq sbin/rabbitmq-server
2. Stream Turbine using port 8989

Netflix Ribbon
Client Side Load Balancing
● Dynamic Routing
● Load Balancer
● Support rules:
○ round robin
○ response time weighted
○ random load balancing
○ Custom rules
● Ribbon does not run as a separate service but instead as an embedded component in each service consumer.

Using Ribbon with Eureka
● Ribbon includes load balancers that are capable of service discovery in a dynamic environment
● Ribbon uses the information available in Eureka to locate appropriate service instances.
● Ribbon will apply load balancing to spread the requests over the available instances.
● Add new instance dynamically -> Registered automatically on discovery services -> Ribbon choose instance to
invoke

Ribbon and RestTemplate
● Inject RestTemplate and use your instance-id
● Spring Cloud Ribbon will handle automatically the load-balancing
@EnableEurekaClient
@RestController
public class Application {
@Autowired
RestTemplate restTemplate
@RequestMapping("/")
String consume() {
Response response = restTemplate.getForObject("http://baggage-service", Response.class)
}
}
Does make sense: Eureka’s service name
The ribbon-loadbalancer must be on the classpath (e.g. via "spring-cloud-starter-
ribbon").
Then you can inject one as a LoadBalancerClient or you can inject a RestTemplate
(it will be load-balancer aware if you have a LoadBalancerClient).

Netflix Zuul - Edge Server
Zuul the Gatekeeper of Gozer

The Gatekeeper
of our microservices world
● Entry point to the microservices
● Zuul is a JVM based router and server side load balancer by Netflix.
● Intelligent Routing
● Using Ribbon to lookup available services and routes the external request to an
appropriate service instance
● By default Zuul set up a route to every service it can find in Eureka

Using Routings
● Routing in an integral part of a microservice architecture.
For example, / -> mapped to your web application,
/api/bookings -> mapped to the flight-booking service
/api/coupon -> mapped to the coupon service.

Zuul harmonically integrated with Netflix stack
● Zuul using Ribbon to locate an instance to forward to via Eureka
● All requests are executed in a hystrix command -> failures will show up in
Hystrix metrics
● once the circuit is open the proxy will not try to contact the service.

Use cases
● Use case where a UI application wants to proxy calls to one or more back end service
●
Other use cases:
● Authentication
● Stress Testing
● Security
● Service Migration
● Active/Active traffic management

Set your routings
On application.yaml you can set your routings and roles
In this example all services are ignored except coupon-service.
[Demo]

How many services being called?

Scenario Example
1. Client calls to our app’s web service ->
2. our webservice need to request multiple micro-services->
3. uses a callbacks interface to pass the successful result to the
next web service call
4. define another success callback- >
5. and then moves on to the next web service request.
Orchestrator
A B C F
Client Request

Looks like that->
Also known as:
“The Callback Hell”

//The "Nested Callbacks" Way
public void fetchUserDetails() {
//first, request the users...
mService.requestUsers(new Callback<GithubUsersResponse>() {
@Override
public void success(final GithubUsersResponse githubUsersResponse,
final Response response) {
Timber.i(TAG, "Request Users request completed");
final List<GithubUserDetail> githubUserDetails = new ArrayList<GithubUserDetail>();
//next, loop over each item in the response
for (GithubUserDetail githubUserDetail : githubUsersResponse) {
//request a detail object for that user
mService.requestUserDetails(githubUserDetail.mLogin,
new Callback<GithubUserDetail>() {
@Override
public void success(GithubUserDetail githubUserDetail,
Response response) {
Log.i("User Detail request completed for user : " + githubUserDetail.mLogin);
githubUserDetails.add(githubUserDetail);
if (githubUserDetails.size() == githubUsersResponse.mGithubUsers.size()) {
//we've downloaded'em all - notify all who are interested!

Async our microservices call
(Using Reactor)
A library for composing asynchronous and
event-based programs by using observable sequences.
● Allow you to compose sequences together declaratively
● Abstracting away :
○ low-level threading
○ synchronization
○ thread-safety
○ concurrent data structures
○ non-blocking I/O.

RxJava for the rescue
● RxJava is single jar lightweight library.
● Using the Observable abstraction and related higher-order functions.
(Support Java6+)
The following external libraries can work with RxJava:
● Camel RX provides an easy way to reuse any of the Apache Camel components, protocols, transports and data
formats with the RxJava API
● rxjava-http-tail allows you to follow logs over HTTP, like tail -f
● mod-rxvertx - Extension for VertX that provides support for Reactive Extensions (RX) using the RxJava library
And ->> Hystrix latency and fault tolerance bulkheading library. (Which is why we here)

Using Observables
“Go do this thing and when that thing is done then invoke
that other code”
public Observable<String> executeObservableBro(String name) {
return new ObservableResult<String>() {
@Override
return executeRemoteService(name);
}
};
}

Feign Client
● It makes writing web service clients easier
● Feign connect your code to http API’S with minimal overhead.
● Maps to the REST API methods that are exposed by a service
● Customizable decoders and error handlings
● Feign works by processing annotations into a templatized request

Feign and Spring-Cloud
● Create interface and annotate
● Support Feign annotations and JAX-RS annotations
● Collaborated with Spring-MVC annotations(Using convertors like: HttpMessageConverters )
● Eureka-aware REST clients that uses Ribbon for client-side load-balancing to pick an available
service instance
● You can call methods on an interface instead of manually handling URLs, Spring Cloud takes
care of auto-instantiating the interface implementation and will use Eureka for discovery.

Building ‘Bootiful’ microservices cloud

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Building ‘Bootiful’ microservices cloud (20)

Building ‘Bootiful’ microservices cloud