Кирилл Толкачев. Микросервисы: огонь, вода и девопс
0 likes320 views
The document discusses microservices and distributed tracing. It notes that without distributed tracing, it can be difficult to trace requests across multiple services to understand where delays are occurring. It then shows how distributed tracing works, with services adding tracing identifiers to logs and calls to other services to enable correlating logs and traces from all involved services for a single request. This allows generating a single, unified trace to understand end-to-end request performance across multiple microservices.
![Payment Service[jar,doc] Insurance Service [jar,doc]
One Point of View
[UberDoc.zip]
Rent Service[jar,doc] Other Service [jar,doc]
Агрегация документации
70](https://image.slidesharecdn.com/random-170603163129/85/-70-320.jpg)
![= Hippo Rent Service
This is documentation for Open API of our hippo renting service
== Methods
=== Rent
==== Request specification
===== Headers
include::{snippets}/rent-hippo/request-headers.adoc[]
===== Example
include::{snippets}/rent-hippo/http-request.adoc[]
==== Response specification
===== Response fields
include::{snippets}/rent-hippo/response-fields.adoc[]
===== Example
include::{snippets}/rent-hippo/http-response.adoc[]
Вставляем их в документ
82](https://image.slidesharecdn.com/random-170603163129/85/-82-320.jpg)
![= Hippo Rent Service
This is documentation for Open API of our hippo renting
service
== Methods
=== Rent
==== Request specification
===== Headers
include::{snippets}/rent-hippo/request-headers.adoc[]
===== Example
include::{snippets}/rent-hippo/http-request.adoc[]
==== Response specification
===== Response fields
include::{snippets}/rent-hippo/response-fields.adoc[]
===== Example
include::{snippets}/rent-hippo/http-response.adoc[]
83](https://image.slidesharecdn.com/random-170603163129/85/-83-320.jpg)
![Payment Service[logs] Insurance Service[logs]
One Log
Rent Service[logs] Other Service[logs]
Агрегация логов
85](https://image.slidesharecdn.com/random-170603163129/85/-85-320.jpg)
![./gradlew deployToArtifactory
ansible -i [stage,dev,test] -t deploy
119](https://image.slidesharecdn.com/random-170603163129/85/-119-320.jpg)
![./gradlew deployToArtifactory
ansible -i [stage,dev,test] -t deploy
“Прибитая молотком” конфигурация
120](https://image.slidesharecdn.com/random-170603163129/85/-120-320.jpg)
Кирилл Толкачев. Микросервисы: огонь, вода и девопс
- 1. 18+ Микросервисы огонь, вода и DevOps
- 5. Agenda 1. Theory 2. Practice 3. Q&A 5
- 6. 6 ● делается с первого раза ● не меняется годами ● не зависит от людей ● не зависит от процессов
- 7. 7 У всех нас конечно же так
- 8. 8 У всех нас конечно же так
- 14. Немного деталей не помешает 14
- 15. Немного деталей не помешает 15
- 16. Архитектура 16
- 18. Что же такое архитектура? 18
- 19. Что же такое архитектура? 19
- 20. Что же такое архитектура? 20 https://www.youtube.com/watch?v=_Kex5hwGE-w
- 21. 21
- 22. 22 Закон Конвея
- 23. Big Ball of Mud 23
- 24. Путь к микросервисам ● Нагрузки ● Масштабирование 24
- 25. Путь к микросервисам ● Нагрузки ● Масштабирование ● Управление ● Разделение 25
- 26. 12 April 1996 Первое упоминание SOA https://www.gartner.com/doc/302868/service-oriented-architectures- 26
- 27. Принципы SOA 1. Standardized service contract 2. Loose coupling 3. Encapsulation 4. Reusability 5. Autonomy 6. Statelessness 7. Discoverability 27
- 28. Принципы SOA 1. Standardized service contract 2. Loose coupling 3. Encapsulation 4. Reusability 5. Autonomy 6. Statelessness 7. Discoverability 28
- 29. 5 January 2009 SOA is Dead http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html 29
- 30. Classic SOA ● centralized orchestration ● complicated (hi, SOAP) ● smart pipes (hi, ESB) 30
- 31. SOA != SOAP (WS-*) 31
- 32. 32
- 33. 33
- 35. Microservices = SOA 2.0 35
- 36. A=F(Rq, FRq, NFRq,..........???............) DevOps OSS Domination 36
- 37. 37 Про тренды
- 38. 38 Про тренды
- 39. In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. -- James Lewis and Martin Fowler Что такое микросервисы? 39
- 40. In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. -- James Lewis and Martin Fowler Что такое микросервисы? 40
- 41. Размер имеет значение? ● Method/Function = Microservice? ● 10-300 LOC = Microservice? ● 1 week = Microservice? ● 1 developer = Microservice? 41
- 42. Размер не имеет значения* ● Single Responsibility ● One capability ● Bounded context “In your organization, you should be thinking not in terms of data that is shared, but about the capabilities those contexts provide the rest of the domain.” – Sam Newman, Building Microservices *до разумных пределов конечно 42
- 44. In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. -- James Lewis and Martin Fowler Что такое микросервисы? 44
- 46. In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. -- James Lewis and Martin Fowler Что такое микросервисы? 46
- 48. In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. -- James Lewis and Martin Fowler Что такое микросервисы? 48
- 52. Problem → So Many Teams 52
- 53. 53
- 55. 55 ● Spring Boot/Spring Cloud ● Ratpack ● Dropwizard ● Vert.x ● Restlet ● Spark ● KumuluzEE ?
- 56. 56 ● Spring Boot/Spring Cloud ● Ratpack ● Dropwizard ● Vert.x ● Restlet ● Spark ● KumuluzEE Выбирайте то, что больше нравится / в чем есть экспертиза
- 57. 57
- 58. 58
- 59. Принцип LSD - L языков программирования - S среднее число фреймворков на язык - D типов источников данных complexity = L * S * D 59
- 60. Немного LSD для вас - три языка программирования - два в среднем фреймворка на язык - семь типов источников данных - legacy WS, mongo db - OLTP, OLAP - elasticsearch, neo4j - Мишкина база %) complexity = 3 * 2 * 7 = 42 (!) 60
- 61. Чем нельзя пожертвовать? min (L * S * D) → ? 61
- 62. min (L * S * D) → max (D) 62
- 63. Problem → Many Teams Again 63
- 64. 64
- 65. 65 t0
- 66. DRY off 66
- 67. @Getter // generate getters @Setter // generate setters @Aspect // we are an aspect @ToString // generate toString() @EnableWs // SOAP is so enterprisy, we definitely need it @Endpoint // Seriously, just read above @EnableWebMvc // we want MVC @EnableCaching // and we want to cache stuff @Configuration // this class can configure itself @RestController // we want some REST @XmlRootElement // this component is marshallable @EnableWebSocket // we want web socket, it's so new-generation @RedisHash("cat") // this class is an entity saved in redis @EnableScheduling // we want scheduled tasks @EnableWebSecurity // and some built-in security @NoArgsConstructor // generate no args constructor @ContextConfiguration // we want context configuration for unit testing @SpringBootApplication // this is a Sprint Boot application @Accessors(chain = true) // getters/setters are chained (ala jQuery) @EnableAspectJAutoProxy // we want AspectJ auto proxy @EnableAutoConfiguration // and auto configuration @EnableRedisRepositories // since it is an entity we want to enable spring data repositories for redis @EnableWebSocketMessageBroker // we want a broker for web socket messages 67 Fluent annotations
- 68. @Getter // generate getters @Setter // generate setters @Aspect // we are an aspect @ToString // generate toString() @EnableWs // SOAP is so enterprisy, we definitely need it @Endpoint // Seriously, just read above @EnableWebMvc // we want MVC @EnableCaching // and we want to cache stuff @Configuration // this class can configure itself @RestController // we want some REST @XmlRootElement // this component is marshallable @EnableWebSocket // we want web socket, it's so new-generation @RedisHash("cat") // this class is an entity saved in redis @EnableScheduling // we want scheduled tasks @EnableWebSecurity // and some built-in security @NoArgsConstructor // generate no args constructor @ContextConfiguration // we want context configuration for unit testing @SpringBootApplication // this is a Sprint Boot application @Accessors(chain = true) // getters/setters are chained (ala jQuery) @EnableAspectJAutoProxy // we want AspectJ auto proxy @EnableAutoConfiguration // and auto configuration @EnableRedisRepositories // since it is an entity we want to enable spring data repositories for redis @EnableWebSocketMessageBroker // we want a broker for web socket messages 68 Fluent annotations
- 70. Payment Service[jar,doc] Insurance Service [jar,doc] One Point of View [UberDoc.zip] Rent Service[jar,doc] Other Service [jar,doc] Агрегация документации 70
- 71. Analyst, PM Developer Tester Docs Word, PDF... 71
- 72. Analyst, PM Developer Tester Docs Word, PDF... Code + Tests 72
- 73. Analyst, PM Developer Tester Docs Word, PDF... Code + Tests Test cases 73
- 74. Analyst, PM Developer Tester Docs :( Word, PDF... Code + Tests Test cases 74
- 75. Analyst, PM Developer Tester Docs :) Markdown/Asciidoctor Docs :) 75
- 76. = Hippo Rent Service This is documentation for Open API of our hippo renting service == Methods === Rent ==== Request specification ===== Headers //тут опишем http-заголовки ===== Example //а здесь будут примеры вызова ==== Response specification ===== Response fields //здесь описание полей ответа ===== Example //ну и пример того, что вообще ожидать в ответе Вот такой документ 76
- 77. ./gradlew ... asciidoc publishDocs 77 Компиляция документа
- 78. document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ) Имя сниппета 78
- 79. document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ) Тестируем заголовки 79
- 80. document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ) Тестируем поля ответа 80
- 82. = Hippo Rent Service This is documentation for Open API of our hippo renting service == Methods === Rent ==== Request specification ===== Headers include::{snippets}/rent-hippo/request-headers.adoc[] ===== Example include::{snippets}/rent-hippo/http-request.adoc[] ==== Response specification ===== Response fields include::{snippets}/rent-hippo/response-fields.adoc[] ===== Example include::{snippets}/rent-hippo/http-response.adoc[] Вставляем их в документ 82
- 83. = Hippo Rent Service This is documentation for Open API of our hippo renting service == Methods === Rent ==== Request specification ===== Headers include::{snippets}/rent-hippo/request-headers.adoc[] ===== Example include::{snippets}/rent-hippo/http-request.adoc[] ==== Response specification ===== Response fields include::{snippets}/rent-hippo/response-fields.adoc[] ===== Example include::{snippets}/rent-hippo/http-response.adoc[] 83
- 84. 84
- 85. Payment Service[logs] Insurance Service[logs] One Log Rent Service[logs] Other Service[logs] Агрегация логов 85
- 86. 86 RentService No TraceId No SpanId TraceId = X SpanId = A
- 87. 87 RentService PaymentService No TraceId No SpanId TraceId = X SpanId = A TraceId = X SpanId = B TraceId = X SpanId = C
- 88. 88 RentService PaymentService BlockchainService No TraceId No SpanId TraceId = X SpanId = A TraceId = X SpanId = B TraceId = X SpanId = C TraceId = X SpanId = D TraceId = X SpanId = D TraceId = X SpanId = F
- 89. 89 RentService PaymentService SecurityServiceBlockchainService No TraceId No SpanId TraceId = X SpanId = A TraceId = X SpanId = B TraceId = X SpanId = C TraceId = X SpanId = D TraceId = X SpanId = D TraceId = X SpanId = E TraceId = X SpanId = E TraceId = X SpanId = F TraceId = X SpanId = G
- 90. 90 RentService PaymentService SecurityServiceBlockchainService No TraceId No SpanId TraceId = X SpanId = A TraceId = X SpanId = B TraceId = X SpanId = B TraceId = X SpanId = C TraceId = X SpanId = C TraceId = X SpanId = D TraceId = X SpanId = D TraceId = X SpanId = E TraceId = X SpanId = E TraceId = X SpanId = F TraceId = X SpanId = G
- 91. 91 RentService PaymentService SecurityServiceBlockchainService TraceId = X SpanId = A No TraceId No SpanId TraceId = X SpanId = A TraceId = X SpanId = A TraceId = X SpanId = B TraceId = X SpanId = B TraceId = X SpanId = C TraceId = X SpanId = C TraceId = X SpanId = D TraceId = X SpanId = D TraceId = X SpanId = E TraceId = X SpanId = E TraceId = X SpanId = F TraceId = X SpanId = G
- 92. 92
- 93. Парадокс централизации Чтобы эффективно разрабатывать распределённые приложения, нам нужны очень хорошие централизованные библиотеки и инструменты Например: логирование, health-checking, метрики, обработка типовых ошибок, автодокументирование 93
- 94. Парадокс централизации Чтобы эффективно разрабатывать распределённые приложения, нам нужны очень хорошие централизованные библиотеки и инструменты Но: не выносите бизнес-логику или доменные объекты! Не размывайте бизнес-контекст вашего API 94
- 96. 96 Одно приложение - одна БД Всё просто
- 97. N сервисов → 1 БД Изоляция на уровне таблиц или схем 97
- 98. N сервисов → 1 БД Изоляция через хранимые процедуры 98
- 99. 99 Transport Layer API API API API API
- 100. 100 Transport Layer API API API API API
- 101. N сервисов → 1 БД Распределённый монолит 101
- 102. N сервисов → N БД 102
- 103. N сервисов → N БД Распределённые транзакции – это больно 103
- 104. 104 Transport Layer API API API API API Очень много кода
- 105. N сервисов → N БД Ваше legacy тянет вас на дно 105
- 106. 106
- 107. M сервисов → 1 БД L сервисов → L БД M + L = N, M << L 107
- 108. 108 Transport Layer APIAPI API API API
- 109. M сервисов → 1 БД L сервисов → L БД M + L = N, M << L 109
- 110. Доставка 110
- 111. war/ear 111 Зависимость от сервера приложений
- 112. executable fatJar 112 Зависимость от системных библиотек
- 113. 113 not war < make jar
- 114. executable fatJar/npm-пакет 114 Разный менеджмент для разных форматов дистрибуции
- 116. 116 not jar < make docker
- 118. scp fat.jar root@prod101:/home/app/ Что делать с консистентостью? Как быть с доступностью? 118
- 119. ./gradlew deployToArtifactory ansible -i [stage,dev,test] -t deploy 119
- 120. ./gradlew deployToArtifactory ansible -i [stage,dev,test] -t deploy “Прибитая молотком” конфигурация 120
- 123. 123
- 124. mesos / kubernetes / ∞ 124
- 126. 126 t Bare Metal
- 130. 130
- 131. Использование группы машин как одной WEB WASTED CACHE WASTED WASTED HADOOP 131
- 132. Представьте, что кластер это просто одна большая машина WEB WASTED CACHE WASTED WASTED HADOOP WASTED WEB CACHE HADOOP FREE FREE 132
- 133. yoman Yo start.spring.io Ss python Py lazybones Lz java Ja spring boot Sb gradle Gr asciidoctor Ad thrift Th docker Dr mesos Ms marathon Ma chronos Ch aurora Au Artifactory Ar Kubernetes Kb eureka Eu consul Cl etcd Ed zookeeper Zk hystrix Hx sleuth Sl zipkin Zn groovy Gy 133 Data Isolation Di Centralization paradox Cp LSD principle Ls DDD Dd Smart Docs Sd Dynamic Sharing Ds Smart Libs Sl SOA So ansible An Conway’s Law Co
- 134. yoman Yo start.spring.io Ss python Py lazybones Lz java Ja spring boot Sb gradle Gr asciidoctor Ad thrift Th docker Dr mesos Ms marathon Ma chronos Ch aurora Au Artifactory Ar Kubernetes Kb eureka Eu consul Cl etcd Ed zookeeper Zk hystrix Hx sleuth Sl zipkin Zn groovy Gy 134 Data Isolation Di Centralization paradox Cp LSD principle Ls DDD Dd Smart Docs Sd Dynamic Sharing Ds Smart Libs Sl SOA So ansible An Conway’s Law Co ?
- 135. A=F(Rq, FRq, NFRq,..........???............) DevOps OSS Domination 135
- 138. «Колодцы»
- 142. Выводы 142 ● Микросервисы - это просто
- 143. Выводы 143 ● Микросервисы - это просто ● Микросервисы - это сложно
- 144. Выводы 144 ● Микросервисы - это просто ● Микросервисы - это сложно ● А DevOps это не человек
- 145. Выводы 145 ● Микросервисы - это просто ● Микросервисы - это сложно ● А DevOps это не человек ● Больше свободы
- 146. Выводы 146 ● Микросервисы - это просто ● Микросервисы - это сложно ● А DevOps это не человек ● Больше свободы и больше ответственности
- 147. Спасибо! Готовы ответить на ваши вопросы @tolkv @aatarasoff 147 @lavcraft @aatarasoff