Apache Camel K - Copenhagen v2

~1 sec build and deploy on Kubernetes
Claus Ibsen
@davsclaus

@davsclaus
About me
● Senior Principal Software Engineer at Red Hat
● Java Champion
● 11 years as Apache Camel committer
● Author of Camel in Action books
● https://medium.com/@davsclaus

@davsclaus
Thanks for having me
Slides https://www.slideshare.net/davsclaus/apache-camel-k-copenhagen

@davsclaus
Agenda
● What is Apache Camel?
● Apache Camel v3
● Apache Camel K
● Quarkus & Camel
● Q & A

@davsclaus
Apache Camel
is an
Integration Framework
based on
Enterprise Integration Patterns

@davsclaus
Integration Framework

@davsclaus
Enterprise Integration Patterns

@davsclaus
Camel Routes
from("file:data/inbox")
.to("jms:queue:order");
<route>
<from uri="file:data/inbox"/>
<to uri="jms:queue:order"/>
</route>
Java DSL
XML DSL

@davsclaus
Camel Routes with Splitter

@davsclaus
from("file:inbox")

@davsclaus
from("file:inbox")
.split(body().tokenize("n"))

@davsclaus
from("file:inbox")
.marshal(customToXml)
Custom data
transformation

@davsclaus
from("file:inbox")
.marshal(customToXml)
.to("activemq:line"); Custom data
transformation

@davsclaus
Camel runs everywhere
Application
Servers
Linux
Containers

@davsclaus
Runs on popular Java Runtimes

@davsclaus
Camel connects everything
Enterprise
Systems
● File
● FTP
● JMS
● AMQP
● JDBC
● SQL
● TCP/UDP
● Mail
● HDFS
● JPA
● MongoDB
● Kafka
● ...
Public Cloud
● AWS
○ S3
○ SQS
○ Kinesis
○ ...
● Google
○ BigQuery
○ PubSub
● Azure
○ Blob
○ Queue
● Box
● Dropbox
● Facebook
● Linkedin
● Salesforce
● SAP
● ServiceNowIoT
● CoAP
● MQTT
● PubNub
SaaS

@davsclaus
300+ Components
ahc ahc-ws amqp apns as2 asn1 asterisk atmos atmosphere-websocket atom atomix avro aws aws-xray azure bam barcode base64
beanio beanstalk bean-validator bindy blueprint bonita boon box braintree cache caffeine cassandraql castor cdi chronicle chunk cmis
cm-sms coap cometd consul context corda core-osgi core-xml couchbase couchdb crypto crypto-cms csv cxf cxf-transport
digitalocean disruptor dns docker dozer drill dropbox eclipse ehcache ejb elasticsearch elasticsearch5 elasticsearch-rest elsql etcd
eventadmin exec facebook fastjson fhir flatpack flink fop freemarker ftp ganglia geocoder git github google-bigquery google-calendar
google-drive google-mail google-pubsub google-sheets gora grape groovy groovy-dsl grpc gson guava-eventbus guice hawtdb
hazelcast hbase hdfs hdfs2 headersmap hessian hipchat hl7 http http4 http-common hystrix ibatis ical iec60870 ignite infinispan
influxdb ipfs irc ironmq jackson jacksonxml jasypt javaspace jaxb jbpm jcache jclouds jcr jdbc jetty jetty9 jetty-common jgroups jibx
jing jira jms jmx johnzon jolt josql jpa jsch jsonpath json-validator jt400 juel jxpath kafka kestrel krati kubernetes kura ldap ldif leveldb
linkedin lra lucene lumberjack lzf mail master metrics micrometer milo mina mina2 mllp mongodb mongodb3 mongodb-gridfs mqtt
msv mustache mvel mybatis nagios nats netty netty4 netty4-http netty-http nsq ognl olingo2 olingo4 openshift openstack opentracing
optaplanner paho paxlogging pdf pgevent printer protobuf pubnub quartz quartz2 quickfix rabbitmq reactive-streams reactor restlet
rest-swagger ribbon rmi routebox rss ruby rx rxjava2 salesforce sap-netweaver saxon scala schematron scr script service servicenow
servlet servletlistener shiro sip sjms sjms2 slack smpp snakeyaml snmp soap solr spark spark-rest splunk spring spring-batch spring-
boot spring-cloud spring-cloud-consul spring-cloud-netflix spring-cloud-zookeeper spring-integration spring-javaconfig spring-ldap
spring-redis spring-security spring-ws sql ssh stax stomp stream stringtemplate swagger swagger-java syslog tagsoup tarfile telegram
test test-blueprint test-cdi testcontainers testcontainers-spring test-karaf testng test-spring thrift tika twilio twitter undertow univocity-
parsers urlrewrite velocity vertx weather web3j websocket wordpress xchange xmlbeans xmljson xmlrpc xmlsecurity xmpp xstream
yammer yql zendesk zipfile zipkin zookeeper zookeeper-master

@davsclaus
A little Camel Example

@davsclaus 29
kamel run file-copier.js
K
A little Camel K example

@davsclaus
Camel Community in Numbers
● Created in 2007
● 139 Releases
● 473 Contributors on github
● ~ 2500 stars ⭐️ on github (a star is appreciated)
● ~ 36000 commits on master branch (~ 3000/year - 8/day)
● ~ 8600 Q&A on stackoverflow
● ~ 500 users on gitter chat
● 552 follows on twitter (https://twitter.com/apachecamel)

@davsclaus
The good, bad & ugly
● Huge and active community
● Continued and stable release cadence
● Open and welcoming community (we love contributions)
● Burden on core developers
● Well established and "old project" may not appear to hipsters
● Balance of "keep as-is" vs "modernize" architecture
● The outdated website
(new site under development: https://camel.apache.org/staging)

@davsclaus
Camel 3 Timeline
● 3.0 Milestone 1 - February 2019
● 3.0 Milestone 2 - March 2019
● 3.0 Milestone 3 - May 2019
● 3.0 Milestone 4 - July 2019
● 3.0 GA - September 2019
● v3.0 Basic Features
● v3.1 Postponed Features
● v3.2 Postponed Features
● ...
Timeboxed
Schedule

@davsclaus
Camel 3 Major Goals
● Backwards compatible (minimal migration for normal use-cases)
● Light-weight & modular camel-core
● Reactive routing engine (non blocking & back-pressure)
● Tidy up APIs & cleanup of technical debt
● Fluent Builder Endpoint configuration (Java & XML)
● Java 8 DSL improvements
● New Cloud EIP Patterns

@davsclaus
Camel 3 Major Goals
● Apache Camel K
● GraalVM/Quarkus Support
● Timeboxed Release
● New website & documentation
● More presence on social/workshop/blog posts
● There is still a lot of unexpressed potential

@davsclaus
Camel 3 Milestone 1
● Released on February 22th 2019
● Migration Guide (work in progress)
https://github.com/apache/camel/blob/master/MIGRATION.md
● Example of migrating Camel in Action 2nd source code
https://github.com/camelinaction/camelinaction2/compare/camel3m1

@davsclaus
Camel 3 Milestone 1
● Modularization of camel-core
○ camel-api API for end users
○ camel-base Internal
○ camel-core Just like Camel 2.x
○ camel-management-api API for JMX management
○ camel-management-impl JMX is optional
○ camel-support API and support classes for end
users / components
○ camel-util General small utilities
○ camel-util-json Internal (Camel tooling and such)
○ camel-core-xml XML DSL support
○ camel-core-osgi OSGi support (no longer in camel-
core)
Mock
Component still
in camel-core
We will attempt to
move it out in M3

@davsclaus
Camel 3 Milestone 1
● Modularization of camel-core

@davsclaus
Camel 3 Milestone 1
● Reactive Core (internal)

@davsclaus
Camel 3 Milestone 1
● Reactive Core
○ All EIPs now fully reactive
○ Blocking vs Non-Blocking code-paths merged into one (Non-Blocking)
● Reactive Core (todo)
○ Backpressure (M3)
○ Client API (M3)
○ Java 9+ Flowable API (M3)
■ M1/M2 currently Java 8
○ Component metadata
■ To know if a component is fully reactive
● Vert.X (todo 3.1/3.2)
○ Better camel-vertx integration
■ To use its fully reactive components
Not all Camel
components are
100% reactive
compliant

@davsclaus
Camel 3 Milestone 1
● Tidy up API and clean technical debt
○ All deprecated code and components removed
○ Some APIs tidied up
...More internal cleanup on the way (M2/M3)
● camel-core-osgi
○ Upgraded to OSGi R6
● camel-activemq
○ Apache ActiveMQ 5.x (do not use activemq-camel)

@davsclaus
Camel 3 Milestone 1
● Improve project build
○ Faster build and re-build of project
○ More source generation of metadata and other files
○ … more to come in M2
● Apache Camel K
○ Camel K 0.3.0 supports
■ Camel 2.x
■ Camel 3 milestone 1

@davsclaus
Camel 3 Milestone 2
● Released on March 31th 2019
● Migration Guide (work in progress)
https://github.com/apache/camel/blob/master/MIGRATION.md
● Example of migrating Camel in Action 2nd source code
https://github.com/camelinaction/camelinaction2/compare/camel3m2

@davsclaus
Camel 3 Milestone 2
● Tidy up API and clean technical debt
○ All deprecated code and components removed
○ Some APIs tidied up
...More internal cleanup on the way (M3)
● Optimized type converters
○ Invoke type converter directly (no reflection method call)
○ Optimized type converter discovery and loading on startup
○ Allows pre-load with GraalVM/Quarkus

@davsclaus
Camel 3 Milestone 2
● Properties component
○ Fallback to lookup property as OS ENV variable
○ To work better with Kubernetes / Camel K
● Camel Main
○ More functionality out of the box
■ dependency injection (IoC) with camel-core Java (RouteBuilder classes)
■ dependency injection (IoC) with camel-spring Java
○ Convention over configuration out of the box
■ application.properties
○ To work better with Camel K

@davsclaus
Camel 3 Milestone 2
● Writeable registry
○ bind beans to registry
camelContext().getRegistry().bind("myName", myBean);
bindToRegistry("myName", myBean); (in RouteBuilder)
@BindToRegistry
○ Easier unit testing
○ Easier Camel standalone
● Step EIP
○ Group together EIPs (logic name)
○ Grouped performance statistics

@davsclaus
Camel 3 Milestone 2
● Quarkus/GraalVM Support
○ Works better with quarkus-camel
○ More minimal/optimized camel-core
○ Works better with Camel K

@davsclaus
Camel 3 Milestone 3
● camel-core-engine
○ Minimal set of dependency
● camel-mock
○ Move Mock out of camel-core (potentially not possible)
● camel-dataset
○ Move dataset & test component out of camel-core DONE
● camel-bean
○ Move Bean component / Language out of camel-core (likely not possible)
● camel-xml
○ Move all related to XML out of camel-core

@davsclaus
Camel 3 Milestone 3
● Quarkus/GraalVM Support
○ Generate additional metadata in JARs
○ Better integration with Quarkus
○ More minimal/optimized camel-core
○ Works better with Camel K

@davsclaus
Camel 3 Milestone 3
● Auto generated Endpoint DSL
○ Fluent Builders for endpoint in Java / XML
from(file("inbox").recursive().withDelay(2000))
.to(jms("cheese").withTimeToLive(5000));
<route>
<from>
<file directory="inbox" recursive="true" delay=”2000”/>
</from>
<to>
<jms queue="cheese" timeToLive="5000"/>
</to>
</route>

@davsclaus
Camel 3 Milestone 4
● Reactive Core
○ Backpressure
○ Client API
○ Flowable API (Java9+)
● Data Shape
○ IN vs OUT contract
■ Route
■ Component
○ Improved Data Mapping support
■ camel-atmasmap

@davsclaus
Camel 3 Milestone 4
● Java 11
○ Builds on Java 11
○ Java 8 to be dropped
● Stabilization and polish
○ Tidy up for GA release
● New website
○ Staging for preview

@davsclaus
Camel 3 - GA September 2019
Camel 3 is our
“Duke Nukem
Forever”
project 😁

@davsclaus
Apache Camel K
K
~1 sec build and deploy on Kubernetes

@davsclaus
Why Camel K
● Building and deploying Camel on Kubernetes
○ fabric8-maven-plugin to build the image
○ s2i on OpenShift to build the image
○ ci/cd pipeline (such as Jenkins) to build the image
● Minute(s) to build and deploy "Hello Camel"
We want to go faster

@davsclaus
Why Camel K
● Low-code / No-code integration
○ Engine in Red Hat Fuse Online
● Cloud Native Camel on Kubernetes
● Camel Serverless
Cloud Native Camel
on Kubernetes

@davsclaus
Why Camel K
● Quick Prototyping
● Simple or little business logic
● Tiny microservices
● Opinionated runtime
Not intended for traditional
bigger projects
-
Instead use regular Camel

@davsclaus
Limitations of Camel K
● Opinionated runtime
● Not a replacement for Apache Camel
● Must run on Kubernetes
● Testing framework currently limited
● Early project
... but Camel K has a lot of
potential

@davsclaus
Overview of Camel K
A lightweight platform for running Camel integration DSL in the cloud.
● Based on operator-sdk
● Works on Openshift and “vanilla” Kubernetes
$ kamel run routes.groovy --dev
from(“telegram:bots/bot-id”)
.transform()...
.to(“kafka:topic”);
from(“kafka:topic”)
.to(“http:my-host/api/path”);
File routes.groovy

Architecture of Camel K
Dev Environment Remote Cloud
kamel CLI
Camel K
Operator
“Integration”
Custom
Resource
Running Pod
Live updates!
Fast redeploy!
Less than 1 second!
Tailored for cloud-native development experience

Camel K in details
kind: Integration
apiVersion: camel.apache.org/v1alpha1
metadata:
name: my-integration
spec:
sources:
- name: source.groovy
content: |-
from(“telegram:...”)
.transform()...
.to(“kafka:...”)
from(“telegram:bots/bot-id”)
.transform()...
This is what people care
about: Camel DSL
Camel K Operator
1. Choose a runtime
2. Create container image
3. Create Kubernetes
resources for deployment
kamel CLI

Camel K Resources Visualization
$ kamel run code.groovyfrom(“telegram:bots/bot-id”)
.transform()...
File code.groovy
<<custom-resource>>
IntegrationPlatform
<<custom-resource>>
IntegrationContext
<<custom-resource>>
Integration
from(“telegram:..”)
.to(“...”)
BuildConfig
ImageStream
ImageStreamTag
Deployment
ConfigMap
.to(“...”)
Pod
On OpenShift.
It uses Kaniko on
“vanilla” Kubernetes.
CRDs allow to abstract
from all this implementation
details!

Camel K Resources Visualization
$ kamel run code.groovyfrom(“telegram:bots/bot-id”)
.transform()...
File code.groovy
<<custom-resource>>
IntegrationPlatform
<<custom-resource>>
IntegrationContext
<<custom-resource>>
Integration
.to(“...”)
BuildConfig
ImageStream
ImageStreamTag
Deployment
ConfigMap
.to(“...”)
Pod
On OpenShift.
It uses Kaniko on
“vanilla” Kubernetes.

Fast Deployment of Camel K
Time to run a integration using different strategies (in seconds)
Lower is better :)
F-m-p is the “fabric8-maven-plugin”
(http://maven.fabric8.io/) deploying a
average spring-boot based
integration on Minishift vs. a remote
OpenShift cluster (accounting time
to upload the fat Jar). Source S2I
build has been measured in Red Hat
Fuse Online.

@davsclaus
Camel K
Demo
Let’s build a Telegram Chat Bot in few minutes.
You’ll need shortly on
your phone:
- Telegram App
- A QR Code reader

@davsclaus
Camel K
Demo
Let’s build a Telegram Chat Bot in few minutes.
@camelkbot

@davsclaus
Build
A pluggable model for
building artifacts, like jar
files, zips or containers
from source code.
Knative Overview - Components
Serving
An event-driven model
that serves the container
with your application and
can "scale to zero".
Eventing
Common infrastructure for
consuming and producing
events that will stimulate
applications.
"...an extension to Kubernetes exposing building blocks to build modern, source-centric, and container-
based applications that can run anywhere".

@davsclaus
Knative Overview - Builds
● A Build is a list of containers run in-order, with source mounted in (TaskRun)
● BuildTemplates provide reusable, parameterized recipes that can be used to
create Builds (Task)
● Pipelines - The future!
○ Group of Tasks
● Build Example
● "Source to URL"
● S2I for OpenShift users
apiVersion: build.knative.dev/v1alpha1
kind: Build
metadata:
name: example-build
spec:
serviceAccountName: build-auth-example
source:
git:
url: https://github.com/example/build-example.git
revision: master
steps:
- name: centos-example
image: centos
args: ["centos-build-example", "SECRETS-example.md"]
steps:
- image: quay.io/example-builders/build-example
args: ['echo', 'hello-example', 'build']

@davsclaus
Knative Overview - Serving
● Configurations represent the ‘floating
HEAD’ of a history of Revisions
● Revisions represent immutable snapshot
of code and configuration
● Routes configure ingress over a collection
of Revisions and/or Configurations
● Services (nope, not K8s services) are top-
level controllers that manage a set of
Routes and Configurations to implement a
network service
Service
(my-function)
Route
(name) Configuration
Revision
Revision
Revision
manages
records
history of
10%
90%

@davsclaus
Knative Overview - Eventing
● Goal is to be a generalized eventing framework
● Many influencers also active on CloudEvents specification
Current-ish API:
● Receive Adapter triggers on EventSources that send Events to a
sink (channel)
● Channels are named endpoints which accept event delivery
(provided by Kafka, AMQP…)
● Subscriptions register to receive traffic from a Channel

@davsclaus
Knative Eventing - Pipeline
Knative Eventing Docs

@davsclaus
Knative Eventing - Flow
Knative Eventing Docs

@davsclaus
Again, beyond functions...
Keep code and configuration separate
Each configuration change triggers the creation
of new revision
Concurrency (scale out via process)
Disposability (fast startup/graceful shutdown)
Build, release, run (separate build and run stages)
https://12factor.net/
Configuration(configuration.serving.knative.dev)
Revision(revision.serving.knative.dev)
Route(route.serving.knative.dev)
Service(service.serving.knative.dev)
Build (buildtemplate.build.knative.dev)

@davsclaus
What about Camel & Knative ?

Camel K and Knative
<<custom-resource>>
Build
Knative building blocks relevance to Camel
<<custom-resource>>
...<<custom-resource>>
...
Knative Build
<<custom-resource>>
Service
<<custom-resource>>
...
Knative Serving
<<custom-resource>>
...
Knative
Eventing
Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications
<<custom-resource>>
Channel
X

Camel & Knative in details
kind: Integration
apiVersion: camel.apache.org/v1alpha1
metadata:
name: my-integration
spec:
sources:
- name: source.groovy
content: |-
from(“knative:channel/a”)
.to(“http:my-host/api/path”)
Camel K Operator
Knative
profile?
kind: Service
apiVersion: serving.knative.dev/v1alpha1
yes
kind: Deployment
# standard one
no

Camel & Knative in details
$ kamel run code.groovy
.transform()...
.to(“slack:#room”);
File code.groovy
<<custom-resource>>
Integration
.to(“...”) <<custom-resource>>
Service
<<custom-resource>>
Channel
<<custom-resource>>
Subscription Pod Pod Pod
events
Same user experience!
Or even no pods!
In the knative profile, the
operator uses Knative
building blocks instead of a
standard Kubernetes
“Deployment”

Camel K: same model for different purposes
Camel K & Knative Eventing
Channel
Channel
Channel
Ext
System
Event
Source
EIP
Integration
Function
250+ components!

@davsclaus
Camel K and Knative - Links
● Apache Camel K
https://github.com/apache/camel-k
● Camel K Telegram Demo Code
https://github.com/nicolaferraro/camel-k-chuck-bot
● Camel K Introduction Blog
https://www.nicolaferraro.me/2018/10/15/introducing-camel-k
● Camel K and Knative tutorial
https://redhat-developer-demos.github.io/knative-tutorial/knative-tutorial-
camelk/v1.0.0/index.html
● Camel Knative Event Sources
https://github.com/knative/eventing-sources/tree/master/contrib/camel/samples

Java Density Problem
CONTAINER ORCHESTRATION
Node Node Node
Traditional Cloud-Native
Java Stack
Java Stack
Java Stack
Java Stack
NodeJS
NodeJS
NodeJS
NodeJS
NodeJS
NodeJS
NodeJS
Go Go Go
Go Go Go
Go Go Go
Go Go Go
Go Go Go
Go Go Go
Go Go Go
https://developers.redhat.com/blog/2017/03/14/java-inside-docker/

SCALE-UP SCALE-DOWN SCALE-TO-
ZERO
Java Fast Scaling Problem

@davsclaus
Supersonic Subatomic Java
https://quarkus.io

@davsclaus
What is Quarkus
A Kubernetes Native Java stack tailored for
GraalVM & OpenJDK HotSpot, crafted from the
best of breed Java libraries and standards

@davsclaus
Minimal Footprint - Quarkus
● Minimal footprint Java applications
○ Native vs Quarkus+OpenJDK JVM vs Traditional JVM

@davsclaus
Container First - Quakrus
● First Class Support for Graal/SubstrateVM
○ Native compiled first-class supported
● Build Time Metadata Processing
○ Pre-Build and "warump" as much as possible during build time
● Reduction in Reflection Usage
○ Reduce startup and memory
● Native Image Pre Boot
○ Most of startup serialized into native image to startup even faster
https://quarkus.io/vision/container-first

@davsclaus
Programming Model - Quakrus
● Unifies imperative and reactive

@davsclaus
Developer Joy - Quarkus
● Unified configuration
● Zero config, live reload
● Streamlined code (80/20 rule)
● Native executable generation

@davsclaus
Standards - Quarkus
● CDI Dependency Injection
● JAX-RS REST
● JPA persistence
● JTA transactions
● Eclipse Microprofile configuration & monitoring
● Vert.X
● Apache Camel
● ... and more
https://quarkus.io/vision/standards

@davsclaus
Quarkus and Camel / Camel K
● Works with Camel K and regular Camel (3.x)
● Fast startup, low memory usage
● Compile Camel routes to native code (Camel K)
● Compile Camel projects to native code (Camel)
● Bring Camel to the serverless world

@davsclaus
Quarkus and Camel / Camel K
● Work to do
○ Quarkus Dev mode
○ More native Camel components
○ More camel-core optimizations
○ Better Camel and Quakrus integration
○ Better Camel and CDI with Quarkus
○ Native Compilation with GraalVM is not rock solid

Apache Camel and Quarkus runs blazingly fast 🚀 with native compiled binaries.
Startup is 7 msec, 28mb binary file, and 15mb RSS memory used.

@davsclaus
Camel and Quarkus - Links
● Quarkus
https://quarkus.io
● Quarkus Camel Extensions
https://quarkus.io/extensions/#integration
https://github.com/quarkusio/quarkus/tree/master/extensions/camel
● Quarkus Camel Demo
https://github.com/quarkusio/quarkus-quickstarts/tree/master/camel-java

@davsclaus
Any Questions ?
🌍 http://www.davsclaus.com
@davsclaus
davsclaus
✉️ claus.ibsen@gmail.com

Apache Camel K - Copenhagen v2

More Related Content

What's hot (20)

Similar to Apache Camel K - Copenhagen v2 (20)

More from Claus Ibsen (15)

Recently uploaded (20)

Apache Camel K - Copenhagen v2

Editor's Notes