Adding Realtime to your Projects
7 likes1,526 views
The document discusses adding real-time capabilities to applications. It begins with an introduction to real-time needs in applications and challenges with existing web technologies. It then covers the history and evolution of the web from documents to more dynamic applications. Various real-time architectures and technologies are presented, including websockets and pub/sub messaging. Examples of using real-time for chat, notifications, and live updates are provided. It emphasizes separating real-time concerns from application logic and stresses the importance of testing real-time capabilities under different load scenarios.
![MADRID · NOV 27-28 · 2015
Authentication: Who are you?
Carotene.authenticate({ userId: "some_user_identifier",
token: "token_for_this_user"
});
Carotene
Your server generates this
Check out JWT
[{carotene, [
% ... Other configuration options
{authenticate_url, "http://mybackend.com/authenticate_carotene/"}
}]}
Your server receives a POST request](https://image.slidesharecdn.com/charla-codemotion-export-151130120553-lva1-app6892/85/Adding-Realtime-to-your-Projects-62-320.jpg)
![MADRID · NOV 27-28 · 2015
[{carotene, [
% ... Other configuration options
{subscribe_authorization, [
{level, anonymous}
]},
}]}
Anonymous allowed
Authorization: Can you do that?](https://image.slidesharecdn.com/charla-codemotion-export-151130120553-lva1-app6892/85/Adding-Realtime-to-your-Projects-63-320.jpg)
![MADRID · NOV 27-28 · 2015
[{carotene, [
% ... Other configuration options
{subscribe_authorization, [
{level, authenticated}
]},
}]}
Only authenticated allowed
Authorization: Can you do that?](https://image.slidesharecdn.com/charla-codemotion-export-151130120553-lva1-app6892/85/Adding-Realtime-to-your-Projects-64-320.jpg)
![MADRID · NOV 27-28 · 2015
[{carotene, [
% ... Other configuration options
{level, ask},
{authorization_url, “http://mybackend.com/authorize-subscribe"}
]},
}]}
Let your logic decide
Publishing follows the same pattern
Authorization: Can you do that?](https://image.slidesharecdn.com/charla-codemotion-export-151130120553-lva1-app6892/85/Adding-Realtime-to-your-Projects-65-320.jpg)
![MADRID · NOV 27-28 · 2015
RT
#
[Id: 9]
subscribe({channel: “user-9”})
!
%!
presence({channel: “user-9”})
presence([9]})
auth({user: 9, token: <token>})
& '](https://image.slidesharecdn.com/charla-codemotion-export-151130120553-lva1-app6892/85/Adding-Realtime-to-your-Projects-78-320.jpg)
![MADRID · NOV 27-28 · 2015
RT
#
[Id: 9] !send({channel: “user-9”,
message: <notification>})
message({channel: “user-9”,
message: <notification>})
& '](https://image.slidesharecdn.com/charla-codemotion-export-151130120553-lva1-app6892/85/Adding-Realtime-to-your-Projects-79-320.jpg)
![MADRID · NOV 27-28 · 2015
RT
!
%!
presence({channel: “user-9”})
presence([]})
& '](https://image.slidesharecdn.com/charla-codemotion-export-151130120553-lva1-app6892/85/Adding-Realtime-to-your-Projects-80-320.jpg)
Adding Realtime to your Projects
- 1. MADRID · NOV 27-28 · 2015 Adding Realtime To your Apps Nacho Martín
- 2. MADRID · NOV 27-28 · 2015 I write software at limenius.com 3/4 of them need some form of Real Time We build web and mobile apps for clients We have tried many things in different battles This year we abstracted carotene-project.com and opensourced it
- 3. MADRID · NOV 27-28 · 2015 A bit of history Websockets? where we are going we don’t need Websockets
- 4. MADRID · NOV 27-28 · 2015 Meet Tim Berners-Lee, 1991
- 5. MADRID · NOV 27-28 · 2015 He worked at CERN CERN’s ideas are rooted in 18th century Enlightenment
- 6. MADRID · NOV 27-28 · 2015 Most important: knowledge
- 7. MADRID · NOV 27-28 · 2015 Humans pass knowledge to next generations!
- 8. MADRID · NOV 27-28 · 2015 We stand on the shoulders of giants!
- 9. MADRID · NOV 27-28 · 2015 Result: The Very First Web It is about documents
- 10. MADRID · NOV 27-28 · 2015 Under this hypothesis 1% 99% Produced before request Produced right now: 1%? Probably more like 0.0000001% Human Knowledge
- 11. MADRID · NOV 27-28 · 2015 Corollary: If something important happens after the HTTP request, it will be filtered and solidified in Knowledge and preserved for ages! Come again next month.
- 12. MADRID · NOV 27-28 · 2015 At CERN they build things pretty well
- 13. MADRID · NOV 27-28 · 2015 Archie News Irc Gopher email WEBFtp
- 14. MADRID · NOV 27-28 · 2015 Under the document paradigm
- 15. MADRID · NOV 27-28 · 2015 Process a Request, call a program serve a Response Serve a Document Under the document paradigm Short lived processes Under the document paradigm
- 16. MADRID · NOV 27-28 · 2015 CGI was born And then… PHP Rails Django Symfony Wordpress MediaWiki Spring CakePHP Perl-cgi Code Igniter Laravel
- 17. MADRID · NOV 27-28 · 2015 This scales very well ! ! ! ! ! And has been VERY battle tested
- 18. MADRID · NOV 27-28 · 2015 25 years have passed Humanity has been exposed to huge loads of information Time to check the enlightement hypothesis What do people express more interest at?
- 19. MADRID · NOV 27-28 · 2015
- 20. MADRID · NOV 27-28 · 2015 Meet Tim Berners-Lee, 2015
- 21. MADRID · NOV 27-28 · 2015 In face of this sad truth What to do as a developer?
- 22. MADRID · NOV 27-28 · 2015 Possibility 1 (Has chances of doing something meaningful and be rememebered)
- 23. MADRID · NOV 27-28 · 2015 Let’s explore possibility 2
- 24. MADRID · NOV 27-28 · 2015 At CERN they build things toowell!
- 25. MADRID · NOV 27-28 · 2015 Stretching the paradigm Backend Html JavaScript
- 26. MADRID · NOV 27-28 · 2015 Stretching the paradigm Backend Html JavaScript API
- 27. MADRID · NOV 27-28 · 2015 Not what it was built for Request overhead Granularity compromise: Requests/min vs Resources Chaining events: waiting times add up. IoT capable? Like asking for a video one frame at a time
- 28. MADRID · NOV 27-28 · 2015 What do we mean by Real Time
- 29. MADRID · NOV 27-28 · 2015 Hard Real Time Maximum response time guaranteed We won’t talk about this
- 30. MADRID · NOV 27-28 · 2015 Soft Real Time Reasonable response time Normally based on human perception
- 31. MADRID · NOV 27-28 · 2015 Soft Real Time For information that loses value with time
- 32. MADRID · NOV 27-28 · 2015 Value with time Your neighbour starts screaming
- 33. MADRID · NOV 27-28 · 2015 Use cases where RT is more beneficial
- 34. MADRID · NOV 27-28 · 2015 Analytics
- 35. MADRID · NOV 27-28 · 2015
- 36. MADRID · NOV 27-28 · 2015 Gaming
- 37. MADRID · NOV 27-28 · 2015 Social & Messaging
- 38. MADRID · NOV 27-28 · 2015 Collaboration
- 39. MADRID · NOV 27-28 · 2015 Amazon Customer Service
- 40. MADRID · NOV 27-28 · 2015 Basically, the less your app looks like this
- 41. MADRID · NOV 27-28 · 2015 For how long can we live in a past paradigm? Kano model
- 42. MADRID · NOV 27-28 · 2015 Architecture
- 43. MADRID · NOV 27-28 · 2015 WebSockets, what are they Hack (Standarized, but hack) to turn HTTP back into a TCP full-duplex connection and call it “upgrade”. GET /chat HTTP/1.1 Host: server.example.com Upgrade: WebSocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Origin: http://example.com Sec-WebSocket-Version: 13 HTTP/1.1 101 Switching Protocols Upgrade: WebSocket Connection: Upgrade Sec-WebSocket-Accept: s3pPL<…>Oo=
- 44. MADRID · NOV 27-28 · 2015 Can I use Websockets? 88.17%
- 45. MADRID · NOV 27-28 · 2015 But Long polling… Make an Ajax Request, leave it open waiting for a Response. Introduces complexity: group chained requests into “connections”. Needs buffering handling. Adds complexity to support the remaining 11.83%.
- 46. MADRID · NOV 27-28 · 2015 Three layers-model Transport Messaging Data Sync Regular HTTP HTTP Request/Response Probably your work Real Time Websockets (or Long polling) Publish/Subscribe (or RPC) Probably your work
- 47. MADRID · NOV 27-28 · 2015 There is a fourth layer Everything our tech model cannot handle, our brain has to make up for it
- 48. MADRID · NOV 27-28 · 2015
- 49. MADRID · NOV 27-28 · 2015 F5F5F5F5F5F5F5F5F5F5F5F5F5F5F5!!!!!!! Human dealing with a protocol that doesn’t map reality well
- 50. MADRID · NOV 27-28 · 2015 Three layers-model Transport Messaging Data Sync Core of the product Where you add value Implemented in a language you know well
- 51. MADRID · NOV 27-28 · 2015 Three layers-model Transport Messaging Data Sync Implemented in a technology very performant Don’t mix your business logic with it
- 52. MADRID · NOV 27-28 · 2015 Aim for Transport Messaging Data Sync Regular HTTP Your work Real Time Your work Nginx/Apache RT Server
- 53. MADRID · NOV 27-28 · 2015
- 54. MADRID · NOV 27-28 · 2015 DIY approach (typically socket.io) Tutorial looks super easy, but: Soon you realise that you must deal with messaging details (who is in a channel? how to do auth?). Multi-server environments are up to you (how to know if a user is connected in a channel in any server?). Tendency is to duplicate code between socket.io realm and your business, fight against this. Transport & messaging may be conditioning the technology you use for your business logic
- 55. MADRID · NOV 27-28 · 2015 Also, may I suggest? If you are going this path, check out Erlang/Elixir. (btw, this is what we did)
- 56. MADRID · NOV 27-28 · 2015 Options available (a lot)
- 57. MADRID · NOV 27-28 · 2015 Options PAAS. Data-centric (talk to a NoSQL DB). OpenSource. Extendible with node.js or Ruby code. PAAS. PubSub channels. RT CDN Infrastructure. Lots of SDKs in different languages. Black box oriented. OpenSource. PubSub channels. Black box oriented. OpenSource. PubSub (&RPC). Platform oriented.
- 58. MADRID · NOV 27-28 · 2015 Black Box Approach RT server like Nginx or Apache (no need to care wether Nginx is written in C or Haskell?). Conflict zones: Authentication and Authorization: Implement in RT server vs in your logic. Communication between regular HTTP server in HTTP Request/Response Fashion. IMHO this makes sense. (You may have other opinion and that is cool)
- 59. MADRID · NOV 27-28 · 2015 How does it work: Client side
- 60. MADRID · NOV 27-28 · 2015 Publish to a channel Carotene.publish({channel: "mychannel", message: "Hello world!"}); channel.trigger("message", "Hello world!"); pubnub.publish({ channel : "mychannel", message : "Hello world!", callback: function(m){ console.log(m); } }); Carotene Pusher PubNub Anything JSON-Serializable
- 61. MADRID · NOV 27-28 · 2015 Carotene Subscribe to a channel Carotene.subscribe({channel: "mychannel", onMessage: function(message) { console.log(message); } }); Pusher PubNub channel.bind('my-event', function(data) { console.log(data.message); }); PUBNUB.subscribe({ channel: 'my-channel', message: function(m){console.log(m)} });
- 62. MADRID · NOV 27-28 · 2015 Authentication: Who are you? Carotene.authenticate({ userId: "some_user_identifier", token: "token_for_this_user" }); Carotene Your server generates this Check out JWT [{carotene, [ % ... Other configuration options {authenticate_url, "http://mybackend.com/authenticate_carotene/"} }]} Your server receives a POST request
- 63. MADRID · NOV 27-28 · 2015 [{carotene, [ % ... Other configuration options {subscribe_authorization, [ {level, anonymous} ]}, }]} Anonymous allowed Authorization: Can you do that?
- 64. MADRID · NOV 27-28 · 2015 [{carotene, [ % ... Other configuration options {subscribe_authorization, [ {level, authenticated} ]}, }]} Only authenticated allowed Authorization: Can you do that?
- 65. MADRID · NOV 27-28 · 2015 [{carotene, [ % ... Other configuration options {level, ask}, {authorization_url, “http://mybackend.com/authorize-subscribe"} ]}, }]} Let your logic decide Publishing follows the same pattern Authorization: Can you do that?
- 66. MADRID · NOV 27-28 · 2015 Examples
- 67. MADRID · NOV 27-28 · 2015 Simple chat
- 68. MADRID · NOV 27-28 · 2015 RT # subscribe({channel: “chat”}) # subscribe({channel: “chat”}) #subscribe({channel: “chat”})
- 69. MADRID · NOV 27-28 · 2015 RT # publish({channel: “chat”, msg:”hi”}) # message({channel: “chat”, msg: “hi”}) # message({channel: “chat”, msg: “hi”}) message({channel: “chat”, msg: “hi”})
- 70. MADRID · NOV 27-28 · 2015 Simple chat with logging
- 71. MADRID · NOV 27-28 · 2015 RT # # # ! subscribe({channel: “chat”}) regular HTTP
- 72. MADRID · NOV 27-28 · 2015 RT # # message({channel: “chat”, msg: “hi”}) # message({channel: “chat”, msg: “hi”}) ! publish({channel: “chat”, msg:”hi”}) message({channel: “chat”, msg: “hi”}) message({ channel: “chat”, msg: “hi”})
- 73. MADRID · NOV 27-28 · 2015 Customer “I want to filter out telephone numbers and emails” Ok
- 74. MADRID · NOV 27-28 · 2015 # subscribe({channel: “chat”}) # subscribe({channel: “chat”}) # subscribe({channel: “chat”}) RT !
- 75. MADRID · NOV 27-28 · 2015 RT ! # # # $message({channel: “chat”, msg:”hi”}) message({channel: “chat”, msg:”hi”}) message({channel: “chat”, msg:”hi”}) publish({channel: “chat”, msg:”hi”}) Ajax publish({ channel: “chat”, msg:”hi”}) Http
- 76. MADRID · NOV 27-28 · 2015 Just because we have RT we don’t need to stop using regular HTTP
- 77. MADRID · NOV 27-28 · 2015 Notify in Real Time vs notify per e-mail or mobile
- 78. MADRID · NOV 27-28 · 2015 RT # [Id: 9] subscribe({channel: “user-9”}) ! %! presence({channel: “user-9”}) presence([9]}) auth({user: 9, token: <token>}) & '
- 79. MADRID · NOV 27-28 · 2015 RT # [Id: 9] !send({channel: “user-9”, message: <notification>}) message({channel: “user-9”, message: <notification>}) & '
- 80. MADRID · NOV 27-28 · 2015 RT ! %! presence({channel: “user-9”}) presence([]}) & '
- 81. MADRID · NOV 27-28 · 2015 Broadcast news to everyone (live scores, new comments, …)
- 82. MADRID · NOV 27-28 · 2015 RT # subscribe({channel: “page”}) # subscribe({channel: “score”}) #subscribe({channel: “score”}) !
- 83. MADRID · NOV 27-28 · 2015 RT # # message({channel: “score”, msg: “1:1”}) #message({channel: “score”, msg: “1:1”}) ! publish({ channel: “score”, msg: “1:1” }) regular HTTP message({channel: “score”, msg: “1:1”}) This case scales very well (no need to store much state in RT)
- 84. MADRID · NOV 27-28 · 2015 Do your tests Very different possible scenarios. Benchmarks tend to test the simplest case and brag about the number of connections. (I like to do this too :). Set up a load test with Tsung or Erlang+Gun to simulate your use case: • Expected #connections per channel • Expected #msgs published per user
- 85. MADRID · NOV 27-28 · 2015
- 86. MADRID · NOV 27-28 · 2015 Thanks! @nacmartin [email protected] http://carotene-project.com http://limenius.com