I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
4 likes2,574 views
The document discusses best practices for consuming and producing HTTP APIs in the Ruby ecosystem, focusing on REST principles, caching, serialization, and authentication using OAuth 2. It emphasizes the importance of separating API and website controllers, utilizing appropriate HTTP status codes, and designing for machine interactions rather than human users. Additionally, it highlights key features for an effective HTTP client and encourages a clean domain model while respecting HTTP and REST constraints.
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
- 1. I can haz HTTP Consuming and producing HTTP APIs in the Ruby ecosystem
- 3. kaiwren
- 4. @ponnappa
- 6. achamian
- 7. @niranjan_p
- 9. Open Source (gem install) wrest (gem install) pox_paginate
- 10. Not about HTTP libs not per se...
- 11. More about...
- 12. REST
- 13. caching
- 15. authentication
- 18. Producer
- 21. digression
- 22. RESTful?
- 24. L0: Plain Old Xml
- 25. L1: Resources L0: Plain Old Xml
- 26. L2: HTTP Verbs L1: Resources L0: Plain Old Xml
- 27. L3: Hypermedia L2: HTTP Verbs L1: Resources L0: Plain Old Xml
- 29. L2 is achievable with minimal effort in Rails
- 30. getting back to the point
- 31. Don’t mix machine stuff with human stuff
- 32. APIs are for machines
- 33. mixing the two causes trouble
- 37. basically, doing this causes trouble def index respond_to do |format| e format.html format.json { render :json => Project.all.to_json } end end
- 38. because, people like usable apps
- 39. separate API controllers from website controllers
- 40. or beyond even that...
- 41. your app only exposes APIs your website is a separate application that consumes these APIs; think NewTwitter
- 42. don’t design exclusively for ActiveResource it isn’t a standard
- 44. rails has nice defaults 201 for creates 401 for auth violations 405 for L2 violations 406 for Content-Type violations 422 for invalid data 500 for server faults 200 for everything else
- 45. let rails help you
- 46. what they missed
- 47. 404
- 48. 409
- 49. assert on status codes gem install rspec-http response.should be_im_a_teapot
- 50. Authentication
- 51. OAuth 2 is nice
- 52. be an OAuth provider consistency makes life easier
- 53. https://
- 54. Caching
- 55. go beyond Rails’ local caches
- 56. the Cache-Control header is your friend
- 57. Versioning
- 59. good versioning is critical publish clear roadmaps and deprecate sensibly
- 60. Content Types
- 62. Accept headers
- 65. rails respects it you should use it
- 66. Important stuff that we’ve never done
- 67. Throttling https://github.com/dambalah/api-throttling (thanks @godfoca)
- 68. Metering
- 69. Consumer
- 70. rails API vs. everything else
- 71. ActiveResource
- 72. 60:40
- 73. the rest of the time you monkeypatch but that isn’t so easy, so be careful
- 74. ActiveResource works best inside a walled garden (IMHO)
- 76. some APIs are weird
- 77. everyone’s standardizing on REST but it’s what they call REST
- 78. suffice to say, one size doesn’t fit all except for OAuth 2, which everyone’s adopting
- 79. building a clean domain model for the API is key
- 80. OAuth 2 is ridiculously easy to write a client for #justSaying https://github.com/kaiwren/wrest/tree/master/ examples/facebook_auth
- 81. authorisation on the other hand still has no standard implementation
- 82. What should a good HTTP client do for you?
- 83. this one needs bullets • Verbs - should support GET, POST, PUT & DELETE. • If it supports PATCH & OPTIONS, epic. • Transparent deserialization of XML and JSON, with the option to easily add more • HTTP caching (RFC 2616) • Keep-alive connections (because negotiating SSL can get expensive) • Thread safety • Transparent compression/decompression using Accept-Encoding • A fluent, easy to use API
- 85. verbs
- 86. serialization I rather like ActiveSupport
- 87. caching RFC 2616
- 88. thread safety
- 89. fluent api is it easy to work with uris, because you’ll do a fair bit of that when building domains
- 90. is it easy to build a domain model using it?
- 92. Shared stuff
- 95. Builder Pure Ruby XML serializer. Pretty Fast. Just does String concats.
- 96. gotcha
- 97. no DOM for Builder so no guarantee that you get valid XML
- 98. testing this can be tricky
- 99. Conclusion
- 100. respect HTTP
- 102. spend less time messing with plumbing
- 103. focus on building a clean domain model
- 104. Photo credits David Blackwell, Elephant in Room: http://www.flickr.com/photos/mobilestreetlife
- 105. Funny picture? cat with mausambi hat
- 106. 42 Sidu Ponnappa Niranjan Paranjape http://twitter.com/ponnappa http://twitter.com/niranjan_p https://github.com/kaiwren https://github.com/achamian
Editor's Notes
- #2: \n
- #3: \n
- #4: \n
- #5: \n
- #6: \n
- #7: \n
- #8: \n
- #9: \n
- #10: Opensource we have done in this area\n
- #11: This talk is not about NetHTTP vs libcurl\n
- #12: \n
- #13: \n
- #14: \n
- #15: \n
- #16: \n
- #17: \n
- #18: This talk does have *some* structure; we’ll look at APIs from the perspective of the producer and the consumer. With a little bit left over for stuff that bridges the gap.\n
- #19: \n
- #20: Everyone talks about how easy it is to produce APIs with Sinatra and such like, but lets be honest. There’s an elephant in the room - those of you that freelance know exactly how many requests we get for APIs on Sinatra versus APIs on...\n
- #21: [Handoff]\n\n...Rails. Love it or hate it, Rails has a standard way of producing APIs, and understanding what Rails can or cannot do for you is important. Sinatra does not have these constraints, so you’re free to roll APIs any way you want, so we’ll focus on the framework that does impose significant patterns.\n
- #22: [achamian]\n\nBut before we launch into Rails’ capabilities, it’s important to put things into context first by talking a little about REST\n
- #23: What is a RESTful API?\n
- #24: For this we turn to Leonard Richardson who came up with a maturity model that has gotten so famous it has an acronym and looks like...\n
- #25: ...this, popularly known as the Richardson Maturity Model, or RMM. \nIt begins with Level 0, which uses plain old xml (or a similar encoding format) to tunnel requests to a HTTP endpoint. Think RPC. Clearly Rails is better than this.\nNext comes Level 1, which introduces the concept of Resources - every URI represents a resource. Easy peasy. Rails makes it easy to do this - but you’re free to deviate.\nThen comes Level 2, which defines a certain set of constraints around how HTTP verbs are used. Rails violates the semantics by using PUT instead of PATCH - this was fixed on May 07 with Pull request #348, so future releases should be L2 compliant.\nFinally, we come to L3 which involves using hypermedia to define the edges of the state graph of an application using hypermedia links. This definitely isn’t in Rails. Yet.\n
- #26: ...this, popularly known as the Richardson Maturity Model, or RMM. \nIt begins with Level 0, which uses plain old xml (or a similar encoding format) to tunnel requests to a HTTP endpoint. Think RPC. Clearly Rails is better than this.\nNext comes Level 1, which introduces the concept of Resources - every URI represents a resource. Easy peasy. Rails makes it easy to do this - but you’re free to deviate.\nThen comes Level 2, which defines a certain set of constraints around how HTTP verbs are used. Rails violates the semantics by using PUT instead of PATCH - this was fixed on May 07 with Pull request #348, so future releases should be L2 compliant.\nFinally, we come to L3 which involves using hypermedia to define the edges of the state graph of an application using hypermedia links. This definitely isn’t in Rails. Yet.\n
- #27: ...this, popularly known as the Richardson Maturity Model, or RMM. \nIt begins with Level 0, which uses plain old xml (or a similar encoding format) to tunnel requests to a HTTP endpoint. Think RPC. Clearly Rails is better than this.\nNext comes Level 1, which introduces the concept of Resources - every URI represents a resource. Easy peasy. Rails makes it easy to do this - but you’re free to deviate.\nThen comes Level 2, which defines a certain set of constraints around how HTTP verbs are used. Rails violates the semantics by using PUT instead of PATCH - this was fixed on May 07 with Pull request #348, so future releases should be L2 compliant.\nFinally, we come to L3 which involves using hypermedia to define the edges of the state graph of an application using hypermedia links. This definitely isn’t in Rails. Yet.\n
- #28: So what level of compliance can we achieve with a minimum of effort?\n
- #29: [Handoff]\n\nI’d say an L2 is quite easily do-able in rails without too much effort. Now lets talk about how.\n
- #30: [kaiwren]\n\nSo after that little digression, lets go back to what we were originally talking about - how to build APIs on Rails.\n
- #31: [kaiwren]\n\nThis is one of the easiest thing to forget - APIs are for machines and software, websites are for people. Mixing the two is Not Good.\n
- #32: \n
- #33: \n
- #34: Hollywood has shown us the consequence of mixing people stuff with machine stuff time and again, and it isn’t good.\n
- #35: \n
- #36: \n
- #37: \n
- #38: and REST cares only about state transitions. If you mix these, then remember that this only works in the most trivial of use cases. Beyond that...\n
- #39: \n
- #40: \n
- #41: Your app doesn’t have a single HTML page. This is especially useful if you expect 3rd parties to use your APIs - you’re dogfooding your own stuff from day one.\n
- #42: [Handoff]\n\nif your api is being consumed by 3rd parties, be aware that they will use a fairly eclectic collection of HTTP libs. What’s convenient/intuitive when using AR may be extremely obscure when using simple HTTP calls.\n
- #43: [achamian]\n\nThe next thing everybody should keep an eye on when building Rails APIs are the HTTP status codes. There are many of them, and your APIs should respect them.\n
- #44: The first thing everybody ignores when building Rails APIs are the HTTP status codes. There are many of them, and your APIs should respect them.\n
- #45: Adhere to these defaults in the code you write. They’re there for a reason. Don’t, for example, return a 200 with an error message in the body - that’s bad form.\n
- #46: There are, however, a couple of codes they deal with weirdly, or not at all\n
- #47: The handling of 404s can be strange, driven as it is in certain cases by the ActiveRecord::RecordNotFound exception. Make sure that if a resource is not found, you’re always returning a 404 status code and you write a spec to expect it.\n
- #48: Have a unique constraint on a resource identifier which is part of the input and not system assigned? When it’s violated, return a 409-Conflict and not a 422-Unprocessable entity.\n
- #49: It’s important to lock these codes in because they mean a lot to the outside world. When building apis, make sure all your controller specs include assertions on the response code. I’ve got a simple gem that can help if you use rspec. Try it out. That asserts that a response has a code of 418.\n
- #50: \n
- #51: \n
- #52: This way your application doesn’t need to deal with your own users through one auth mechanism and those that come through fb/twitter/github/sf through another\n
- #53: Oauth 2 requires SSL, so do keep that in mind.\n
- #54: \n
- #55: \n
- #56: If you have resources that don’t need to be refreshed as soon as they change, you should use an appropriate expires header to allow intermediate nodes on the network to cache them for a specified period of time. This can massively reduce load on your servers.\n
- #57: \n
- #58: \n
- #59: Just like any library or gem, your users require visibility and stability. Whether you use URL based versioning (by including v1/v2/v3) in the API url, or by headers that specify the version, make sure your versioning roadmap is clear, consistent and sensible. Try maintaining backward compatibility as far as is reasonable.\n
- #60: Content Types are rarely done wrong, so I won’t talk about them for too long. There’s only one thing that I’ve observed out here...\n
- #61: ... is folks tunneling application/x-www-form-urlencoded over application/xml or application/json.\n
- #62: \n
- #63: \n
- #64: a suffix of .xml or .json in the URI is to make it easier to look at api responses in the browser. While consuming an API you should stick to the Accept header.\n
- #65: There are a couple of important things when dealing with APIs that I’ve never done and have no first-hand experience with\n
- #66: Solved using rack middleware\n
- #67: [Handoff]\n
- #68: [kaiwren]\n
- #69: Are you talking to a rails app (typically within your own walled garden) or to an API out on the internets?\n
- #70: Let’s talk about the consumer that ships with Rails - ActiveResource\n
- #71: ActiveResource follows the 60/40 rule not 80/20 - it works for the straightforward use cases, and does so smoothly and well\n
- #72: Trying accessing the header information for a response\n
- #73: \n
- #74: \n
- #75: Passing parameters for a post request. Accepting access token as a parameter to a Get over https \n
- #76: \n
- #77: \n
- #78: \n
- #79: You don’t need gems for OAuth2 clients. Just roll your own - it takes 20 minutes tops.\n
- #80: \n
- #81: \n
- #82: \n
- #83: \n
- #84: \n
- #85: \n
- #86: \n
- #87: \n
- #88: \n
- #89: \n
- #90: Can I figure out the verb, the connection (if keep alive), corelate the req and response a situation where parallel threads are logging to the same log\n
- #91: These are the bits that both producers and consumers deal with\n
- #92: A significant proportion of the time an API takes to process something is spent doing serialisation and deserialization\n
- #93: Remember to switch your deserialization backend from REXML to Nokogiri which uses the native libxml library. It’s way way faster.\n
- #94: The default XML serializer in Rails is written in pure Ruby. It’s pretty fast, and there are drop in native alternatives based off libxml that are upto 50% faster, but that aren’t particularly popular.\n
- #95: The big gotcha with builder occurs when you’re generating custom XML\n
- #96: you want to run all the custom xml you create using builder’s DSLs through Nokogiri’s deserialise to make sure its valid XML in your specs\n
- #97: In addition to controller specs, it makes sense to have full, live tests hitting a real server, especially if you’re integrating across multiple APIs that you’ve authored\n
- #98: \n
- #99: \n
- #100: \n
- #101: \n
- #102: \n
- #103: \n
- #104: \n
- #105: \n