Hijacking Ruby Syntax in Ruby
4 likes44,216 views
This document summarizes a presentation given by @joker1007 and @tagomoris on hijacking Ruby syntax. It discusses various Ruby features like bindings, tracepoints, and refinements that can be used to modify Ruby's behavior. It then demonstrates several "hacks" that leverage these features, including finalist, overrider, and abstriker which add method modifiers, and binding_ninja which allows implicitly passing a binding. Other hacks discussed are with_resources and deferral which add "with" and "defer" statements to Ruby. The presentation emphasizes how these modifications are implemented using hooks, bindings, tracepoints and other Ruby internals.
Hijacking Ruby Syntax in Ruby
- 1. HIJACKING RUBY SYNTAX IN RUBY RubyKaigi 2018 Sendai #rubykaigi 2018/05/31 (Fri) @joker1007 & @tagomoris
- 2. Self-Intro Joker ▸ id: joker1007 ▸ Repro inc. CTO ▸ I’m familiar with Ruby/Rails/Fluentd/ECS/Presto. ▸ I ♥ “Jojo’s Bizarre Adventure” so much. ▸ I’m very happy to talk in Sendai (Moriocho)
- 3. Satoshi Tagomori (@tagomoris) Fluentd, MessagePack-Ruby, Norikra, Woothee, ... Treasure Data, Inc.
- 5. Agenda ▸ Ruby Features ▸ Hacks ▸ finalist, overrider, abstriker ▸ binding_ninja ▸ with_resources, deferral
- 6. Ruby Features: Binding Binding ▸ Context object, includes: ▸ Kernel#binding receiver ▸ local variables ▸ For template engines (?) ▸ Methods: ▸ #receiver, #eval, #local_variables, #local_variable_get, #local_variable_defined?, #local_variable_set
- 7. テキスト
- 8. テキスト
- 9. テキスト
- 10. Ruby Features: Binding Binding#local_variable_set ▸ Method to ▸ add a variable only in a binding instance ▸ overwrite values of existing variables in original context
- 11. Ruby Features: TracePoint TracePoint ▸ Tracing events in VM ▸ For various events: ▸ :line , :raise ▸ :class , :end ▸ :call , :return , :c_call , :c_return , :b_call , :b_return ▸ :thread_begin , :thread_end , :fiber_switch ▸ "We can use TracePoint to gather information specifically for exceptions:" (from Ruby doc) ▸ This is COMPLETELY WRONG statement...
- 12. Ruby Features: TracePoint TracePoint Methods ▸ Methods: ▸ Control: disable, enable, enabled? ▸ Event what/where: event, defined_class, path, lineno ▸ Method names: method_id, callee_id ▸ Event special: raised_exception, return_value ▸ And: binding ▸ So what? ▸ We can use TracePoint ▸ to gather information ▸ to overwrite everything
- 17. Ruby Features: Refinements ▸ Refinements provide a way to extend a class locally ▸ Useful use case. (Safety monkey patching) Ruby Features: Refinements
- 18. Another use case ▸ Super private method Ruby Features: Refinements
- 19. Hacks 1 Method modifiers ▸ final (https://github.com/joker1007/finalist) ▸ forbid method override ▸ override (https://github.com/joker1007/overrider) ▸ enforce method has super method ▸ abstract (https://github.com/joker1007/abstriker) ▸ enforce method override These method modifiers work when Ruby defines class. It is actually runtime, but in most case, before main logic. Hacks: method modifiers
- 20. Sample code: finalist Hacks: method modifiers
- 21. Sample code: overrider Hacks: method modifiers
- 22. Sample code: abstriker Hacks: method modifiers
- 23. How to implement method modifiers I use so many hook methods. `included`, `extended`, `method_added`, and `TracePoint`. And I use `Ripper`. In other words, I use the power of many black magics !! Hacks: method modifiers
- 24. Ruby Features: Method Hooks ▸ Ruby has six method hooks ▸ Module#method_added ▸ Module#method_removed ▸ Module#method_undefined ▸ BasicObject#singleton_method_added ▸ BasicObject#singleton_method_removed ▸ BasicObject#singleton_method_undefined Ruby Features: method hooks
- 25. Sample: #method_added Ruby Features: method hooks
- 26. Sample code: #method_removed Ruby Features: method hooks
- 27. Sample code: #method_undefined Ruby Features: method hooks
- 28. Use case in finalist (Simplified sample) Ruby Features: method hooks
- 29. call method_added -> call verify_final_method
- 30. Usage: Method Hook ▸ Method Hook provides a way to implement declarations (like protected, private) ▸ With class inheritance, Method Hook seems magic !! ▸ But it is not enough to implement “finalist” actually ▸ Ruby has so many cases of method definition ▸ `def` or `define_method` ▸ `include` module ▸ `extend`, `prepend ▸ Each case calls different hooks Ruby Features: method hooks
- 31. “include” does not add method ▸ Because `include` insert module to method lookup hierarchy, it does not touch method entry table. Ruby Features: method hooks
- 32. “include” changes only chain of method discovering Foo Object Bar Insert module to hierarchy It is different from to add method Class#ancestors displays class-module hierarchy Ruby Features: method hooks
- 33. Hooks in finalist gem ▸ `method_added` to detect override by subclass ▸ `singleton_method_added` to detect override class method ▸ `included` to detect override by module include ▸ `extended` to detect override by module extend
- 34. And I talk about TracePoint too
- 35. overrider and abstriker uses TracePoint ▸ `inherited` and `included` to start `TracePoint`` call main logic call main logic Tracepoint in overrider, abstriker
- 36. Tracepoint in overrider, abstriker Check abstract method existence in class hierarchy
- 37. Why use TracePoint? ▸ In order to verify method existence at the end of class definition. ▸ Ruby interpreter needs to wait until the end of class definition to know a method absence. ▸ override and abstract cannot detect violation just when they are called. ▸ In ruby, The only way to detect the violation is `TracePoint`.
- 38. Advanced TracePoint: Detect particular class end Advanced Tracepoint :end event cannot trace definition by `Class.new`. Use :c_return and return_value to detect particular class end
- 39. Advanced TracePoint: Ripper combination Advanced Tracepoint Detect target Sexp node by TracePoint#linen Sexp node type expresses the style of method cal
- 40. Ripper empowers TracePoint ▸ `Ripper.sexp` returns token position ▸ And TracePoint returns lineno an event occurs. ▸ Ripper provides more information about event context. ▸ ex. power_assert gem is implemented by this combination. Advanced Tracepoint
- 41. Hacks 2 (binding_ninja) ▸ https://github.com/joker1007/binding_ninja ▸ It is method wrapper to pass binding of method caller implicitly. ▸ I talked about it at Rubykaigi 2017 LT. Hacks: binding_ninja
- 42. Sample code: binding_ninja Hacks: binding_ninja Pass binding of caller implicitly
- 43. How To Implement ▸ Binding is based on Ruby level control frame. ▸ When Ruby calls c-method, Ruby does not change Ruby level control frame. ▸ It is core logic of binding_ninja create binding in C level, binding keeps caller stack in Ruby level Hacks: binding_ninja
- 44. Use case: binding_ninja ▸ I implement scala like implicit parameter in Ruby. ▸ https://github.com/ joker1007/ implicit_parameter Hacks: binding_ninja
- 45. Hacks: binding_ninja Wrap method by `define_method` And wrap method by binding_ninja
- 46. Black Magic is dangerous actually, but it is very fun, and it extends Ruby potential These gems is for proof of concept. But these work and decent practical.
- 47. Ruby Quiz Break (25-30min) What the difference between: - #undef_method - #remove_method
- 48. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo def foo Bar.new.foo()
- 49. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo Bar.new.foo() undef_method(:foo) NoMethodError remove_method(:foo)
- 50. Hack: with_resources Add "with" Statement in Ruby ▸ Safe resource allocate/release ▸ Ensure to release resources ▸ after a lexical scope ▸ in reverse order of allocation ▸ Idioms used very frequently ▸ Other languages: ▸ Java: try-with-resources ▸ Python: with ▸ C#: using
- 51. Hack: with_resources Safe Resource Allocation/Release Statement in Ruby ▸ Open method with blocks ▸ File.open(path){|f| ... } ▸ Ruby way (?) ▸ More indentation ▸ Not implemented sometimes (e.g., TCPSocket) ▸ Simple begin-ensure ▸ Anomaly cases can't be handled
- 52. Hack: with_resources with_resources.gem ▸ Safe resource allocation ▸ Top level "with" ▸ via Kernel refinement ▸ Resource allocation as lambda ▸ Multi statements to allocate resources ▸ to release first resource if second resource allocation raises exception ▸ Block to define scope for resources https://github.com/tagomoris/with_resources
- 53. Hack: with_resources Implementing "with" in Ruby ▸ TracePoint ▸ "b_return": pass allocated resources to block arguments ▸ "line": identify allocated resources in lambda ▸ Binding ▸ detect newly defined local variables in allocation lambda ▸ Refinements ▸ introduce "with" in top-level without side effects https://github.com/tagomoris/with_resources
- 54. Hack: deferral Alternative: defer? ▸ Multi-step resource allocation in a method ▸ using "with" ▸ Nesting! ▸ not so bad ▸ many nesting looks a bit messy :( ▸ Alternative? ▸ "defer" in golang
- 55. Hack: deferral Making "defer" in Ruby ▸ Block based "defer" ▸ Should work ▸ Requires 1-level nesting *always* ▸ Defer.start, end (+ nesting) look too much (than golang) ▸ Abnormal case: reassigning variables
- 56. Hack: deferral deferral.gem ▸ Safe resource release ▸ Top level "defer" ▸ via Kernel refinements ▸ Deferred processing to release resources ▸ at the end of scope (method, block) ▸ or exception raised
- 57. Hack: deferral Implementing "defer" in Ruby ▸ #defer ▸ Enable TracePoint if not yet ▸ Initialize internal stack frame ▸ TracePoint ▸ Monitor method call stack ▸ Get the snapshot of local variables in defer block ▸ Call release blocks at the end of scope ▸ Binding ▸ save/restore local variables of release block ▸ Refinements ▸ introduce "defer" in top-level without side effects stack level 0 stack level 1
- 58. The Hard Thing in Magical World: Debugging!
- 59. "Are You Ready? I'm OK." Blono Bucciarati Thank You! @joker1007 & @tagomoris