Post-Mortem Debugging and Web Development
2 likes4,783 views
The document discusses various methodologies and tools for post-mortem debugging and web development, emphasizing the importance of logging errors effectively and using debugging tools. It highlights the use of frameworks like Backlash, Pyramid, and Django for error handling, as well as the advantages of interactive debuggers. Additionally, it covers techniques for monitoring performance issues and suggests tools for profiling in production environments.
![Ready-Made Email Reporting
● Looking for a stand-alone solution?
○ Backlash can be used by itself, not bound to
TurboGears.
○ Only dependency is “WebOb”
● Supports both Python2 and Python3
from backlash.trace_errors import EmailReporter
email_reporter = EmailReporter(smtp_server="localhost",
error_email="email@host.com",
from_address="errors@host.com")
app = backlash.TraceErrorsMiddleware(app, [email_reporter])](https://image.slidesharecdn.com/fosdem2014-post-mortemdebuggingandwebdevelopment-140202060842-phpapp01/85/Post-Mortem-Debugging-and-Web-Development-11-320.jpg)
![Try Sentry
● Gathers data and detects duplicates
● Provides “Reply Request” button
● Mark exceptions as solved to keep track
● Can log multiple events, not only
exceptions
from backlash.trace_errors.sentry import SentryReporter
sentry_reporter = SentryReporter(sentry_dsn="http://public:secret@example.com/1")
app = backlash.TraceErrorsMiddleware(app, [sentry_reporter])](https://image.slidesharecdn.com/fosdem2014-post-mortemdebuggingandwebdevelopment-140202060842-phpapp01/85/Post-Mortem-Debugging-and-Web-Development-12-320.jpg)
Post-Mortem Debugging and Web Development
- 1. POST-MORTEM DEBUGGING AND WEB DEVELOPMENT Alessandro Molina @__amol__ [email protected]
- 2. Who am I ● CTO @ Axant.it, mostly Python company (with some iOS and Android) ● TurboGears2 dev team member ● Contributions to web world python libraries ○ MING MongoDB ODM ○ ToscaWidgets2 ○ Formencode
- 3. Why ● Debugging is a core part of the development process. ● You can try to prevent issues as much as possible, but users will find new ones. ● Gathering informations to replicate issues is required to fix them
- 4. Debugging
- 5. At least know you have an issue. ● Debugging is the “process of finding and reducing the number of bugs” ● Users are already doing half of the work. ● When users find a bug for you, make sure to be aware that they found it ● Log it or it will be forgotten
- 6. Log what you really need ● Log to replicate ○ Then you are able to write tests that actually verify the issue has been solved. ● You probably want to log: ○ WSGI Environ ○ Traceback ○ Last N stack frames local variables ○ Request Headers & Body (when size permits)
- 7. Log in an useful way ● Log data in an easy to use way ○ Log request in a way it’s quick to replay it ○ Log in a computer friendly format for test units ○ Request dump is good: netcat to replay it and is already understood by computers. ● Organize your informations ○ Use a tool to group and avoid duplicates ○ Know what got solved and what didn’t ○ Log by email, so you don’t have to check yourself
- 8. Log or...
- 9. But log in a separate thread or...
- 10. Crash Report ● Many middlewares and frameworks provide exception reporting by email: ○ pyramid_exclog (Pyramid) ○ WebError (Pylons) ○ BackLash (TurboGears) ○ Django & Flask, framework provided ● Logging module has what you need: Logger.exception + handlers.SMTPHandler
- 11. Ready-Made Email Reporting ● Looking for a stand-alone solution? ○ Backlash can be used by itself, not bound to TurboGears. ○ Only dependency is “WebOb” ● Supports both Python2 and Python3 from backlash.trace_errors import EmailReporter email_reporter = EmailReporter(smtp_server="localhost", error_email="[email protected]", from_address="[email protected]") app = backlash.TraceErrorsMiddleware(app, [email_reporter])
- 12. Try Sentry ● Gathers data and detects duplicates ● Provides “Reply Request” button ● Mark exceptions as solved to keep track ● Can log multiple events, not only exceptions from backlash.trace_errors.sentry import SentryReporter sentry_reporter = SentryReporter(sentry_dsn="http://public:[email protected]/1") app = backlash.TraceErrorsMiddleware(app, [sentry_reporter])
- 13. Sentry
- 14. See what’s happening now ● On development and test environments receiving errors by email is not convenient ● In case of freezes you don’t get an exception that can be reported ● Some problems need to be troubleshooted on the fly
- 15. Interactive Debugger ● Python provides built-in Post Mortem debugging through pdb module try: return app(environ, start_response) except: import pdb pdb.post_mortem() ● While it’s ready to use and works great it’s not the most comfortable tool when working on web development
- 16. Browser Debugger ● Many frameworks have browser debugger ○ Pyramid DebugBar ○ Werkzeug DebuggedApplication ○ TurboGears ErrorWare ● BackLash provides a stand-alone version of the Werkzeug debugger. import backlash app = backlash.DebuggedApplication(app)
- 18. Browser Debugger Console ● Browser debuggers usually implement a special URL with an interactive console that runs in the context of the application. ● Use it to see what’s happening right now ○ for threadId, stack in sys._current_frames().items() ● Try /__console__ on Werkzeug or Backlash
- 19. Production Debugging ● In-browser debugger must be disabled on production environments ○ Unless you are PythonAnyware you want to block people from running code on your servers. ● So we are back again at the starting point ○ How do I debug issues that don’t provide an exception traceback on production?
- 20. Attaching to running processes To inspect running applications you can rely on tools like ispyd and pyrasite that are able to attach to a running python process from ispyd.plugins.wsgi import WSGIApplicationWrapper app = WSGIApplicationWrapper(app)
- 21. ispyd (wsgi:18630) requests No active transactions. (wsgi:18630) requests ==== 67 ==== thread_id = 140735076232384 start_time = Mon Apr 9 21:49:54 2012 duration = 0.013629 seconds HTTP_HOST = 'localhost:5000' QUERY_STRING = '' … File: "wsgi.py", line 19, in hello time.sleep(0.05)
- 22. Pyrasite
- 23. Not only Debugging ● Debugging tools can also help in finding and solving performance issues ● When a request is taking a lot of time, just attach ipsyd and see what it’s happening ● There are specific tools that can proactively notify you of slow requests
- 24. Dogslow ● Dogslow is a Django tool created by Bitbucket guys to monitor slow requests ● Notifies you by email, no need to check ● Really small codebase, it’s easy to port it to other environments ○ Upcoming BackLash version will provide a port of dogslow for frameworks apart Django
- 25. New Relic ● Full stack tracing ○ Controllers ○ SQLAlchemy queries ○ Background Tasks ○ External Services ○ Groups informations by controller, not URL ● On-Demand profiling ○ Turn On / Off controllers profiling from remote
- 26. New Relic
- 27. Slow requests stack trace might not be enough
- 28. Profiling ● Full profiling has a big cost, so it is usually constrained to development environment ● If you want to profile on production, perform sampling, don’t profile every request ● Use a Low Overhead profiler like PLOP, not built-in profile module.
- 29. Questions?