Talk at FullStack 2016: Automating documentation on JavaScript projects

Editor's Notes

• #2: Hi, Thanks for coming!! I am super excited about being here at FullStack2016 It has been an amazing conference so far and I hope to contribute to it with my 2 cents My name is Marcos Iglesias and I am here to talk about documenting JavaScript projects
• #3: But, first a bit about me: I am Senior Software Engineer at Eventbrite, where I build User Interfaces and Dashboards for our event organizers.
• #4: Let’s start off with a story, It is a software story
• #5: It all started when they named me Sheriff of JavaScriptTown I didn’t want the job, you know, a city at the edge of technology, with continuous changes, new projects arriving every day and some of them dying of starvation But I had to pay some debts so I was forced to accept the position
• #6: Once I became Sheriff, I understood that I would have to deal with three different types of code
• #7: The first specimen was the Good code It was new, clean and well-tested code [You know], A Greenfield project, the kind of project we developers dream of working on.
• #8: Next up was the Bad code. It was entangled old code without documentation and few, if any, tests. You may have heard its other name: “Legacy Code”
• #9: The last code I had to work with was the Ugly It was code that was neither old nor new, and it was hard to understand and hard to learn It was optimized for reusability and fully tested but it didn’t have any documentation
• #10: My mission as Sheriff of JSTown was to put them straight and get them into good shape And as you can imagine, working with such different codebases meant different challenges
• #11: Let’s talk about the good code first. We wanted to open source it, so it needed to be kept clear and simple. It required good documentation because we wanted a high quality project.
• #12: The challenge with the bad code was that it was easy to get lost in it. There were a number of non-obvious or directly misleading relationships between modules, and it featured a high level of indirection
• #13: Finally, when working with The Ugly code, I found a lot of new elements and new ways of doing things without a lot of examples to reference. It also featured a high level of indirection and a complicated hierarchy. (It was a tough one)
• #14: In all three cases it seemed that I would benefit from some documentation but Software developers do not document their code very often, and that's especially true for JavaScript. It´s a shame because docs are great: - Just writing them is a useful reviewing process - They help other developers as well as your future self - And they amplify the shared code ownership within your team So why don’t we document more?
• #15: In my opinion we don’t do it enough because docs are broken (I have worked in a bunch of different places in different countries and really, nobody got it right) Thinking more about it, and in how docs bugged me in my day to day work, I summarized this issue as 3 main problems:
• #16: The first is that most of the times, the docs are out of date. A typical problem of documentation is that unless approached with methodology and care, it can easily get out of date. And not only that, when you find out that some docs are out of date you lose confidence in the rest of the docs and you let even more errors slip through unnoticed (Broken windows effect).
• #17: Another problem is that docs are boring to write. We are software developers, we like to write code, not docs!
• #18: The third problem is that they are really inconvenient, as they usually live far from where they are needed. Sometimes you don’t even know if a piece of documentation exists before you search for it! (context-switching) Taking into account all these problems, I thought about changing my approach and tried another solution:
• #19: This isn’t new, right? Adding comments to the source is something we have been doing for a while. That’s true, however, the way we have been writing Javascript has evolved in the last few years. And I think these recent developments could inform our solution to the problem of documentation. The first of these developments is the use of Code Reviews
• #20: The fact that the JavaScript community relies on Code Reviews and Pull Requests to monitor the quality of code means that it can also act as an enforcer of up-to-date comments. (This makes sense, right? Pushing the code and the docs in the same review.)
• #21: The second big change is the popularity of ESLint. I love ESLint! It has saved me a lot of time just by giving me warnings about typos or errors before I have even saved the code. It’s that effective! Well, now we can enforce comments and also give warnings about malformed comments using Eslint rules
• #22: Up until now, we have seen the three kinds of code I met in JavascriptTown and the challenges they presented. We also saw what’s wrong with documentation of JavaScript projects and a possible approach to solving it: writing the docs as comments near the source code In the rest of this talk I will show you how I used this approach to work with those 3 kinds of code: First I will talk about JSDoc comments Then I will show you two popular ways of generating docs: DocStrap and Docco We’ll also see when to choose each one And finally I will talk about the benefits of adding docs into the source code [itself].
• #23: Now let me introduce you to JSDoc comments
• #24: The first use of this syntax to document JavaScript was in 1999 with the Mozilla project Rhino, that was a JS runtime written in Java.
• #25: Let’s see how it works: First we, as developers, add comments on the source code Then we run the JSDoc tool and it gathers all the docs To generate a documentation website with them
• #26: JSDoc is based on a series of tags that you fill with information Its syntax and semantics are similar to those of Javadoc, which is used for documenting code written in Java.
• #27: I have separated the tags in two groups: those for functions and those for modules
• #28: We have @param that is by far the most common and which specifies the input parameters and types of the method. A typical line will include the tag, the type of the variable, the name and a description of it
• #31: We also have @return, that follows a similar pattern
• #32: [This indicates] Whether the function is private or public This one is important because it defines what comments would go into the docs, as we will see later
• #33: If it is deprecated, you can use this tag.
• #34: We can add links to references with @link
• #35: This is an example of a private function with some input params and a return of type function
• #36: Here is another example with a link reference, note how first we have the URL and then the text
• #37: Regarding modules or classes, we will add more tags like:
• #38: [Use] @file for the file’s description
• #39: You can mark files as a module with @module
• #40: Or state dependencies with @requires
• #41: Use @version to specify versions.
• #42: and add examples of use.
• #43: You can link to demos with @tutorial by adding the name of the demo page html file
• #44: The @typedef tag is useful for documenting custom types, particularly if you wish to refer to them repeatedly on @param or @return tags
• #45: Here we have an example of how to document a module
• #46: We have a type definition, in this case a D3 selection
• #47: Also the description, module name, version, requirements and tutorial file name
• #48: And finally an example of use
• #49: We also have some help from plugins for our editors Helper plugins for your editor: Sublime Text, Atom, Vim, PyCharm
• #50: Sublime Text
• #51: Atom
• #52: Vim
• #53: Emacs
• #54: In the case of pyCharm
• #55: And Webstorm, JSDoc comments are supported by default
• #56: Here we can see how the Sublime Text plugin helps us to fill the info using the tab button Hit the tab key to bounce between the elements we need to fill Note how effectively we are doing a kind of informative type definition
• #57: Now let me go back to my 3 friends, and the cat
• #58: Back in JavascriptTown, I decided to use JSDoc to document The Good, The Bad and The Ugly codebases
• #59: Commenting with JSDoc on the good code... Helped me to keep my methods small Forced me to review my code one more time In general it made me more mindful when writing my code
• #60: For the bad code, writing JSDoc comments: Helped me label the methods that didn’t belong to each module Highlighted dependencies between objects And allowed me to avoid re-reading convoluted methods
• #61: Lastly, I commented the Ugly code on a top-to-bottom basis, starting with the high level artifacts This process made it easier for me to understand the big picture Helped other developers to understand it And again I saved some reading time and effort.
• #62: Cool! Are you still with me? We have seen what JSDoc comments look like, How commenting is a great tool for understanding other people’s code And for communicating about your own code But there is more, with just a little bit of extra work, we can generate docs from those same comments! At Eventbrite we have experimented with two different flavors, One is API-style documentation and the other is a literate programming approach Each one has been useful for different reasons, but the two are not mutually exclusive.
• #63: The first flavor I want to talk about is an API-style documentation generator
• #64: You could ask why do you need this? Isn’t JSDoc supposed to generate documentation for you? Paradoxically, the default output of JSDoc generated documentation is not really beautiful (Actually, not even the reference docs are pretty in any sense) So we are gonna use DocStrap to create a better looking output for our docs
• #65: The process would be something like this: And debug! Yeah, you didn’t really think it was so easy right? Luckily the kind of debugging you will need to do is to fill some missing types or similar easy problems And now we can use eslint rules to validate the comments beforehand Let me show you an example:
• #66: This is the main documentation page of Britecharts, that is a reusable charting library based on D3.js we have been working on at Eventbrite. We are now preparing to open source it and this is the main page, basically derived from the README.md file of the repository.
• #67: This is the API reference page for one of the components of Britecharts. It has a list of the public methods that the line component exposes on the right sidebar, And it includes descriptions of each one and its parameters. Here we won’t see the methods marked with an @private This is the main page that Docstrap generates for us.
• #68: We can add tutorials like this one, that is the demo page for that same Line component
• #69: And we can also add as many additional pages like this Getting Started section. It’s pretty neat and complete.
• #70: The other docs generation flavor we have tried is Docco This is A project by Jeremy Ashkenas, the creator of coffeescript, backbone.js and underscore.js
• #71: It produces this split view with your comments on the left and the code on the right I am sure you have seen this kind of documentation before
• #72: Annotated source code of Backbone
• #73: Annotated source code of Underscore.js
• #74: Annotated source code of Marionette
• #75: One of the main critiques of JSDoc is that their comments are too verbose. On the other hand, docco comments are pretty simple:
• #76: Simple inline comment
• #77: Simple inline comment
• #78: The process is pretty standard
• #79: Just if you were guessing Here we can see both type of comments working together
• #80: So now we’ll see briefly how to choose between these two documentation styles
• #81: I would use Docstrap… - When building a project based on an API, when I need a lookup-style documentation - On public-facing projects that need tutorials and demos, it is a good choice - When you need high levels of control and customization (specially on the look and feel side) On the other hand, I would use Docco - When describing complex pieces of code - Or when the code is based on a flow or that’s the thing we want to describe - When explaining decisions in your code is important
• #82: If neither of these options fit in your requirements there are more projects
• #83: For Sass we have SassDoc if your platform is heavy on sass magic
• #84: Also for Css we have DocumentCSS
• #85: And it’s counterpart on the JS side
• #86: Lastly for React we can use the React-styleguidist project to generate component libraries This list is not comprehensive and new OS projects are appearing every day...
• #87: making me think it’s really an exciting time for documentation generators
• #88: Now let me tell you what happened at the end of my time as Sheriff of JavaScriptTown, and after a couple of brawls and rough times with those guys
• #89: As we saw it before on the Britecharts project, We managed to create great docs with Docstrap for The Good code I was able to customize the docs and I ended up really proud of our project
• #90: At first I didn’t want to create docs for the bad code (It didn’t make sense at that point) However, now that this code got refactored, maybe we want to document it I will ask myself: how would I prefer to learn about this system?
• #91: Adding comments was useful for me and for the next developers that worked with it If I spend more time working with this codebase, I would probably choose Docco to document it
• #92: We’ve arrived at the end of my talk and You have seen how documenting near the source code can help you deal with different kinds of code I showed you 2 styles of creating documentation from comments and how to choose between them Now I want to close this talk enunciating the benefits of following this approach to commenting right in the source code
• #93: The first and primary benefit of consistently adding comments is that it helps you communicate about your code. The mere task of having to figure out the comment you will write for a method will help you think about it and clarify its purpose. It also promotes a shared ownership of the code, reducing the ‘bus factor’ problem.
• #94: Also, writing comments on other developer’s code helps you understand it because it forces you to find out why something was written in a given way Plus, it helps you dodge some extra reading when you need to work with it
• #95: Another benefit is that it generates documentation for you, we just saw two different styles: API-style references of DocStrap split-screen style of Docco
• #96: By far my favorite outcome is that this docs don’t get out of date so easily We are going to review the comments alongside the code when doing code reviews, (Also, when you remove code from the repo, the docs go with them!)
• #97: Commenting out your methods is not hard; it’s actually easy and fast! Regardless of your code editor, there are helper plugins that help you And this system will be easier to maintain as the comments are close to the source You can also complete this task any time: while building containers in your dev environment, while running your tests or while waiting for your CI tool to finish the build. It’s an easy way to fill your time productively and it has a huge pay-off for your engineering team.
• #98: Lastly, having the code explanation near the source code that is being commented is much more convenient. This way, developers won’t need to open a new browser tab to search for a piece of documentation that maybe doesn’t even exist. [Our teams are wasting hours of developer time just in this context-switching.] [Just think of all the hours of developer time wasted on context switching that you can save with my advice!]
• #99: Here I leave you some resources about JSDoc
• #100: And Docco
• #101: I want to encourage you to give it a try! Maybe you can start with a small project or library, adding comments on the important pieces. Use those comments to communicate about your code, generate documentation and keep it as up-to-date as your code is. You will see that it doesn't take too much work to achieve a codebase that you can be proud of!