Porting 100k Lines of Code to TypeScript

tiny.cloud
PORTING 100K LINES OF
CODE TO TYPESCRIPT
MILLIE MACDONALD
SOFTWARE ENGINEER @ TINY

The Talk
● About Tiny and me
● The Conversion Project:
○ Motivation
○ Why TypeScript?
○ How we converted our JavaScript to TypeScript
○ Benefits, lessons learned and ongoing work
TL;DR: Types help. Gradual typing really helps.

About Tiny
● TinyMCE is a popular open source project
● A rich text editing platform that helped launch
and scale the adoption of Medium, WordPress,
Shopify, Tumblr, Atlassian, EventBrite & more
● In 2015, we launched a startup with offices in
Palo Alto, Australia and Sweden
By the numbers:
● 1 million+ users on Cloud
● 125,000 downloads/week
● 1,000 paying customers
● 232 contributors

About Me
● From Brisbane, Australia
● Bachelor of Engineering
(Honours), Software Engineering
● Intern then QA Engineer then
Software Engineer at Tiny
● Admin of the Australian gaming
community for Splatoon

“100k Lines of Code”?
TinyMCE is:
● Core is ~30k LoC
● User Interface is ~30k LoC
● 44 core plugins + several
premium plugins with ~40k
LoC
Plus various other projects
and libraries…The point of
conversion!

Before
● Over the years, we tried and rejected a lot of frameworks
● We also use a functional programming style
● So we wrote our own libraries and frameworks
○ Avoided upgrade problems and framework churn
○ Easier for us to fix problems
○ FP + JS means defensive code, which resulted in code size and runtime performance
issues
● Then ES2015 appeared...

Motivation for the Conversion Project
● ES2015 features - especially a standard module system - were
very enticing, so in 2017 we decided to convert our code.
● We had also been considering switching to an alternative JS
language. Our wish list was:
○ static typing
○ pattern matching

Why TypeScript?
● We looked at ReasonML, PureScript and TypeScript
● We considered:
○ how much of the functional programming feature set we needed
○ the learning curve for each language
○ the upfront development cost to switch

Why TypeScript?
● ReasonML
○ Rewrote our PowerPaste plugin for TinyMCE, replacing complicated regex with strong types and
pattern matching
○ Syntax is similar to JS so easy to learn
● PureScript:
○ Used for some of our internal prototypes
○ Side-effect tracking speeds up code reviews and training, and optics improve runtime efficiency
○ Lots of FP features we don’t need on the client side
● Neither is an easy drop-in replacement for the editor code

Why TypeScript?
● Unfortunately...
○ No pattern matching
● But...
○ Vanilla JS is valid TypeScript
○ Due to already using a module system with strict syntax, converting our code
could be mostly done with a script
○ Types!
○ Gradual types!

What is Gradual Typing?
● Process of starting with minimal types and adding more over time
● Requires a mix of static and dynamic types - like TypeScript
● Requires minimal initial work
● Can invest as however much time and developers as you like in
increasing types each week/month/etc.
● Can start with simple types then increase complexity as you learn and
adjust to the type system
● Low initial cost of adding types, flexible ongoing cost

Conversion Process for each Library
1. Run script to convert modules to ES2015 and TypeScript
2. Manually check the script converted each file correctly
3. Update with changes from already-converted dependencies
4. Fix problems found by TypeScript
5. Run tests and fix any problems
6. Update build process

The Conversion Script
1. Transpile all modules using an Abstract Syntax Tree (AST)
a. Change import syntax
b. Pull code from inside the wrapping function to the root level
c. Change export syntax and apply export default <any>
2. Move files and rename to .ts
3. Copy templates for configuration files
4. Generate Main.ts from api folder
5. Update package.json and .gitignore

Module Conversion Example
define(
'Example Module',
[ ‘example.Dependency' ],
function (Dependency) {
var bar = 0;
var foo = function () { … };
return {
foo,
bar
};
}
);
import { Dependency } from 'example';
var bar = 0;
var foo = function () { … };
export default <any> {
foo,
bar
};

Adding Types
● The script doesn’t add types
● We started adding types to libraries, starting with the libraries
most commonly depended on
● We add types with each PR
○ New code must be at least partially typed
○ “If you touch a file, type it” was the rule for a while
● Now, many of our libraries are at least partially typed

Benefits of TypeScript and
Gradual Typing

Types
● Gained type-based tools e.g. find all usages and
automated refactoring
● Compile time type checking
● Even just adding simple types caught some serious bugs
● Typing our most common dependencies caught many
more

Example Bugs
Incorrect function argument list length
Missing comment resulted in accidental global

Gradual Typing
● Most of the team was able to go back to normal development work
quickly
○ Sometimes devs take a couple days to type a library
○ We add some types with most PRs
● Conversion script used export default <any> so minimal errors
and developer work upfront
● Gradually removing any from modules allowed us to type modules
slowly and methodically

Increased Readability
● Types make it easier to follow code
○ Especially in our giant projects where variables sometimes get passed around a lot
● Types make it easier to debug code
○ Type errors are generally nicely worded
○ Sometimes easier to match weird values to types, then track down which variable or function
they come from
● Types make it easier for new developers to get up to speed and learn
the code

Convert Common Libraries First
● Because of our modular architecture we have a lot of
libraries, and many depend on other libraries
● We mostly converted the most common dependencies first,
so we could cascade their types and bug fixes through other
libraries
● But some other libraries were done at the same time, and
they have duplicate type definitions, clashing types, etc.

Gradually Introduce Compiler Features
A couple times we enabled compiler features without testing them
properly on every library...

Write a Script to Test Everything
● Write a script that does every kind of check it can - types,
linting, build, tests, etc.
● Run it when you make a big change
● Ours is called the Really Cool ScriptTM
○ Contains a list of all our libraries
○ Clones each one, installs its dependencies, runs tsc and npm test

Developer Adjustments
● Discuss changes in coding style, tools and processes
● Communicate changes
● Script pre-commit or pre-build type checks to help developers
remember what to check

Continuing Work
● More types!
● Enabling more compiler features
● Building a type definition file for TinyMCE’s editor API

Check out TinyMCE
● TinyMCE is open source
○ https://github.com/tinymce/tinymce
○ Contains TS files and most config and build files
○ Master branch is TinyMCE 4
○ 5.x branch for TinyMCE 5
● Found a bug? Win some swag!
https://go.tiny.cloud/blog/tinymce-5-developer-challenge/

Porting 100k Lines of Code to TypeScript

Recommended

More Related Content

What's hot (20)

Similar to Porting 100k Lines of Code to TypeScript (20)

More from Tiny (16)

Recently uploaded (20)

Porting 100k Lines of Code to TypeScript