33
tiny.cloud PORTING 100K LINES OF CODE TO TYPESCRIPT MILLIE MACDONALD SOFTWARE ENGINEER @ TINY
tiny.cloud
PORTING 100K LINES OF
CODE TO TYPESCRIPT
MILLIE MACDONALD
SOFTWARE ENGINEER @ TINY
The Talk
● About Tiny and me
● The Conversion Project:
○ Motivation
○ Why TypeScript?
○ The conversion
○ Benefits, lessons learned and ongoing work
TL;DR: Types help. Gradual typing really helps.
About Tiny
● TinyMCE is a popular open source project
● A rich text editing platform that helped launch and scale the adoption of Medium, WordPress, Shopify, Tumblr, Atlassian, EventBrite & more
● In 2015, we launched a startup with offices in Palo Alto, Australia and Sweden
By the numbers:
● 1 million+ users on Cloud
● 125,000 downloads/week
● 1,200 paying customers
● 270 contributors
About Me
● From Brisbane, Australia
● Bachelor of Engineering (Honours), Software Engineering
● QA Intern then QA Engineer then Software Engineer at Tiny
● @metricjs on Github and Twitter
“100k Lines of Code”?
At the time of the conversion, TinyMCE was:
● Core = ~30k LoC
● User Interface = ~30k LoC
● 44 core plugins + several
premium plugins = ~40k LoC
Plus various other projects and libraries that get bundled in…
JS -> TS
The Conversion Project
Before
● We tried and rejected many frameworks
● We use functional programming
● So we wrote our own libraries and frameworks
+ Avoided upgrade problems and framework churn
+ Easier for us to fix problems
− FP + JS means defensive code -> code size and runtime performance issues
● Then ES6 was announced...
Motivation for the Conversion Project
● Decided to convert to ES6 in 2017
● Had been discussing alternative JS languages for a while
● Decided to do both conversions at once
● Our original wish list for an alt JS was:
○ static typing
○ pattern matching
Why TypeScript?
● We looked at ReasonML, PureScript and TypeScript● For each language we considered:
○ how much of the feature set we needed○ the learning curve○ the upfront development cost to switch
Why TypeScript?
● ReasonML
○ Trialed it by rewriting our PowerPaste plugin for TinyMCE in ReasonML, replacing complicated regex with strong types and pattern matching
+ Syntax is similar to JS so easy to learn
− Requires a full rewrite
● PureScript:
○ Was already used for some of our internal prototypes
+ Side-effect tracking speeds up code reviews and training, and optics improve runtime efficiency
− Lots of FP features we don’t need on the client side
Why TypeScript?
● Typescript
− No pattern matching
+ Vanilla JS can be valid TypeScript
+ ES6 modules == TypeScript modules
+ We were already using our own module system with a strict syntax so conversion could be done with a script
+ Types! Even better: gradual types!
What is Gradual Typing?
● Start with minimal types and add more over time
● Requires static and dynamic types - like TypeScript
● Minimal initial cost of conversion
● Flexible ongoing time/dev investment
● Can start with simple types and improve later
The How
Conversion Process for each Library
1. Run script to convert modules to ES6/TypeScript syntax
2. Manually check the script converted each file correctly
3. Update with changes from already-converted dependencies
4. Fix TypeScript problems found
5. Run and fix tests
6. Update build process
The Conversion Script
1. Transpile modules using an Abstract Syntax Tree (AST)
a. Change import syntax
b. Format body of the file correctly
c. Change export syntax and apply export default <any>
2. Move files and rename to .ts
3. Copy templates for configuration files
4. Generate Main.ts from api folder
5. Update package.json and .gitignore
Module Conversion Example
define( 'Example Module', [ ‘example.Dependency' ], function (Dependency) {
var bar = 0; var foo = function () { … };
return { foo, bar };
});
import { Dependency } from 'example';
var bar = 0;
var foo = function () { … };
export default <any> {
foo,
bar
};
Adding Types
● The script doesn’t add types
● We gradually started adding types to libraries
● We add types with each PR
○ New code must be at least partially typed
○ “If you touch it, type it” was the rule
● Invested time to fully type our most used libraries
● Now, many of our libraries are fully or at least partially typed
Benefits of TypeScript and Gradual Typing
Types
● Gained type-based tools e.g. find all usages and automated refactoring
● Compile time type checking
● Even simple types caught some serious bugs
● Typing our most used dependencies caught many more
Example Bugs
Incorrect function argument list length
Missing comment resulted in accidental global
Gradual Typing
● Allowed the team to go back to normal tasks quickly
● Able to type modules slowly and methodically by gradually removing placeholder types
● Can leave adding types until we have spare time
● Can type new code without having to type the old code around it
● Type inference tools allow for some automated typing
● Types make it easier to read and debug code, and to train new devs
Lessons Learned
Convert Common Libraries First
● Our modular architecture means we have a lot of libraries and dependencies
● We mostly converted the most used dependencies first which allowed us to cascade their types and bug fixes through other libraries
● But some other libraries were done standalone and they have duplicate type definitions, clashing types, etc.
Gradually Introduce Compiler Features
A couple times we enabled multiple new compiler features without testing them properly on the libraries...
Write a Script to Test Everything
● Write a script that does every kind of check it can - types, linting, build, tests, etc.
● Run it when you make a big change
● Ours is called the Really Cool ScriptTM
○ Contains a list of all our libraries
○ Clones each one, installs its dependencies, runs tsc and npm test
Developer Adjustments
● Discuss changes in coding style, tools and processes
● Communicate changes and their impact
● Add scripts that check types, etc. without devs having to remember complicated commands
Continuing Work
Continuing Work
● More types!
● Enabling more compiler features
● Building a type definition file for TinyMCE’s editor API
After Typescript
● Typescript is great but it is still mostly Javascript● Sometimes you really need strict types, pattern matching,
etc. to ensure code quality● So...
RTC and ReasonML
● Tiny is working on Real Time Collaboration!● Client-side is entirely in ReasonML● More info coming soon!
Check out TinyMCE!
● tiny.cloud
● github.com/tinymce/tinymce
Keep in touch:
● metricjs on Twitter and Github
Thank you!
Questions?
