Web heresies

transcript

5 Heresies

James AylettKittens with Jetpacks

Thursday, 27 October 2011

5 Heresies

James AylettKittens with Jetpacks

For A Better World

1: Don’t use a web framework

Web frameworks make it easier to do the repetitive tasks involved in creating a web app that weren’t terribly difficult in the first place. Which saves time, but has some problems. So what actually happens when you start using the latest, sexiest framework?

Whee! This Nightshade framework is so cool. I wrote my web app in 10 minutes. Give me lots of money!

Then you do the usual things with a new site - you squash bugs...

and get distracted by dinosaurs.

And next you start getting some real users (ie not your mum), and things get a bit slower, and your investors come calling because they can’t get on your site to look at the attractive coeds. So you do the sensible thing and add some more machines in, praying to god that you understood the idea of shared-nothing and that your database isn’t too shabby.

For bonus points, you read Cal’s book.

Then you get more bugs, and more users, and more bugs - and eventually you’re not fixing your own bugs, you’re diving into the framework and fixing theirs.

At least they’re nice people who say thank you.

And, if you’re really lucky, you get enough users that you end up with a room full of servers, and learn about all the nasty details of how mysql prevents deadlocking, or how your storage server distributed file blocks across spindles, or how web browsers fuck up TCP congestion control algorithms. And of course you start running into those weird rare bits of the framework that don’t actually scale for you.

Because of aliens.

And this goes on until your database freezes solid, your site goes down for days on end, and your users remember they didn’t actually need it in the first place. Your investors start screaming...

... and if you’re really famous, so does Michael Arrington.

In the best case scenario, one of your investors was Y Combinator, and you rewrite the entire thing in arc.

Frameworks give you a number of really import things: a templating system which isn’t the one you want, a URI dispatcher, which doesn’t match your URI scheme, an ORM that’s less powerful than the one you would have used otherwise (and that’s certainly less powerful than just using your database directly), and finally...

... an octopus higher up your call stack. The problem is that while frameworks save you a tonne of time, YOU DON’T UNDERSTAND YOUR FRAMEWORK, unless you wrote it. At some point it will stop being a help and start being a hindrance, because their problem space is not yours.

Besides, it’s lazy. Standing on the shoulders of giants is pretty awesome, but it’s really a waste of time unless you’re looking for something high up.

Most websites aren’t up there, and while you’re looking around, your competitors are eating you. Or the shrubs, depending on how good they are.

So let’s go back to our new framework Nightshade. It’s got all the standard stuff...

and then people start adding new features.

Many of these are really important.

But many are increasing complexity of the framework without actually giving YOU any advantages.

Features become optional, which generally either increases the length of the callstack before it hits your code or at best makes it harder to just pick up an instance of the framework and figure out what’s going on, because some core section has been replaced somewhere in a config file.

In any case, people keep on adding new features...

Until we pass the Zawinski threshold, and it starts reading your email.

Of course, that’s pretty helpful in a web app, so people don’t really notice.

And features keep on being added until someone has a REALLY smart idea of adding

a user scripting system. At the most extreme, this is itself a framework that allows you to use a DSL rather than writing code in your base language. Typically this starts in the templating layer, but if it’s designed well, hell you might as well start using it for triggers in your models, or for flexible configuration, or to run deployment scripts.

And then someone has the REALLY smart idea of just reimplementing the entire framework in terms of the scripting system...

Which of course makes perfect sense. Except that then you’ve just obeyed Greenspan’s 10th Law, which really just means ...

you’ve built emacs. And we’ve probably got enough of them already.

However NONE of this actually helps you once you start to hit serious scale. For that you’re going to need architecture.

And I’m afraid for that you’re going to have to think.

2: Don’t build an API

Look, you could have built an API. But you chose to build a website. If you have decent URI design, and use microformats, and you were only thinking of building a read-only API anyway, then just don’t bother. Which is probably just as well, because you’d only get it wrong.

And everyone is watching.

In fact, APIs are notoriously difficult to get right. Here’s are just a few ways you can get them wrong.

Firstly, you could use XML. It’s not that XML’s a bad language for moving data around; it’s really not. But there are lots of exciting ways of getting it wrong. Actually, it’s difficult to see why. This guy’s Tim Bray, one of the guys who invented XML. He said:

“Anyone who can’t make [...] well-formed XML is an incompetent fool”

- Tim Bray

Which is true. Unfortunately we’re all incompetent fools. I am, you are, and lots of important people reckon XML’s too hard to generate. They include James Clark, and this guy:

Henri Sivonen, who went to the lengths of coming up with a list of things to do and not to to make sure you get XML right. Turns out there are 19 points! So maybe trying to get all them right isn’t such a great idea.

Another popular choice for moving data around these days is JSON. JSON’s pretty simple, although there are a couple of simple mistakes you can make. But there are good libraries for it (although to be fair, if everyone used good XML libraries, that wouldn’t be a problem either). However there’s an interesting wrinkle that comes out of using JSON.

JSON isn’t HTML. It’s NOTHING TO DO WITH HTML. But these little HTML named entity escapes have a tendency to creep into JSON outputs of APIs, often because the APIs were originally intended for internal use in loading little chunks of data to quickly spit out into innerHTML. This doesn’t help very much when people start using it for other things.

And then there’s REST. No one understands REST. Okay, Roy T. Fielding probably understands it, but given that he makes it looks like industrial plumbing, we’re probably not going to get it right. The trouble is, you really don’t want to use SOAP (or you’ll make Mark Baker cry), so you’re going to have to try. If you get it wrong...

... and REST purists will burn you in effigy. And you’re limiting the ability of larger sites to use you without killing your servers.

But there is a chance that some big site will come along, have a look at your lovely API, and

Suck all of your data out. But there aren’t many that will do that, except in Russia and China, so you can probably ignore that. Of course, lots of other startups probably want your data, but they’re going to be subtle about it - they’ll buy a couple of hackers, and a botnet. In Russia or China. So you could just block those countries. And if you do that, of course you don’t really need to bother internationalising and localising your API. Or web site.

Oh - except that Richard Ishida will eat your children. Here he is, demonstrating how to direct traffic in both France and Japan at the same time.

So APIs require you to think. This may become a theme. Of course, you don’t have to think.

You could not build an API, as I suggested. Or I suppose you could hire me.

3: Don’t make ′em pretty

Lots of people can do this. Maybe not that many of them work for MSN. But more important than making things pretty is making them useful - ignore the flashy bits for a moment and concentrate on the CONTENT. The DATA.

MSN doesn’t have much of that either.

This is your website if you haven’t thought about the data. It’s a disaster.

This is if you make it pretty.

Of course, if you take this perfectly-designed site, suited to its purpose, there is a risk that by making it pretty THEN

you’ll throw away the utility.

Yeah, you’re going to have to think again.

4: Don’t build websites

You know, it’s just a lot easier not to bother building websites. I’m not even talking about using Flash, or Silverlight, or JavaFX. They still live within the browser. There’s some problems serving stuff over HTTP in the first place, and certainly one of these problems is the browsers. Or should I say the user agents.

You’ve all heard of these user agents. Hopefully you make sure your websites work with them. But you don’t have to worry too much about them implementing HTTP differently.

This is still true as we think about some more unusual web browsers.

Although they may surprise you with their support, or lack of, for CSS, Javascript, bits of HTML, SSL support.

Then we have automated processes, one of which is probably the most important user agent your site will encounter.

This is the point where you really want to be speaking HTTP correctly yourself, because Google will reward you. You’ll get cookies or something.

There are a whole load of other things that talk to your website. Here’s a couple of interesting ones.

Package managers for your operating system, be it Windows, Mac OS, Linux, whatever. apt is interesting because it deliberately has an unusual interpretation of HTTP which bit me recently when I tried to reorganise a URI tree. It still seems to obey the rules though.

These guys might not. If they’re not raping your data, they might be trying to take your site down if you’re successful enough. It does happen. Buffer overruns, web process exploits - and that’s just at the HTTP layer. You’ll also get lower-level attacks at the TCP and IP layers. If you think you haven’t got competitors who’ll stoop to this, you still have to cope with random hackers.

We’re starting to get interesting things on the internet as well. Clocks, refrigerators, robots...

... rabbits. At least some of these can’t speak HTTP properly. Some of them can’t speak TCP properly, because they have abbreviated network stacks because they only have 1K of memory. Some of them are rabbits.

Oh, and there are these crazy people who’ll just telnet straight to your web server or load balancer and start typing away.

One of them’s me.

And we can make mistakes; instead of typing GET /girlfriend I could make a typo and instead try to PET your girlfriend. Although that might not be a mistake. The correct response in this case is 405. Or 301 if you’ve sent her to a convent.

Then there’s stuff we haven’t even dreamed of. Or we’ve dreamed it, but we don’t believe it yet. And this is beginning to feel a bit like a medieval map of places we don’t know very well, which is a fairly accurate description of all the things that can hammer on your web server.

And of course here be lizards.

This guy, if I remember correctly, is the roman God Janus. He’s a gatekeeper. His head points in both directions, and he enjoys close harmony singing. He’s a pretty good analogy for the routers and gateways that make up the internet. I had this whole way of explaining packet routing using a banana, but it kept on getting more complex, and in the end it was going to take half an hour all by itself and at the end we’d have had something closer to a pie than a banana, so you’re going to have to take my word for it that Janus is sometimes going to chew up your packets, drop them on the floor, or drown out all the local Januses with his off-key singing.

Then there’s the topology of the network between you and the people trying to use your website.

And those users might have their computers set up weirdly. Most of these things will bite you even if you don’t build websites, if you just build desktop software that functions over the network. Life would be easier if you just built little desktop applications that ignore the network. Or you could just play guitar in a band, because you only really need three chords.

Yet again, you’re going to have to think. Unless you get really good at guitar.

5: Be evil; get a cat

Firstly, because being evil is fun.

And cats are evil. This one’s about to steal your watch.

The good news is that cats are smarter than you, so actually you don’t have to think after all. Unfortunately they are also more evil, so when they rev the social structure of the universe...

... we’ll all be out of jobs.

http://www.flickr.com/photos/jimgris/http://www.flickr.com/photos/thomashawk/http://www.flickr.com/photos/pip/http://www.flickr.com/photos/protohiro/http://www.flickr.com/photos/psd/http://www.flickr.com/photos/leralle/http://www.flickr.com/photos/martinalvarez/http://www.flickr.com/photos/maguisso/http://www.flickr.com/photos/m-i-k-e/http://www.flickr.com/photos/samuraifiction/http://www.flickr.com/photos/ericasimone/http://www.flickr.com/photos/johnmueller/http://www.flickr.com/photos/millzero/

Questions?

Web heresies

Technology