Dupes, Speedhacks and Black Holes

How Players Cheat in MMOs

No Super-Secret Info

Most of the people in this room have already worked on UO :-)

The cheaters are the ones who discovered the vulnerabilities in the first place.

All the exploits we will be talking about today have been fixed.

Who The !@#$% Are You?

Server Lead & Lead Designer on UO Live.

Worked on the product for 7 years. This is not navel-gazing.

Why Do Players Cheat?

Fun Profit Sadism Industrial Espionage

Ultimately, it doesn’t matter!

General Classes of Cheats

Grief Tactics DOS attacks Shortcuts and minor game

advantages. Account theft.

We’re Not Talking About Client Hacks Art replacement (invisible tree

canopies, anyone?) Lighting hacks. Client scripting. Auto-aiming, wall hacks etc. for FPS

games.

We’re Not Talking About Client Hacks Have been discussed extensively by

others. Should be obvious to anyone

working in the industry today.

Rule #1

“The client is in the hands of the enemy”

– Raph Koster

Griefing: Item Theft

“Overload scam” General principle: to trick someone

into picking up more weight or items than they can legally carry.

When the system realizes they are overloaded, it forces them to drop something.

Black Bag Scam

Scammer initiates a trade. Places a bag dyed black into the trade

window – most of the time the other person will not notice this.

When the trade completes, the person receiving the bag is at their item limit.

When they try to unequip an item, their pack cannot contain it, so it falls to the ground.

Design Flaws & How to Avoid Be conscious of the value of player

information to scammers. Don’t give players items that look just like

the background of the UI! The unequip action didn’t have a way to

query its target container & fail gracefully. Don’t have flexible inventory. Test that item “drop sprites” contrast

sufficiently with UI elements. Ideally, the UI should have its own separate palette.

The Candle Trick

Scammer finds a player selling artifacts. Gets them to equip one.

“I use a candle to mine, can you equip one?”

Candle goes in the left hand, but . . . When he lights it, it switches to the

right.

Game Advantage: The Stabled Pack Animal Extra storage The ability to use items anywhere – since

the stable was a special backpack, items were always 0 tiles away.

Player drags an item out of the pet pack, holds it while stabling the pet with macros, and then drops it on an “illegal” container.

Item reverts to its last known good location.

Axiom:

If you don’t give a designer what they are asking for, they will figure out a way to do it on their own . . . and that way will probably be wrong.

Logic Accretion (aka “Monkey Patching”) “Can I put this thing in this other thing?”

if inMyInventory(item)

And so on . . .

&& !isInStabledPetPack(container)

&& bankerNearIfBank(container, me)

&& currentCount(container) + 1 < MAX_CONTAINER_ITEMS

&& currentWeight(container) + weight(item) < MAX_CONTAINER_WEIGHT

A Better Way

// Fires event on all scripts on me,

// the item, and the container

// Any script has a chance to reject

success = TestPutItem(me, container, item)

if (!success):

Bail()

else:

PutItemInContainer(item, container)

“Can I put this in you?”

Why Is It Better?

Consuming code no longer has to be aware of the internal working of other objects.

Emphasizes handling the failure case! When creating new code, you’re less aware

of old code. When you are writing the script that handles being

dead, you are more likely thinking about the things you can’t do while you’re dead.

Code ends up cleaner, easier to read and more maintainable.

Why Is It Better?

Many exploits occur when action should have been disallowed, but the special case was missed.

In an MMO, failure to catch special cases is the norm.

Allow nothing by default. Every permitted act is a special case.

Missing Special Cases = Normal? Truly new features are orthogonal to

existing features. Interaction testing burden scales

nonlinearly (n2-n)/2 10 features = 45 potential interactions 20 features = 190 potential interactions 30 features = 435 potential interactions 40 features = 780 potential interactions

Quadrupled our feature set, but the pool of potential interactions (which must be tested) grew more than 17x !!!

Missing Special Cases = Normal? Unless you can afford to scale your QA

staff commensurately, or you have fantastic automated test coverage, there will be holes

Implement the game to reject any behavior you haven’t specifically allowed.

Players will report behavior that inconveniences them.

Example: The Disguise Quest Designer creates a quest that involves

making and wearing a disguise that prevents aggro from a class of mob.

Assume the disguise code doesn’t exist! When you die, mount, harvest a

resource node etc. the disguise is stripped off.

The Way of the Monkey (Patch)

if DoingDisguiseQuest(me):

RemoveDisguise(me)

death.py

if DoingDisguiseQuest(me):

RemoveDisguise(me)

harvest_resource.py

if DoingDisguiseQuest(me):

RemoveDisguise(me)

mount.py

A Better Way

def HandleEvent(me, event):

switch (event.id):

case DeathEvent:

case HarvestResourceEvent:

case MountEvent:

RemoveDisguise(me)

default:

# Do nothing

disguise.py

An Even Better Way

def HandleEvent(me, event):

switch (event.id):

case InnocuousEvent:

case HarmlessEvent:

# Do nothing

default:

RemoveDisguise(me)

disguise.py

The Evil Flip Side of Item Scams Why not just let them overload

themselves? “A guy with 70K items in his

backpack walks across a server boundary . . . ”

Design Flaws & How to Avoid Them “So they overload themselves by a

couple of items . . . what could happen?”

Don’t default to dodgy behavior in the hopes that players won’t find it or won’t find a way to exploit it!

Network code wasn’t robust & didn’t red flag extreme circumstances.

Why Do Players Crash the Server? Sheer malice and/or revenge. Rollback undesirable events. Puts the game world into an

unstable state, which is a useful tactic for creating dupes.

Duping

“Our game won’t have dupes. We use a database.”

Duping occurs in code, not in data storage.

Duping is a hard case to generalize. Most dupes are caused by failure to

handle a delete.

Duping: Evil Furniture

Items animated through object replacement.

When the house containing the objects was packed up (for house customization), objects were “redeeded”.

Except, oops, we were looking for the wrong object type.

Design Flaws & How To Avoid Them No general-purpose object

animation code. If you write it once, it’s a one-off. If

you write it twice it’s a system. Spend some quality engineering time on it.

DOS: Black Holes

Player drops thousands of non-stackable items onto a single tile.

Get to the point where the size of the update exceeds the size of the network buffer.

Logically, we disconnect the user!

Design Flaws &How to Avoid Them Your network infrastructure should

support arbitrary packet sizes. However, packet sizes above a certain

threshold should set off alarm bells! If players are allowed to drop items in the

world, impose limits. It’s better to crash the server than

inflict cruel and unusual punishment on your players.

Account Theft: The Spam Tile UO only has “bubble chat” The “chat log” area is only used for

messages from the system. Scammers discovered a way to generate

messages that appear in the system message area

“Please send your username & password to this ICQ number”

Design Flaws &How to Avoid Them Caused by an off-by-two (!) error in

the server boundary mirroring code. Server boundaries are the devil. Use standard containers and

algorithms!

remove_if(player_id, mirror_list.begin(), mirror_list.end();

Game Advantage: Para Flag Paralyzing effects cause you to be

frozen. Taking damage unparalyzes you. Players carried trapped pouches to do

slight damage to themselves & unpara. Guess what? You’re frozen when you

cast spells, too!

Design Flaws &How To Avoid Them Single flag used for multiple “frozen”

effects. Again, code manipulates the flag directly

(“too much information about implementation”) & you have to monkey patch all the special cases to make them interoperate correctly.

Route everything through a code library that centralizes conflict resolution logic.

Game Advantage: Speed Hacking Player sends one move packet per

tile of movement. DON’T DO THIS. Speed hack #1: send more packets.

Solution: Throttle Movement Throttled distance over time. Speed hack #2: When players ran in

a circle they bypassed the distance checks.

Solution: Movement Queue Final solution was to buffer user movement,

discarding movement packets when the queue was full.

Was tremendously difficult to get to “feel right”

Still lets players get short bursts (<1 second) of fast movement.

Movement is probably about as good as it’s going to get without rearchitecting.

Griefing: Stealing Insured Items In UO, you can “insure” your items so they

are not lost if you die. UI is implemented with a context menu. When you invoke the context menu, it

checks to ensure that you are the owner of the item . . .

But guess what? Griefers figured out you can just send a “use this context option on that object” packet.

Design Flaws &How To Avoid Them Again, be selective in what you

allow! “Can I use this context menu

option?”“NO! Unless you meet this and this and this”

Be wary of the TOCTOU problem.

TOCTOU

“Time of check, time of use.” Many exploits involve creating conditions

post “time of check” that allow you to bypass the terms of that check.

Encapsulate validation in functions and call them at every critical juncture.

Commonly (as in this case), that will be when the UI is accessed and when the command is invoked.

Conclusions

No documentation of any of these incidents except in my own notes! Have post-mortems!

Track the design flaws that led to these problems and document them for new team members.

They can be entertaining, so people will probably read them.

If you see the same design flaws recurring, it’s time for an architectural improvement.

Conclusions

Anytime you put control of a system resource (memory, CPU, storage) into the hands of players, someone will abuse it.

Don’t rely on obscurity. Griefers & scammers will find a weakness, no matter how weird the series of actions.

Fin.

Tim Keating, Director of Development, Heatwave Interactive

tim.keating@heatwaveinteractive.com

http://mrtact.com/blog