Date post: | 13-Jul-2015 |
Category: |
Technology |
Upload: | jorgeortiz85 |
View: | 3,484 times |
Download: | 1 times |
foursquare runs on...
• Scala 2.8.0
• Lift 2.1
• MongoDB
• Twitter open source projects
• ~46k lines of Scala
foursquare server team
• 9 out of 11 had no major prior Scala experience
• Previously: Java, Python, C#
• Most exposed to: ML, Lisp
foursquare loves Scala
• Pimped: Option, List, Map
• Type-safe equality: =?
• BackgroundOperation { ... }
• Throttle(“secret feature!”) { ... }
• Any Java library
2.7.7 => 2.8.0(painless migration)
• Compile: 2.7.7 with -deprecation
• Branch
• Compile 2.8.0 without -deprecation
• Compile 2.8.0 with -Xmigration
• Compile 2.8.0 with -Xwarninit
• Switch
foursquare on Lift
• REST API
• Out-of-the-box:
• Rich Ajax/Comet support
• ORMs for PostgresQL, MongoDB
• Random: Email, Facebook, JSON, utils...
GC issues
• Stateless API machines
• Tens of millions of requests per day
• Only a dozen or so servers
• 12 GB heap => 15sec GCs *ouch*
GC issues
• GC tuning?
• “There's no way to get around the fact that we're allocating faster than we can collect.”
• Allocating ~200 MB/s
• Opportunity to reduce wasteful allocations
• Better load-balancing strategy
• Cap # of concurrent connections
GC issues
• Stateful WWW machines
• Stateful Lift + Scala closures + Java GC
• Lots of variables getting caught in Session
• val venues: RequestCache[List[Venue]] =
RequestCache(Venue.findAll(...))
MongoDB in Scala
• Some support built-in to Lift
• No expressive query language, so we wrote one:
Venue where (_.mayor eqs userid) modify (_.tags push “I’m mayor!”) updateMulti