Date post: | 28-Oct-2014 |
Category: |
Technology |
Upload: | poornima-vijayashanker |
View: | 10 times |
Download: | 0 times |
The Evolution of a Scrappy Startup to a Successful Web ServicePoornima Vijayashanker – Software Engineer
November 7, 2008
Confidential & Proprietary
What is Mint.com? My Bio Came up with the name and the second employee
Getting Mint.com off the ground Used open source tools: Eclipse, mySQL, Apache Tomcat No prior experience at a startup or in web services
Mint.com today
Confidential & Proprietary
Creating a prototype
Prototype: a rudimentary working model of a product or information system,
build for demonstration purposes or as part of the development process.
Confidential & Proprietary
What doesn’t belong in a prototype?
Don’t waste time spec’ing out a complete feature set “Everyone by now presumably knows about the danger of premature
optimization. I think we should be just as worried about premature design - designing too early what a program should do. “
- Paul Graham
Mint.com’s mission statement: “Do more with your money.” Aggregate checking, savings, and credit card accounts Show balances Auto-categorize transactions
Confidential & Proprietary
What does belong in a prototype.
Started small by focusing on features that differentiated our productFocus on solving critical user problems. Engineering problems arise from their solutions. e.g. Financial data is a sensitive matter, needed a good security model. Handle concurrency amongst our 100+ users, and making sure we
were encrypting stored data. Bare-bones implementation for algorithms is sufficient. Don’t get tied
down to any one particular solution. Refine in subsequent releases and based on product specifications.
Simple unit test framework, no system tests; focus was to get the product out there and have real users test it with real data!
Confidential & Proprietary
What did our application server look like?
Single Server Web layer and analysis engine, wired together using RMI. UI -> Business Logic -> Data processing engines Single data base that consisted of less than 50 tables
Confidential & Proprietary
What did our code base look like?
A few key choices during prototyping molded our software’s architecture, and affected the longevity of our code base e.g. Choice in messaging system: JMS, RMI, or none Re-factored code into logical modules to avoid spaghetti code, a task
that would have been much harder to do later on, especially with a growing team and code base.
Confidential & Proprietary
Web Application to Web Service
Prototype -> Product likened to Web Application to Web Service. Application is a point tool used to complete a simple tasks Web services is more than features Goal is to create a product with growing user base you have to
broaden thinking from features to logistics
Confidential & Proprietary
Prime mover of Software’s Architecture
User Growth Rate To solve the business problem of becoming profitable you have to
grow the user base
More users -> Growing Pains Need to be addressed in a timely manner in order to continue to grow
and keep the existing users happy.
Confidential & Proprietary
Computer Architecture 101
Latency: a reasonable response timeThroughput: satisfying many requests without compromising latencyQuality: accurate data and reliable service Address points of failure
Confidential & Proprietary
How do you achieve each?
Latency: affected by amount of data that needs to be retrieved, in order to satisfy the incoming requests. e.g. computed data and persisted to db, as users increased
reading/writing to the db took forever. Switched to loading user data into a cache upon login and then
computing.
Confidential & Proprietary
Throughput: handle multiple requests/processes Separate into code into tiers (UI, web, business/service logic, DAO, db)
in parallel data processing/retrieving tier Separate data engine from web engine
• One server for handling data processing• One server to process user requests and serve user pages
Tune each server based on its needs (web vs. analysis)
Confidential & Proprietary
Quality: low bug count, which is directly proportional to incoming number of customer service requests Testing (unit, load, and system) Beware of the vocal minority, by measuring the number of users
impacted by a bug.
Confidential & Proprietary
Optimization: improving performance of code at runtime in order to satisfy latency and throughput requirements. “…premature optimization is the root of all evil." – Donald Knuth
Confidential & Proprietary
How to Measure Created internal tools to measure the
performance of our code base, which helps figure out where to optimize.
Product will continue to evolve in approximately 6 month cycles.
Don't waste time optimizing everything, or before you see a demand for a feature. Remember its a startup; resources are scarce and time is critical.
Confidential & Proprietary
What to measure
How quickly a data set is going to grow when designing tables, foreign key associations, retrieving data, and frequency with which data is accessedSharding: break up a large database into smaller pieces that contains redundant information or a parent db can map data to separate dbs. Implementation was based on user id lookup Easy to shard because we were dealing with financial data that is
restricted to a single person, each user and their data was limited to a single shard, unless services like Twitter or Facebook that have lots of interactions amongst users
Confidential & Proprietary
Review Decisions
We didn’t cache user data from the start because synchronizing data across nodes was difficult and had no mechanism for centralized locking, but once this was put in place we switched to loading data on demandWe didn't shard databases from the start because of overhead and more impending issues.In the future we might only show most recent data instead of all data).
Confidential & Proprietary
Summary
Prototype with limited featuresAddressing CS 101 Basics: Latency, Throughput, and QualityMaking architectural decisions based on the time frameMeasure and Optimize that which is criticalCheck it out: www.mint.comMy blog: www.femgineer.com