UC Berkeley

CS 98/198:Intro. to Web 2.0

Development Using Ruby on Rails

Armando Fox, Will Sobel, Dave Patterson

CS98-10/198-10, Spr08

RESERVED: MW 4–5:30

271 Soda is RESERVED for CS 98-10/198-10, Intro. to Web 2.0 Development Using Ruby on Rails,MONDAYS & WEDNESDAYS 4–5:30 PM.

Space permitting, students not enrolled in the class may use the space during those times if their activities don’t interfere with

running the class. Thanks.

Questions: fox@cs.berkeley.edu

CS98-10/198-10, Spr08

Today

• About the course– administrivia, prereqs, course format, projects

– about the RAD Lab– course projects & Entrepreneurship tie-in

• Movies– zero to blogging in 58 lines of code– Rails “TV ads”

• Background concepts review– Web app anatomy & brief history of Web programming

– TCP/IP, HTTP, HTML, Web servers

CS98-10/198-10, Spr08

Why this course?

• Important trends in “Web 2.0” style applications– Reinventing client-server: the datacenter is the computer

– New models of software development & deployment– Rich, sophisticated UI’s– Modular apps (mashups, etc.)

• Attract “end users” to RAD Lab platform– Student-conceived apps generate traffic– We provide “hosting service”

• Exercise RAD Lab infrastructureOpportunity: combine sound pedagogy with “real world”

skills & gratification of crafting a useful artifact early in your CS career.

CS98-10/198-10, Spr08

Web 1.0

• Static and some dynamic content– lots of ad-hoc scripting languages, cgi-bin– “homegrown” sites not yet standardized on a stack (like LAMP); eg, variants of Berkeley DB or filesystem for storage

– Big companies: $$$ relational DB (eg Oracle)

• Primitive web page GUIs– 1 interaction == 1 roundtrip to server for page view

• “Services” available but have to be a company to get them (small sites use libraries instead)– Doubleclick (ads)– Inktomi et al. (web search on your site)– Visa/MC (accept credit cards)

CS98-10/198-10, Spr08

Software as a Service (SaaS)

• Run your own database => Salesforce• Email Word files around => Google Docs

• Shrinkwrap mapping software => Google Maps, Google Earth

• Buy a hard disk => Amazon S3• Buy servers => Amazon EC2• Desktop GUI app => AJAX (Javascript-based) GUI in browser

CS98-10/198-10, Spr08

Web 2.0 Trends

• Company => Individual can get services– DoubleClick => Google AdSense; Visa => Paypal

• Sophisticated UI: AJAX decouples interactions/page updates from server roundtrips– Effect 1: “desktop-like” interactive Web apps– Effect 2: Desktop => cell phone (Browser w/Javascript == universal platform)

• “Mass customize” to 1 => Wisdom of crowds– global search, etc are more compelling than local because learns from everybody

• Libraries => Services, thanks to arch. standards– REST/CRUD for state management– Javascript/DOM for advanced browser interactions

CS98-10/198-10, Spr08

Effect: Complexity Curve

• History: startups can’t compete effectively because of cost of developing large scale or complex infrastructure– Full-text index for 200MM documents– Geocoded real-time messages– Quickly index and navigate mesh of millions of friends, compute shortest-path distance

– Validate identity for secure transactions

• Services provide components and provide pay as you go model

• Scalable costs allow small companies to focus on core competency

CS98-10/198-10, Spr08

New models of software development

• Process: SupportDADO Evolution, 1 group• Waterfall: Static

Handoff Model, N groups

Develop

Assess Deploy

Operate

Develop

Assess

Deploy

Operate

CS98-10/198-10, Spr08

Agile (in 5 steps)

Client Focused DevelopmentLight Weight Process for Small Teams

1. Write Stories not Requirements2. Estimate Stories using Points3. Compute Your Velocity in Points per

Day4. Scope Your Project Based on Your

Velocity5. Spike Where You Can’t Estimate

CS98-10/198-10, Spr08

Real World Practices• Development vs. deployment environments & tools– Real version control (Subversion)– Automated deployment tools (Capistrano)

– Regression testing techniques (Rspec, Selenium)

– Database schema migration

• User Stories. Customer focused requirements.

• CRC cards (Class – Responsibility – Collaborator). Minimalist Object Modeling.

• Iterations and velocity to scope releases.

• Discussion of RoR and Web app entrepreneurship toward end of sem.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

CS98-10/198-10, Spr08

Prereqs & course format

• “Guided individual study”, 2-4 units C/NC

• Prereqs: CS 61A or equiv.; OOP concepts

• Hybrid lecture/lab format• Realistic development environment

– develop primarily on your own machines– Deploy to “production ISP” (RAD Lab cluster), also available for development use

– Each project gets an external sub-domain– Option to purchase your own domain

CS98-10/198-10, Spr08

Required Team Project

• You pick the app– Tie-in to Enterpreneurship (Eng 190C) to help brainstorm

• We help you scope & design the project

• We help you DADO the project• You give live demo & 5 minute “pitch” at end of semester

• “Best” project team members offered RAD Lab internships if they wish

• Some previous projects already .com’s

CS98-10/198-10, Spr08

Some projects from previous iterations

• Spring 07 Projects: heavy comments; test rigs; Good GUI; Fine in browser; 1k to 3k LOC– WeJoinIn (real users!): staff your volunteer events

– Calentopia: organize your academic life– 5um (next generation forum): organize & stay on top of all forums you read

• Fall 07 Projects with real users (used 5um in class);– Yumify: where go to dinner? http://www.yumify.com/

• 2.5K LOC, 0.3K Test rig, 1K Lines of comments– Hesperian: multilingual community health access http://hesperian.rorclass.org/• 0.5K LOC, 0.3K Test rig, 0.2K Lines of comments

Students from “award winning” projects already working in RAD Lab as ugrad research assistants

CS98-10/198-10, Spr08

Datacenter is the Computer

• Program == Web search, Mail,…• Computer == Building full of 1000s of

computers, networking & storage devices• Warehouse-sized facilities and

workloads may be unusual today but are likely to be more common in the next few years

• Re-inventing Client/Server Computing• The Datacenter is the (Server) Computer• The Laptop/Handheld is the (Client)

Computer– HP shipped more laptops than desktops in

2007– Million cells phones per day, increasing

functionality• Apple iPhone as one example

CS98-10/198-10, Spr08

Datacenter is the Computer

• Interesting design point: datacenter composed of 20 ft. containers– Power/cooling for 200 KW of racked HW– External taps for electricity, network, water– ~250 Servers, 7 TB DRAM, or 1.5 PB disk

• BlackBox to arrive next to Soda Spring 2008?

Sun Project Blackbox10/17/06

CS98-10/198-10, Spr08

RAD Lab research goal: Address Datacenter management

challenges• Management == efficient use of resources under dynamic demand, cost model, reliability– Resources: Order(s) of magnitude more devices– “Dynamic demand”: peaks 5-10X averages, provisioning hard

– “Reliability”: SW/HW failures commonplace– “Software Churn”: Google search rewritten twice in last 5 years; Ebay rewritten 4 times in 9 years

– “Dynamic cost model”: nonlinear cost models for power, cooling, network traffic

Too large scale, too complex for manual administration by people use

Statistical Machine Learning to tackle

CS98-10/198-10, Spr08

RoR as Datacenter Programming Language

• We believe RoR will have lasting impact– Tastefully chosen features in language & framework combine best ideas from programming language literature

– Intellectual/academic value in reinforcing key ideas of programming structures

– Bootstrap RoR expertise within Berkeley EECS

• Exceptionally rapid prototyping of Web 2.0 apps – Incremental/agile development support– Sophisticated UI support (Ajax, etc.)– Rapid bootstrapping doesn’t require learning bad habits

• Maximizes programmer productivity, which is increasingly a scarce resource

CS98-10/198-10, Spr08

So, why this course?

• We (researchers) are great at creating technologies, terrible at creating apps

• You create popular apps, your friends & colleagues flock to them

• We get apps/data to test our ideas• You get fame, etc.• Example: two recent RAD Lab MS grads...

CS98-10/198-10, Spr08

CS98/198 Development & Deployment Environment

Shared MySQL instance

Mongrel

Apachereverseproxy

Internet

Mongrel

Apachereverseproxy

.

.

.

Staticcon-tent

MySQLDev & Testing

Mongrel,Webrick,or IIS Capistrano

Sub-versionrepo-sitory

Your laptop

RAD Lab/ Millennium clusterwww.MyVanityURL.com

CS98-10/198-10, Spr08

Toolz U Will Uze

• Lab 0: install these items from class page• RoR development environment• Capistrano deployment tools• An editor (emacs, vi, vim...) or IDE (Eclipse)

• Subversion client for source code management (with SSL & Kerberos support compiled in)

• MySQLAdmin or CocoaMySQL database GUI• and signup for class forum on 5um.rorclass.org

CS98-10/198-10, Spr08

Movie time...

• radlab.cs.berkeley.edu/wiki/RoR

CS98-10/198-10, Spr08

Today: technical stuff review

• Web Apps 101, from Web 1.0 to Web 2.0– TCP/IP, HTTP, HTML, dynamic content (CGI)

– Web 1.0 vs Web 2.0– Model/view/controller

• Lifecycle of a Web app• REST• Get RoR install, etc. running on your computer

CS98-10/198-10, Spr08

The Web as remote procedure call

• RPC protocol == HTTP– ASCII based request/reply protocol run over TCP/IP

– protocol headers specify metadata about the request

– Stateless: notion of “session” must be synthesized separately

• RPC arguments == URL’s, HTML-form contents– URL names a server & resource– URL may embed “argument values”, or these can be “uploaded” as encoded HTML form submission

– REST goes a long way toward cleaning up this sorry state

browser serverThe Internet

CS98-10/198-10, Spr08

A conversation with a Web server

• Browser opens TCP connection to server on port 80 (default) and sends:

GET /index.html HTTP/1.0User-Agent: Mozilla/4.73 [en] (X11; U; Linux 2.0.35 i686) ...other boring headers...Cookie: B=2vsconq5p0h2n

• Server replies:HTTP/1.0 200 OKContent-Length: 16018Content-Type: text/html

<html><head><title>Yahoo!</title><base href=http://www.yahoo.com/>

…etc.

• Repeat for any embedded images, scripts, ...

CS98-10/198-10, Spr08

HTML in one slide

• Roughly hierarchical collection of elements that make up a viewable page– inline (headings, tables, lists...)– embedded (images, video, Java applets, JavaScript code...)– forms—allow user to submit simple input (text, radio/check

buttons, dropdown menus...)• Each element can have attributes (many optional)

– of particular interest are id and class attributes– CSS (Cascading Style Sheets) allow specification of visual

appearance of HTML pages based on the id’s and/or classes of elements

• Old “styling” elements (like <b>, <i>, etc.) are deprecated; use CSS instead, or where possible, logical styling like <emph>, <strong>

• Current incarnation, XHTML, more device-portable by being strict about syntax that went to pot in HTML– RoR and many other frameworks generate XHTML-compliant

code

CS98-10/198-10, Spr08

XHTML/CSS in 1 slide

• Declarative language to specify visual appearance of your HTML

• Style attributes include font color/size, background color/picture, border colors, padding, spacing, justification/position...

• Attributes can apply to specific elements, elements of a given class...

.bannerHead { background-color: #883333; color: white; padding: 8px; margin: 4px; border: 2px solid #d3bf8e; text-align: center; } .bannerHead a { color: white; text-decoration: none; }

pre { background-color: #eee; padding: 10px; font-size: 11px;}

CS98-10/198-10, Spr08

Web 0.9 and Web 1.0• Web <1.0: web page == file (“static content”)

– Web server (eg Apache) maps URL to file name• Web 1.0: web page == output of running a

program– URL or HTML form specifies name of program and

parameters to pass to it, via CGI (Common Gateway Interface) API

– example: http://www.foo.com/search?term=white%20rabbit&show=10&page=1

– App runs and generates HTML– Server passes HTML back to client via HTTP

• How to identify same user returning?– Server gives Web browser a cookie on 1st visit– Browser passes cookie back to server on each

request– Server can modify cookie before returning it to

browser– Cookie typically used to look up session info

in database or other store• Various frameworks now capture this common

structure

apache

your app

CGI

Filesystemor databasemysql

Rubyinterp.

firefox

CS98-10/198-10, Spr08

The MVC Design Pattern

• Goal: separate organization of data (model) from UI & presentation (view) by introducing controller – mediates user actions requesting access to data– presents data for rendering by the view

• Web apps are “sort of” MVC by design

Controller

View Model

• User actions• Directives for rendering data

• Read data• Update data

• Data provided to views

.rb (Ruby) code

.rhtml template(or .rjs, .rxml...)

SQL table + Ruby class

CS98-10/198-10, Spr08

What is Ruby on Rails?

• Ruby is a language that is...– dynamically typed, interpreted, object-oriented, functionally-inspired

• Rails is a web application framework that...– embodies the MVC design pattern– emphasizes convention over configuration – leverages Ruby language features incl. dynamic typing, metaprogramming, & object-orientation to provide elegant support for both goals

• Rails handles everything up to the point where your code is called

• And everything past the point where your code delivers stuff to the user.

CS98-10/198-10, Spr08

Web 1.0

• Web 1.0: user interaction == server roundtrip– Other than filling out form fields– Every user interaction causes server roundtrip– Every roundtrip causes full page redraw

• Web 1.5: user interactions without contacting server– e.g. form validation before submit– e.g. selecting something from menu A causes contents of menu B to change

– But every roundtrip still causes full page redraw

CS98-10/198-10, Spr08

Web 2.0

• Separation of server roundtrip from page rendering– Initial load of page & draw page– User interaction causes background roundtrip to server

– Response from server “captured” and passed to a programmer-defined JavaScript function

– That function can redraw part of the page in place (using same mechanisms as Web 1.5)

• Result: “desktop-like” responsive UI’s that can contact server– Auto completion– “Lazy” fetch of complicated parts of page– etc

CS98-10/198-10, Spr08

Managing Your Source

• Source/configuration management (SCM)– a/k/a revision control, version control– history in Unix world: sccs => rcs => cvs => svn

• Idea: take “snapshots” of code at different points– roll back to snapshot if screw up– everyone can bring their working copies up-to-date relative to some recent snapshot

– everyone can contribute changes to new snapshot (supports multiple developers working on same tree)

– “conflicts” detected at check-in time

CS98-10/198-10, Spr08

Basic operations of source code management

• First time: Check out a copy of current revision (or some previous revision) of code from repository

• Subsequent times: Update your working copy from repository (to get latest changes by others)

• Edit code, etc.• Commit your changes• Conflicts during commit? (if others modified same files as you)– identify bug in communication among developers– 1 person manually edits file to resolve conflict; marks file "resolved" and commits definitive version

– others update their version from newly-committed file

CS98-10/198-10, Spr08

Lifecycle of Web app

Shared MySQL instance

Mongrel

Apachereverseproxy

Internet

Mongrel

Apachereverseproxy

.

.

.

Staticcon-tent

Mongrel,Webrick,or IIS Capistrano

Sub-versionrepo-sitory

Your laptop

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

IDE or editor1. Checkoutor update

2. edit & test 3. commit

4. deploy

MySQLDev & Testing

migrate

update

CS98-10/198-10, Spr08

REST is a Software Architectural Style

• What's software architecture?– subdivides complex systems into components– assigns each component specific responsibilities

– dictates how components communicate and which ones communicate with which others

– Models the information (domain) into entities and specifies how it will be transformed and flow through the application

– State management

CS98-10/198-10, Spr08

REST

• REST: Representational State Transfer– Documents are transferred around– Application state is contained entirely in the documents

– URI's name documents and actions to be performed on them

– The definitive document on REST is Roy Fielding's dissertation:

http://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf

CS98-10/198-10, Spr08

Web Architecture

• Network vs. Distributed– Distributed: A single system that distributes its

work across multiple machines– Networked: A system that is utilized across a

network

• We are discussing networked architecture

• A networked architecture can have a distributed architecture at the server

CS98-10/198-10, Spr08

REST and CRUD

• CRUD = basic operations on data objects (documents)

• REST maps CRUD actions to HTTP methods (commands)

CRUD HTTP

Create POST

Read GET

Update PUT

Delete DELETE

CS98-10/198-10, Spr08

URI Refers to Resources

• A resource is an object that can be operated upon

• A URI will always refer to the same semantic resource– "customer with ID # 27"– "first order belonging to customer with ID # 27"

• Resources are delivered in the manner best suited for the user agent– HTML– XML– WML– Etc…