S1 2GX 2011 - Using Grails on a public facing Fortune 500 website

transcript

CLICK TO EDIT MASTER TITLE STYLE

Click to edit Master text styles– Second level

Third level

Using Grails on a public facing Fortune 500 website

10/26/2011

INTRODUCTION

Jim Shingler• Lead Technical Architect for Big Lots• Beginning Groovy and Grails – Co-Author• Griffon In Action – Co-Author• Co-Creator of FallME (IOC Container for JavaME)• Grails and Griffon plugin contributor• Speaker

AGENDA

• Background Biglots.com• Stats• Why Grails• Adopting Grails• Architecture• Plugins• Development Tools and Process• Performance• JVM Tuning• Monitoring

BACKGROUND – WWW.BIGLOTS.COM

Biglots is the nation's largest broadline closeout retailer, Big Lots has the power to negotiate the best deals in the business. We sell a broad range of high-quality, brand-name products, including consumables, seasonal items, furniture, housewares, toys, electronics, home decor, tools and gifts.

Lots of Items, over 325KNot all items in all 1400 storesWebsite has to handle this

VERY Dynamic, High frequency of content change.

Business Partner – Marketing

Marketing owns website, sets functional direction, sets functional priorities, responsible for content creation and management.

Business Functional Requirements / Objectives

Bring Website development in-house

Faster content creation and deployment

Give Business Partner more control

No Style, Layout Changes

IT Non-Functional Requirements / Objectives

Minimize IT Support requiredSupportable by ITMust run on Windows ServersMust use SqlServer DBGive Business Partner tools they

needJava based

http://rachelmariepaquin.files.wordpress.com/2011/02/tardis_2.jpg

BACKGROUND – WWW.BIGLOTS.COM12/2003

Source: http://wayback.archive.org/web/*/http://www.biglots.com

12/2004

12/2005

12/2006

12/2007

12/2008

12/2009

02/2011

Biglots.com Corporate Lots2Give.com

KioskStoreLocator Services

Store LocatorAdvanced JavaScript

Google Maps

Weekly AdI2O Integration

Flash AdHTML Ad

Rewards ClubBack Office Integration

Web Services

Rewards ClubAccount Maint.

Back Office IntegrationWeb Services

Search Results Searchable / Elastic

Search Plugin

DepartmentDynamic Changing

Images

CatalogDynamic Items

PagingSorting

Breadcrumbs

Item DetailsSharing: Email, Twitter,

Facebook, . . . Might also like

CAMPAIGNS - EMAIL

WWW.BIGLOTS.COM

http://g-ecx.images-amazon.com/images/G/01/ciu/55/9c/5663228348a072b6d4473110.L._AA300_.jpg

Page ViewsPer Hour / Day / Week / Month / Qtr

Unique visitorsPer Hour / Day / Week / Month / Qtr

Frequency of Content Change5-15 / Week (Old Site 3-5 / Week)

Third level

PAGE VIEWS

STATS – PAGE VIEWS (HOUR)

STATS – PAGE VIEWS (DAY)

STATS – PAGE VIEWS (WEEK)

STATS – PAGE VIEWS (MONTH)

Third level

UNIQUE VISITORS

STATS – UNIQUE VISITORS (HOUR)

STATS – UNIQUE VISITORS (DAY)

STATS – UNIQUE VISITORS (WEEK)

STATS – UNIQUE VISITORS (MONTH)

Third level

MORE STATS

What happened?

STATS – WEBPAGETEST.ORG

WHY GRAILS

Why Not?

WHY GRAILS

Background: Java ShopSmall Shop (140 People in all of IT, 15 Java

Limited experience w/ Web Technologies

Use Spring and Hibernate

WHY GRAILS

Factors: Must leverage existing skill sets and

experiences Has to be easy Must be rapid High developer productivity required Must scale easily Tool support Vendor support

WHY GRAILS

Some Options Considered: Struts JSF / Seam Wicket Rails PHP Grails

Choose Grails:Experience implementing Wholesale Handheld System

- Flex, Handheld, and Web interfaces- Connected to Local DB and SAP

Winner

WHY GRAILS

experiences

Groovy leverages Java EcosystemGroovy leverages Java skill setsGrails leverage Spring, Hibernate and

many other industry standard frameworks

WHY GRAILS

Factors: Has to be easy

Can use Groovy or JavaGroovy is easyGrails conventions Grails easy to move outside of

conventions when you need to

WHY GRAILS

Factors: Must be rapid

Grails development is very addictive, You get lots done with very little coding compared to the old ways

Fast resultsWARNING: Grails development is very

addictive. MUST TEST

WHY GRAILS

Factors: High developer productivity requiredGroovy and Grails allow great

developer productivityDevelopers produce a lot of

functionality quicklyConsistently beating effort estimates

WARNING: Grails development is very addictive. MUST TEST

WHY GRAILS

Factors: Must scale easily

Grails scales like any other Java Web application, Same approach.

WHY GRAILS

Factors: Tool support

IDE – STS (Eclipse)Proven industry standard frameworksProven industry standard infrastructure

WHY GRAILS

Factors: Vendor support

The company behind Groovy and Grails

WHY GRAILS

experiences Has to be easy Must be rapid High developer productivity required Must scale easily Tool support Vendor support

Third level

ADOPTING GRAILS

Multipronged approach– Jumpstart– Books– Other Resources

ADOPTING GRAILS – GRAILS JUMPSTART TRAINING

http://upload.wikimedia.org/wikipedia/commons/4/4a/CBC_Classroom_1932.jpg

ADOPTING GRAILS – GRAILS JUMPSTART TRAINING

Grails Course Taught by Jeff Brown

- 4 Day On-Site- Elected +1 Day Consulting- 8 developers

Video sample of coarse:http://www.springsource.com/training/free-online-video-metaprogramming-groovy

ADOPTING GRAILS - BOOKS

http://upload.wikimedia.org/wikipedia/commons/thumb/f/f2/Statistics_norway_library.jpg

ADOPTING GRAILS – BOOKS

Definitive Guide to GrailsDecember, 2006Graeme Rocher

January, 20092nd Edition

Graeme Rocher, Jeff Brown

Beginning Groovy and Grails

June, 2008Christopher Judd, Joseph Nusairat, James Shingler

Grails Quick Start

October, 2009Dave Klein

The Pragmatic Bookshelf

Grails In Action

May, 2009Glen Smith and Peter

Ledbrook

Groovy In Action

January, 2007Dierk Koenig with Andrew

Glover, Paul King, Guillaume Laforge and Jon Skeet

Summer, 20122nd Edition

MEAP 10 Chapters AvailableDierk König, Guillaume Laforge, Paul King, Jon

Skeet, and Hamlet D'Arcy

Other Books

ADOPTING GRAILS – OTHER RESOURCES

Groovy Mag http://groovymag.com/

MichaelKismal

Grails Podcast http://www.grailspodcast.comThe Grails Podcast is a bi-weekly podcast about the Groovy scripting language, the Grails Web Framework and more. It is run by Glen Smith from Canberra, Australia and Sven Haiges

Groovy Blogs http://groovyblogs.org

Mail Groups and Forumshttp://grails.org/Mailing+listshttp://grails.org/Forums

ADOPTING GRAILS – OTHER RESOURCESTwitter – Some people to follow

Graeme Rochere - @graemerocher

Guillaumm Laforge - @glaforgeJeff Scott Brown -

@jeffscottbrownDierk Konig- @mittiePeter Ledbrook - @pledbrookBurt Beckwith - @burtbeckwithPaul King- @paulk_asertScott Davis - @scottdavis99Dave Klein - @daveklein

This is a good starting place, . . . . There are many more.

ARCHITECTURE

http://www.besthousedesign.com/wp-content/uploads/2008/10/openhouse-xten-02.jpg

ARCHITECTURE

ARCHITECTURE - CDNA new user request www.biglots.com

Akamai looks for cookie. Since it isn’t set send the request to BigLots Load Balancer

BigLots Load Balancer sends the request to an application server

Application Server returns response to BigLots Load Balancer with cookie set

BigLots Load Balancer returns response to Akamai

Akamai returns response to user

User makes additional request in same session

Akamai looks for cookie. Since it is found, it is mapped to the appropriate application server

Akamai sends request to the appropriate application server

Application server returns response to Akamai

Akamai returns response to user

ARCHITECTURE – FUNCTIONAL PLUGINS

Spring Security Core Spring Cache Elastic Search (Previously Searchable) Quartz Mail JQuery JQuery UI CKeditor Export Ref Code Sanitizer Recaptcha

ARCHITECTURE – DEVELOPMENT PLUGINS

Code NarcGMetricsCode CoverageMelody

DEVELOPMENT TOOLS AND PROCESS

IDE: Springsource Tool Suite (STS)

CI: BambooIssue Tracking: JiraWiki: ConfluenceSCM: SubversionBrowsers: All, Favor Firefox and

ChromeLocal Dev

- Tomcat, MySQL, HSQL, Grails 1.3.7

DEVELOPMENT TOOLS AND PROCESS

Environments– Local– Dev– QA– Staging– Prod

PERFORMANCE & TUNING

http://www.greatoccupations.com/wp-content/uploads/2010/02/measure.jpg

Measure !!!Don’t Guess

javamelody

JMXGoogle Page

HTTPFox

Reduce DB CallsTune Garbage CollectionManage Static Content / AssetsIn efficient code

Reduce DB CallsHibernate Caching

http://grails.org/doc/latest/guide/single.html#5.5.2.2 Caching Strategy

Examine Lazy vs Eager Fetch http://grails.org/doc/latest/guide/single.html#5.3.4 Eager and Lazy Fetching

Higher Level Caching using Spring Cache

Indexes (DB Access Analysis) http://grails.org/doc/latest/ref/Database%20Mapping/indexColumn.html

Tune Garbage CollectionTools: Visual VM, Hyperic, OthersAvoid / Reduce GC Pause

Easier with multi core server

Additional Resources http://www.slideshare.net/kensipe/debugging-your-production-jvm

PERFORMANCE & TUNING - JVM TUNING

PERFORMANCE & TUNING - JVM TUNINGLog It

-Xloggc:D:\apps\...\logs\gc_hnwip00109_ent1.log-XX:+PrintGC -XX:+PrintGCDetails-XX:+PrintGCTimeStamps-XX:+PrintHeapAtGC-XX:+PrintTenuringDistribution

Manage Sizing-XX:MaxPermSize=512m <Permanent Gen Size>-Xms4096m <Old Gen Initial Heap Size> -Xmx4096m <Old Gen Maximum Heap Size>-XX:NewSize=1792m <New Gen (nursery) Size>-Xss384k <Stack Size>

Collectors-XX:+UseConcMarkSweepGC <Old Gen – Concurrent Collector (Low Pause)>-XX:+UseParNewGC <Young Gen - Parallel Collect (Low Pause)>-XX:+CMSClassUnloadingEnabled <GC Sweep PermGen – Remove Unused Classes>

Manage Static Content / AssetsUse a Web Server and

Application ServerMove Static Content to Web

Server and Content Delivery Network (CDN)

What happened?

Manage Static Content / AssetsUse Best Practices

http://stevesouders.com/hpws/rules.php

http://developer.yahoo.com/performance/rules.html

http://juddsolutions.com/downloads/10_performance_tips.pdf

In efficient codelist.each{item -> …} vs for(item

in list) ClosureVs

Code Block

Advice: You might want to see how it impacts your

application

In efficient codeBest Practices

Cobertura

In efficient codeLearn more about Groovy and

Grails

Tuning Your Grails Applications

http://www.youtube.com/watch?v=gl83-80Qq94

http://blog.springsource.com/2010/06/23/gorm-gotchas-part-1/http://blog.springsource.com/2010/07/02/gorm-gotchas-part-2/http://blog.springsource.com/2010/07/02/gorm-gotchas-part-3/

Third level

MONITORING

MONITORING - OPERATIONAL

Internal Monitoring: Detailed information on the server (Box), Web Server (IIS), Application

Server (TC), Applications

Internal Monitoring: Alerting server (Box), OS, Server Logs, Application Logs

External Monitoring: User Experience, Runs scripts around the country.

External Monitoring: Constantly monitoring security of the web sites

MONITORING - ANALYTICAL

External Monitoring: All of the analytics that the business people want

External Monitoring: More analytics

External Edge Monitoring: Unique Visitors, Popular pages, Errors, …

SECURITY

External Monitoring: Constant Security Monitoring

Internal Monitoring: QA Time Security Check

Q & A DISCUSSION

Question and Answer

Discussion

@JShinglerhttp://www.linkedin.com/in/jimshingler

http://jshingler.blogspot.com/ShinglerJim at gmail.com

Third level

Thank You

S1 2GX 2011 - Using Grails on a public facing Fortune 500 website

Technology