Date post: | 14-Jan-2015 |
Category: |
Technology |
Upload: | elizabeth-leddy |
View: | 2,575 times |
Download: | 2 times |
unloading plone
Elizabeth Leddy@eleddy
Tools, Tips, and Techniques for Approaching Scalability
am I in the right room?Symptoms Include
inability to grow with traditional caching solutions, icky system performance, crashitis, log-o-phobia,
hardware anemia, mashup mayhem, diminishing confidence in your abilities as plone integrator or
administrator, where do I start migraines, all the rest of the talks look boring, this room has better internet connectivity, I
have no clue what I’m doing here
A responsive, scalable Plone setup has little to do with optimizing code
CONFUCIUSSAY
Measure, Measure, MeasureMeasure, Measure, Measure
Measure, Measure, MeasureMeasure, Measure, Measure
Measure, Measure, MeasureMeasure, Measure, Measure
Act
Measure, Measure, MeasureMeasure, Measure, Measure
Measure, Measure, Measure
CUSTOM PLUGINS
AUTO-RESTART
Monitoring Tools
ALERTS
MUNIN MONIT
GANGLIA ZOPE HEALTH WATCHER
ABNORMALITIES
TIME BASED GRAPHS
TIMEOUTS
I quit...
AFFECTION
Watching Bad PerformancesWhale watchers are triggered by system abnormalities, and grab relevant information to help you diagnose what happened.
Simple(r) Error MonitoringSludge Through Logs
Escalate with Email
Google Analytics for Timeouts, 4XX & 5XX
4 Stages of System Grief
HTTPD
CACHE
PROXY
ZOPE
DB CACHE
ZEO
Anger BargainingDenial
ZOPE++ ZOPE++++
HTTPD^N
CACHES
PROXY
ZOPE^N
DB CACHES
ZEO & CO.
Acceptance
LB
LB
ZEO SHARD
HTTPD HTTPD
PROXY
ZEO SHARD
HTTPD
ZOPE
ZEO
ZODB CACHE
VARNISH
SQUID
HAPROXY
MEMCACHE
POUND
RAM
CPU DISK
ZEO CACHE?
HARDWARE
OW MUCH
NGINX
Two’s Company, Three’s a Crowd
DB CACHE ZOPE++
How many zopes should I have?
2 zopes/cpu50% avg util
api/async instances?
ZOPE++
Complex by Default
The CacheFu Conundrum
Oh, CacheFu you have been a great leader and I wish I would have taken the
time to figure out how caching really works. I’m afraid I will fail you, never fully
understanding the 304. Shame!
The browser is the cheapest, easiest caching tool at your disposal, Daniel-son. It’s time to show the world you are the developer I know you can be...
user.gif link_icon.gif poo.gif
5200
2600
2100
2700
1500
1800
httpdplone
RewriteRule ^/(.*)portal_skins/mysite_images/(.*) /mysite_images/$2 [L]
Serve STATIC content from a STATIC web server to unload plone, provide faster response time to initial requests,and enable simpler cachingstrategies
HTTPD
.contenttype-folder{ background-image: url(folder.gif);}
30 requests, ~200 kb
CSS & Sprites
.contenttype-folder{ background-image: url(all-my-images.png); background-position: 75px 25px;}
1 request, ~18 kb
UNLOAD
HTTPD
Picking a Proxy or:How I learned to stop worrying and love HAProxy
graceful reloads
backend health
distribution algorithms
warmup time
preserve keepalives
web based stats
PROXY
if python < 2.6: socket.settimeout(2)else: urllib2.urlopen(url, timeout=2)
ZEO & CO.
Playing Nice with Others
ZOPE++
Pickle Drop
PickupProcess
Callback
Pickle Me Some Asynchronicity
ZOPE++
tal:on-errorZEO & CO.
EXTERNAL DATA ST.
ACCEPTANCEZEO is perfect for object
oriented access, but it needs a little help with other data
access paradigms
ZEO & CO.
RESTARTing zope is NOT a SIN
a PACKED zodb is a HAPPY zodb
CRON is fabrics’ HOMEBOY
FACTS
beware of backup side effects
deploy with 1-step rolling blackouts
sometimes its just easier...
ZOPE
ZEO & CO.
ON BACKUPS
watch the disk
use repozo
chunked rsync
ZEO & CO.
Plone != Drupal Scalability = Talent + Hardware
The Oath:
I will automate a modular installation of everything
I will not do anything manually that could be done automatically, which is everything
I can trigger unscheduled processes anytime, from anywhere
I appreciate that scaling is hard: the cloud offers options, not solutions
Living in the Cloud
‣ Always setup system and error monitoring first
‣ Choose the right software for YOUR hardware
‣ Use as many zopes as possible but no more
‣ Do it like a flickr engineer
‣ Don’t make zope handle unnecessary requests
‣ Never underestimate the importance of a proper disk-RAM partnership
‣ Design processes to be able to run from anywhere, at anytime, on any machine
SUMMARY
Thank You!
All this and more athttp://scalingplone.pbwiki.com