10 sql tips to speed up your database | CatsWhoCode.com http://www.catswhocode.com/blog/10-sql-tips-to-speed-up-your-database[08/29/2012 11:38:07 AM] 10 sql tips to speed up your database On most websites, content is stored in a database and served to visitors upon request. Databases are very fast, but there’s lots of things that you can do to enhance its speed and make sure you won’t waste any server resources. In this article, I have compiled 10 very useful tips to optimize and speed up your website database. Design your database with caution This first tip may seems obvious, but the fact is that most database problems come from badly-designed table structure. For example, I have seen people storing information such as client info and payment info in the same database column. For both the database system and developers who will have to work on it, this is not a good thing. When creating a database, always put information on various tables, use clear naming standards and make use of primary keys. Source: http://www.simple- talk.com/sql/database- administration/ten-common- database-design-mistakes/ Know what you should optimize If you want to optimize a specific query, it is extremely useful to be able to get an in- depth look at the result of a query. Using the EXPLAIN statement, you will get lots of useful info on the result produced by a specific query, as shown in the example
Transcript
10 sql tips to speed up your database | CatsWhoCode.com
10 sql tips to speedup your databaseOn most websites, content is stored in adatabase and served to visitors uponrequest. Databases are very fast, but there’slots of things that you can do to enhance itsspeed and make sure you won’t waste anyserver resources. In this article, I havecompiled 10 very useful tips to optimize andspeed up your website database.
Design your databasewith cautionThis first tip may seems obvious, but the factis that most database problems come frombadly-designed table structure.For example, I have seen people storinginformation such as client info and paymentinfo in the same database column. For boththe database system and developers whowill have to work on it, this is not a goodthing.When creating a database, always putinformation on various tables, use clearnaming standards and make use of primarykeys.Source: http://www.simple-talk.com/sql/database-administration/ten-common-database-design-mistakes/
Know what youshould optimizeIf you want to optimize a specific query, it isextremely useful to be able to get an in-depth look at the result of a query. Using theEXPLAIN statement, you will get lots ofuseful info on the result produced by aspecific query, as shown in the example
The fastest query… Isthe one you don’tsendEach time you’re sending a query to thedatabase, you’re using a bit of your serverresources. This is why, on high traffic sites,the best thing you can do in order to speedup your database is to cache queries.
There’s lots of solutions to implement aquery cache on your server. Here are a few:
AdoDB: AdoDB is a databaseabstraction library for PHP. It allows youto use the database system of yourchoice (MySQL, PostGreSQL, Interbase,and way much more) and it is designedfor speed. AdoDB provides a simple,yet powerful caching system. And lastbut not least, AdoDB is licenced underthe BSD, which means that you can usefreely on your projects. A LGPL licenceis also available for commercialprojects.
Memcached: Memcached is adistributed memory caching systemwhich is often used to speed updynamic database-driven websites byalleviating database load.
CSQL Cache: CSQL Cache is anopen-source data caching infrastructure.Never tested it personally, but it seemsto be a great tool.
EXPLAIN SELECT * FROM ref_table,other_table WHERE ref_table.key_column=other_table.column;
Don’t select what youdon’t needA very common way to get the desired datais to use the * symbol, which will get allfields from the desired table:
Instead, you should definitely select only thedesired fields as shown in the examplebelow. On a very small site with, let’s say,one visitor per minute, that wouldn’t make adifference. But on a site such as Cats WhoCode, it saves a lot of work for thedatabase.
Use LIMITIt’s very common that you need to get only aspecific number of records from yourdatabase. For example, a blog which isshowing ten entries per page. In that case,you should definitely use the LIMITparameter, which only selects the desirednumber of records.Without LIMIT, if your table has 100,000different records, you’ll extract them all,which is unnecessary work for your server.
Avoid queries inloopsWhen using SQL along with a programminglanguage such as PHP, it can be tempting touse SQL queries inside a loop. But doing so
SELECT * FROM wp_posts;
SELECT title, excerpt, author FROM wp_posts;
SELECT title, excerpt, author FROM wp_posts LIMIT 10;
10 sql tips to speed up your database | CatsWhoCode.com
Use join instead ofsubqueriesAs a programmer, subqueries are somethingthat you can be tempted to use and abuse.Subqueries, as show below, can be veryuseful:
Although subqueries are useful, they oftencan be replaced by a join, which is definitely
foreach ($display_order as $id => $ordinal) { $sql = "UPDATE categories SET display_order = $ordinal WHERE id = $id"; mysql_query($sql);}
UPDATE categories SET display_order = CASE id WHEN 1 THEN 3 WHEN 2 THEN 4 WHEN 3 THEN 5 ENDWHERE id IN (1,2,3)
SELECT a.id, (SELECT MAX(created) FROM posts WHERE author_id = a.id)AS latest_post FROM authors a
Be careful when usingwildcardsWildcards are very useful because they cansubstitute for one or more characters whensearching for data in a database. I’m notsaying that you shouldn’t use them, butinstead, you should use them with cautionand not use the full wildcard when the prefixor postfix wildcard can do the same job.In fact, doing a full wildcard search on amillion records will certainly kill yourdatabase.
Use UNION instead ofORThe following example use the ORstatement to get the result:
SELECT a.id, MAX(p.created) AS latest_postFROM authors aINNER JOIN posts p ON (a.id = p.author_id)GROUP BY a.id
#Full wildcardSELECT * FROM TABLE WHERE COLUMN LIKE '%hello%';#Postfix wildcardSELECT * FROM TABLE WHERE COLUMN LIKE 'hello%';#Prefix wildcardSELECT * FROM TABLE WHERE COLUMN LIKE '%hello';
The UNION statement allows you tocombine the result sets of 2 or more selectqueries. The following example will returnthe same result that the above query gets,but it will be faster:
Source:http://www.bcarter.com/optimsql.htm
Use indexesDatabase indexes are similar to those youcan find in libraries: They allow the databaseto find the requested information faster, justlike a library index will allow a reader to findwhat they’re looking for without loosing time.An Index can be created on a single columnor a combination of columns in a databasetable. A table index is a database structurethat arranges the values of one or morecolumns in a database table in specificorder.
The following query will create an index onthe Model column from the Product table.The index is called idxModel:
Bart Jacobs said: March 8,2010 at 11:49 amThis article is very useful foranyone working with mySQL(and with databases ingeneral). For high trafficwebsites especially, databasequeries can be the bottleneck(and often are). Thanks forthis great post!
Reply
Can Aydoğan said: March8, 2010 at 1:27 pmGreat article. Thanks forsharing!
Metric Stormtrooper said:March 8, 2010 at 3:27pmyour table type can also makea lot of difference. MyISAM isbetter if you expect a lot ofreads, InnoDB if you needmore writes in a table (seealsohttp://www.softwareprojects.commysql-storage-engines-1470.html ).
And don’t forget to enable thequery cache in your my.ini
Reply
Kovica said: March 8, 2010at 3:29 pm“Use Indexes” should be #1.
Reply
Jean-BaptisteJung said:March 9,2010 at4:20 amTips aredisplayedwithoutimportanceorder, but Iagree thatindexes aredefinitelyone of thebest way tooptimizeyour DB!
wxianfeng said: March 8,2010 at 8:57 pmvery helpful~!
Reply
10 sql tips to speed up yourdatabase | [codepotato]said: March 9, 2010 at4:00 am[...] 10 sql tips to speed upyour database Posted March9th, 2010 in Blog by Garethhttp://www.catswhocode.com/blosql-tips-to-speed-up-your-database [...]
Anonymous said: March 9,2010 at 8:13 amOne thing anyone thinkingabout optimising their queriesshould bear in mind is not todo it straight away. Get yourcode working and then goback to it and improve ifnecessary.
‘may’ when you mentionedUNION. Your source states“may run faster than”, not “willbe faster”. I’ve found that ORis faster than UNION and aquick google on the subjectconfirms this:
Your source,bcarter.com/optimsql.htm wasapparently last updated in1996 and I’d like to thinkMySQL had changed quite abit since then.
If anyone’s looking intospeeding up their MySQLstuff thenmysqlperformanceblog.com ishighly recommended, theEnglish isn’t always greatthough and the navigationsucks but give it a try.
To expand on what Dirk sorightly said, Indexes are greatfor tables where there will bea lot of read (SELECT)operations. But if you’re oftenwriting to a table then theIndexes can slow it downbecause of course the indexmust be updated.
Could you please consider anupdate Jean, which reflectswhat Dirk and I havementioned?
Jean-BaptisteJung said:March 9,2010 at8:49 amYou’re right
about thebcartersource: Ididn’tchecked the“last update”date whenwriting thearticle, sorryfor this.
I rarelyupdateposts, but I’llrewrite someparts thisweek,according tomy busyschedule.
Thanks a lotfor yourinterest!
Timothy said: March 9,2010 at 8:47 amAlso, here are a few moredetails that could help:
1) Be wise with column types(VarChar, Blob, Clob / LongText, etc). When usingsomething like int, varchar,etc with a set length the SQLserver will know what toexpect when conductingqueries. If you use somethinglike Clob / Long Text, whichdoes not have a set limit,then the server will not knowwhat to expect, so the querieshave to way of beingoptimized. Basically, it meansthat every query that involvesa column with such a type willbe slow. Only use these
2) Using trims or other stringmodifiers will greatly slowdown your queries. Theserver will have to do this oneach row. Pretty logical whenyou think about it. Any timeyou are modifying a string orother variable type within aquery you are adding a greatdeal of overhead. If you can,put these kind of things onthe server script side (PHP,Ruby, etc).
3) If you have two tables, with1 to 1 for every (key wordthere) row, then why not usejust one table? Don’t makemore tables than you need.Just because you’ll havemore columns doesn’t meanyou will have to SELECTthem all.
4) If you can, avoid puttingcertain things in a table. Likeimages. I often find out thatpeople store certain images ina table as BLOBs and acolumn for the imagemimetype. Then they querythe image and use somethinglike PHP to spit it out onto thepage. The only time I’ve reallyseen this to be necessary iswith CAPTCHA images.Where the image is stored asa BLOB alongside a hashrepresenting the text in theimage. Though, you couldprobably put the images in afolder with random filenamesand have a table matchingfilenames to hash values. Butthat can open up somesecurity risks (like black-hatscreating their own look-uptable so that they can producescripts).
10 sql tips to speed up your database | CatsWhoCode.com
Daniel said:March 9, 2010at 6:25 pmin reply to your tip3:
i disagree.sometimes it canmake sense to splitup a table into two,because searchingin very wide (lots ofcolumns) tablesalso has aperformanceimpact. splitting uptables is prettyuseful if you onlysearch on a coupleof fields in thistable, and themajority of theother fields is just“additional info”such astimestamps,user_ip, filelocations etc. younever considerwhen looking uprecords.this also meansthat it is easier tokeep your “index-
table” free fromvariable length datatypes as describedin tip 1
Reply
Tomasz Kowalczyk said:March 9, 2010 at 1:53pmVery good list, I would alsoadd using ORMs such asDoctrine / Propel [included inframeworks such as symfony]that are caching everything –no need to remember thatthings.
memoryusage* I evenheard aboutan option tobuffer queryexecution, todo it in onetime. That isnot onlystupid. But itis also verydangerous! Iusetransactionsallot andusing anORMsystem willbreak this.
Some one Iknowdecided onusing anORM, and isnot veryhappy withit.But he can’tswitch sincethat willbreak thecoding.
mithu said: March 9, 2010at 4:35 pmLife saving tricks. Thankx.
Reply
CoryMathews said: March9, 2010 at 5:07 pm“Use join instead of subqueries”
Depends on if the sub queryis in the select from or wherestatement. So this advicewhile normally true for selectsis not always true for the fromor where statement. Thus isfalse.
“Use UNION instead of OR”
His benchmark says that butevery db I have worked withsays otherwise..
Reply
James Edinburgh said:March 9, 2010 at 6:26pmThanks for this information.
I’m just getting started in theworld of SQL and databasesand this is some solid advice.I’ve been playing around withWordPress and other PHPbased CMS systems and Ireally really want to getstarted and build my ownsystem, i’ll be sure to putthese tips to work when I do.
Mes favoris du 8-03-10 au10-03-10 » Gilles Toubianasaid: March 10, 2010 at9:03 am[...] 10 sql tips to speed upyour database [...]
Reply
» ce que j’ai vue depuismon retour ! oxynel, blogcommunication - Agence decommunication Montpelliersaid: March 10, 2010 at11:03 am[...] Du SQL, du SQL ou l’artd’optimiser ses requêtes [...]
Reply
10 Astuces SQL pouraméliorer vos requêtes |KubX said: March 11,2010 at 7:51 am[...] 10 sql tips to speed upyour database. Tags:sql [...]
Harsh Athalye said: March11, 2010 at 11:38 amGood SQL tips. Optimizingqueries is such a huge topicin itself, that a book can bewritten around it. But basicallytips mentioned here are goodenough. I would add somemore:
1. Choose wise columnswhen creating indexes.Creating indexes on datatypes of char, varchar or suchother sizable data types cankill the performance2. If you have indexes on
columns, try to avoidwrapping column inside afunction3. Convert IN clause to joins4. Set based queries definitelybeat procedural approach likecursors in terms ofperformance
Reply
Brian said: March 12, 2010at 2:03 amGreat tips there, I’m suprisedyou’ve not mentioned themysql query cache, qcache.This can dramatically speedup mysql especially on hightraffic sites.
I must admit i’d not givenmuch thought to design, Ilumped all the fields in onetable which was fine whenthe site had 100 entries butnow it has over 3000 thingsare really starting to sufferand I’m having to strip out thedatabase and organise it.
It goes to show that a little bitof haste can really turn into alot of waste later on.
Reply
LISA TORRES said: March13, 2010 at 10:18 amThe server will have to dothis on each row. Prettylogical when you think aboutit. Any time you are modifyinga string or other variable typewithin a query you are addinga great deal of overhead.
quicoto said: March 17,2010 at 11:03 amNice I never though how badcan be ask for “SELECT *”without no LIMIT or withoutasking for certain fields.
Regards
Reply
TDAH said: March 19,2010 at 1:25 amMan, I just tried these on mydb and they worked great,thanks
Reply
Christian said: March 19,2010 at 5:02 amWow, I did not know theupdate query with the nice“CASE” syntax until now.Very helpful. Thanks.
Reply
Jonathan Good said: March20, 2010 at 1:50 pmGreat article thanks forsharing – we can all usefaster dbs!
Reply
Agnideb Mishra said:March 20, 2010 at 10:32pmexcellent article! The queriesare particularly very helpful fora newbie like me. Thanks forthe awesome information.
Jack colt said: March 22,2010 at 11:56 pmThanks for the tips. Most ofmy websites are really slow inloading. I tried using WPsuper chache without muchsuccess. It appears now thatthe culprit lies in my designand database.
Reply
Shukrit Verman said: March27, 2010 at 11:15 pmThanks a bunch for thisarticle. I will try to fix my slowblog using these tips
Reply
Chimpu Sharma said:March 27, 2010 at 11:22pmI have always had to take myweb host’s help for runningsql queries. Thanks to thisarticle, I guess I would nolonger have to! Thanks again
Reply
Bob said: March 27, 2010at 11:36 pm“I tried using WP super cachewithout much success.”
It is not working for me either.Actually, while it is supposedto speed up my blog, it isactually slowing the blogdown I would try using thedatabase optimization tips laidout in this article.
Vinith Shukla said: March27, 2010 at 11:39 pmThank you very much forthese details; but for them, Iwould not have known how tooptimize my SQL database!
Reply
Dan - LogoFoo.com said:March 31, 2010 at 8:41pmVery helpful. By the way, Ithink using SELECT * is notreally that harmful if youcombine it with the LIMIToperator.
Reply
Blogger said: April 1, 2010at 9:10 amHu, i had never thought thatthose changes could speedup my DB.Thanks for collecting theseinformations!
Reply
SQLを早くする10個の項目 |アイビースター said: April 7,2010 at 2:02 am[...] 10 sql tips to speed upyour database [...]
Reply
fireRoxy said: April 13,2010 at 4:45 amvery useful tips. thank youvery much.
Savita Bisht said: April 22,2010 at 11:20 pmyeah, it is a nice article,worked great for me.
Reply
Mundu said: April 29, 2010at 11:06 amThanks so much for thisinformative article. I now knowhow to fix my extremely slowblog
Reply
8 consigli e trucchi sql perottimizzare e rendereperformante un Database |Pecciola said: April 30,2010 at 12:33 am[...] Fonte : Catswhocode [...]
Reply
Edward said: May 13, 2010at 10:59 amThis post is very usefull,thank youI also heard about somethingcalled stored queries so youjust call it by its name whenyou do the query for example“getnews()” but I’m acomplete newbie !
Michael said: May 13, 2010at 11:37 amThanks for sharing thisinformation, it is very useful tomy visual basic programming.
Jamie said: May 25, 2010at 10:51 amThis is indeed very helpful. Iappreciate the fact that youare providing screenshots sowe would know we are on thesame track while followingyour tutorial. Thanks so muchfor sharing.
Reply
Johnson said: May 31,2010 at 7:04 pmGreat tips, i’ve beenwondering if i could customisemy wordpress database, thisis too slow as postsaccurelates, anyone haveinterests ?
Reply
Jon said: June 11, 2010at 1:05 pmI may have to come back tothis page very shortly…a lotof good info here.
Reply
Jennifer R said: June 12,2010 at 10:34 amCould use DB cache reloadcan increase the performanceof mySQL database?
Büyü said: June 18, 2010at 10:12 amCould use DB cache reloadcan increase the performance
Alex Wolf said: June 20,2010 at 11:16 amThanks for the limit tip, Ididn’t know about it.
Reply
Charlie said: August 3,2010 at 1:52 pmAll good tips, particularly onthe use of UNION over OR.
As far as I know, in mostSQL databases, some uses ofOR prevent the use of anindex. So for large tables, it isfaster to scan the indicesmultiple times rather thansearching the entire tableonce.
Reply
Sumanta Sinha said:August 20, 2010 at 5:09amLately one of my PHP scriptsis making way too many sqlqueries while it could easilydo the job by making fewerqueries. The reason: thescript developer did notbother to optimize the scriptsso it could use the leastpossible resource. He hasalso stopped supporting theproject a year ago. Thanks tomy web host, I would havenever had known about this. Iwould try your tips and see ifit helps at all. thanks
Leave a ReplyYour email address will not be published.Required fields are marked *
Name *
Reply
Felipe said: November 4,2010 at 6:47 pmAlways refreshing to seesomeone pop open the hoodand show how it is done inthe age of “just-use-my-cuttie-pretty-web-2.0-framework-and-it-will-take-care-of-everything-for-you-in-a-pinch”. Congrats.
Reply
Shawn said: June 21, 2012at 11:49 amI know it’s a bit of a late postbut…..”Avoid queries inloops,” while i agree with youthat you shouldn’t keepsending SQL to a databasewhile looping, i think yourexample is *way* to simple tobe of use. You are essentiallyhard coding the case valueswhich 1. Shouldn’t be hardcoded and 2. onlymanageable for a very smallset. I believe a betteralternative is to go ahead anduse the looping statement andGENERATE the SQL needed.Ex. Update value 1; Updatevalue 2; update value 3,update value N. That is, toaggregate the SQL into astring variable, AND THENsend it to the database onlyonce and after it is generated.
1,075 people like Cats Who Code.1,people like Cats Who Code.
Confirm
Phil Ahmed
Like You lik Page You lik Page
Facebook social plugin
Latest postsjQuery plugins for awesome webtypographyAwesome CSS3 generators tosimplify front end developmentPractical tips to save bandwidth andreduce server load10 useful typography tips to improveyour website readabilityWordPress dashboard hacks fordevelopers and freelancers
CatsWhoCode.com is hosted byVidaHost. Use our exclusive couponCATSWHOCODE to get a 10% discounton hosting!
Search
Latest snippetsGet current url with PHPAutomatically creates variables withthe same name as the key in thePOST arrayjQuery 5 row accordionDetect double click in CSSCreate a page top shadow in CSS3Display a black&white version of animage using CSSDelete directory in PHPDetecting Mozilla App Install
