Date post: | 18-Aug-2015 |
Category: |
Technology |
Upload: | simone-di-maulo |
View: | 192 times |
Download: | 1 times |
— D O N A L D K N U T H — “ S T R U C T U R E D P R O G R A M M I N G W I T H G O T O S TAT E M E N T S ”
[…] premature optimization is the root of all evil.
— D O N A L D K N U T H — “ S T R U C T U R E D P R O G R A M M I N G W I T H G O T O S TAT E M E N T S ”
[…] premature optimization is the root of all evil.
K E E P I N
M I N D
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
A PA C H E B E N C H M A R K
ab -n 200 -c 50 -k -g bench.tsv http://my.api.test/v1/pro... ^[1] ^[2] ^[3] ^[4] ^[5]
1. Number of requests 2. Concurrent requests 3. Use the keep-alive connection 4. Write the output ready for gnuplot 5. The URL to call
A PA C H E B E N C H M A R K ## Graph configuration ## # Set the output format and size set terminal jpeg size 1280,800
# Set the aspect ratio set size 1, 1
# Title set title "Benchmark testing"
# The legend/key position set key left top
# Draw gridlines on the y axis set grid y
# Label the x-axis set xlabel 'seconds'
# Label the y-axis set ylabel "response time (ms)"
## I/O Configuration # Specify that the x-series data is time data set xdata time
# The output file set output "bench.jpg"
# Specify the *input* format of the time data set timefmt "%s"
# Specify the *output* format for the x labels set format x "%S"
# Use tabs as the delimiter instead of spaces set datafile separator '\t'
## Run the Plot # Plot the data plot "bench.tsv" every ::2 using 2:5 with points
exit
• https://blackfire.io • https://tideways.io • http://xhprof.io/ • https://symfony.com/doc/current/cookbook/profiler/
index.html
$ curl -I http://my-api.com/users/toretto460
{ “id”: da34gtyu50-lo983, “username”: “toretto460”, . . . }
$ curl -I http://my-api.com/users/toretto460
{ “id”: da34gtyu50-lo983, “username”: “toretto460”, . . . }
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Sat, 28 Jul 2015 20:12:45 GMT
Cache-Control: private max-age=600
ETag: 88493f3-4afd-507dd8e0aa030
{ “id”: da34gtyu50-lo983, “username”: “toretto460”, . . . }
$ curl -I http://my-api.com/users/toretto460
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Sat, 28 Jul 2015 20:12:45 GMTCache-Control: private max-age=600
ETag: 88493f3-4afd-507dd8e0aa030
{ “id”: da34gtyu50-lo983, “username”: “toretto460”, . . . }
$ curl -I http://my-api.com/users/toretto460
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Sat, 28 Jul 2015 20:12:45 GMT
Cache-Control: private max-age=600ETag: 88493f3-4afd-507dd8e0aa030
{ “id”: da34gtyu50-lo983, “username”: “toretto460”, . . . }
$ curl -I http://my-api.com/users/toretto460
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=utf-8Date: Sat, 28 Jul 2015 20:12:45 GMTCache-Control: private max-age=600ETag: 88493f3-4afd-507dd8e0aa030
{ “id”: da34gtyu50-lo983, “username”: “toretto460”, . . . }
$ curl -I http://my-api.com/users/toretto460
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json; charset=utf-8Date: Sat, 28 Jul 2015 20:12:45 GMTCache-Control: private max-age=600ETag: 88493f3-4afd-507dd8e0aa030
Avoid same request until 28 Jul 2015 20:22:45 GMT
P L AY W I T H H E A D E R S
public function getAction($userIdentifier, Request $request) { $response = new Response();
$user = $this->fetchUser($userIdentifier); $response->setETag($user->caclulateETag()); // $response->setLastModified($user->getLastUpdateDate()); if ($response->isNotModified($request)) {
return $response; } ... }
T O O M U C H M A G I C I N S I D E
https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching
HTTP Compression is a set of rules defined by the protocol which enable network
entities (Client and Server) to exchange compressed data.
# Apache Configuration File
# ------------------------------------------------------------------------------ # | Compression | # ------------------------------------------------------------------------------
<IfModule mod_deflate.c>
<IfModule mod_filter.c> AddOutputFilterByType DEFLATE application/javascript \ application/json \ text/css \ text/html \ text/plain \ text/xml </IfModule>
</IfModule>
E N A B L E C O M P R E S S I O N
# Apache Configuration File
# ------------------------------------------------------------------------------ # | Compression | # ------------------------------------------------------------------------------
<IfModule mod_deflate.c>
<IfModule mod_filter.c> DeflateFilterNote Input input_info DeflateFilterNote Output output_info DeflateFilterNote Ratio ratio_info LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate CustomLog /var/log/apache2/deflate_log deflate
</IfModule>
</IfModule>
L O G C O M P R E S S I O N
H T T P C O M P R E S S I O N
GET /users/toretto460 HTTP/1.1 Host: www.example.com Accept-Encoding: gzip, deflate
HTTP/1.1 200 OK Date: mon, 27 Jul 2015 22:38:34 GMT Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux) Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT Accept-Ranges: bytes Content-Length: 438 Connection: close Content-Type: text/html; charset=UTF-8 Content-Encoding: gzip
H T T P C O M P R E S S I O N
GET /users/toretto460 HTTP/1.1 Host: www.example.com
HTTP/1.1 200 OK Date: mon, 27 Jul 2015 22:38:34 GMT Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux) Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT Accept-Ranges: bytes Content-Length: 438 Connection: close Content-Type: text/html; charset=UTF-8
Accept-Encoding: gzip, deflate
Content-Encoding: gzip
T H E P O W E R O F H T T P
Client ServerISP Proxy
Client cache
Public cacheReverse ProxyData compression
T I P S
• AUTO_INCREMENT / SEQUENCE are not so cheap
• Understanding the TRACKING POLICY
• Use READONLY entities
• Let’s CACHE
A U T O _ I N C R E M E N T & S E Q U E N C E S A R E N O T S O C H E A P
class User { /** * @ORM\Id * @ORM\Column(name="ID", type="string", length=37, nullable=false) * @ORM\GeneratedValue(strategy="AUTO") */ private $id;
}
A U T O _ I N C R E M E N T & S E Q U E N C E S A R E N O T S O C H E A P
class User { /** * @ORM\Id * @ORM\Column(name="ID", type="string", length=37, nullable=false) * @ORM\GeneratedValue(strategy="AUTO") */ private $id;
public function __construct(UserInfoDTO $userInfo) { $this->id = (string) Uuid::uuid4(); $this->name = $userInfo->name; ... } }
U N D E R S TA N D I N G T H E T R A C K I N G P O L I C Y
— M A R T I N F O W L E R —
“Maintains a list of objects affected by a business transaction
and coordinates the writing out of changes and the resolution of concurrency problems.”
UNIT OF WORK
D E F E R R E D I M P L I C I T
With this policy, Doctrine detects the changes by a property-by-property comparison at commit time and
also detects changes to entities or new entities that are referenced by other managed entities …
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
D E F E R R E D I M P L I C I T
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
class User { /** * @ORM\Id * @ORM\Column(name="ID", type="string", length=37, nullable=false) */ private $id; /** * @ORM\Column(name="FIRST_NAME", type="string", length=100, nullable=false) */ private $firstName;
/** * @ORM\Column(name="LAST_NAME", type="string", length=100, nullable=false) */ private $lastName;
/** * @ORM\Column(name="BIRTH_DATE", type="date", nullable=false) */ private $birthDate; }
UOW
$em->flush();
D E F E R R E D E X P L I C I T
… the difference is that Doctrine 2 only considers entities that have been explicitly marked for change
detection through a call to EntityManager::persist(entity)
or through a save cascade.
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
D E F E R R E D E X P L I C I T
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
class User { /** * @ORM\Id * @ORM\Column(name="ID", type="string", length=37, nullable=false) */ private $id; /** * @ORM\Column(name="FIRST_NAME", type="string", length=100, nullable=false) */ private $firstName;
/** * @ORM\Column(name="LAST_NAME", type="string", length=100, nullable=false) */ private $lastName;
/** * @ORM\Column(name="BIRTH_DATE", type="date", nullable=false) */ private $birthDate; }
UOW
$em->persist($user); $em->flush();
N O T I F Y
This policy is based on the assumption that the entities notify interested listeners of changes to their properties.
For that purpose, a class that wants to use this policy needs to implement the NotifyPropertyChanged
interface
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
N O T I F Y
<?php
use Doctrine\Common\NotifyPropertyChanged, Doctrine\Common\PropertyChangedListener;
/** * @Entity * @ChangeTrackingPolicy("NOTIFY") */ class User implements NotifyPropertyChanged { // ... private $_listeners = array();
public function addPropertyChangedListener(PropertyChangedListener $listener) { $this->_listeners[] = $listener; }
}
N O T I F Y
<?php
// . . . protected function _onPropertyChanged($propName, $oldValue, $newValue) { if ($this->_listeners) { foreach ($this->_listeners as $listener) { $listener->propertyChanged($this, $propName, $oldValue, $newValue); } } }
public function setData($data) { if ($data != $this->data) { $this->_onPropertyChanged('data', $this->data, $data); $this->data = $data; } } }
U S E R E A D O N LY E N T I T I E S
This means that the entity marked as read only is never considered for updates
U S E R E A D O N LY E N T I T I E S
/** * @ORM\Entity(readOnly=true) */ class NewsTag { /** * @ORM\Id * @ORM\Column(name="ID", type="string", length=37, nullable=false) */ private $id; }
L E T ’ S C A C H E
/** * Configure the Doctrine ORM with a result cache provider. */ $config = new \Doctrine\ORM\Configuration();
// The APC way $apcCacheDriver = new \Doctrine\Common\Cache\ApcCache(); $config->setResultCacheImpl($apcCacheDriver);
// The memcache way $memcache = new Memcache(); $memcache->connect('memcache_host', 11211); $memCacheDriver = new \Doctrine\Common\Cache\MemcacheCache(); $memCacheDriver->setMemcache($memcache); $config->setResultCacheImpl($memCacheDriver);
L E T ’ S C A C H E
doctrine: orm: metadata_cache_driver: apc query_cache_driver: apc result_cache_driver: type: memcache host: localhost port: 11211
L E T ’ S C A C H E
class UserRepository { /** * @return User[] */ protected function findLocked() { $qb = $this->createQueryBuilder('user'); $qb->where($qb->expr()->eq('user.locked', true)); $query = $qb->getQuery(); $query->useResultCache(true, 60 * 2 /* 2 minutes */);
return $query->getResult(); } }
R E A L U S E C A S E - # 1 D ATA I M P O R T > 2 0 0 K E N T I T I E S
SEQUENCE UUID $em->flush(get_class($entity)); $em->clear(get_class($entity));
R E A L U S E C A S E - # 1 D ATA I M P O R T > 2 0 0 K E N T I T I E S
208k Entities loaded in 2h 25m
😟
R E A L U S E C A S E - # 1 D ATA I M P O R T > 2 0 0 K E N T I T I E S
AFTER THE CURE 208k Entities
loaded in 28 minutes 😊
S L O W TA S K
The user avatar should be resized to 300x300px
The administrator should upload up to 250k orders in a single cvs file
Send an email when a new post has been created
S L O W TA S K
A crawler should find and aggregate product info
Every X minutes the rss feed should be updated
The user can export all the orders
R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst I WANT TO request for a new monthly report
R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst I WANT TO request for a new monthly report
CREATE TABLE TMP_CFMS_R7_7844 AS SELECT ROWNUM AS ID, x.* FROM (SELECT COD_DEALER, COD_PDV, TOTAL_ACQUISIZIONI, CONFRONTO_ACQUISIZIONI, AVG_ACQUISIZIONI, PERCENTUALE_INCREMENTO, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL, TOTAL_SOSPENSIONI, SOSPENSIONI_CONFRONTO, AVG_SOSPENSIONI, PERCENTUALE_INCREMENTO_SOSP, CAST( NULL AS NUMBER ) QTA_SOGL_SOSP, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL_SOSP, PERCENTUALE_SOSPENSIONI_ACQU, PERC_SOSP_ACQU_CONF, PERCENTUALE_SUPERAMENTO, D_RAG_SOC, D_IND, D_LOCALITA, D_CAP, D_PROVINCIA, D_PIVA, INDIRIZZO V_IND, LOCALITA V_LOCALITA, CAP V_CAP, PROVINCIA V_PROVINCIA FROM ANAGRAFICA_DEALER, (SELECT COD_DEALER, COD_PDV, TOTAL_ACQUISIZIONI, CONFRONTO_ACQUISIZIONI, AVG_ACQUISIZIONI, PERCENTUALE_INCREMENTO, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL, TOTAL_SOSPENSIONI, SOSPENSIONI_CONFRONTO, AVG_SOSPENSIONI, PERCENTUALE_INCREMENTO_SOSP, CAST( NULL AS NUMBER ) QTA_SOGL_SOSP, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL_SOSP, PERCENTUALE_SOSPENSIONI_ACQU, PERC_SOSP_ACQU_CONF, PERCENTUALE_SUPERAMENTO, RAGSOC D_RAG_SOC, INDIRIZZO D_IND, LOCALITA D_LOCALITA, CAP D_CAP, PROVINCIA D_PROVINCIA, PIVA D_PIVA FROM ANAGRAFICA_DEALER, (SELECT A.COD_DEALER,A.COD_PDV,NVL(A.TOTAL_ACQUISIZIONI,'0') TOTAL_ACQUISIZIONI,NVL(A.CONFRONTO_ACQUISIZIONI,'0') CONFRONTO_ACQUISIZIONI,NVL(B.AVG_ACQUISIZIONI,'0') AVG_ACQUISIZIONI,NVL(B.PERCENTUALE_INCREMENTO,'0') PERCENTUALE_INCREMENTO, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL ,NVL(C.TOTAL_SOSPENSIONI,'0') TOTAL_SOSPENSIONI,NVL(C.SOSPENSIONI_CONFRONTO,'0') SOSPENSIONI_CONFRONTO, NVL(D.AVG_SOSPENSIONI,'0') AVG_SOSPENSIONI,NVL(D.PERCENTUALE_INCREMENTO_SOSP,'0') PERCENTUALE_INCREMENTO_SOSP, CAST( NULL AS NUMBER ) QTA_SOGL_SOSP, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL_SOSP ,NVL(E.PERCENTUALE_SOSPENSIONI_ACQU,'0') PERCENTUALE_SOSPENSIONI_ACQU, NVL(PERC_SOSP_ACQU_CONF,'0') PERC_SOSP_ACQU_CONF,NVL(PERCENTUALE_SUPERAMENTO,'0') PERCENTUALE_SUPERAMENTO FROM (SELECT ROWNUM AS ID, x.* FROM (SELECT COD_DEALER, COD_PDV, TOTAL_ACQUISIZIONI, CONFRONTO_ACQUISIZIONI, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL, D_RAG_SOC, D_IND, D_LOCALITA, D_CAP, D_PROVINCIA, D_PIVA, INDIRIZZO V_IND, LOCALITA V_LOCALITA, CAP V_CAP, PROVINCIA V_PROVINCIA FROM ANAGRAFICA_DEALER, (SELECT COD_DEALER, COD_PDV, TOTAL_ACQUISIZIONI, CONFRONTO_ACQUISIZIONI, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL, RAGSOC D_RAG_SOC, INDIRIZZO D_IND, LOCALITA D_LOCALITA, CAP D_CAP, PROVINCIA D_PROVINCIA, PIVA D_PIVA FROM ANAGRAFICA_DEALER, ( SELECT A.COD_DEALER,A.COD_PDV,A.TOTAL_ACQUISIZIONI,NVL(B.TOTAL_ACQUISIZIONI,'0') CONFRONTO_ACQUISIZIONI, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL FROM (SELECT A.COD_DEALER,A.COD_PDV,COUNT(A.COD_DEALER) TOTAL_ACQUISIZIONI, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL FROM CFMS_ATT A WHERE A.DATA_ATTIVAZIONE between TO_DATE('2015-06-15 00:00:00','yyyy-mm-dd HH24:MI:SS') AND TO_DATE('2015-06-15 23:59:59','yyyy-mm-dd HH24:MI:SS') AND A.COD_PDV IS NOT NULL GROUP BY A.COD_DEALER,A.COD_PDV) A LEFT JOIN ( SELECT COD_DEALER,COD_PDV,COUNT(COD_DEALER) TOTAL_ACQUISIZIONI FROM CFMS_ATT WHERE DATA_ATTIVAZIONE between TO_DATE('2015-06-16 00:00:00','yyyy-mm-dd HH24:MI:SS') AND TO_DATE('2015-06-16 23:59:59','yyyy-mm-dd HH24:MI:SS') AND COD_PDV IS NOT NULL GROUP BY COD_DEALER,COD_PDV ) B ON A.COD_DEALER=B.COD_DEALER AND A.COD_PDV=B.COD_PDV) WHERE COD_DEALER = COD_ID (+)) WHERE COD_PDV = COD_ID (+) ORDER BY COD_DEALER) x ) A FULL OUTER JOIN (SELECT ROWNUM AS ID, x.* FROM (SELECT COD_DEALER, COD_PDV, ACQUISIZIONI, AVG_ACQUISIZIONI, ACQUISIZIONI_CONFRONTO, AVG_ACQUISIZIONI_CONFRONTO, PERCENTUALE_INCREMENTO, D_RAG_SOC, D_IND, D_LOCALITA, D_CAP, D_PROVINCIA, D_PIVA, INDIRIZZO V_IND, LOCALITA V_LOCALITA, CAP V_CAP, PROVINCIA V_PROVINCIA FROM ANAGRAFICA_DEALER, (SELECT COD_DEALER, COD_PDV, ACQUISIZIONI, AVG_ACQUISIZIONI, ACQUISIZIONI_CONFRONTO, AVG_ACQUISIZIONI_CONFRONTO, PERCENTUALE_INCREMENTO, RAGSOC D_RAG_SOC, INDIRIZZO D_IND, LOCALITA D_LOCALITA, CAP D_CAP, PROVINCIA D_PROVINCIA, PIVA D_PIVA FROM ANAGRAFICA_DEALER, ( SELECT A.COD_DEALER,A.COD_PDV,A.ACQUISIZIONI,TO_CHAR(A.AVG_ACQUISIZIONI,'99990D99') AVG_ACQUISIZIONI,NVL(B.ACQUISIZIONI_CONFRONTO,'0') ACQUISIZIONI_CONFRONTO, CASE WHEN (B.AVG_ACQUISIZIONI_CONFRONTO
R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst I WANT TO request for a new monthly report
R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst I WANT TO request for a new monthly reportSO THAT the system will send me an email with the attached csv report
S C A L I N G T H E A S Y N C
Y O U R A P P I S
H E R E
W O R K E R
W O R K E R
W O R K E R
W O R K E R
Q U E U E
POST /rss
PATCH /users
POST /export
A S Y N C H R O N O U S TA S K
• https://github.com/uecode/qpush-bundle
• https://github.com/videlalvaro/php-amqplib
• https://github.com/videlalvaro/RabbitMqBundle
• http://gearmanbundle.readthedocs.org/en/latest/configuration.html
• https://github.com/chrisboulton/php-resque