Date post: | 11-Feb-2017 |
Category: |
Software |
Upload: | betclic-everest-group-tech-team |
View: | 355 times |
Download: | 0 times |
MAXIME LEMAITRE – 9/7/2015
Microsoft Azure Storage… on the road to a scalable architecture …
Agenda
• Introduction• What is a Storage Account ?• Storage Services• Quotas & Pricing• Working with Azure Storage• Demo• Conclusion• Question
Standard Storage Accounts
Blob storage stores file data. A blob can be any type of text or binary data, such as a document, media file, or application installer.
Table storage stores structured datasets. Table storage is a NoSQL key-attribute data store, which allows for rapid development and fast access to large quantities of data.
Queue storage provides reliable messaging for workflow processing and for communication between components of cloud services.
File storage (Preview) offers shared storage for legacy applications using the standard SMB 2.1 protocol. Azure virtual machines and cloud services can share file data across application components via mounted shares, and on-premise applications can access file data in a share via the File service REST API.
A standard storage account gives you access to Blob storage, Table storage, Queue storage, and File storage.
4
How to create a Storage Account ?(in less than 5 minutes)
Note : To use this new account, you just need the AccountName and Access Keys
5
Storage QuotasRESOURCE DEFAULT LIMITTB per storage account 500 TBMax size of a single blob container, table, or queue 500 TBMax number of blob containers, blobs, shares, tables, queues, entities, or messages per storage account
Only limit is the storage account capacity
Max size of a file share 5 TB (Unlimited number of files)Max 8 KB IOPS per persistent disk (Basic/Standard) 3001/5001Total Request Rate (assuming 1KB object size) per storage account
Up to 20,000 IOPS, entities per second, or messages per second
Target Throughput for Single Blob Up to 60 MB per second, or up to 500 requests per second
Target Throughput for Single Queue (1 KB messages) Up to 2000 messages per secondTarget Throughput for Single Table Partition (1 KB entities) Up to 2000 entities per secondTarget Throughput for Single File Share (Preview) Up to 60 MB per secondMax ingress2 per storage account (Europe) 5 Gbps if GRS3 enabled, 10 Gbps for LRSMax egress2 per storage account (Europe) 10 Gbps if GRS3 enabled, 15 Gbps for LRS
6
Pricinghttp://azure.microsoft.com/en-us/pricing/details/storage/
Additional €0.0027 per 100,000 transactions for all Standard storage types
Page Blobs & Disks
Block Blobs
Tables & Queues
7
• Build apps with first class client library support for .NET, Java, Android, C++, and Node.js.
• Data in Azure Storage is also accessible via REST API, which can be called by any language that makes HTTP/HTTPS requests
Clients
Also available in Node.js, Java, C++, Python, Xamarin, Ruby, …
8
• Azure Blob storage is a service for storing large amounts of unstructured data, such as text or binary data, that can be accessed from anywhere in the world via HTTP or HTTPS. Common uses include :– Serving images or documents directly to a browser– Storing files for distributed access– Streaming video and audio– Performing secure backup and DR– Storing data for analysis by
an on-premises or Azure-hosted service
• Concepts– Container : A container provides a grouping of a set of blobs. All blobs must be in a container.An
account can contain an unlimited number of containers. A container can store an unlimited number of blobs.
– Blob : A file of any type and size. There are two types of blobs that can be stored in Azure Storage: block and page blobs. Most files are block blobs. A single block blob can be up to 200 GB in size. This tutorial uses block blobs. Page blobs, another blob type, can be up to 1 TB in size, and are more efficient when ranges of bytes in a file are modified frequently. For more information about blobs, see Understanding Block Blobs and Page Blobs.
– URL format: http://<storage account>.blob.core.windows.net/<container>/<blob
Blob Storage
9
• A NoSQL datastore for large amounts of structured data. Accepts authenticated calls from inside and outside the Azure cloud. Azure tables are ideal for storing structured, non-relational data. Common uses include :– Storing TBs of structured data for web– Storing datasets that don't require complex joins,
foreign keys, or stored procedures and can be denormalized for fast access
– Quickly querying data using a clustered index– Accessing data using the OData protocol and LINQ queries with WCF Data Service .NET Libraries
Concepts– Table : A table is a collection of entities. Tables don't enforce a schema on entities, which means a
single table can contain entities that have different sets of properties. The number of tables that a storage account can contain is limited only by the storage account capacity limit.
– Entity: An entity is a set of properties, similar to a database row. An entity can be up to 1MB in size.– Properties: A property is a name-value pair. Each entity can include up to 252 properties to store
data. Each entity also has 3 system properties that specify a partition key, a row key, and a timestamp. Entities with the same partition key can be queried more quickly, and inserted/updated in atomic operations. An entity's row key is its unique identifier within a partition.
– URL format: http://<storage account>.blob.core.windows.net/<table>
Table Storage
10
• Azure Queue storage is a service for storing large numbers of messages that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. A single queue message can be up to 64 KB in size, and a queue can contain millions of messages, up to the total capacity limit of a storage account. Common uses include :– Creating a backlog of work
to process asynchronously– Passing messages from
an Azure Web role to an Azure Worker role
Concepts– Queue: A queue contains a set of messages. All messages must be in a queue.– Message: A message, in any format, of up to 64KB.– URL format: http://<storage account>.blob.core.windows.net/<queue>
Queue Storage
11
• Offers shared storage for applications using the standard SMB 2.1 protocol. Azure virtual machines and cloud services can share file data across application components via mounted shares, and on-premises applications can access file data via the File storage API. Common uses include :– Migrating on-premises applications that rely on file shares to run on Azure virtual machines or cloud
services, without expensive rewrites– Storing diagnostic data such as logs,
metrics, and crash dumps in a shared location
– Storing tools and utilities needed for developing or administering Azure VM or cloud services
– Storing shared application settings, for example in configuration files
Concepts– Share: A File storage share is an SMB 2.1 file share in Azure. All directories and files must be created
in a parent share. An account can contain an unlimited number of shares, and a share can store an unlimited number of files, up to the capacity limits of the storage account.
– Directory: An optional hierarchy of directories.– File: A file in the share. A file may be up to 1 TB in size.– URL format: https://<storage account>.file.core.windows.net/<share>/<directory/directories>/<file>
Files Storage
12
Partitions affect load balancing and scalability for each of the storage services.
• Blobs: The partition key for a blob is container name + blob name. This means that each blob has its own partition. Blobs can therefore be distributed across many servers in order to scale out access to them. While blobs can be logically grouped in blob containers, there are no partitioning implications from this grouping.
• Messages: The partition key for a message is the queue name, so all messages in a queue are grouped into a single partition and are served by a single server. Different queues may be processed by different servers to balance the load for however many queues a storage account may have.
• Entities: The partition key for an entity is table name + partition key, where the partition key is the value of the required user-defined PartitionKey property for the entity.
All entities with the same partition key value are grouped into the same partition and are stored on the same partition server. This is an important point to understand in designing your application.
Partitions in Azure Storage
13
• Provides a local environment that emulates the Azure Blob, Queue, and Table services for development purposes (File Shares is not supported)
• Uses a local MS SQL Server instance (LocalDB) and the local file system to emulate the Azure storage services
• a single fixed account and a well-known authentication key
• Service endpoints
Working with Storage emulatorhttps://azure.microsoft.com/en-us/documentation/articles/storage-use-emulator/
14
Usefull toolshttp://storagetools.azurewebsites.net/
Visual Studio + Azure SDK Azure Storage Explorer
15
Example #1a better front-end process to optimize images
Images (PNG/JPG/GIF/SVG)
Block Blob
CDN endpoint
Gulp process (Imagemin + upload to Azure)
16
Example #2Optimazer
Azure BlobStorage optimized
images
Worker RoleScan web pages
and optimize images
Azure TableStore pages &
images results/stats
Azure Web SiteWeb Api +
Angular
SchedulerAutomatically add
page to scanAzure Queue
Pending scans, optimizations …
Demo
Questions
References
• https://azure.microsoft.com/en-us/documentation/articles/storage-introduction/• http://azure.microsoft.com/en-us/documentation/services/storage/• https://azure.microsoft.com/en-us/documentation/articles/storage-introduction/• https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-tables/ • https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-queues/ • https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-files/ • https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/
About Us• Betclic Everest Group, one of the world leaders in online
gaming, has a unique portfolio comprising various complementary international brands: Betclic, Everest Poker/Casino, Bet-at-home, Expekt, Imperial Casino, Monte-Carlo Casino…
• Through our brands, Betclic Everest Group places expertise, technological know-how and security at the heart of our strategy to deliver an on-line gaming offer attuned to the passion of our players. We want our brands to be easy to use for every gamer around the world. We’re building our company to make that happen.
• Active in 100 countries with more than 12 million customers worldwide, the Group is committed to promoting secure and responsible gaming and is a member of several international professional associations including the EGBA (European Gaming and Betting Association) and the ESSA (European Sports Security Association).
We want our Sports betting, Poker, Horse racing and Casino & Games brands to be easy to use for every gamer around the world. Code with us to make that happen.
Look at all the challenges we offer HERE
Check our Employer Page
Follow us on LinkedIn
WE’RE HIRING !