Ignite North America 2017Orlando, 25-29 Sep

Major Releases and Announcementsin Azure Data & Infrastructure

Keeping the world in context

Soul Nebula6500 light years away100 light years across

https://techcommunity.microsoft.com/t5/Microsoft-Ignite-Content-2017/bd-p/MicrosoftIgniteContent2017

https://news.microsoft.com/hitrefresh/

https://buildazure.com/2017/09/25/microsoft-azure-gets-a-new-logo-and-a-manifesto/

• Interactive and clickable diagrams showing Azure… Compute, Networking, Storage, Web + Mobile, Containers, Databases, Data + Analytics, AI + Cognitive Services, Internet of Things, Enterprise Integration, Security + Identity, Developer Tools, Monitoring + Management

• Interactive and clickable diagrams showing Azure…

http://azureinteractives.azurewebsites.net/Azure101Cards/default.html

http://azureinteractives.azurewebsites.net/CloudDesignPatterns/default.html

• Azure Storage: EastUS, WestUS, WestCentralUS, WestUS2, AustraliaEast, AustraliaSouthEast

• Azure SQL Database: EastUS, WestCentralUS, WestUS2

https://azure.microsoft.com/en-us/updates/public-preview-virtual-network-service-endpoints-for-azure-storage-and-azure-sql/

• https://azure.microsoft.com/en-us/pricing/reserved-vm-instances/

• https://azure.microsoft.com/en-us/pricing/hybrid-benefit/

• fault-isolated locations within an Azure region, providing redundant power, cooling, and networking.

applications with higher availability and fault tolerance to DC failures.

• preview in two regions, East US 2 and West Europe + others by year end

https://azure.microsoft.com/en-us/blog/introducing-azure-availability-zones-for-resiliency-and-high-availability/

• DDoS Protection in preview - no cost for the service during preview.

• Need to register for preview

• available now in East U.S., West U.S. and West Central U.S

https://azure.microsoft.com/en-us/blog/azure-ddos-protection-service-preview/

https://azure.microsoft.com/en-us/blog/announcing-larger-higher-scale-storage-accounts/

• supported in a variety of scenarios – VM to a VM, VM to an external endpoint, and VM to an on-premise endpoint

https://docs.microsoft.com/en-us/azure/network-watcher/network-watcher-connectivity-powershell

https://azure.microsoft.com/en-us/blog/azure-network-watcher-introduces-connectivity-check-preview/

• Peering virtual networks in same region is GA already – HOWEVER - Peering virtual networks in different regions currently in preview (US West Central, Canada Central, and US West 2. )

• virtual network peering enables seemless connection across virtual networks.

• Once peered virtual networks appear as one. VM in peered networks can communicate with each other directly.

• traffic between VM in the peered networks is routed through Microsoft backbone infrastructure. Traffic going through virtual network peerings is completely private.

https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-peering-overview

• keeps Azure File share in-sync with on-premises Window Servers shares

• ability to tier files between on-premises file server and Azure Files

• multi-master sync solution - makes it easy to solve global access by replicating data between Azure File shares and servers anywhere in the world

• helps leverage Azure to get control over on-premises data - cloud tiering moves old and infrequently accessed files to Azure

https://azure.microsoft.com/en-us/blog/announcing-the-public-preview-for-azure-file-sync/

• Integration with VM enables update management, inventory, and change tracking for Windows and Linux computers directly from VM blade in portal

• Can see compliance status for Windows + Linux, can create scheduled deployments to orchestrate update installation within maintenance window

• ability to exclude specific updates is available, with detailed troubleshooting logs to identify issues during deployment.

https://azure.microsoft.com/en-us/blog/automation-and-configuration-of-your-azure-on-premises-and-hybrid-cloud-resources-at-scale/

https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview

• get started on-boarding monitoring solutions (1) Azure alerts, (2) Log Analytics (OMS), and (3) Application Insights – and (4) Azure Activity Log.

• overview page gives starting point for navigation, and shows notable issues from 3x different core services

https://azure.microsoft.com/en-us/blog/azure-monitor-new-services-and-capabilities-for-metrics/

https://azure.microsoft.com/en-us/blog/get-started-with-monitoring-in-azure/

• Cloudyn (Cloud Cost Management solution) is free for Azure customers to manage Azure spend

• SaaS - sign up 30-day trial, then simply continue the product for free.

• track cloud usage + expenditures for Azure and other cloud providers (AWS and Google)

• Track usage + cost trends, Detect usage inefficiencies, Create alerts for unusual spending or overspending and Forecast future spending

• Optimizer reports = improve efficiency, optimize usage, and identify ways to save money on cloud resources.

• Helpful cost-effective sizing recommendations intended to help reduce idle or expensive VMs.

• cost projection help forecast spending based on usage over time

https://azure.microsoft.com/en-us/blog/use-azure-cost-management-for-free/

https://azure.microsoft.com/en-us/services/cost-management/

• [Hive on LLAP] in Apache Hive 2.1, brings the interactivity to your complex data warehouse style queries on large datasets stored on commodity cloud storage.

• enables end-users to consume data from rich BI tools, such as PowerBI, Tableau, Excel, Hive View 2.0, Beeline, Hive CLI, and Visual Studio

• attach an Interactive Query cluster to existing HDI metastore and data storage, and start querying the data right away.

• Interactive Query Dynamic Text Cache, converts CSV or JSON data into optimized in-memory format on-the-fly, so queries determine what data is cached.

• create an interactive query cluster joined to an Active Directory domain

https://azure.microsoft.com/en-au/blog/general-availability-of-hdinsight-interactive-query-blazing-fast-data-warehouse-style-queries-on-hyper-scale-data-2/

• database workloads like SQL Server or Oracle often require high memory, storage, and I/O bandwidth, but not a high core count

• latest versions of popular VM sizes (DS, ES, GS, and MS), constrain the vCPU count to ½ or ¼ of original VM size, while maintaining memory, storage, IOPS.

• licensing charged for SQL or Oracle will be constrained to the new vCPU count

• VM compute cost (including OS licensing) remains the same as original size *

• results in 50% - 75% decrease in SQL licencing costs for database workloads

https://azure.microsoft.com/en-us/blog/announcing-new-azure-vm-sizes-for-more-cost-effective-database-workloads/

• AML Workbench a cross-platform client for AI-powered data wrangling and experiment management

• AML Experimentation service help data scientists increase their rate of experimentation with big data

• AML Model Management service host, version, manage and monitor machine learning models

https://azure.microsoft.com/en-us/blog/tools-for-the-ai-driven-digital-transformation/

https://azure.microsoft.com/en-us/blog/diving-deep-into-what-s-new-with-azure-machine-learning/

• lift SQL SSIS packages into Azure using new managed SSIS hosting capabilities in Data Factory.

• Ability to execute, manage, monitor, deploy SSIS packages

• use SQL Server Data Tools (SSDT) or SQL Server Management Studio (SSMS) to deploy SQL Server Integration Services (SSIS) packages to Azure “Integration runtime” (IR) hosted in ADF service

https://blogs.technet.microsoft.com/dataplatforminsider/2017/09/25/new-managed-sql-server-integration-service-hosting-capability-in-data-factory/

https://docs.microsoft.com/en-us/azure/data-factory/tutorial-deploy-ssis-packages-azure

• SQL Server can run on Windows, Linux and Docker containers

• AI built-in = R and Python analytics plus native scoring in T-SQL

• R is also built in into Azure SQL Database (preview only)

• Supports graph tables, supports Adaptive Query Processing

• Resumeable onine reindexing

• Machine Learning Server for Hadoop, formerly R Server, brings R and Python scalable analytics to Hadoop / Spark environments - now available to all SQL Server EE customers as SA benefit

• SSRS installer is now separate from SQL Server – and is now single instance only

• Clusterless availability groups is now available

• Together with Red Hat provide up to 30% off SQL Server 2017 through an annual subscription

https://blogs.technet.microsoft.com/dataplatforminsider/2017/09/25/microsoft-for-the-modern-data-estate/

https://docs.microsoft.com/en-us/sql/sql-server/what-s-new-in-sql-server-2017

• moving massive data to cloud is slow – even fast networks can take months to move terabytes petabytes

• Data Box is secure, human-portable, easy-to-get Microsoft appliance that enables offline transfer of large data sets to Azure cloud

• 45 pound, ruggedized, tamper-resistant, human-manageable appliance

• Plugs into your network, can store 100 TB, uses NAS protocols

• Order it + plug it in + fill it up + return it

• supports 256 bit AES encryption

https://azure.microsoft.com/en-us/blog/announcing-the-preview-for-the-azure-data-box-achievements-will-be-unlocked/

Text Analytics GA + in 4 new regions - South Central US, East US, West Europe, Southeast Asia

Azure SQL Database supports in database Machine Learning (R – in preview)

Azure SQL Database elastic pools supports 4TB

Rename R Server to Machine Learning Services (all flavors)

Python support Machine Learning Services / Server (all flavors)

Private Preview of Read Replicas for Azure Database for MySQL (the ability to setup Read Replicas for Azure Database for MySQL servers in Azure, or MySQL server’s on-prem or in other cloud environments)

Azure Portal supports in portal querying using PowerShell Commands

Additional Azure services moved into the Azure Free list of services

Intelligent insights for Azure SQL Database: Performance degradation diagnostics auto-detection

Power BI Embedded has gone GA

Azure Analysis Services adds firewall support

HDInsight is now fully intenerated with Azure Log Analytics (OMS) for easy monitoring

Azure Stream Analytics has ability to output to Azure Functions, with built-in anomaly detection

New PCI DSS reference architecture