Lonny Bastien Kevin CunnaneSenior Program Manager Senior Software EngineerSQL Server SQL Server

Enable Application Lifecycle Management for SQL Server Database Development

Objectives and TakeawaysObjectives: 1. Learn how to setup and use SQL Server database projects on Team Foundation Server.2. Learn how to use SQL Server database project unit testing as part of TFS continuous

integration.3. Learn how to setup continues integration with database projects and Visual Studio

Online.

Key Takeaways:4. SQL database projects easily integrate with on-premises TFS and Visual Studio Online.5. SQL Database Unit Testing complements continuous integration.

AgendaWhat is SQL Server Tooling in Visual Studio?Common Development PatternsDemo

Project setupSetting up Source Control with GITAdding Database Unit Test IntegrationTFS Integration

Setup Continuous IntegrationVisual Studio Online Integration

Q/A

SQL Server Tooling in Visual StudioProject system for database developmentBuild time validationOnline/offline editingExtensible source code analysis engineDatabase publishing with extensibilitySchema comparisonDatabase Unit Testing

Connected Database Development

SQL

Dev

SSMS

Execute .bak

backup

Connected Database Development

SQL

Dev

SSMS

File Server

T-SQL

Read

Write

Execute

Development with Visual Studio Tooling

Dev

VS

File Server

GIT Share

Sync

Check-in

Publish

TestDeploy

SQL

SQL

Development with Visual Studio ToolingDev

VS

File Server

GIT Share

Sync

Check-in

Build

TestDeploy

SQL

SQL

DBA

SQLPackage

DacPac

Build SharePublish

T-SQL

Build Share DBA

SSMS

Execut

e

Script

Demo Part 1Visual Studio database project integration with GITKey Concepts:

Moving from connected development to project based developmentSource Code Control using GITSimple Database Unit Testing

Demo Part 1 NotesVisual Studio database project integration with GITCreate project from connected database

Add project to GIT source control through Visual Studio

Push your initial checkin

Create a shared git repository on the fileshare

Link local and remote repositories

Team members can clone the shared repository and collaborate

Sources for the steps in these slides:

http://www.gitguys.com/topics/creating-a-shared-repository-users-sharing-the-repository/

http://elegantcode.com/2011/06/18/git-on-windows-creating-a-network-shared-central-repository/

Option 1: Import DatabaseRight-click a database in SQL Server Object Explorer and choose “Create new project”

Choose “Add to source control” and choose GIT as your source code provider

Automatically sets directory up as a local GIT repository

Option 2: Add existing project to source controlOpen an existing SQL Project or

solution

Right-click on the solution, choose “Add solution to Source Control…” and choose GIT as your source code provider

Automatically creates a local GIT repository for your solution

Option 3: Create Local Git Repository firstIn Team Explorer choose “New” and enter a local file path.

Double-click on the new repository

Go to “Changes”, type in a commit comment and choose “Commit”

Create projects / solutions under this directory and they will be detected as changes that you can check in

Checking in locallyCreating a GIT repository doesn’t check in any projects or code you may have created.

Go to the Changes page in Team Explorer to check code into the local repository. This allows you to have a local repository with all your changes.

To share with other team members or back up, create a shared repository that the team can sync their changes to and from.

Create a shared (remote) repositoryRequires Git command line tools to be installed - see http://msdn.microsoft.com/en-us/library/dd286572.aspx

To create a git repository for sharing with git clone/pull/fetch/push, Use the --bare option to git init.

The easiest way to open a command prompt is by right-clicking on your local repository and choosing “Open Command Prompt” pushd \\remoteFileshare\root

git init --bare AdventureworksDb.git popd

This will create an empty Git repository in \\remoteFileshare\root\Adventureworks.git. Note that shared repositories should end in .git by convention.

Publish to remote repositoryThe final step before you can share your projects with your team is to publish to the shared repository you created.

Go to the Unsynced Commits page in Team Explorer and enter the URL of the remote repository to publish to.

Note that the fileshare path has to be added in Unix notation (// and /) rather than windows (\\ and \), for example:

//remoteFileshare/root/AdventureworksDb.git

Demo Part 1 NotesDatabase unit test creationCreate unit tests for stored procedures and functions in a SQL Project

Create blank unit test project and add SQL Unit Test class template

Sources for the steps in these slides:

Database Unit Testing Documentation

Getting started blog post

Creating tests for stored procedures There is built in support for creating unit test templates against stored procedures, functions and table triggers.

Right-click on a file in Solution Explorer that contains one of these objects and choose “Create Unit Tests”.

Then choose to create a C# or VB unit test project, and a new database unit test file that calls your procedure will be created

Configuring SQL TestsThe first time you create a SQL Unit Test you will be asked to specify the database to test against and what project to deploy.

We recommend choosing a different test DB to the one you use during regular F5 deployment of a SQL project, since you want tests to run in a clean environment.

We also recommend always deploying the project that your test runs against. This will be deployed on each test run to the database you choose in this dialog.

Note that in team environments each team member may need to configure a different database connection. We’ll cover this scenario later in the presentation.

Once you have a basic unit test created, you’ll need to add some test conditions and update the SQL to specify the inputs and outputs you want to validate.

The template is useful for defining required parameters, their types, etc. but it’s up to you to define the expected behavior

All test conditions are based on examining the results of SELECT statements, and a single test can examine multiple result sets.

Creating tests for stored procedures

First remove the inconclusive test.

Next, validate the expected behavior by adding one or more conditions. In this case, we want to validate 3 cases: Validate NULL input returns ‘**Invalid**’

Validate 3 input returns ‘Backordered’

Validate 7 input returns ‘**Invalid**’ as it’s outside the expected range

Tests work against ResultSets, so we create 3 queries that output to a result set, one for each input. Note ‘select @rc as rc’ is the statement returning a ResultSet.

We then add 3 scalar value conditions and change the ResultSet number and Expected Value for each condition. These are configured in the Properties tab.

See the example for testing ‘3’ – it’s the 2nd query and so ResultSet is set to ‘2’

Defining test conditions

Select “Test -> Run -> All Tests” to run the tests.

You’ll notice that the first time you run a “TestMethod1” might appear. This is because the template for creating a C#/VB unit test project creates a “UnitTest1.cs” file. Delete this file as it’s not needed for DB unit tests.

To view test results, you must open the Test Explorer window (Test -> Windows -> Test Explorer). You can choose from there what tests you wish to run, see test results and understand what went wrong if a test fails.

Running your tests

Development with Visual Studio ToolingDev

VS

TFS Server

GIT/TFS

Sync

Check-in

TestDeploy

TestDeploy

Build Agent

MSBuild/MSTest

Sync

SQL

SQL

DacPac

Build Share

SQL

Copy

DBA

SQLPackage

Publish

Demo 2:TFS Integration

Key Concepts:TFS and GIT IntegrationCreating Build DefinitionCreating Team ProjectSetup of Continuous IntegrationConfiguration of Database Unit Testing

Demo: TFS SetupInstall Visual Studio Team Foundation Server 2013

Use Basic configurationCreate a team collectionAdd users to collectionCreate Build ConfigurationInstall Visual Studio 2013 Express for

WebInstall latest update for SQL Server

Tooling

Demo: SQL Server SetupCreate logins for Network service for Build Agent(s)

Integrated Authentication is recommended for On-prem databases

SQL Auth required for SQL IaaS and Microsoft Azure SQL databases

DEMO

Demo Part 2 NotesSetup TFS IntegrationCreate new Team Project in TFS

Add TFS as a remote for your Git projects

Setting up Continuous Integration

Configuring database unit tests as part of Continuous Integration

Create new team project (1/2)In Team Explorer choose “Connect to Team Projects”, the “Select Team Projects…” to add a connection to your TFS Server. If there is an existing project connect to it.

Go back to “Connect to Team Projects” and choose the new “Create Team Project…” option and enter your project name and other details.

Be sure to select GIT as the version control system.

Create new team project (2/2)

Add TFS as remote: find team project URLTo sync with a team project that uses Git you need to know the repository URL.

Double click on the team project you created to view the project information. Click “Clone this repository” to view and copy the URL.

Note: If you are creating a brand new project and don’t have a local repository yet, you can just clone the repository to get started.

Add TFS as remote for existing projects (1/3)If you never set up a remote for your existing project setting up TFS is easy.

Go to Unsynced Commits and enter the URL you just copied. Then click “Publish” and you’re done!

Note that after doing this, your “Local” project info in Team Explorer will be merged with the Team project. This will change the project name to match the team project, e.g. in this example “ImportedProject2 (Local)” will become “TechReady”

Add TFS remote for existing projects (2/3)

git remote remove origin git remote add origin

http://mytfsserver:8080/tfs/collectionName/_git/projectName

If you already have a remote set up, you either need to replace this or add the TFS team project as an extra remote via the command prompt. The easiest way to open a command prompt is by right-clicking on your local repository and choosing “Open Command Prompt”.

To move directly to TFS and stop syncing to the fileshare, remove the existing remote and add the TFS URL instead.

Alternatively you can keep the existing fileshare remote and just add the new remote. Visual Studio has some support for multiple remote repositories, but it is normally simpler to just keep 1 shared remote repository for your team.

Add TFS remote for existing projects (3/3)Once you have updated your remote repository, you will need to publish the master branch to TFS.

The easiest way to do this is to go to the Branches page in Team Explorer and choose to publish the branch. Once you’ve done this your project will be available to share with other teammates.

Setting up Continuous Integration 1/4One or more Builds can be set up to support continuous integration. This can be configured using the Builds page in Team Explorer.

Choose “New Build Definition” and give your build a name, for example “Master branch build”.

Setting up Continuous Integration 2/4On the Trigger tab, choose “Continuous Integration” as the trigger type. This ensures a build is triggered for each check-in.

You leave Source Settings at their default values if you just wish to build the master branch.

Setting up Continuous Integration 3/4On the Build Defaults tab, choose “Copy build output to server” to use a default drop location. If you prefer to copy these to a specific network share location you can use a different configuration.

Setting up Continuous Integration 4/4On the Process tab, use the default v12 template (GitTemplate.12.xaml).

Under Build click on the “…” button near “Projects” and choose the solution or project file you wish to build in your local filesystem. This will be automatically mapped to a relative path in TFS (e.g. “\Adventureworks\Adventureworks.sln”)

Under Advanced specify “/p:VisualStudioVersion=12.0” as the value for “MSBuild arguments”. This is required since the default is 11.0 but you will have installed the v12.0 SQL Tooling (Visual Studio 2013 == v12.0)

Disable tests for now as these require additional configuration

Save the build settings and make a check-in!

Running unit tests on a build serverRe-enable unit tests in the build configuration and save this.

To configure unit tests to run on the build server, we need to overcome a few issues:

The TFS server will normally connect to a different SQL Server instance for testing.

The path to the project we deploy before testing is different on a build server

For VS2013 you also need to update the VisualStudioVersion environment variable in code

We need multiple configurations, since you will still want to run locally using your original settings

Creating an override file for your userLet’s create an override file for yourself so that you can keep running tests locally. Copy the app.config file and paste it into the unit test project.

Rename this file as “username.sqlunittest.config”, where username is your Windows Authentication alias. If you don’t have Windows Auth enabled, rename it to “machinename.sqlunittest.config” instead.

In the Properties tab ensure the file is copied to the output directory (Copy if newer / Copy always) so unit test runs can find it

Finally, you must remove everything except the SqlUnitTesting_VS2013 section from the file. You also must be sure not to have whitespace at the start – the end result should look like the screenshot below

Specify that config overrides are allowedNow open up the original app.config file and enter ‘AllowConfigurationOverride=“true”’ as an attribute on the SqlUnitTesting_VS2013 section

Save the file – now your settings from your override file can be picked up instead of the settings in the app.config file. This is important as that is where the TFS Build Server settings will be saved.

For every team member you’ll need a separate username.sqlunittest.config file set up. This enables each user to use a different server/database for testing, so it can be useful even if you’re not setting up unit testing as part of the build process

Changing the project pathDatabase unit tests deploy the contents of a specific SQL database project. The tests look up the project relative to the unit test DLL that’s built, then call the MSBuild Deploy target for that project to actually deploy. One your local machine the default DLL location is <Project path>\bin\Debug, whereas on a build server it’s in a separate directory structure completely.

For the project deployment to run successfully on the server, we therefore need to change the project path in the app.config file. As long as you use the defaults shown here, you need to change "..\..\..\Adventureworks\Adventureworks.sqlproj" to "..\src\Adventureworks\Adventureworks.sqlproj".

Setting VisualStudioVersion in test codeAs we just mentioned, the test code calls MSBuild on the SQL database project to deploy. One side effect is that the same VisualStudioVersion environment variable we set in the build definition must also be set in the test code. If we don’t do this MSBuild will try to use the v11.0 tools and fail to deploy since they aren’t installed on disk

Note that this applies only if you installed VS2013 on your build server instead of VS2012.

Add the following script to the SqlDatabaseSetup.cs file:

static SqlDatabaseSetup() { Environment.SetEnvironmentVariable("VisualStudioVersion", "12.0"); }

Specifying the Database for TFS testingThe last thing to change is the database you wish to test against when running tests on the TFS Build Server. How you configure this will depend on whether you intend to use Windows Authentication (preferred) or SQL Authentication to connect to that server.

For on-prem TFS instances and SQL Servers you can use Windows Authentication. In this case, you need to add a login for your build agent (DOMAIN\TFS_BUILD_SERVER_NAME$) on that SQL Server instance.

For Azure VM or Microsoft Azure SQL Database instances you must use SQL Authentication. In that case you will have to define the username and password for this connection in the app.config file. Note that this is strongly discouraged for any production or other sensitive servers – this is appropriate only for test servers.

To change the database we recommend updating SQL Server Test Configuration – choose New Connection there and input your server information. Right-click on the test project and choose “SQL Server Test Configuration…” to change the connection string.

If you are using SQL Authentication you’ll need one more step after doing this. By default we don’t store password information for connections, so after updating the connection string you need to update the 2 connections strings in app.config:

Remove “Persist Security Info=True;” if this is present. This assumes an encrypted password would be present in the registry, which isn’t true on the build server

Add “Password=xyzMyPwd;” so that the connection string can successfully be used to connect to the DB

Check in your changes and run testsOnce you’ve made these changes you’ll need to Commit and Sync them to push the updated configuration to TFS. This will automatically kick off a build, and you should see your tests running and passing!

Go to the Builds tab in Team Explorer to see recent builds and their status

If there are any problems, expand the “test run completed” section and click the test information link to see what went wrong. Common problems are that the path to the database project was incorrect or you forgot to add the VisualStudioVersion code to the unit test project.

If all goes well you’ll see that the build succeeded and all tests passed.

Demo Learnings:VS 2013 Requires setting /p:VisualStudioVersion=12.0 in build definition

Database Unit TestingRequires custom app.config to support different dev and TFS configurationsRequires setting VisualStudioVersion in database test code for VS2013 installation

DemoIntegration with Visual Studio OnlineKey Concepts:

It works the same as TFS On-PremSQL Server access requires SQL authenticationMultiple DB platforms supported

Microsoft Azure SQL Database (PaaS)SQL on Azure VM (IaaS)

DEMO

Demo Learnings:VS 2013 By default VS2012 and VS2013 bits are installed on Visual Studio Online build servers, so you can just use the 2012 bits and avoid setting /p:VisualStudioVersion=12.0 in build definition

Database Unit TestingStill requires custom app.config to support different dev and TFS configurationsIt is much easier to use an IaaS (Azure VM with SQL Server) or PaaS (Microsoft Azure SQL Database) instance for testing than your on-prem SQL Server. This is because you don’t need to add a special firewall exception for the Visual Studio Online build server in Azure. Instead just turn on “Windows Azure Services” in the allowed services section of the server configuration in the Azure management portal.

Session Objectives and TakeawaysSession Objectives: 1. Learn how to setup and use SQL Server database projects on Team Foundation Server.2. Learn how to use SQL Server database project unit testing as part of TFS continuous

integration.3. Learn how to setup continues integration with database projects and Visual Studio

Online.

Key Takeaways:4. SQL database projects easily integrate with on-premises TFS and Visual Studio Online.5. SQL Database Unit Testing complements continuous integration.

Related ContentContactLonny Bastien – Lonnyb@Microsoft.com

Kevin Cunnane – KCunnane@Microsoft.com

Talks & Downloads:White Paper: SQL Server Database Projects and Team Foundation BuildDownload : http://msdn.microsoft.com/en-us/data/hh297027 Techready18: Integrated tooling for SQL Server 2014 and Windows Azure SQL database in Visual Studio 2013https://techreadytv.com/tr18/session?sCode=DP328TechEd 2012: Microsoft SQL Server Data Tools: Database Development from Zero to Sixtyhttp://channel9.msdn.com/events/teched/Europe/2012/dbi311

Team Blog and Forumshttp://blogs.msdn.com/b/ssdt/http://social.msdn.microsoft.com/Forums/sqlserver/en-US/home?forum=ssdt

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.