26 February 2016

T-SQL: Converting rows to columns

A common task of a database developer is to convert rows to columns from a query result set.

T-SQL has the PIVOT that facilitates the process.

A simple example:

DECLARE @myValues TABLE (Quantity INT, Category VARCHAR(100))

INSERT INTO @myValues (Quantity, Category)
VALUES 
 (10, 'Fruits'),
 (200, 'Vegetables'),
 (40, 'Meats')


SELECT *
FROM @myValues

SELECT [Fruits], [Vegetables], [Meats]
FROM @myValues
PIVOT
(
  MAX(Quantity)
  FOR Category IN ([Fruits], [Vegetables], [Meats])
) Piv

17 November 2015

How to fix visual studio 2015 error "The package did not load correctly"

When a visual studio extension is installed, a error can occur that gives the following error:

The 'zebre' package did not load correctly.

The problem may have been caused by a configuration change or by the installation of another extension. You can get more information by examining the file 'C:\Users\[user name]\AppData\Roaming\Microsoft\VisualStudio\14.0\ActivityLog.xml'.

Restarting Visual Studio could help resolve this issue.
Continue to show this error message?


The error can cause intellisense not working or other erratic issues.

A possible fix is to uninstall the extension that is causing the error.

The issued can be fixed executing the following command as an administrator:

devenv.exe /setup

If the previous solution does not work, reset the visual studio preferences by issuing the command (Execute as an administrator):

devenv.exe /resetuserdata

02 October 2015

Dapper: Powerful micro object mapper for .Net

Introduction

Dapper.Net or Dapper is a simple and powerful micro object mapper for .Net, that eases the development of a Data Access Layer (DAL).
Dapper is in production used by Stack Overflow, helpdesk and other companies.
Performance is the major focus in Dapper. It is very fast and the execution time is near equal to a hand coded DAL using the SqlDataReader.
Dapper uses the POCO (Plain Old CLR Object) approach.

Setup

Dapper has several installation options:
1) Download from github
Dapper can be downloaded from the github
After the download the cs class files must be added to your project or class library.

2) Nuget package
Dapper is also available as a Nuget package that can be added to you Visual Studio Project references.

Helpers

Dapper extend the IDbConnection interface with three Helpers:
1) Execute a query and map the results to any strongly type
public static IEnumerable Query(this IDbConnection cnn, string sql, object param = null, SqlTransaction transaction = null, bool buffered = true)

2) Execute a query and map the results to a Dynamic type
public static IEnumerable Query (this IDbConnection cnn, string sql, object param = null, SqlTransaction transaction = null, bool buffered = true)

3) Execute a command that returns no results
public static int Execute(this IDbConnection cnn, string sql, object param = null, SqlTransaction transaction = null)

A simple example is to retrieve user information from a database.
First define the POCO class that represents the user. The names must be equal to the database table names.
public class User
{
     public int UserID { get; set; } 
     public string Name { get; set; }
}
To select a list of all the users:
using (SqlConnection connection = new SqlConnection(myConnectionString))
{
     connection.Open();
     List users = connection.Query("SELECT * FROM [User]").ToList();
     connection.Close();
}
Not that "myConnectionString" must be a connection string accepted by the SqlConnection.
Dapper also works with LINQ to return a list (ToList) or single objects (Single / SingleOrDefault)

Parameterized queries

The parameters of a query are passed as anonymous classes.
The anonymous classes must have the query parameter name and it's value.

Example:
To select a list of all the users that the name ends with "Doe":
using (SqlConnection connection = new SqlConnection(myConnectionString))
{
     connection.Open();
     List users = connection.Query("SELECT * FROM [User] WHERE Name LIKE @name", new {name = "%Doe'"}).ToList();
     connection.Close();
}
In this example the name will be matched with the parameter @name of the query.

Stored Procedures

Dapper has extensive support for stored procedures.

Example:
using (SqlConnection connection = new SqlConnection(myConnectionString))
{
     connection.Open();
     List users = connection.Query("UserSearch", 
                                               new {name = "%Doe"}, 
                                               commandType: CommandType.StoredProcedure).ToList();
     connection.Close();
}
In this example, the stored procedure has a parameter named "name" that will receive the value "%Doe".
The parameters can also be passed using the DynamicParameters class, but the anonymous class approach works fine and is more simple.

Table Value Parameters

Dapper supports Stored Procedures Table Value Parameters (TVP):
System.Data.DataTable countryDataTable = new System.Data.DataTable();
countryDataTable.Columns.Add("Id", typeof(long));

countryDataTable.Rows.Add(1);
countryDataTable.Rows.Add(2);

using (SqlConnection connection = new SqlConnection(myConnectionString))
{
     bool isValid = connection.Query(@"IsValid", 
                                      new {
                                             UserID = 674,
                                             countries = countryDataTable.AsTableValuedParameter("[dbo].[IdList]")
                                           },
                                           commandType: CommandType.StoredProcedure
                      ).Single();
}

Where IdList is the sql server table value type with the column Id of type INT
CREATE TYPE [dbo].[IdList] AS TABLE(
 [Id] [int] NULL
)

Multiple Result Sets

Dapper has support for multiple result sets in a single query.
In this case use the QueryMultiple extension method:
var sql = @" select * from Users where UserID = @id ";
sql += @" select * from Roles where UserID = @id"

using (var multi = connection.QueryMultiple(sql, new {id = myId }))
{
     var users = multi.Read<User>().Single();
     var roles = multi.Read<Role>().ToList();
} 

CRUD - Create, Read, Update and Delete


Dapper has no native support for Create, Read, Update e Delete (CRUD)
These operations must be manually developed in queries.

There are however third party extensions that easy the implementation of these operations.
One of those is SqlMapperExtensions the can be found on the dapper github on the Dapper.Contrib Folder.
The file must be added to your project or class library.
This extension uses the POCO (Plain Old CLR Object) approach, so you must create a class that has it's properties with the same name and corresponding data type with the database table.
Take some time to read the extension code and verify that it implements the code accordingly to your needs.

The SqlMapperExtension implements the following extension methods:

The entity identifier must have the attribute [Key] to identify it.
Example:
public class User
{
     [Key]
     public int UserID { get; set; } 
     public string Name { get; set; }
}

1) Get
Get returns a single entity by it's Id.
using (SqlConnection connection = new SqlConnection(myConnectionString))
{
     connection.Open();
     User user = connection.Get(200);
     connection.Close();
}

2) Insert
Inserts the entity and returns it's Id.

using (SqlConnection connection = new SqlConnection(myConnectionString))
{
     connection.Open();
     User user = new User();
     user.Name = "John Doe";

     connection.Insert(user);
     connection.Close();
}
3) Update
Updates the entity.

4) Delete
Delete the entity by it's Id.

5) DeleteAll
Deletes all the entities in the table with the same name as the POCO class.
The best documentation for this extension is at the Test cases

12 May 2015

Time testing: How to prevent time syncing

A common task when performing tests on data sets that vary over time is to advance the windows time.
The server or the workstation can however be part of a domain and the time is synced with it.
In this situation after the date/time is changed for the tests, the sync process will update them to the Domain Controllers value.
This will reset the date/time and the test environment is lost.

A question will then arise:
How to prevent time syncing with Domain Controllers?

The best solution that works in my test environment, is to change the registry key Type (REG_SZ value) located at
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters

The Type can have the following values:
Value Description
NoSync No Sync with Domain Controllers
Nt5DS Sync with Domain Controllers

Note that the Nt5DS value is my configuration. Keep your Type value in the registry to restore to it.

The value can be updated with the registry editor (regedit) or by the command line, executing the following command:

reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters /v Type /d NoSync /f

Note that the value NoSync forces the time not being synced with the Domain Controllers. Change the value to you configuration to restore the time sync.

This solution is great, but be aware that if the server/workstation date/time gets out of sync with the Domain Controllers, authentication will fail and you can loose access to them.
While testing, keep the server/workstation logged-in so that you can keep the date/time in sync with the Domain Controllers.

To automated the process of setting the test ambient you can create a cmd file with:

reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters /v Type /d NoSync /f
net start W32Time
sc config "W32Time" start= disabled
net stop W32Time
date 01-04-2090

In this example the date is set as 01-04-2090

To return to the previous environment:
reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters /v Type /d Nt5DS /f
sc config "W32Time" start= auto
net start W32Time

You may also need to set the date, if you computer is not syncing the date with and time server.

03 March 2015

Fix: Property PopulationStatus is not available for FullTextCatalog

After restoring a database with full text search active, on Sql Server 2014, on a staging server I got the following error when opening the Full Text Catalog:

Property PopulationStatus is not available for FullTextCatalog '[SearchDB]'. This property may not exist for this object, or may not be retrievable due to insufficient access rights. (Microsoft.SqlServer.Smo)

The error is not self explanatory, but after some investigation I descovered that Full Text Search Service isn't installed on the staging server.

The easy way to validate if  Full Text Search is installed on a Sql Server is to execute the following T-SQL statement on Management studio:

SELECT FULLTEXTSERVICEPROPERTY ('IsFulltextInstalled')

28 November 2014

Management Studio: The type initializer for ‘PropertyMetadataProvider’ threw an exception

Sql Server Management Studio, gave me the following error after I rebooted my computer:
The type initializer for ‘PropertyMetadataProvider’ threw an exception



I installed the windows updates and started relating the problem to the updates.

After some research I found a interesting artice in Sql Blog about the problem that states that the DLL versions of the cumulative update 4 for SQL Server 2014 had bad DLL versions.

The temporary workaround is:
1) Close all instances of Management studio and visual studio
2) Open a administrator command prompt
3) Execute the following commands :

         (i) cd %WINDIR%\assembly\GAC_MSIL\Microsoft.SqlServer.Smo
         (ii) rmdir /Q /S 12.0.0.0__89845dcd8080cc91

4. Run the SQL Server 2014 setup
5. Select the Maintenance option
7. Press the Repair link
8. Pass all the per-requirement steps and click Repair button
9. Once the repair is finished launch Management studio and the error should be fixed

References:
Sql Blog

08 July 2014

SQL Server Table Value Constructor

The latest versions of Sql Server (SQL Server 2008 or above) implement the Table Value Constructor.

Definition

The Table Value Constructor defines a set of row value expressions to be added to a table.
It allows us to create tables of values and/or expressions.
The Table Value Constructor allows us to simplify the T-SQL syntax.

Syntax

The Table Value Constructor is defined as:
VALUES (<row value expression>),(<row value expression>), (<row value expression>), ...
The Table Value Constructor must start with the VALUE keyword and that it can have one or more row value expression.
The <row value expression> can have one of the following values:
1) NULL
2) DEFAULT. If there isn't a default defined for the column NULL is inserted.
3) A constant, variable or an expression. The expression cannot have the EXECUTE statement.
The expression only allows single scalar values. Sub-queries that return multiple columns are not supported.
An example of an invalid sub-query is:
SELECT ID, Name FROM [User]
Note that the values list can have a maximum of 1000 rows. The sql server error 10738 is thrown if the number of rows is greater than the 1000 rows limit.
To insert more row use the classic approaches like a multiple INSERT statements or a BULK INSERT.

Examples

1) Insert multiple rows
CREATE TABLE Product
(
Id INT IDENTITY(1,1) NOT NULL,
Name VARCHAR(1000)
)
INSERT INTO Product
VALUES ('John Doe'), ('Tim Burton'), ('Tom Thomas')

This example can be used to initialize table values.

2) SELECT using a derived table
SELECT x,y
FROM (VALUES (1, 'row 1'), (2, 'row 2'), (3, 'row 3')) AS myTable(x,y)

This example uses the Table Value Constructor as a derived table, to create a table named myTable with the columns x and y.

3) SELECT the maximum value between two variables
DECLARE @val1 MONEY = 20
DECLARE @val2 MONEY = 5

SELECT MAX(Val)
FROM (VALUES (@val1), (@val2)) AS MyTable(Val)

The query returns the value 20. This can be very useful to easily select the maximum value between two or more variables, since sql server doesn't have a built-in function to do this operation.

Table Value Constructor can also be used with the MERGE statement.

01 July 2014

SSAS: OLAP PivotTable Extensions

Excel Pivot tables are a reporting tool that makes it easy to extract information from data sets without the use of formulas.
Excel Pivot tables are great for browsing Analysis Services cubes, since they allow to easily move, pivot and analyze data using drag and drop to see the same data in a number of different ways
OLAP PivotTable Extensions add advanced features to the pivot tables.
OLAP PivotTable Extensions are an Excel add-in which extends the functionality of PivotTables on Analysis Services cubes.
It works with Excel 2007, Excel 2010, and Excel 2013.
The Excel API has certain PivotTable functionality which is not exposed in the user interface. OLAP PivotTable Extensions provides an interface for some of this functionality and it also adds some new features like searching cubes, configuring default settings, and filtering to a list in your clipboard.



For more information and to install the OLAP PivotTable Extensions go to the codeplex website:

http://olappivottableextend.codeplex.com

30 June 2014

Turn off Visual Studio 2013 Preview of files

Visual studio 2013 previews the file content when it is selected.
The feature can be very annoying and luckily it can be disabled.

To disable the feature, in the visual studio menu, select Tools -> Options -> Environment -> Tabs
Uncheck all the Preview tab checkboxes.

06 June 2014

Fix SSDT error" Could not load type Microsoft.SqlServer.TransactSql.ScriptDom.OnOffStatisticsOption"

The SQL Server Data Tools for SQL Server 2014 (SSDT) are now available for Visual Studio 2012 and 2013.
The SQL Server Data Tools - Business Intelligence for Visual Studio 2013 (SSDT-BI) are also available.
I installed both and after a while the schema compare started failing with the error:

Error 6 Could not load type 'Microsoft.SqlServer.TransactSql.ScriptDom.OnOffStatisticsOption' from assembly 'Microsoft.SqlServer.TransactSql.ScriptDom, Version=12.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91'

The solution that worked for me was to uninstall some components of SSDT and then repair the installation.
The detailed steps are:
1) Open Control Panel and then Programs and Features
2) Uninstall the following items:
    Microsoft SQL Server 2014 Management Objects version 12.0.2299.1
    Microsoft SQL Server 2014 Management Objects (x64) version 12.0.2299.1
    Microsoft SQL Server 2014 Transact-SQL ScriptDom version 12.0.2299.1
3) Select Microsoft SQL Server Data Tools 2013, right click to select Change

press the repair button


Note that this solution fixes the SSDT but may have side effect on the SSDT-BI.

02 June 2014

Undocumented sp_MSforeachdb: Iterate through each database

The stored procedure sp_MSforeachdb is used to iterate through each database that exists in SQL Server, including the system databases.

The stored procedure is undocumented and can be removed at any time, so use it carefully.

It is used to run a command over a set of databases in a server.
The stored procedure receives a parameter with the command to execute. The ? is used as a placeholder to identify the current database name listed by the stored procedure.

Some common scenarios where it can be used are:

1. Print all the database names, excluding the master,tempdb, model and msdb:
EXEC sp_msforeachdb 
"IF '?' NOT IN ('master','tempdb', 'model', 'msdb')   
BEGIN
        PRINT '?'
END"


2. Show the size of all the database
EXEC sp_MSforeachdb 'USE ?; EXEC sp_spaceused'


3. Check the integrity of all objects in the database
sp_MSforeachdb 'DBCC CHECKDB(?)' 


4. Retrieve database physical files information excluding the master,tempdb, model and msdb:
EXEC sp_msforeachdb 
'IF ''?'' NOT IN (''master'',''tempdb'', ''model'', ''msdb'')   
BEGIN
        SELECT name,physical_name,state,size FROM [?].sys.database_files
END'


5. Retrieve database physical files information excluding the master,tempdb, model and msdb to a table:

DECLARE @DbSize TABLE
    (
      mame NVARCHAR(50),
      physical_name NVARCHAR(500),
      size INT,
      growth INT,
   is_percent_growth BIT,
   type_desc NVARCHAR(10)
    )

INSERT  INTO @DbSize
EXEC sp_MSforeachdb 'IF ''?''  NOT IN (''master'', ''tempDB'',''model'',''msdb'')
BEGIN
       SELECT name,physical_name,size, growth, is_percent_growth, type_desc
       FROM ?.sys.database_files
END'

SELECT * FROM @DbSize

27 May 2014

Entity Frameword 6 Error: Unable to update the EntitySet 'X' because it has a DefiningQuery and no element exists in the element to support the current operation.

I was inserting an object in entity framework 6 using the usual code:
context.X.Add(x); context.SaveChanges();
The entity framework returned the error:
Unable to update the EntitySet 'X'because it has a DefiningQuery and no element exists in the element to support the current operation.

The solution to fix this error was to create a primary key in the table and update the entity model.

19 May 2014

Poor Mans T-SQL formatter

A database developer must have a set of SQL coding Standards to avoid each statement being written differently.

Steven Bates has witten the SQL Server 2005 coding Standards in a series of blog posts at the MSDN blogs.
The rules are implemented by a SQL Server Management Studio add-in that is named Poor Mans T-SQL formatter.
The Poor Mans T-SQL formatter is a open-source T-SQL formatter that is available in several distibutions: Stand alone application, Notepad++ add-in, online,...
For more information visit the Poor Mans T-SQL formatter site, by pressing here

As for now there is no support to Management Studio 2014.
To make it work with Management Studio 2014:

1) Run the setup for the SQL Server Management Studio 2012 add-in available at the site.

2) Create the folder
%SystemDrive%\ProgramData\Microsoft\SQL Server Management Studio\12.0\Addins\
if it doesn't already exists.

3) Copy the file from:
%SystemDrive%\ProgramData\Microsoft\SQL Server Management Studio\11.0\Addins\PoorMansTSqlFormatterSSMSAddIn.AddIn
To
%SystemDrive%\ProgramData\Microsoft\SQL Server Management Studio\12.0\Addins\PoorMansTSqlFormatterSSMSAddIn.AddIn

And now the add-in should be available at the Tools menu:
Note: You may need to enable show Hidden Items, in windows explorer View Menu, for the folder ProgramData to be shown.

06 February 2014

Visual Studio 2013 connection to TFS very slow

Visual Studio 2013 connects to Team Foundation Server (TFS) with the TFS client.
In some situations, out of our control, the connection to TFS is verty slow. The source control operations take ages to complete.

The problem appears to be network related, since at the same time one computer is very slow executing a source control operation and the others are working at a normal pace.

The TFS client usually connects to TFS using HTTP. The .Net Framework is used to make the connection.
Visual studio has a configuration of the Hypertext Transfer Protocol (HTTP) proxy server to perform the connection.
To avoid a slow speed the proxy must be disabled.

The configuration file for visual studio 2013 is located in
C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv.exe.config

Open the file and add / Update the following entry:


    
    
        
        
    


03 February 2014

Faster Android emulator on Intel Architecture

The Eclipse is the main stable development environment for Android.
The performance of the Android emulator is really bad, but Intel has a solution for computers on a Intel Architecture.

The recent computers with an Intel processor have the Intel Virtualization Technology enabled, that allows the acceleration of the the Android Emulator.
In win Windows and Apple Mac OS there is the Intel Hardware Accelerated Execution Manager (Intel HAXM) or KVM for Linux.

In this document I am focusing on the configuration of the Intel HAXM in a windows environment.

The prerequisites are:
1) Android SDK must be installed
2) An Intel processor with support for Intel VT-x, EM64T and Execute Disable(XD) Bit functionality enabled from the BIOS


To install the Intel HAXM:
1) Open the Android SDK manager where you installed it
2) Go to Extras in the tree-view and select the Intel x86 Emulator Accelerator(HAXM) and press the Install button

3) After the installation the SDK manager displays that the extra is installed, but it only downloaded a executable named IntelHaxm.exe that must be executed manually.
The file is tipically where you insalled the Android SDK in sdk\extras\intel\Hardware_Accelerated_Execution_Manager. If you can't find it search in windows explorer for the file.
4) Execute the IntelHaxm.exe and set the amount of memory that you want to reserve for the emulator and install it.
If your computer doesn't meet the requirements and error will occur, so validate that you BIOS settings are enabled for the Virtualization Technology.


The next step is to create an Android Virtual Device (AVD) with hardware accelerated emulation:
1) Launch Eclipse
2) Go to the AVD Manager and create a new device
3) Select Intel Atom (x86) as the CPU/ABI. This option is only available for Intel x86 system images, so you must install it.
You can install the Intel x86 Atom system images using the Android SDK manager.

4) Switch on the GPU emulation



The memory options of the AVD are also important. RAM values greater than 768M usually may fail on windows depending on the system load. Progressively lower the RAM values until the AVD is stable.

On windows 8.1 there is a side effect: The error dump CRITICAL_STRUCTURE_CORRUPTION
Intel has released a hotfix for windows 8.1 that is not included on the Android SDK Manager download.

Download it here

18 July 2013

WCF and the Try-Catch-Abort Pattern

Proxy Classes are used to talk to Windows Communication Foundation (WCF) Services.
A communication channel is open to the WCF Service that must be closed after the calls to the service.
The proper way to close the communication channel is very important.
The WCF proxy is usually generated using Visual Studio or the svcutil tool.
The generated proxy inherits from a base class System.ServiceModel.ClientBase that implements the IDisposable interface.

The service can be called in a using block that automatically calls the Dispose() method.

using (MyService client = new MyService())
{
...
} // Dispose is called here
Dispose() calls the proxy close() method, that sends a message from the client to the service indicating that the connection session is no longer needed.
A problem can arise with this approach if there is an exception when calling the close() method. This is why the using approach is not recommended when calling WCF methods.

If the communication channel is in a faulted state Abort() should be called and not close();
The recommended approach is the Try-Catch-Abort Pattern.
This is a simple pattern where a try...catch block is used to call the service and in the catch of an exception the connection is aborted or closed.

The recomendation by Microsoft in MSDN is:
 
try
{
    ...
    client.Close();
}
catch (CommunicationException e)
{
    ...
    client.Abort();
}
catch (TimeoutException e)
{
    ...
    client.Abort();
}
catch (Exception e)
{
    ...
    client.Abort();
    throw;
}

A generic class for creating and initializing WCF proxy clients can be created to implement the pattern and replace the using block:
 
public class WCFProxy
    {
        public static void Using<t>(Action<t> action)
        {
            ChannelFactory<t> factory = new ChannelFactory<t>("*");

            T client = factory.CreateChannel();

            try
            {
                action(client);
                ((IClientChannel)client).Close();
                factory.Close();
            }
            catch (Exception ex)
            {
                IClientChannel clientInstance = ((IClientChannel)client);
                if (clientInstance.State == System.ServiceModel.CommunicationState.Faulted)

                {
                    clientInstance.Abort();
                    factory.Abort();
                }
                else if (clientInstance.State != System.ServiceModel.CommunicationState.Closed)
                {
                    clientInstance.Close();
                    factory.Close();
                }
                throw (ex);
            }
        }
    }
To use the class is as simple as:
 
WCFProxy.Using((delegate(IMyService client)
{
  client.DoWork();
});

21 March 2013

Fixing SSDT unresolved reference to object error

The SQL Server Data Tools (SSDT) for Visual Studio 2012 are a great set of tools for database development, but as any tool they have some undesirable "features".
One of those "features" is the "unresolved reference to object" error.
The "unresolved reference to object" in a stored procedure is a warning, but for the function it is an error. The error fails the build and consequently the the schema compare and update of the database.
The error normally is caused by a query that references an object on other database.

One example of the error is:
Error 190 SQL71561: Function: [dbo].[myFunction] has an unresolved reference to object [myDatabase].[dbo].[Product]
In the example the function [myFunction] is using the table [Product] from the database [myDatabase], that isn't the same database being managed by the database project of [myFunction].

A possible solution is to add a database reference to the database that has the missing object.
The reference needs a Data-tier Application (dacpac file) that can be easily generated on the solution with the database project that has the missing object. Press the right mouse button over the database project and selected Snapshot Project. The dacpac file is created on the Snapshots folder.
The file should then be copied to a common folder for re-usability.

In the project with the error press the right mouse button over the References and selected Add Database Reference.
The Add Database Reference dialog appears:
1) Select the dacpac file
2) Select the database location. The most common option is "Different database, same server"
3) Confirm that the Database name field is as expected
4) Clear the "Database variable" field in the dialog. If this field has a value the queries must use this variable and not the database name.



Look at the "Example usage" text and verify that it looks as expected. Click "OK" to add the reference and that should take care of the 'unresolved reference' errors.

The database reference resolves the schema comparison issue, but trying to build the project produced the following error:
Error 408 SQL00208: Invalid object name 'db.schema.table'.

To fix this error, go to the project properties and uncheck "Enable extended Transact-SQL verification for common objects".



15 February 2013

Dropdown list autopostback not working when validators fire

The Dropdown server control of the webforms can automatically post back by setting the autopostback property to true and setting the change event.
It works as expected, except when you have validators on the form with client-side validation active. The post back causes the validators to fire and prevent the autopostback if the form is invalid.

If there is the need to post back even if the form is invalid, the Microsoft documentation states that the CausesValidation property must be set to false.
However the issue still persists and the client side onchange event must execute the code Page_BlockSubmit=false
Page_BlockSubmit controls whether the form should be submitted or not.
Here is a full example:

<asp:dropdownlist autopostback="true" 
                 causesvalidation="false"
                 id="myDropDown" 
                 onchange="Page_BlockSubmit = false;"
                 onselectedindexchanged="myDropDown_SelectedIndexChanged"
                 runat="server">  
</asp:dropdownlist>

Note that to apply the best practices the client side event onchange should be set on the code behind.
myDropDown.Attributes.Add("onchange", "myDropDownOnchange();");
function myDropDownOnchange() {
    Page_BlockSubmit=false;
}

07 February 2013

Query Optimization and the SQL Server Cache

Query optimization is a very important task to assure that the server resources are not heavily consumed by a bad performing query.
One of the important tasks when optimizing a query  is to clean the SQL Server Cache so that the results are not influenced by the caching mechanism.

The T-SQL commands, that clean the Sql Server cache, will cause severe performance problems in a production environment, so they are to be used for testing purposes on a development or staging environment only.
They can be executed in a production server in special and controlled conditions. An example is to remove an individual plan of a query that has a bad performance because of a bad plan cache.

This article explains the main concepts involved and then the possible options to Clean the Sql Server Cache.


Execution Plan

The execution plan is the result of the query optimizer attempt to calculate the most efficient way to process the request represented by the SQL query statement.
Sql Server has to build an execution plan for each Transact-SQL (T-SQL) statement it has to execute.
The  execution plan defines how the T-SQL statement can be executed by Sql Server to produce the desired results.

The execution plan is built based on several considerations:
1) The tables it needes to join
2) The Indexes to use 
3) The sub-queries it has to execute
4) How aggregations of Group By are calculated
5) The estimated cost and load the operations place on the system
6) other even more complex considerations

The execution plan is also known as query plan.

As can be easily understandable, SQL has to put a lot of work to build a Execution Plan, so it caches the execution plan in memory to avoid having to do the same work over and over again.
Sql Server uses the Plan Cache to reuse plans. In this way, SQL Server can avoid the overhead of calculating the execution plan for each T-SQL statement and in this way speed up the execution of the queries.

 
Plan Cache

The plan cache is used by Sql Server to store the Execution Plans of the queries it has run.
The plan cache allows Sql Server to reuse Execution Plans for subsequent requests. It stores plans and it's associated information. There are metrics about the number of times a query was executed and the resources it used for example.

The Plan cache can be flush totally in some situations, the principal are:
1) Sql Server Service Restarts
2) Statistics of an object changing
3) Restores of a database
4) Executing T-SQL commands to clean it
5) Insufficient Memory on the server, causing memory pressure
6) Detaching a database
7) Some T-SQL Commands like for example RECONFIGURE, ALTER DATABASE ... MODIFY FILEGROUP or modifying a collation using ALTER DATABASE … COLLATE command

 The Plan cache of a database can also be flushed totally in some situations, where the principal are:
1) Some operations like for example DROP DATABASE or ALTER DATABASE … MODIFY NAME
2) If the database auto-closes
3) The database is set online or offline


Recompilations

Sql Server checks for correctness and for the optimality of a query plan before it executes it.
If one of the checks fails, the statement is compiled again and new query plan is produced.
These compilations are named as recompilations.

The recompilations are necessary to:
1)  Ensure statement correctness
2)  Obtain potentially better query execution plans as data changes

The recompilations can also have the side effect that they can slow down executions considerably.
In this case it is necessary to reduce the number of recompilations.

Parameter sniffing

Parameter sniffing is a complex topic, but according to Microsoft:

"Parameter sniffing" refers to a process whereby SQL Server's execution environment "sniffs" the current parameter values during compilation or recompilation, and passes it along to the query optimizer so that they can be used to generate potentially faster query execution plans. The word "current" refers to the parameter values present in the statement call that caused a compilation or a recompilation.

The query optimizer uses the parameters passed to the statements for performing estimates  when evaluating possible execution plan options. The final effect is that the plan is optimized for those specific parameter values. 
This feature allows more efficient stored procedure execution plans in most cases. There is however an assumption that the parameter values are "typical".
The main problem with this approach is when a parameter is atypical. The asymmetry usually is in the distribution of the data values or how the value is distributed across where the data is a non-uniform distribution, this is called skewed data.
 One example of skewed data is when there is a table with two million records and a column of type BIT. That column has the value 0 for only 1000 records and all the others have the value 0.
The LIKE clause is also very prune to this issue.

Parameter sniffing affects the performance of a query since the execution plan that is generated by the query optimizer depends on parameter sniffing.

Parameter values are sniffed for:
1) Stored Procedures
2) Queries executed using sp_executesql
3) Prepared queries

The following statements help to control the parameter sniffing performance problems:
1) WITH RECOMPILE - the stored procedure
2) Dummy local variables that are set equal to a parameter
3) OPTION(RECOMPILE) - This query hint is used to extended the behavior to queries (SELECT, INSERT, UPDATE, or DELETE). In this case,  both the parameter values and the current values of local variables are sniffed.
4) OPTION(OPTIMIZE FOR ())

The explanation of these options are out of the scope of this article, but you can search for a detailed explanation if you wish to learn more.


Clean the buffers

Sql Sever buffers the data before it is written to disk, this can cause dirty pages.
To guarantee that all these dirty pages, for the current database, are be written to disk and buffers are clean use the CHECKPOINT statement.
CHECKPOINT forces all dirty pages for the current database to be written to disk and cleans the buffers. After you do this, you can issue DBCC DROPCLEANBUFFERS command to remove all buffers from the buffer pool.

The CHECKPOINT is to guarantee that if you performed an operation or if you are in a collaborative environment that you are also testing you don't end up with dirty pages in the buffers.

Use DBCC DROPCLEANBUFFERS to test queries with a cold buffer cache without shutting down and restarting the server.  It serves to empty the data cache. Any data loaded into the buffer cache due to the prior execution of a query is removed.

Example:
CHECKPOINT
GO
DBCC DROPCLEANBUFFERS
GO


Clean the Plan Cache

There are several options to clean the plan cache. The next sections are going to explain the principal options, but don't forget to always Clean the Buffers first.


Clean the entire Plan Cache

The more drastic method is to clean the entire Sql Server plan cache, using the statement FREEPROCCACHE.
This will free the entire plan cache and causes the recompilation of subsequent ad-hoc SQL statements or Stored Procedures.

Usage:
DBCC FREEPROCCACHE


Clean a Database Plan Cache

A less drastic option, is to clear only the entire plan cache for a specific database and not the full Sql Server Plan Cache.

Usage:
DECLARE @dbId INTEGER
SELECT @dbId  = dbid FROM master.dbo.sysdatabases WHERE name = ‘myDatabase’
DBCC FLUSHPROCINDB (@dbId)

Where myDatabase is database to clear the entire plan cache.
The @dbId parameter is the database the number (database ID) to be affected by the FLUSHPROCINDB  command.


Clean a Compiled Plan

In some situations it is useful to clean only a specific compiled execution plan for an Stored Procedure or a Ad-Hoc query.
One example is when some queries are executed in production for testing or to Extract, Transform, Load (ETL) data and should be removed form the Plan Cache so that they don't occupy cache space.

Usage:
SELECT [text], CachedPlans.size_in_bytes, CachedPlans.plan_handle, CachedPlans.objtype, CachedPlans.usecounts
FROM sys.dm_exec_cached_plans AS CachedPlans
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
WHERE CachedPlans.cacheobjtype = N'Compiled Plan'

This query returns all the compiled plans.
The column Text identifies the T-SQL statement executed (Stored Procedure: proc or Ad-Hoc Query: Adhoc for example) and  the plan_handle can be passed to FREEPROCCACHE to remove it:

DBCC FREEPROCCACHE (plan_handle)


Clean  Stored procedure or Trigger Plan cache

The sp_recompile statement causes the recompilation of stored procedures and triggers the next time that they are run.It drops the existing plan from the procedure cache.

Usage:
EXEC sp_recompile N'myObject'';

Where myObject can be a stored procedure, trigger, table, or view in the current database.
If is the name of a stored procedure or trigger, the stored procedure or trigger will be recompiled the next time that it is executed.
If object is the name of a table or view, all the stored procedures or triggers that reference the table or view will be recompiled the next time that they are executed.

12 September 2012

Undoing a checkout that is from another user

A common task in TFS isto to undo someone else pending change for several reasons.
The TFS client doesn't have this option in the GUI, so we have to go to the command line as an administrator of TFS and enter the command:

tf undo /workspace:UserWorkspace;Username $/Project/file.cs /s:http://yourtfsserver:8080/tfs

Where:
- UserWorkspace is the workspace of the user to undo pending changes
- Username is the user name of the user we wanto to undo the checkout
- $/Project/file.cs is the TFS path to the file to undo the checkout
- /s:http://yourtfsserver:8080/tfs is the connection to the TFS Server