Monday 5 November 2012

Create a Cursor using Dynamic SQL Query

Even though we are advised not to use or to minimize the usage of cursors in SQL, there are times which, it’s the only available option, depending on the task and the situation. But there are times you are required to build the cursor using a dynamic SQL select statement.

To illustrate this, i will create two tables and populated it with sample data.

 
--==== Create sample tables ====--
create table sample_table_a(
    id        int
    ,name    varchar(50)
)
 
create table sample_table_b(
    id            int
    ,country    varchar(50)
)
 
--==== Populate with sample data ====--
insert into sample_table_a (id,name)
values(1,'Windows'),(2,'Mac OS'),(3,'Linux')
 
insert into sample_table_B(id,country)
values(1,'Austria'),(2,'Australia'),(3,'USA'),(4,'Singapore')

 


When creating a cursor using dynamic query, initially the cursor should be declared and passed on to the dynamic query execution.



 
/* ==== Variable Declaration ==== */
declare @objcursor as cursor 
 
declare 
    @vsql        as nvarchar(max)
    ,@vquery    as nvarchar(max)
    ,@id        as int
    ,@value        as varchar(50)
    
 
/* ==== Sample Table A ==== */
set @vquery = 'select id, name from sample_table_a'
set @vsql = 'set @cursor = cursor forward_only static for ' + @vquery + ' open @cursor;'
 
exec sys.sp_executesql
    @vsql
    ,N'@cursor cursor output'
    ,@objcursor output
 
fetch next from @objcursor into @id,@value
while (@@fetch_status = 0)
begin
    print cast(@id as varchar) + ' - ' + @value
    fetch next from @objcursor into @id,@value
end
 
close @objcursor
deallocate @objcursor
 
/* ==== Sample Table B ==== */
 
set @vquery = 'select id, country from sample_table_b'
set @vsql = 'set @cursor = cursor forward_only static for ' + @vquery + ' open @cursor;'
 
exec sys.sp_executesql
    @vsql
    ,N'@cursor cursor output'
    ,@objcursor output
 
fetch next from @objcursor into @id,@value
while (@@fetch_status = 0)
begin
    print cast(@id as varchar) + ' - ' + @value
    fetch next from @objcursor into @id,@value
end
 
close @objcursor
deallocate @objcursor


 


Also you can create a procedure which will return you a cursor taking the selection query as a parameter.



 
CREATE PROCEDURE [dbo].[Gsp_Create_GenericCursor]
    /* Parameters */
    @vQuery        NVARCHAR(MAX)
    ,@Cursor    CURSOR VARYING OUTPUT
AS
BEGIN
    SET NOCOUNT ON
    
    DECLARE 
        @vSQL        AS NVARCHAR(MAX)
    
    SET @vSQL = 'SET @Cursor = CURSOR FORWARD_ONLY STATIC FOR ' + @vQuery + ' OPEN @Cursor;'
    
   
    EXEC sp_executesql
         @vSQL
         ,N'@Cursor cursor output'  
         ,@Cursor OUTPUT;
END


Use the following syntax to use the above mentioned procedure



DECLARE @obj AS CURSOR
DECLARE @i AS INT    
 
 
    EXEC dbo.Gsp_Create_GenericCursor 
        @vQuery = N'SELECT 1 AS FLD1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4'
        ,@Cursor = @obj OUTPUT
        
        FETCH NEXT FROM @obj INTO @i
        
        WHILE (@@FETCH_STATUS = 0)
        BEGIN
            PRINT @i
            
            FETCH NEXT FROM @obj INTO @i
        END
        
        CLOSE @obj
        DEALLOCATE @obj

Wednesday 19 September 2012

Introduction to SSDT (SQL Server Data Tools)

What is SSDT ?

SQL Server Data Tools (SSDT) is a toolset which provides an integrated environment for database developers to carry out all their database design work for any SQL Server platform (both on and off premise) within Visual Studio. Database developers can use the SQL Server Object Explorer in VS to easily create or edit database objects and data, or execute queries.

In a previous blog entry I have described on how to install SSDT into the VS environment. During this I will briefly describe few of it’s features.

SSDT’s intention is not to replace the SQL Server Management Studio, but provide a developer a complete development environment, which the developer need not required to leave Visual Studio IDE to do any database related development. The tool does not contain all the features which you find in SSMS, yet it’ll provide sufficient functionality which will required to most of the developers during their development tasks.

I find following features, pretty  much interesting and helps to increase the productivity of the developer/team.

 

1. Design & Code view in a single screen.

img_screen_001

Ability to see the design view and the code view in a single screen is a wonderful thing. You do not have to move across screens. And the changes you do to the design view will be affected to the code immediately and vice versa.

 

2. Ability to add Constraints, Indexes, Foreign Keys & Triggers without changing the screen

Adding constraints, indexes, etc.. are much easier. You can add those by right clicking and choosing the ‘add <object>’ menu like shown below.

img_screen_02

 

3. It uses ‘Declarative – Model Based Development’

What this means is that there is always an in-memory representation of what a database looks like—an SSDT database model— and all the SSDT tools (designers, validations, IntelliSense, schema compare, and so on) operate on that model. This model can be populated by a live connected database (on-premise or SQL Azure), an offline database project under source control, or a point-in-time snapshot taken of an offline database project (you will work with snapshots in the upcoming exercises). But to reiterate, the tools are agnostic to the model’s backing; they work exclusively against the model itself. Thus, you enjoy a rich, consistent experience in any scenario—regardless of whether you’re working with on-premise or cloud databases, offline projects, or versioned snapshots.

 

4. Connected Development

Although SSDT places great emphasis on the declarative model, it in no way prevents you from working imperatively against live databases when you want or need to. You can open query windows to compose and execute T-SQL statements directly against a connected database, with the assistance of a debugger if desired, just as you can in SSMS.

img_screen_03

 

5. Disconnected Development

The new SQL Server Object Explorer lets you connect to and interact with any database right from inside Visual Studio. But SSDT offers a great deal more than a mere replacement for the connected SSMS experience. It also delivers a rich offline experience with the new SQL Server Database Project type and local database runtime (LocalDB).

Actually this is one great feature it has. Because due to many reasons, most of the time we are requested to work offline, or to do our development locally and later on merged. So this feature allows us to maintain a local database and do all the development using that. And later on merged/published to other database. And by doing a schema comparison, it’s possible to find out the changes we have done.

img_screen_04

 

6. Schema Comparison

This is a very valuable feature. This allows us to do a schema comparison between our development environment and the physical database (or vice versa) and find out the changes very quickly.

img_screen_05

 

7. Saving snapshots of the Database

Sometimes it’s required to keep snapshots of your database at different stages of the development. And the best thing is, it even allows you to compare between two database snapshots. So it’s easy to see the changes you have done compared to the previous stages.

img_screen_06

8. Ability to find any errors or reference issues in design time

When SSDT is used it’s easy to identify any syntax issue or any reference issue before deploying it to the database. E.g. assume we have one view which is referring to few columns of a table. Usually if someone change or remove any columns from the table which this view is referring, there is no way of identify that, till the view is used in our application. But when SSDT is used, it will show you these issues, when building the project. So these can be eliminated before we apply these to the deployment server.

img_screen_07

These are some features which I have find very interesting and most developers expect. But having said that I am not saying it’s the complete set of features. There are few missing functionalities which I felt that it would have been even nicer, if those were there.

And you can find a good article in here (http://www.codeproject.com/Articles/357905/Evaluating-SQL-Server-Data-Tools) regarding the SSDT. You can find things in more depth.

Friday 31 August 2012

Installing SQL Server Data Tools (SSDT)

What is SQL Server Data Tools?

SQL Server Data Tools (SSDT) is a toolset which provides an integrated environment for database developers to carry out all their database design work for any SQL Server platform (both on and off premise) within Visual Studio. Database developers can use the SQL Server Object Explorer in VS to easily create or edit database objects and data, or execute queries.

More details are available regarding it’s features at http://blogs.msdn.com/b/ssdt/archive/2011/11/21/what-is-sql-server-data-tools-ssdt.aspx

SSDT is not intended to be a replacement for SSMS, but instead can be viewed much more as a greatly evolved implementation of DbPro. Indeed, SSMS is alive and well in SQL Server 2012, and it continues to serve as the primary management tool for database administrators who need to configure and maintain healthy SQL Server installations.

However SSDT does not get installed with either Visual Studio or SQL Server. Instead, SSDT ships separately via the Web Platform Installer (WebPI).

  1. Download SSDT from http://go.microsoft.com/fwlink/?LinkID=241405
  2. Once it’s downloaded, open a command window with administrative privileges (run cmd.exe as Administrator), and execute the following command :

SSDTSetup.exe /layout <destination>

<destination> is the path which the WebPI will download all the necessary installation files and create the administrative installation point. This can be a location in either LAN, USB or your Local Drive.

img_screen_01

Then it’ll start to download the required files to the given location.

img_screen_02

Once everything is downloaded you can execute the ‘SSDTSetup.exe’ from the destination location. (without any arguments). Once the installation is completed successfully, you can see the tool in Visual Studio 2010 development environment. Select it from the ‘View’ menu.

img_screen_04

img_screen_05

Wednesday 11 April 2012

Resolved ~ How to get back SSMS ‘Find and Replace’ Alert

 

As you know in SQL Server Management Studio (SSMS), once you have cleared the check box value of ‘Always show this message’, from the ‘Find and Replace’ alert, there’s no way of getting it back.

img_scr_01

If you remove the check box, instead of the alert you will get a message no the status bar as shown below.

img_scr_02

But there’s a workaround to fix this problem. Do the following steps

1. Close all the SSMS environments. (If you have opened)
2. Open the registry editor and locate the key '\HKEY_CURRENT_USER\Software\Microsoft\Microsoft SQL Server\100\Tools\Shell\Find'.
3. Change the value of 'HitReplaced' to '1'

img_scr_03

Thursday 8 March 2012

Locks and Duration of Transactions in MS SQL Server

 

It is a common argument which I hear among developers these days, regarding SQL locks. Some say that the ‘locks are held for the duration of the entire transaction’. But others debate that ‘locks will be only held for the duration of the statement execution’. But who is correct ?

Well both parties are correct up to a certain point. Actually lock durations are depend on the Isolation Levels.

As mentioned in the SQL-99 Standards, there are 4 Transaction Isolation Levels

  • Read Committed (Default)
  • Read Uncommitted
  • Repeatable Read
  • Serializable

SQL Server** provides following two additional isolation levels (** SQL Server 2005 & Upwards)

  • Snapshot
  • Read Committed Snapshot

There are several concurrency issues which can occur in a DBMS when multiple users try to access the same data. Each isolation level protects against a specific concurrency problem.

  • Lost Update
  • Dirty Read
  • Non-Repeatable Read
  • Phantom Reads

 

Lost Update – This can take place in two ways. First scenario: it can take place when data that has been updated by one transaction (Transaction A), overwritten by another transaction (Transaction B), before the Transaction A commits or rolls back. (But this type of lost update can never occur in SQL Server** under any transaction isolation level)

img_screen_02

The second scenario is when one transaction (Transaction A) reads a record and retrieve the value into a local variable and that same record will be updated by another transaction (Transaction B). And later Transaction A will update the record using the value in the local variable. In this scenario the update done by Transaction B can be considered as a ‘Lost Update’.

img_screen_04

 

Dirty Read – This is when the data which is changed by one transaction (Uncommitted) is accessed by a different transaction. All isolation levels except for the ‘Read Uncommitted’ are protected against ‘Dirty Reads’.

img_screen_05

 

Non Repeatable Read – This is when a specific set of data which is accessed more than once in one transaction (Transaction A) and between these accesses, it’s being updated or deleted by another transaction (Transaction B). The repeatable read, serializable, and snapshot isolation levels protect a transaction from non-repeatable reads.

img_screen_03

 

Phantom Read – This is when two queries in the same transaction, against the same table, use the same ‘WHERE’ clause, and the query executed last returns more rows than the first one. Only the serializable and snapshot isolation levels protect a transaction from phantom reads.

img_screen_06

 

In order to solve the above mentioned concurrency issues, SQL Server uses the following type of locks.

  • Shared or S-locks - Shared locks are sometimes referred to as read locks. There can be several shared locks on any resource (such as a row or a page) at any one time. Shared locks are compatible with other shared locks.
  • Exclusive or X-locks - Exclusive locks are also referred to as write locks. Only one exclusive lock can exist on a resource at any time. Exclusive locks are not compatible with other locks, including shared locks.
  • Update or U-locks - Update locks can be viewed as a combination of shared and exclusive locks. An update lock is used to lock rows when they are selected for update, before they are actually updated. Update locks are compatible with shared locks, but not with other update locks.

Please refer to the following link to get more information regarding lock types. http://msdn.microsoft.com/en-us/library/ms175519.aspx

As I have mentioned earlier, the type of lock which the SQL server will be acquired depends on the active transactions isolation level. I will briefly describe each isolation level a bit further.

Read Committed Isolation Level – This is the default isolation level for new connections in SQL Server. This makes sure that dirty reads do not occur in your transactions. If the connection uses this isolation level, and if it encounters a dirty row while executing a DML statement, it’ll wait until the transaction which owns that row has been committed or rolled back, before continuing execution further ahead.

img_screen_07

 

Read Uncommitted Isolation level - Though this is not highly recommended by experts, it's better to consider about it too. It may result in a 'dirty read', but when correctly used it could provide great performance benefits.

You should consider using this isolation level only in routines where the issue of dirty reads is not a problem. Such routines usually return information that is not directly used as a basis for decisions. A typical example where dirty reads might be allowed is for queries that return data that are only used in lists in the application (such as a list of customers) or if the database is only used for read operations.

The read uncommitted isolation level is by far the best isolation level to use for performance, as it does not wait for other connections to complete their transactions when it wants to read data that these transactions have modified. In the read uncommitted isolation level, shared locks are not acquired for read operations; this is what makes dirty reads possible. This fact also reduces the work and memory required by the SQL Server lock manager. Because shared locks are not acquired, it is no problem to read resources locked by exclusive locks. However, while a query is executing in the read uncommitted isolation level, another type of lock called a ‘schema stability lock’ (Sch-S) is acquired to prevent Data Definition Language (DDL) statements from changing the table structure. Below is an example of the behavior of this isolation level.

img_screen_08

 

Repeatable Read Isolation Level - In this isolation level, it guarantees that dirty reads do not happen in your transaction. Also it makes sure that if you execute/issue two DML statements against the same table with the same where clause, both queries will return the same results. But this isolation level will protect against updates and deletes of earlier accessed rows, but not the inserts, which is known as ‘Phantom’ rows concurrency problem. Note that phantom rows might also occur if you use aggregate functions, although it is not as easy to detect.

img_screen_09

 

Serializable Isolation Level – This guarantees that none of the aforesaid concurrency issues can occur. It is very much similar to the ‘repeatable read isolation level’ except that this prevents the ‘phantom read’ also. But use of this isolation level increases the risk of having more blocked transactions and deadlocks compared to ‘Repeat Read’. However it will guarantee that if you issue two DML statements against the same table with the same WHERE clause, both of them will return exactly the same results, including same number of row count. To protect the transaction from inserts, SQL Server will need to lock a range of an index over a column that is included in the WHERE clause with shared locks. If such an index does not exist, SQL Server will need to lock the entire table.

 

Snapshot Isolation Level – In addition to the SQL’s standard isolation levels, SQL 2005 introduced ‘Snapshot Isolation Level’. This will protect against all the above mentioned concurrency issues, like the ‘Serializable Isolation Level’. But the main difference of this is, that it does not achieve this by preventing access to rows by other transaction. Only by storing versions of rows while the transaction is active as well as tracking when a specific row was inserted.

To illustrate this I will be using a test database. It’s name is ‘SampleDB’. First you have to enable the ‘Snapshot Isolation Level’ prior using it

alter database SampleDB set allow_snapshot_isolation on;
alter database SampleDB set read_committed_snapshot off;


Now we’ll create a sample table and insert few records.


create table SampleIsolaion(
id int,
name varchar(20),
remarks varchar(20) default ''
)

insert into SampleIsolaion (id,name,remarks)
select 1, 'Value A', 'Def' union
select 2, 'Value B', 'Def'


 


img_screen_10


 




Read Committed Snapshot Isolation Level – This can be considered as a new implementation of the ‘Read Committed’ isolation level. When this option is set, this provides statement level read consistency and we will see this using some examples in the post. Using this option, the reads do not take any page or row locks (only SCH-s: Schema Stability locks) and read the version of the data using row versioning by reading the data from tempdb. This option is set at the database level using the ALTER DATABASE command


I will illustrate the use of this isolation level with a sample. First enable the required isolation level.


alter database SampleDB set read_committed_snapshot on;
alter database SampleDB set allow_snapshot_isolation on;



Now lets create a table and populate it with few sample data.


create table sample_table(
id int,
descr varchar(20),
remarks varchar(20)
)

insert into sample_table
select 1,'Val A','Def' union
select 2,'Val B','Def'


Now open two query windows in SQL Server Management Studio.


--Window 1
begin tran
update sample_table set descr = 'Val P', remarks = 'Window 1' where id = 1


 


Without committing execute the following in the second window



--Window 2
begin tran
set transaction isolation level read committed
select * from sample_table



And you can see, even without committing, it’ll read from the older values, from the row versions which were created in the tempdb. If it was only the ‘Read Commited’ isolation level without the ‘Read Committed Snapshot’ option turned on, this select statement would have been locked.

Monday 16 January 2012

Exclusive access could not be obtained because the database is in use ~ Resolved

 

img_screen_001

Sometimes this is a common error message that we encounter, when we try to restore a SQL database, which is being used by other users.

This can occur due to various reasons. But the most common incident is, users not closing the Management Studio’s query window after they have finished the query task.

There are few ways of resolving this and restore the database.

1.    Find all the active connections, kill them all and restore the database
2.    Get database to offline (And this will close all the opened connections to this database), bring it back to online and restore the database

Method 1

Use the following script to find and kill all the opened connections to the database before restoring database.

declare @sql as varchar(20), @spid as int

select @spid = min(spid) from master..sysprocesses where dbid = db_id('<database_name>')
and spid != @@spid

while (@spid is not null)
begin
print 'Killing process ' + cast(@spid as varchar) + ' ...'
set @sql = 'kill ' + cast(@spid as varchar)
exec (@sql)

select
@spid = min(spid)
from
master..sysprocesses
where
dbid = db_id('<database_name>')
and spid != @@spid
end

print 'Process completed...'



Method 2


Use the following code to take database offline and bring back to online so that all the active connections will be closed. And afterwards restore the database.



alter database database_name
set offline with rollback immediate
alter database database_name
set online
go

Friday 25 November 2011

Deploy/Use assemblies which require Unsafe/External Access with CLR and T-SQL

 

What is an Unsafe Assembly?

Assemblies which are built using normal computational functions are considered as safe assemblies. But when assemblies do external operations such as reading file information, creating files, etc.… they are categorized as unsafe/external assemblies.

**Visual studio creates safe assemblies by default.

We will create an assembly which access the external file system, so that it will need external access. We will create a simple function which will return the file size for a given file, using the ‘FileInfo’ class.

using System;
using Microsoft.SqlServer.Server;
using System.IO;

public partial class UserDefinedFunctions {
[SqlFunction]
public static long GetFileSize(string FileName) {
FileInfo fi = new FileInfo(FileName);
return fi.Length;
}

};



Now right click the project (from the solution explorer) and go the properties tab. Form the ‘Database’ tab select the permission level to ‘External’. (Default value is ‘Safe’)

img_scr_001

Open MS SQL Server Management Studio (Run it as Administrator since you are going to assign permission to the current user), and log in as a different user than the one you are trying to provide access permission (For this example I am logging as ‘sa’). Execute the following script.


use master;
grant external access assembly to [Domain\UserID];
use SampleCLR;


Use the appropriate values for ‘Domain’ and ‘UserID’. (And ‘SampleCLR’ is the database that I will be using)

**Please note that this is a server wide permission. Therefore user can create any external assembly in any database on the SQL Server.

Above script will grant permission the user to create external assemblies on the executed server. But it is not sufficient. The database should be allowed to have external access assemblies.

There are two methods of doing so.


  1. Making the database trusted.
  2. Using sign assemblies.

Method 1 ~ Making Database Trusted

Execute the following script (using Management Studio) in order to make the database trusted.

alter database SampleCLR set trustworthy on;



And if you inspect the database properties, you can see that the database’s ‘Trustworthy’ property value is changed to ‘True’

img_scr_002

Now go to visual studio and deploy the solution. It will succeed without any issues.

However if you try to deploy without doing the above mentioned steps, you will get the following error.


CREATE ASSEMBLY for assembly <AssemblyName> failed because assembly <AssemblyName> is not authorized for PERMISSION_SET = EXTERNAL_ACCESS. The assembly is authorized when either of the following is true: the database owner (DBO) has EXTERNAL ACCESS ASSEMBLY permission and the database has the TRUSTWORTHY database property on; or the assembly is signed with a certificate or an asymmetric key that has a corresponding login with EXTERNAL ACCESS ASSEMBLY permission


We will check the deployed assembly by executing the following script. (I have a file on my D:\ drive with the mentioned name.)


select dbo.GetFileSize(N'D:\data.csv')

img_scr_003


img_scr_004



And if you try to execute the function without making the database trusted you will get the following error


Msg 10314, Level 16, State 11, Line 1
An error occurred in the Microsoft .NET Framework while trying to load assembly id 65540. The server may be running out of resources, or the assembly may not be trusted with PERMISSION_SET = EXTERNAL_ACCESS or UNSAFE. Run the query again, or check documentation to see how to solve the assembly trust issues. For more information about this error:
System.IO.FileLoadException: Could not load file or assembly 'sqlclrproject, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. An error relating to security occurred. (Exception from HRESULT: 0x8013150A)
System.IO.FileLoadException:
   at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
   at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
   at System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
   at System.Reflection.Assembly.Load(String assemblyString)


Method 2 ~ Using Sign Assemblies

Set the trustworthy to false by using the following script (You only have to do this if you have made the database a trusted one in the previous example, and I am keeping it false for illustrated purpose)


alter database SampleCLR set trustworthy off;



In order to sign an assembly we need a public/private key file (.snk file). We will create one using the ‘sn.exe’.

img_scr_005


sn -k "D:\Sample CLR\SampleCLRKey.snk"



And sign the assembly using the key file that we have created now.

To sign an assembly: go to project properties => select the ‘Signing’ tab and check the ‘Sign the assembly’ check box and browse and select the created file. Save the project.

In order to deploy the assembly,


  1. Need to create an asymmetric key using the key file which we have created in the SQL Server
  2. Need to create a login using that asymmetric key
  3. Giving that login the permission for external access assemblies

img_scr_006


Use the following script to create the asymmetric key using SQL Server Management Studio. (** Please note that the key should be created on the master database)


use master;
create asymmetric key CLRExtensionKey
from file = 'D:\Sample CLR\SampleCLRKey.snk'
encryption by password = '@Str0ngP@$$w0rd'



Now create the login using the above created key (*Please note that the login should be created on the database which you want to publish the assembly to)


use SampleCLR;
create login CLRExtensionLogin from asymmetric key CLRExtensionKey;



Give the login permission for external access assemblies.


use master;
grant external access assembly to CLRExtensionLogin;



Now go to visual studio and deploy the solution. And you can use the following statement which we used in Method 1.


select dbo.GetFileSize(N'D:\data.csv')


img_scr_007