SQL Server DBA

How to create a Guid.Empty in SQL

Sabrina C. — Tue, 10 Aug 2010 20:53:20 GMT

Last friday my colleague Luca was “fighting” with a problem in managing null values and GUIDs. A Guid is a value type so it is not nullable. In my stored procedures when using parameters for filtering, I use the NULL value as the default value to ignore a filter parameter.

Changing the code of several classes to be able to change a Guid.Empty into a DbNull before passing it to a SELECT command was decisely not the best thing to do, so we searched a way to recreate a Guid.Empty value in SQL.

We were able to create the small snippet of SQL in the following function:

CREATE FUNCTION Guid_Empty()
RETURNS UniqueIdentifier
AS
BEGIN
RETURN cast(cast(0 as binary) as uniqueidentifier)
END
GO

After the creation of the above function and a simple table on my Paperinik Database, I was able to write the following query:

SELECT [ID]
      ,[Descrizione]
      ,[Prezzo]
      ,[Quantità]
      ,[Umi]
      ,IsNull(AltGuid, dbo.Guid_EMPTY()) AS Alt
  FROM [paperinik].[dbo].[TbRighe]

The query produces the following output:

I can use it also to build a query like the one below:

SELECT [ID]
      ,[Descrizione]
      ,[Prezzo]
      ,[Quantità]
      ,[Umi]
      ,IsNull(AltGuid, dbo.Guid_EMPTY()) AS Alt
  FROM [paperinik].[dbo].[TbRighe]
WHERE
IsNull(AltGuid, dbo.Guid_EMPTY()) = dbo.Guid_EMPTY()

That allows me to filter only the Guid Empty rows, or I can use the <> to find only the rows with a non Empty Guid. This way of writing the query, is giving me the possibility to use Guid.Empty in my Code as a SQLParameter value without the need to use a Nullable Guid or to implement complicated control expressions in my C# classes.

Technorati Tag: SQL,C#,Guid

SQL Server Update a Table with values selected from another table

Sabrina C. — Mon, 12 Jul 2010 16:40:51 GMT

Today, on another step of the job started with the last post, I had the following problem, I created a table with some informations and I needed to add more informations taken from other tables.

A an Example, I created an inventory table, with Article Code, Description, Quantity, to create it I used an INSERT query from the Articles table, then the users of my system inserted the Inventory values and now I need to add Sales Price and Cost of every article, reading them from the PriceList.

For Simplicity my tables are as follows:

TbInventory
   IDInventory int
   IDArticle nvarchar(32)
   Quantity decimal(18,5)
   Price decimal(18,5)
   Cost decimal(18,5)

TbListini
   IDPricelist int
   IDarticle nvarchar(32)
   Price decimal(18,5)
   Cost decimal (18,5)

To update price and cost I’lluse, an Update query, with a Join.

UPDATE TbInventory
    SET Price = isnull(lis.Price,0)
      , Cost = isnull(lis.Cost,0)
FROM TbInventory inv
INNER JOIN
    TbPriceList lis
ON
  inv.IDArticle = lis.IDArticle

This way, I’m setting a relation between the two tables usin ga JOIN thenI update the Inventory Table with The Price List values.

Tags: SQL, UPDATE

Use of Rank function to extract a list of rows with the highest date

Sabrina C. — Wed, 07 Jul 2010 21:45:13 GMT

Today I had the problem to create a query to get the most recent price from a pricelists table. Every pricelist in the table can contain the same article any number of times with different startup dates. I made some search to find a way to obtain what i needed and I found the RANK function, it was used in a similar scenario. I publish this post as usual to remember how to use this function next time I need it and in the hope to help someone else solving similar problems.

The table:

CREATE TABLE [dbo].[TbPricelistsRg](
    [IDPriceListRg] [int] NOT NULL,
    [IDPriceList] [nvarchar](10) NOT NULL,
    [IDArticle] [nvarchar](32) NULL,
    [Price] [decimal](18, 5) NULL,
    [StartDate] [date] NULL,
 CONSTRAINT [PK_TbPriceListsRg] PRIMARY KEY CLUSTERED 
(
    [IDPriceListRg] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  
     = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

Every row is linked to a PriceList through the IDPriceLIst field, we don’t need to use it for the example so it is omitted.

INSERT INTO [dbo].[TbPricelistsRg]
           ([IDPriceList]
           ,[IDArticle]
           ,[Price]
           ,[StartDate])
     VALUES
           ('CLD'
           ,'ABC'
           ,10
           ,CONVERT(date, '20080105', 112))
GO

INSERT INTO [dbo].[TbPricelistsRg]
           ([IDPriceList]
           ,[IDArticle]
           ,[Price]
           ,[StartDate])
     VALUES
           ('CLD'
           ,'OCQ'
           ,14
           ,CONVERT(date, '20080105', 112))
GO

INSERT INTO [dbo].[TbPricelistsRg]
           ([IDPriceList]
           ,[IDArticle]
           ,[Price]
           ,[StartDate])
     VALUES
           ('CLD'
           ,'FYS'
           ,21
           ,CONVERT(date, '20080105', 112))
GO

INSERT INTO [dbo].[TbPricelistsRg]
           ([IDPriceList]
           ,[IDArticle]
           ,[Price]
           ,[StartDate])
     VALUES
           ('CLD'
           ,'MUI'
           ,13
           ,CONVERT(date, '20080105', 112))
GO

INSERT INTO [dbo].[TbPricelistsRg]
           ([IDPriceList]
           ,[IDArticle]
           ,[Price]
           ,[StartDate])
     VALUES
           ('CLD'
           ,'SIO'
           ,42
           ,CONVERT(date, '20080105', 112))
GO

INSERT INTO [dbo].[TbPricelistsRg]
           ([IDPriceList]
           ,[IDArticle]
           ,[Price]
           ,[StartDate])
     VALUES
           ('CLD'
           ,'GOU'
           ,18
           ,CONVERT(date, '20080105', 112))
GO

INSERT INTO [dbo].[TbPricelistsRg]
           ([IDPriceList]
           ,[IDArticle]
           ,[Price]
           ,[StartDate])
     VALUES
           ('CLD'
           ,'HOU'
           ,20
           ,CONVERT(date, '20080105', 112))
GO

The previous script inserts some articles, we can repeat it changing the PriceLIst, the Dates and prices to be able to show how the query meets our needs.

SELECT [IDPriceListRg]
      ,[IDPriceList]
      ,[IDArticle]
      ,[Price]
      ,[StartDate]
      ,RANK() OVER (PARTITION BY IDPriceList, IDArticle ORDER BY StartDate DESC) DATERANK
  FROM [paperinik].[dbo].[TbPricelistsRg]

The RANK function, allows us to indicate one or more grouping fields on which calculate ranking, this is done using the PARTITION BY keywords, in our query we indicate the two fields on which ranking has to break. The ORDER BY keyword, instead indicates on which field we have to measure ranking. while the DESC keywords tells that we want to start from the most recent date..

This is the result of our query.

IDListinoRg	IDListino	IDArticolo	Prezzo	ValidoDal	DATERANK
107	CLD	ABC	1.000.000	24/10/2010	1
86	CLD	ABC	1.000.000	15/05/2010	2
65	CLD	ABC	1.000.000	08/01/2010	3
44	CLD	ABC	1.000.000	08/09/2009	4
23	CLD	ABC	1.000.000	16/04/2009	5
1	CLD	ABC	1.000.000	05/01/2008	6
2	CLD	ABC	1.000.000	05/01/2008	6
109	CLD	FYS	2.100.000	24/10/2010	1
88	CLD	FYS	2.100.000	15/05/2010	2
67	CLD	FYS	2.100.000	08/01/2010	3
46	CLD	FYS	2.100.000	08/09/2009	4
25	CLD	FYS	2.100.000	16/04/2009	5
4	CLD	FYS	2.100.000	05/01/2008	6
112	CLD	GOU	1.800.000	24/10/2010	1
91	CLD	GOU	1.800.000	15/05/2010	2
70	CLD	GOU	1.800.000	08/01/2010	3
49	CLD	GOU	1.800.000	08/09/2009	4
28	CLD	GOU	1.800.000	16/04/2009	5
7	CLD	GOU	1.800.000	05/01/2008	6
113	CLD	HOU	2.000.000	24/10/2010	1
92	CLD	HOU	2.000.000	15/05/2010	2
71	CLD	HOU	2.000.000	08/01/2010	3
50	CLD	HOU	2.000.000	08/09/2009	4
29	CLD	HOU	2.000.000	16/04/2009	5
8	CLD	HOU	2.000.000	05/01/2008	6

So to obtain the most recent prices, we need just to add a filter on DATERANK = 1, and to obtain the less recent price we just need to delete the DESC keyword and filter on rank 1.

Technorati Tag: SQL,RANK,DATE

FILESTREAM feature in SQL server 2008

Sabrina C. — Thu, 04 Mar 2010 18:19:20 GMT

One of the most important new features in SQL Server 2008 is certainly the possibility to use File System to store BLOB data. So if you work with images, binary files and so on and you need to store them in a SQL server database, it is possible to use this feature. It is important for those who are using SQL Server Express edition, because it avoids the problem of the 4GB maximum size for the database giving virtually no limit to the BLOB data growth.

Let’s see how to activate this feature on our server:

First step is to activate the feature at server protocol level, so, from Programs menu > Microsoft SQL Server 2008 > Configuration Tools – launch the SQL Server Configuration Manager (icon with the red toolbox).

Select SQL Services in the left treeview
On the right window, select SQL Server (MSSQLSERVER) for the default instance or the name of the instance to configure.
Right click on it, and select properties.
On the property window there is a series of TABS, select FILESTREAM, activate the T-SQL access and if necessary the API access and the remote client access (I haven’t done a full test so I don’t know if this is necessary, so you have to make some tests on this.)

Second step is in SQL Management Studio, connect to the server with an account with Sysadmin rights, on the server, right click and select properties, select Advanced on the left options list and on the first row of the property grid on the right part of the window, activate the Filestream option in T-SQL mode or Full Access.
Create a Database able to memorize data on FILESTREAM:

USE [master]
GO

/****** Object:  Database [Paperinik]    Script Date: 03/04/2010 18:09:07 ******/
IF NOT EXISTS (SELECT name FROM sys.databases WHERE name = N'Paperinik')
BEGIN
CREATE DATABASE [Paperinik] ON  PRIMARY 
( NAME = N'Paperinik', FILENAME = N'C:\SQL.DIR\Data\Test\Paperinik.mdf' , 
    SIZE = 3072KB , MAXSIZE = UNLIMITED, FILEGROWTH = 1024KB ), 
 FILEGROUP [Binari] CONTAINS FILESTREAM  DEFAULT 
( NAME = N'Paperinik_Bin', FILENAME = N'c:\sql.dir\data\test\PaperinikFS\paperinik_bin' )
 LOG ON 
( NAME = N'Paperinik_log', FILENAME = N'C:\SQL.DIR\Data\Test\Paperinik_log.ldf' , 
   SIZE = 1024KB , MAXSIZE = 2048GB , FILEGROWTH = 10%)
END

this example made with my favourite database :D creates the database with the Paperinik.mdf as master data file and the file Paperinik_log.ldf as a log file. Adding a FILEGROUP named Binari which defines the FILESTREAM zone, telling the server which is the filesystem folder where to put the data. in my case, c:\sql.dir\data\test\PaperinikFS\ pay attention that the Service account needs full control on this folder.

To proceed with test we need to create a table with a Varbinary(MAX) field with option FILESTREAM

USE [Paperinik]
GO

CREATE TABLE [dbo].[TbFiles](
    [IDFile] [uniqueidentifier] ROWGUIDCOL  NOT NULL,
    [IDFileSerial] [int] NOT NULL,
    [DDFile] [nvarchar](255) NULL,
    [FileData] [varbinary](max) FILESTREAM  NULL,
 CONSTRAINT [PK_TbFiles] PRIMARY KEY NONCLUSTERED 
(
    [IDFile] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  =     OFF, 
    IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON,
    ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY] FILESTREAM_ON [Binari]
GO

USE [Paperinik]
/****** Object:  Index [UQ_TbFileSerial]    Script Date: 03/04/2010 17:23:33 ******/
CREATE UNIQUE CLUSTERED INDEX [UQ_TbFileSerial] ON [dbo].[TbFiles] 
(
    [IDFileSerial] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, 
    SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, 
    DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, 
    ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY] FILESTREAM_ON [Binari]
GO

Now we need to insert the test data in the table, it is very simple using T-SQL:

INSERT INTO [Paperinik].[dbo].[TbFiles]
           ([IDFile]
           ,[IDFileSerial]
           ,[DDFile]
           ,[FileData])
     VALUES
           (NEWID()
           ,1
           ,'Nullo'
           ,null)
GO

INSERT INTO [Paperinik].[dbo].[TbFiles]
           ([IDFile]
           ,[IDFileSerial]
           ,[DDFile]
           ,[FileData])
     VALUES
           (NEWID()
           ,2
           ,'Vuoto'
           ,cast('' as varbinary(max)))
GO

INSERT INTO [Paperinik].[dbo].[TbFiles]
           ([IDFile]
           ,[IDFileSerial]
           ,[DDFile]
           ,[FileData])
     VALUES
           (NEWID()
           ,3
           ,'Stringa'
           ,cast('Questa stringa è binaria' as varbinary(max)))
GO

INSERT INTO [Paperinik].[dbo].[TbFiles]
           ([IDFile]
           ,[IDFileSerial]
           ,[DDFile]
           ,[FileData])
     VALUES
           (NEWID()
           ,4
           ,'La quarta'
           ,cast('Questa riga contiene questa stringa' as varbinary(max)))
GO

If we open our root folder after this operation on c:\sql.dir\data\test\PaperinikFS\ we will find some new folders, with a GUID as name, digging into the content we will find some files with names like nnnnnnnn-nnnnnnnn-nnnn that when opened with notepad will show the data in our varbinary field, the NULL field has no file. I haven’t yet tried to memorize a binary file inside this field, I’ll tell you what happens in a future post.

What comes out in the end from this test? If we need to store unstructured data in a database, this can be a good solution, because SQL Server gives us transactions, logs, managed purge the possibility to activate indexing, I’ve to try Full text indexing on NON Sql Express servers, the possibility to backup these data as a normal file system directory, eliminates the 4GB database limit but pay attention, even though the BLOB data can be accessed from the file system it is not easy to find things inside this file system, it is very simple to find things and check out/check in binary data using the T-SQL or the direct acces APIs. So if you thought to use FILESYSTEM in SQL Server to be able to access data both from the database and directly from file system, this is not the right way.

If you want to build a database to index informations on your non structured data and be able to keep files on the filesystem, and use them directly from where they are stored, well, you have to build something manual, and put just the “Path” info inside your database.

Tags: SQL, FILESTREAM, SQLServer 2008

How to search text inside Views SQL definition on a SQL Server database

Sabrina C. — Fri, 29 Jan 2010 11:28:58 GMT

Scenario: There is a problem in SQL 2005 and 2008, due to Legacy functionalities of SQL 2000. In SQL 2000 it was possible to use an ORDER BY clause inside a VIEW putting a SELECT TOP 100 PERCENT in the View definition. Obviously We all used it massively :D, after the SP2 or SP3 in SQL 2005 this feature available for all databases set in SQL 2000 compatibility mode does not work anymore.

I've discovered this on a production site where a nice small Access Function Developed in 2001 stopped working on October 15 2009.

There is an Hot Fix (Made after SP2 but not contained in SP3) that fixes the problem available here, but of course it is opportune for us to find out and change all views using the Order By feature and modify then the code to order things after the View.

After the problem has been discovered and the fix made on the Function, the question was:

How many views with an Order By do I have in this database?

Searching on the internet I've found a nice small script on Stack overflow Forums so I put it here just in case:

SELECT 
    v.name, 
    m.definition 
FROM  
    sys.views v 
INNER JOIN  
    sys.sql_modules m ON v.object_ID = m.object_id 
WHERE m.definition LIKE '%TOP%'

This script allowed me to search the TOP clause in all my views definitions, and can be customized to search probably any piece of sql contained in all object definitions in a database.

Thanks to Mark S. for sharing it.

Technorati Tag: SQL,Order by,SortedViews

Some Database Modification tests

Sabrina C. — Thu, 31 Dec 2009 10:43:26 GMT

If you, like me, have the misfortune to work with very creative software architects :D:D:D, you probably need to write Database Modification scripts on a daily base. It is a good practice be sure that a script executed two times in a row doesn’t throw exceptions, mainly if the script can be executed by your customers DBA (or the customers themselves). I write down here some of the most common tests, you can use when you have to Add tables, Drop Tables, Add Columns, Drop Columns or Foreign Keys.

IF NOT EXISTS (
    SELECT 1 FROM sysobjects WHERE xtype='u' 
       AND name='MyTable')
    BEGIN
       CREATE TABLE Mytable …;
    END

Without the NOT this can be used also to Drop Tables.

IF EXISTS (SELECT * FROM sys.foreign_keys WHERE object_id = 
    OBJECT_ID('[FK_MyForeignKeyConstraintName]') 
    AND parent_object_id = OBJECT_ID('[MyTable]'))
    ALTER TABLE [MyTable] DROP CONSTRAINT [FK_MyForeignKeyConstraintName]

If you need to delete a table with Relations, it is better drop the Constraints before dropping the table. If you Add a Constraint you can set a NOT before Exists and avoid trying to create a constraint already present.

IF NOT EXISTS (SELECT 1 FROM sys.columns WHERE object_id=object_id('MyTable') AND name='MyColumnName')
    ALTER TABLE MyTable ADD MyColumnName int NULL;

This one is useful when Adding, or without NOT, Dropping columns.

Tags: SQL, DBMaintenance

SQL Server, plain security

Sabrina C. — Sun, 29 Nov 2009 10:55:52 GMT

A colleague, moving it’s first steps into .NET and SQL Server to move it’s applications in 21st century, taking a look at a database I made asked me the following questions:

Why do I found the SQL User (MyUser) both in Server Security and in your Database? And Why did you tell me you granted permission to a Database Role (db_MyRole) instead of giving them to the user?

How the security works in SQL Server:

To answer his questions I’ve decided to explain the basics here in the hope to be helpful to more beginners.
In order to be able to connect an Application to SQL Server, the Application needs to know a Username and a Password (in Sql Server mode) or the Windows Domain User that uses the application or a Domain Group of which it is member needs to be mapped as a trusted User or Group in SQL Server.

This can be made manually by the SQL Server Administrator using SQL Management Studio and going to the Security Folder Using the Add Login feature, or it can be done using the following code in a query (connecting to the SQL Server with Administrator rights).

CREATE LOGIN [MyUser] WITH PASSWORD=N'MyPassword'
    , DEFAULT_DATABASE=[master]
    , DEFAULT_LANGUAGE=[us_english]
    , CHECK_EXPIRATION=OFF, CHECK_POLICY=OFF

CREATE LOGIN [MyWindowsUserOrGroup] 
    FROM WINDOWS WITH DEFAULT_DATABASE=[master]
    , DEFAULT_LANGUAGE=[us_english]

But this is not enough, with this credentials an Application can open a connection to the SQL Server, but the server gives no automatic access to it’s data to a new login.

To access a Database, the logins (Sql or trusted) first need to be mapped on the database (or databases), this can be achieved from SQL Management Studio, going to the Database Security Folder and adding the user, or through the Usermapping feature in the Login Mask on server security, otherwise it can be done using the following T-SQL queries.

CREATE USER [MyUser] 
    FOR LOGIN [MyUser] WITH DEFAULT_SCHEMA=[dbo]

CREATE USER [MyWindowsUserOrGroup] 
    FOR LOGIN [MyWindowsUserOrGroup] WITH DEFAULT_SCHEMA=[dbo]

After this operation is done, the application can connect to the server and access the database, but still it can’t see any data, because we didn’t grant it any permission on the database objects, such as Tables, Stored Procedures, Views, Functions and more.

We can grant permission on the objects to our logins in two ways, mapping them directly to the User or Windows User, generate a database Role, give permissions to the role and then assign the role to the users. The second way is the one I choose because a user can be changed for any reason, so the role can be passed to another user and we are sure the user has all permissions he needs. The role can also be assigned to more than one user and again we are sure the users have all the correct permissions and if we need to add some permissions or remove some permission we do it on the role and all users benefit of the modifications.

Again, this can be done both from the SQL management studio and by code.

USE [MyDatabase]
GO
CREATE ROLE [MyRole] AUTHORIZATION [sa]
GO
use [MyDatabase]
GO
GRANT SELECT ON [dbo].[MyTable1] TO [MyRole]
GO
use [MyDatabase]
GO
GRANT EXECUTE ON [dbo].[MySpSelect1] TO [MyRole]
GO
use [MyDatabase]
GO
GRANT EXECUTE ON [dbo].[MySpSelect2] TO [MyRole]
GO
use [MyDatabase]
GO
GRANT SELECT ON [dbo].[MyTable2] TO [MyRole]
GO

After the role is made, we made the users member of the role and we have reached our goal. Also this operation can be done using SSMS or the following query.

USE [FyRepository]
GO
EXEC sp_addrolemember N'MyRole', N'Myuser'
GO

In Short

To access the data of a SQL Server database, an Application needs to be able to Connect to the Server through a Login, the Login needs to be mapped as a Database User on the database (s) needed, and the Database User mapped needs to be member of one or more Database Roles that grant it permissions on the database objects.

Technorati Tag: SQL,T-SQL,Security

How to generate rapidly a copy of a table in SQL Server

Sabrina C. — Thu, 26 Nov 2009 10:35:46 GMT

Today I've found a helpful little piece of SQL I wasn't aware of so I share this with the whole Internet in the hope to help someone else. The scenario was the following, I have a remote server from which I need to import a few tables on my local SQL Server for further processing. The Two servers are connected through a VPN. The connection is good but I can’t kill the band with a long run series of data transferts through the network. So I first set a linked server from the local machine to the remote server that allows me to query the remote server on the Database I'm interested in. Then, to transfer the table I was interested in I wrote this code:

USE MyNewLocalDb

SELECT FIELD1, FIELD2, FIELD3 .... FIELDn INTO dbo.MynewTable FROM
[MyRemoteServer].[MyRemoteDb].[Dbo].[MyremoteTable]

In this case I had just a simple select from table, but If you need to build the table from a a Join, in the from clause you can write a complex select with joins or whatever you need. The Table is very raw, but then you can set PK, indexes and whatever else you need on it.

Obviously you need to have the proper rights on the two databases to make this work, you have to be at least Db_Owner of the destination db and Db_Datareader on the source db.

Technorati Tag: SQL