Friday, 27 January 2017

SQL SERVER – SPID status is KILLED/ROLLBACK state. Unable to kill the process.

he first question which would come to mind would be “what is killed/rollback command?” The answer is very simple – sometimes, DBA/Developer use “KILL” command to end processes that appear in a hung state. Once that happen, if SQL knows how to rollback that work, it would start “real” rollback process and would undo the work done. There are situations where SQL doesn’t know and has no control of work done of the work done and it has no control to roll back the work. These are the cases where its stays in KILLED/ROLLBACK state forever.
How would you decide which action to take? It completely depends on the action done by the SPID which was killed. There are only a few things, which you can do:
  1. Have patience and wait for rollback to finish. Some large operations may take a long time to rollback. Such SPID should show a change in CPU, Memory and IO column in size. sys.sysprocesses DMV (query in the next step).
  2. If the SPID is not showing any change in CPU, Memory and IO column then it is doing nothing. Here is the query which can tell us more about the SPID and from where the connection might have originated.
                                  SELECT spid , kpid, login_time, last_batch, status, hostname
                                  ,nt_username, loginame, hostprocess, cpu, memusage, physical_io
                                    FROM sys.sysprocesses WHERE cmd = 'KILLED/ROLLBACK' 
If the process is originated from different machine, then using task manager and doing end process on that machine might help. This decision has to be taken very carefully as it might be a process of a 3rd party application.
After figuring out the SPID we can run the DBCC INPUTBUFFER (<SPID>) to know what is the batch running under this session. 
  1. Restart SQL Service if it was killed and doing nothing.


Wednesday, 11 January 2017

'SetEndOfFile' error during Backup

Below is the error.

Message
Executed as user:. Date and time: 2016-12-11 04:10:11  Server:  Version: 11.0.6537.0  Edition: Enterprise Edition: Core-based  Procedure: [master].[dbo].[Database]  Parameters: @Databases = 'DB_Name', @Directory = 'Backup path1', @BackupType = 'FULL', @Verify = 'N', @CleanupTime = 240, @CleanupMode = 'AFTER_BACKUP', @Compress = 'Y', @CopyOnly = 'N', @ChangeBackupType = 'Y', @BackupSoftware = NULL, @CheckSum = 'Y', @BlockSize = NULL, @BufferCount = 256, @MaxTransferSize = 2097152, @NumberOfFiles = 4, @CompressionLevel = NULL, @Description = NULL, @Threads = NULL, @Throttle = NULL, @Encrypt = 'N', @EncryptionAlgorithm = NULL, @ServerCertificate = NULL, @ServerAsymmetricKey = NULL, @EncryptionKey = NULL, @ReadWriteFileGroups = 'N', @OverrideBackupPreference = 'N', @NoRecovery = 'N', @URL = NULL, @Credential = NULL, @MirrorDirectory = NULL, @MirrorCleanupTime = NULL, @MirrorCleanupMode = 'AFTER_BACKUP', @LogToTable = 'Y', @Execute = 'Y'  Source: https://ola.hallengren.com [SQLSTATE 01000] (Message 50000)  Date and time: 2016-12-11 04:10:11  Database: []  Status: ONLINE  Standby: No  Updateability: READ_WRITE  User access: MULTI_USER  Is accessible: Yes  Recovery model: FULL  Differential base LSN: 151862000023530400068  Last log backup LSN: 151862000025621600001 [SQLSTATE 01000] (Message 50000)  Date and time: 2016-12-11 04:10:11  Command: DECLARE @ReturnCode int EXECUTE @ReturnCode = [master].dbo.xp_create_subdir N'\\' IF @ReturnCode <> 0 RAISERROR('Error creating directory.', 16, 1) [SQLSTATE 01000] (Message 50000)  Outcome: Succeeded  Duration: 00:00:00  Date and time: 2016-12-11 04:10:11 [SQLSTATE 01000] (Message 50000)  Date and time: 2016-12-11 04:10:11  Command: BACKUP DATABASE [EDDS2623458] TO DISK =  WITH CHECKSUM, OMPRESSION, BUFFERCOUNT = 256, MAXTRANSFERSIZE = 2097152 [SQLSTATE 01000] (Message 50000)  Processed 388889392 pages for database '', file '' on file 1. [SQLSTATE 01000] (Message 4035)  Processed 56314928 pages for database '', file '' on file 1. [SQLSTATE 01000] (Message 4035)  Processed 2 pages for database '', file '' on file 1. [SQLSTATE 01000] (Message 4035)  The operating system returned the error '1359(An internal error occurred.)' while attempting 'SetEndOfFile' on ''. [SQLSTATE 42000] (Error 3634)  BACKUP DATABASE is terminating abnormally. [SQLSTATE 42000] (Error 3013)  Outcome: Failed  Duration: 01:45:31  Date and time: 2016-12-11 05:55:42 [SQLSTATE 01000] (Message 50000)  Date and time: 2016-12-11 05:55:42 [SQLSTATE 01000] (Message 50000).  The step failed.


When performing a backup dump using compression we may get an error like this.

It doesn’t always happen every backup. The problem appears to be related to the way that SQL server creates the backup file. When using compression, the SQL server pre-allocates space for the backup file and the data is then written into it. Since the data is compressed the backup takes less space than was allocated and SQL server instructs the filesystem to truncate the file (set the end of the file). On local disk this can happen quickly but over a network to a share it can take longer.


When the backup file is written to a SMB share the truncation can take longer than expected, especially if the amount of data to be truncated is quite large (i.e. it was compressed quite alot) and the request can timeout.


Resolution :

We can tweak the timeout by adding the following registry entry:

1. Click Start, click Run, type regedit.exe, and then click OK.
2. In Registry Editor, locate and then right-click the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters
3. On the Edit menu, point to New, and then click DWORD Value.
4. Type SessTimeout, and then press ENTER.
5. On the Edit menu, click Modify.
6. In the Edit DWORD Value dialog box, click Decimal under Base, type 300 under Value data, and then click OK.

The default SessTimeout setting is 50 seconds. This changes it to 300 seconds (5 minutes).

For more information check out:

The function used to truncate the file is SetEndOfFile :

Tuesday, 3 January 2017

Check the Active Cluster Node

 SELECT srvname as InstanceName , (CONVERT(varchar(200), (SERVERPROPERTY('ComputerNamePhysicalNetBIOS')))) ActiveNode , GETDATE ( ) CurrentDate from sys.sysservers where srvid=0 

Monday, 2 January 2017

Send an alert email whenever SQL Cluster Fail over happens

Pre requisites :

1. Database Mail should be configured
2. Need to have a valid SMTP server to send the mail

Steps as below.

1. Configure the Database Mail

      Refer Below Link

2. Create a Procedure

3. Create Table

3. Create a Job
         Step 1. Execute procedure to update the Table
         Step 2. EXEC msdb.dbo.sp_send_dbmail.


Concept :

                 One should get an alert email, whenever SQL Cluster fail over to other Node.
I have created a Table and hard coded the existing Instance Name and Active Cluster Node information in a table. There is an additional column with value as 1 for all the Active node instance and rest with value zero. So, whenever fail over happens Zero value for the respective Active node will get value as 1 and existing 1 as zero. While sending a mail, all active node instances will be selected.


Table is created on a centralized Server. All other servers will update data on the same. 

USE [DBA_Admin]
GO

/****** Object:  Table [dbo].[Current_AcitveCluster_Node]    Script Date: 1/2/2017 7:11:46 AM ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

SET ANSI_PADDING ON
GO

CREATE TABLE [dbo].[Current_AcitveCluster_Node](
[ActiveNode] [char](30) NULL,
[InstanceName] [char](30) NULL,
[Status] [bit] NULL
) ON [PRIMARY]

GO

SET ANSI_PADDING OFF
GO

Procedure is created on all individual instance. 

USE [DBA_Admin]
GO
/****** Object:  StoredProcedure [dbo].[usp_CheckActiveNode]    Script Date: 1/2/2017 7:14:52 AM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[usp_CheckActiveNode]
AS
BEGIN


SELECT srvname InstanceName
, (CONVERT(varchar(200), (SERVERPROPERTY('ComputerNamePhysicalNetBIOS'))))            NewNodeName
,  GETDATE ( )  CurrentDate
into #NewData
from sys.sysservers
where srvid=0

update [LinkedServer].DBA_Admin.dbo.Current_AcitveCluster_Node
set status = 0
where instancename = (select InstanceName from #NewData)
and status = 1

update [LinkedServer].DBA_Admin.dbo.Current_AcitveCluster_Node
set status = 1
from [LinkedServer].DBA_Admin.dbo.Current_AcitveCluster_Node O
join #NewData N
on N.InstanceName = O.InstanceName
and O.activeNode = N.NewNodeName


if object_id('tempdb..#NewData') is not null
drop table #NewData

END




Job is created on all individual instance. 


USE [msdb]
GO

/****** Object:  Job [SQL Alert for Cluster Failover]    Script Date: 1/2/2017 7:16:40 AM ******/
BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
/****** Object:  JobCategory [[Uncategorized (Local)]]    Script Date: 1/2/2017 7:16:40 AM ******/
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'[Uncategorized (Local)]' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

END

DECLARE @jobId BINARY(16)
EXEC @ReturnCode =  msdb.dbo.sp_add_job @job_name=N'SQL Alert for Cluster Failover',
@enabled=1,
@notify_level_eventlog=3,
@notify_level_email=0,
@notify_level_netsend=0,
@notify_level_page=0,
@delete_level=0,
@description=N'Send an alert mail whenever Failover happen on the SQL Cluster.',
@category_name=N'[Uncategorized (Local)]',
@owner_login_name=N'sa', @job_id = @jobId OUTPUT
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
/****** Object:  Step [Update the Table]    Script Date: 1/2/2017 7:16:40 AM ******/
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'Update the Table',
@step_id=1,
@cmdexec_success_code=0,
@on_success_action=3,
@on_success_step_id=0,
@on_fail_action=3,
@on_fail_step_id=0,
@retry_attempts=0,
@retry_interval=0,
@os_run_priority=0, @subsystem=N'TSQL',
@command=N'EXEC  [dbo].[usp_CheckActiveNode]',
@database_name=N'DBA_Admin',
@flags=0
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
/****** Object:  Step [Send an alert mail]    Script Date: 1/2/2017 7:16:40 AM ******/
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'Send an alert mail',
@step_id=2,
@cmdexec_success_code=0,
@on_success_action=1,
@on_success_step_id=0,
@on_fail_action=2,
@on_fail_step_id=0,
@retry_attempts=0,
@retry_interval=0,
@os_run_priority=0, @subsystem=N'TSQL',
@command=N'EXEC msdb.dbo.sp_send_dbmail
@profile_name = ''SQLProfile'',
@recipients=''DBTeam@XYZ.com'',
@query = ''set nocount on select InstanceName, ActiveNode from [LinkedServer].[DBA_Admin].[dbo].Current_AcitveCluster_Node_backup where status=1'',
@Body=''SQL Server Failed over to other Node. Find the Cluster Node details below.'',
@from_address= ''DBTeam@XYZ.com'',
@subject=''SQL Cluster Failover Alert'',
@execute_query_database = ''DBA_Admin''',
@database_name=N'DBA_Admin',
@flags=16
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id=@jobId, @name=N'Starts when SQL Server Agent starts on Passive Node',
@enabled=1,
@freq_type=64,
@freq_interval=0,
@freq_subday_type=0,
@freq_subday_interval=0,
@freq_relative_interval=0,
@freq_recurrence_factor=0,
@active_start_date=20161223,
@active_end_date=99991231,
@active_start_time=0,
@active_end_time=235959,
@schedule_uid=N'6dbda10f-3d49-4da7-9bcc-1d44a7eef1a6'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
    IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:

GO









How to move the User database file to different location

Run the following statement.

  1. Verify the file change by running the following query.

SELECT db_name (database_id) DBName,
physical_name AS CurrentLocation,
name as LogicalName, state_desc 
FROM sys.master_files 
WHERE database_id=100

2.     Take the Database Offline.

CHECKPOINT

USE master;
GO
ALTER DATABASE AdventureWorks2012
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE;
GO

Detach the database


3.     Move the file or files to the new location.

Physically move the file or use ROBO copy command

  1. For each file moved, run the following statement.

ALTER DATABASE AdventureWorks2012 MODIFY FILE (NAME = AdventureWorks2012_2, FILENAME = 'D:\USER_DATA\DATA_07\ AdventureWorks2012.ndf'); -- We have change the actual physical path

  1. Bring the Database online.

Attach the database; 

USE master;
GO
ALTER DATABASE AdventureWorks2012
SET READ_ONLY;
GO
ALTER DATABASE AdventureWorks2012
SET MULTI_USER
WITH ROLLBACK IMMEDIATE;
GO


  1. Verify the file change by running the following query.

SELECT db_name (database_id) DBName,
physical_name AS CurrentLocation,
name as LogicalName, state_desc 
FROM sys.master_files 
WHERE database_id=100