Splitting up a large MDF file

I wanted to break-up a single 500GB MDF file into 5 files of around 100GB each, so created 4 NDF files.

I set autogrowth to 1024 MB for the NDF files and OFF for the MDF file.

In a SQL Job I used code like …

DBCC SHRINKFILE (N'MDFLogicalFileName', EMPTYFILE);

Which after the second weekend left about 88GB in the MDF file.

--datafiles.sql

select Physical_Name,
	ROUND(CAST((size) AS FLOAT)/128,2) Size_MB,
	ROUND(CAST((FILEPROPERTY([name],'SpaceUsed')) AS FLOAT)/128,2) Used_MB,
	convert(int, ROUND(CAST((FILEPROPERTY([name],'SpaceUsed')) AS float)/128,2) / ROUND(CAST((size) AS float)/128,2) * 100) Used_Pct
FROM sys.database_files
where physical_name not like '%ldf%'
order by physical_name;

I cancelled and deleted the job.

Over the next 3 week nights I reduced its physical size to 300GB, 200GB, then 100GB using a Job step like …

DBCC SHRINKFILE (N'MDFLogicalFileName', 100000);

I set MDF autogrowth to match the NDF files, so the five would naturally balance (size wise) over time.

Lastly I set up a nightly job to rebuild the most fragmented indexes (Thanks again Ola).

Update Reporting from Live (part 1 of 2)

I can hardly believe I am revisiting my old “Smart Diff backup / restore” project from 2015 – in this – the age of the cloud!

This time around I chose to use native commands instead of Ola’s scripts as the architecture was simpler. And SQL Server 2008r2 (yes, I know).

To start, I made a SQL job on Live “Backup for Reporting” to create a local backup …

Step-1 “Jump to Diff or Full backup”

This step attempts to manage smart diff-backups. If it is Friday, or there is no current full backup, or if some 3rd party has taken a full backup (rendering our full backup obsolete), then a full backup is taken via step-2.

If none of the above are true, over an hour can be saved by jumping directly to Step-3.

(NOTE: Step-1 was set to go to Step-3 on success, or the next-step on failure).

/* 1. Do a FULL backup if its Friday */

	SET DATEFIRST 7 /* FirstDayOfWeek = Sunday, so Friday = 6 */
	IF (SELECT DATEPART(WEEKDAY, GETDATE())) = 6 /* its Friday */	
	BEGIN
		RAISERROR ('Force this Job-Step to Fail - Its Friday', 16, 1)
		Return
	END

/* 2. Do a FULL backup if there aren't any */
 
	DECLARE @backups char(1)
	EXEC @backups = [titan].[master].[sys].[xp_cmdshell] 'DIR /b X:\SQLBackups\CMI_ProDB_Audit_Live\*FULL.bak'
	IF (select @backups) > 0 /* 0 = found, 1 = not found */
	BEGIN
		RAISERROR ('Force this Job-Step to Fail - There are no FULL backups', 16, 1)
		Return
	END

/* 3. Do a FULL backup if the LSNs don't match */

	IF	(SELECT isnull(MAX(differential_base_lsn), 1)
		FROM [Titan].[master].[sys].[master_files]
		WHERE physical_name like '%Audit%')
		!=
		(SELECT isnull(MAX(differential_base_lsn), 1)
		FROM [SQLREPORTING].[master].[sys].[master_files]
		WHERE physical_name like '%Audit%')
	BEGIN
		RAISERROR ('Force this Job-Step to Fail - The LSNs do not match', 16, 1)
		Return
	END

/* Else jump to the DIFF-backup step */

	Select 'We can do a DIFF backup'
	Print 'We can do a DIFF backup'

Step-2 “Create a new FULL backup”

This step creates a Full backup locally, overwriting any that already exist. When complete a Diff backup is taken via step-3.

A Diff backup immediately after a Full backup is not strictly necessary, but ensures that a) the restore process can be simple and robust (IE: a Full backup is always restored, then a Diff backup is always restored), and b) that we never have a Diff backup older that the partnering Full backup.

BACKUP DATABASE [LiveDatabaseName] 

TO DISK =  N'V:\SQLBackups\LiveDatabaseName\LiveDatabaseName_FULL.bak'

WITH CHECKSUM, COMPRESSION, INIT;

Step-3 “Create a new DIFF backup”

This step creates a Diff backup (containing changes since the last Full backup), overwriting any Diff backup for this database that already exists.

BACKUP DATABASE [LiveDatabaseName]

TO DISK =  N'V:\SQLBackups\LiveDatabaseName\LiveDatabaseName_DIFF.bak'

WITH CHECKSUM, COMPRESSION, DIFFERENTIAL, INIT;

Step-4 “Start the Restore job”.

Kicks-off the restore job on the Report Server.

EXECUTE [ReportServerName].[msdb].[dbo].[sp_start_job] 'Restore LiveDatabaseName';

Next I will detail the steps in the “restore” job on the report server.

Update Reporting from Live (part 2 of 2)

… I created a SQL Job on the Reporting server called “Restore from Live”.

Step-1 “Kill any connections”

Before a database can be restored it needs to be unused. Removing connections in this way is more reliable then changing the database to “Single-user-mode”.

DECLARE @kill VARCHAR(8000) = '';

SELECT  @kill = @kill + 'kill ' + CONVERT(VARCHAR(5), spid) + ';'
	FROM [master].[dbo].[sysprocesses]
	WHERE dbid = DB_ID('LiveDatabaseName')
	AND spid > 50;

EXEC (@kill);

Step-2 “Full Restore”

This step restores a Full backup. Changing the database name and file locations as required.

When complete the database will be left in a “Restoring” state. *It can be brought online either by completing the next step or by manual recovery EG: “Restore Database [LiveDatabaseName] with Recovery;”.

RESTORE DATABASE [LiveDatabaseName] 

FROM DISK = N'\\LiveServerName\SQLBackups\LiveDatabaseName\LiveDatabaseName_FULL.bak' 

WITH NORECOVERY, REPLACE;

Step-3 “Diff Restore”

This step restores a Diff backup similar to the last step, however it brings the database back online after completion. If this step ever fails see * above.

RESTORE DATABASE [LiveDatabaseName] 

FROM DISK = N'\\LiveServerName\SQLBackups\LiveDatabaseName\LiveDatabaseName_DIFF.bak' 

WITH RECOVERY;

Step-4 “Switch to Simple Recovery Mode”

This step changes the database to Simple recovery mode where the log-files are re-used and do not require management (EG: regular log backups). This is appropriate for report servers where the data is already backed up from live “for recovery” (IE: outside of the backups detailed in these 2 posts).

ALTER DATABASE [LiveDatabaseName] set recovery SIMPLE;

Step-5 “Remove Orphans”

This deprecated command changes the password for the “mssql” login to match the password from the “mssql” user. Login passwords are not captured by backups.

sp_change_users_login 'auto_fix', 'mssql';

Footnote

I wrote these 2 jobs using minimal variables and dynamic SQL, and using a generous number of jobs and job-steps, in the hope that this will be robust and easy to manage. (And because I really like simplifying such things)