Tuesday, September 20, 2011

SCVMM "Migration Failed"- how to clear

When working with Microsoft SCVMM (System Center Virtual Machine Manager) in a Hyper-V virtualization environment, we have run across scenarios in which the failed migration of a VM from one host to another may leave the VM in a failed state. Specifically, in the SCVMM console, the VM has a red 'X' beside it with the "Migration failed" under the status column. While the VM is in this failed state, SCVMM will only allow you to Repair, Delete, or View networking of the VM. Furthermore, if you right-click the VM and select 'Repair', the option to 'Ignore' is unavailable, and the 'Retry' and 'Undo' options fail. Even though you may have corrected the problem outside of VMM, for example using the Hyper-V Manager console, the repair options yield no results.

We have read many articles and posts on how to deal with this scenario, such as removing the host from VMM and adding it back. Or using SQL Server Management Studio to hide failed jobs (NOT a good idea as it can eventually lead to poor SCVMM performance since the VMM cleanup process will skip old jobs that are hidden!). Once you have corrected the problem and exhausted all possibilities to remedy the situation, here is a handy PowerShell script that will help you out of this jam, which we pieced together specifically for this situation.

First, a little peek under the hood:

- When the migration fails, the ObjectState field in SQL for the VM gets set to 220, indicating a failed state.
- If the ObjectState field for a VM is set to 0 (zero) or 1, SCVMM will re-check the VM state. If you have repaired the VM and set the ObjectState in the VirtualManagerDB to 0, VMM will immediately notice the VM has been fixed, and all is well.

WARNING
I have created this script and used it in a production environment, but use it at your own risk. Be sure to back up the VMM database just before using this so you have an out.

Script name: Clear-VMMFailState.ps1
PS Requirements: I have only tested this under PowerShell 2.0, and have verified that it doesn't require any special PS modules such as the VMM or SQL modules. It worked fine for us in the out-of-box PowerShell in Win2008R2 with no modules loaded. This should also work remotely with no problem.

We encourage feedback on your experience with this script, and any suggestions you may have on how to improve it.

Script Help (get-help ./Clear-VMMFailState.ps1 -Detailed)
=====================================================================
Synopsis:
This script will clear the failed state of a Virtual Machine in VMM.
Description:

!!!WARNING!!! back up your VMM SQL database before using this script. You have been warned!
This should only be used under the following circumstances:
- If a failure in a VMM job has a VM in a 'Failed' state
- AND -
- The repair function in VMM won't allow you to 'Ignore'
- AND - 
- The repair function in VMM fails when performing a 'Retry' or 'Discard'
- AND -
- You have corrected the issue outside of VMM (such as with Hyper-V Manager MMC etc.)
- AND -
- You have already tried refreshing the VMM Server, VM Host, and VM from within the VMM console or shell

Parameters:
-VMname <String> - the name of the Virtual Machine (as it appears in SCVMM)
-VmmDBserver <String> - this is the FQDN of the server that hosts the SQL instance of the SCVMM database
- VmmDBname <String> - this is the Database name of the SCVMM database; if not specified, this defaults to the value of 'VirtualManagerDB'
-IntegratedAuth <Switch> - if this switch is specified, integrated Windows authentication is used for the connection to the SCVMM database. If not specified, you must use the -UserName and -Password switches to authenticate to SQL
-UserName <String> - The username used to authenticate to the SCVMM database. If not using the -IntegratedAuth switch, this is required
-Password <String> - Password for the user specified with -UserName

Example 1:
./Clear-VMMFailState.ps1 -VMname TestVM01 -VmmDBserver VmmSQLserver.domain.com -IntegratedAuth
>> Connects to the SQL server that hosts the SCVMM database (VmmSQLserver.domain.com) and the default VMM database (VirtualManagerDB) using Integrated Authentication with currently logged on credentials, then clears the failed state of the Virtual Machine named 'TestVM01'.


Example 2:
./Clear-VMMFailState.ps1 -VMname TestVM01 -VmmDBserver VmmSQLserver.domain.com -VmmDBname MyVmmDB -UserName domain.com\vmmAdmin -Password MyPassWord
>> Connects to the SQL server that hosts the SCVMM database (VmmSQLserver.domain.com) and the VMM database named 'MyVmmDB' using the specified username and password, then clears the failed state of the Virtual Machine named 'TestVM01'.


And now for the script... http://powershell.com/cs/media/p/12426.aspx