Storage vMotion manual repair (ESX 3.5)

Two years ago (approx) VMware released Storage vMotion. At the time it was a remote-command-line-only utility and kind of rough. There were not many checks plus it had a bug that was triggered by another bug – for whatever reason, some VMs created via GUI would occasionally get their .vmx configuration file tagged as if they had thin-provisioned disks when they didn’t.
No big deal, except Storage vMotion v1 would crash during the final hand-off when it found that flag, leaving the VM crashed with the hard drive in an unusable state. “Failed to unstun VM after disk reparent”
Storage vMotion makes a snapshot of the VM, copies all the file over (except the new snapshot) to the new directory, makes another snapshot and copies the first snapshot, quiesces the file system and copies the last snapshot, then points the VM to the new location. The bug keeps it from pointing the VM to the new location.

These are my notes at the time to recover from that error.

Two Storage VMotion (DMotion) tasks failed with an error :

To resolve this, first verify it is the same problem by checking the files in the old and new directories. To verify the LUNs, check the properties of the VM:

Then use SCP (or open an SSH session and check manually) to look at the VM directory on each LUN.
The old directory should have only VMDK files :

The New Lun will look like a normal VM directory with vmx, vmsd, nvram and vmdk files:

At this point you simply need to edit the vmx file to point to the new location and the VM should launch. First copy the new directory in the clipboard to save having to type it.

Next open a SSH session to the host (any host with access to the LUN) and change to the new directory. It is also a good idea to backup the vmx file, just in case.

Now use VI and check the old directory entries.

Delete the old directory using “x” up to the two forward slashes “//” then delete the “DMotion-scsi0:0x_” information up to the vmdk filename. The entries should now look like this.

Now paste the new directory information. Use “i” to enter insert mode, position the cursor after the first quote and then paste the directory info into the window. Hit “esc” then “:wr” to save and “:q” to exit.

You should then be able to start the VM.

A lot has changed for vSphere; Storage vMotion is now available from the vSphere client as well as the remote command line and there is a new file (vmsd) used to track the snapshot tree. I’m told the thin-provision-flag bug only occurred for a short period of time, I don’t know if anything was done to correct the flag in exiting VMX files.

This entry was posted in Computing, Virtualization, VMware and tagged , . Bookmark the permalink.

Leave a Reply