Tag Archives: failover cluster

Using cluster.exe with Hyper-V 2012 R2

I have a newer 2012 R2 Failover Cluster on which I was not able to move one of the Cluster Shared Volumes. I kept getting an error indicating that no other node could own the CSV:

Error draining node / moving CSV to another node
Error draining node / moving CSV to another node
"Error Code: 0x80071398 The operation failed because either the specified cluster node is not the owner of the group, or the node is not a possible owner of the group"

So I searched online and found website after website indicating that I needed to use cluster.exe in order to add my second node as a possible owner of the CSV, with the following syntax:

cluster resource “csv_a1” /listowners

I was not able to use cluster.exe and eventually found that it had been deprecated.

Error when running cluster.exe in Server 2012 R2
Error when running cluster.exe in Server 2012 R2

It took me a while of searching and using the wrong PowerShell cmdlets to find this post on Installing the Failover Cluster Feature and Tools in Windows Server 2012

In the blog post it indicates that you can install the Windows Feature RSAT-Clustering-CmdInterface which “Includes the deprecated cluster.exe command-line tool for Failover Clustering. This tool has been replaced by the Failover Clustering module for Windows PowerShell.”

Well I already had the Failover Clustering module for Windows PowerShell installed, and I couldn’t figure out how to make this second node a possible owner of the CSV, so I installed RSAT-Clustering-CmdInterface to use cluster.exe

Installing RSAT-Clustering-CmdInterface to use cluster.exe
Installing RSAT-Clustering-CmdInterface to use cluster.exe

Once this was installed I was able to use cluster.exe to add my second node as a possible owner of “csv_a1”. Then I could drain the Node, and finally reboot it which was what I was trying to do before running into this issue.

Hopefully this helps someone else with Windows Server 2012 R2 Hyper-V Failover Clustering if they have this issue and are also getting errors when trying to use cluster.exe

Error Draining Roles in Hyper-V 2012 R2 Failover Cluster

I was trying to do some maintenance on my Hyper-V 2012 R2 Failover Cluster and I was unable to drain one of the nodes in order to install Windows Updates.

An error occurred pausing node 'RDU-HV01'
Error Code: 0x80071748
"The requested operation can not be completed because a resource has locked status"
I love specific error messages.
I love specific error messages.

In Hyper-V Manager the VM was stuck in a “Backing Up” status, and this was after I manually Live Migrated all other VMs to my second node.

status-backing-up

When trying to manually Live Migrate this VM I was prompted to override the locked resources and try again… and just like any good System Administrator I saw an opportunity to try and force something to work, while potentially producing an extremely horrible outcome, so I naturally clicked “Yes”. YOLO.

The virtual equivalent of kicking the computer.
The virtual equivalent of kicking the computer.

But it still failed with an error!

Failed to Live migrate virtual machine 'VM_name'
The action 'Move' did not complete.
Error Code: 0x80070057
The parameter is incorrect
#fail
#fail

I restarted the VM but that had no effect so I shut it down.

Now that the VM was shut down I could restart the node which I did from a command prompt utilizing the shutdown command. However the node would not restart – it was stuck somewhere in the shutdown process. I could still see it in Server Manager and when I did a systeminfo from the command prompt the System Boot Time told me that it had not restarted yet. Since I was doing this remotely, I could not go into the server room to shut down the server and I had no OoB management configured so I had to do a little digging. I found that others with this issue were able to fix it by restarting the Hyper-V Virtual Machine Management service. I tried stopping this service (vmms is the Service Name) from Server Manager of my Windows 8.1 laptop but it did not seem to work.

I then opened a command prompt to try using SC.exe to stop or restart the vmms service. By the time I figured out the correct syntax, I noticed that the node had just gone down for a restart. Maybe it timed out, or maybe my command from Server Manager just took a minute to go through.

The correct syntax would have been:

sc \\rdu-hv01 query vmms
sc \\rdu-hv01 stop vmms

The VM which was stuck in the “Backing up…” state was automatically moved to my second node and the first node restarted itself. The VM which was stuck started properly on the second node and the status for “Backing up…” was no longer showing.

Once the first node came back up from its restart I was able to Pause and Drain Roles to go on with my maintenance.

If this happened again I would suggest shutting down the VM which is stuck in the “Backing up…” status. Then Live Migrate everything else (don’t forget your storage!) so that the only thing on this node is the VM that is off. I would then attempt to restart the vmms service. If that does not work, restart the node.