Reinstalling Xenserver and Preserving Virtual Machines on the Local Disk(s)

Summary

This document describes how to reinstall a XenServer host and preserve virtual machines (VMs) on the local storage.

Requirements

• Basic XenServer knowledge

• Basic Logical volume Manager (LVM) knowledge

• XenServer host (4.1 or 5.0) with two or more local hard drives

Background

Sometimes reinstallation of XenServer is required to obtain a clean system state or to recover from a serious failure (such as a database file corruption), where VMs residing on local storage must be preserved and recovered within a reasonable amount of time.

Procedure

To achieve the above goal, perform the following steps:

1. If at all possible, collect VM metadata and store it offline (such as on a USB drive). This step requires a working XAPI stack – that is, that you can run xe commands.
xe vm-export metadata=true --multiple filename=VMEXPORT

This creates a set of files (VMEXPORT0 and so on) containing VM metadata. Having this metadata available speeds up VM recovery considerably because it creates the VM itself and restores the mappings between VMs and their disks (VDIs) – something that otherwise must be done manually.

2. Using XenServer installation media, reboot the host and launch the XenServer installer.

3. Choose Install or upgrade XenServer host ,as shown in the screen shot below:

4. You are prompted to choose between performing a clean reinstallation or freshening the existing installation. Choose the clean reinstall option.

5. Proceed to the point where the installer asks which drive to use for Dom0 installation (See Note on XenServer hosts with one hard drive section in the end of this document).

6. Select the drive to use for Dom0 installation (usually the first detected physical hard drive). Press Ok to proceed.

7. Select the storage for virtual guests. It is important NOT to select any drives in this screen so that the installer skips those during drive/partition initialization and thereby preserves the data on those partitions, as shown below (See Note on XenServer hosts with one hard drive section in the end of this document.

8. Press Ok to proceed.

9. A warning appears advising that no VM storage has been chosen. This is normal and can be ignored. Press Continue.

10. Follow the rest of the installation process as usual.

11. Once reinstalled, log in to the host using an SSH client (such as PuTTY). One can perform the following commands from XenCenter, however given the console constraints the output may be difficult to read as compared to an SSH client.

12. Restore the Physical Volume (PV) (introduce Storage Repository [SR]) using the commands below.

a. Using pvscan, identify the Volume Group (VG) name on the local drive:
# pvscan

As you can see from the output above, the VG name of the local drive /dev/sda3 is VG_XenStorage-39baf126-a535-549f-58d6-feeda55f7801 . The VG name contains the SR Universally Unique Identifier (UUID) that resides on this storage media. In this case, the UUID is 39baf126-a535-549f-58d6-feeda55f7801. 

b. Re-introduce the SR using the UUID detected above:
# xe sr-introduce uuid=39baf126-a535-549f-58d6-feeda55f7801 type=lvm name-label=”Local storage” content-type=user 
The command above sets up database records for the SR named “Local storage”.


c. Locate the SCSI ID of the device/partition where the SR data is stored:
# ls -l /dev/disk/by-id/

In this case, the SCSI ID of the device /dev/sda3 is scsi-SATA_ST380815AS_6QZ5Z1AM-part3. This is the device name to be used in the next command where a PBD (physical block device – a connector between the XenServer host and the SR) is created.

d. Create a PBD (see above) for the SR. Start with finding out the host UUID for the local host:
# xe host-list

e. Create the PBD using the device SCSI ID, host UUID and SR UUID detected above:
# xe pbd-create sr-uuid=39baf126-a535-549f-58d6-feeda55f7801 device-config:device=/dev/disk/by-id/scsi-SATA_ST380815AS_6QZ5Z1AM-part3 host-uuid=83f2c775-57fc-457b-9f98-2b9b0a7dbcb5
aec2c6fc-e1fb-0a27-2437-9862cffe213e

f. Attach the PBD:
# xe pbd-plug uuid=aec2c6fc-e1fb-0a27-2437-9862cffe213e
At this time, the SR should be connected to the XenServer host and be visible in XenCenter.

13. To restore VM-to-VDI mappings (which VMs use which disks), two alternatives are available:

• Importing VM metadata from backup copies

• Restoring the mappings manually, if metadata backup is not available

This subsection describes how to restore the mappings using VM metadata backup.

a. Run the following command to reinstate the VM metadata (VM records, VCPU/RAM allocation, VDI mappings):
# xe vm-import metadata=true filename=<METADATA-BACKUP-FILE>


b. If a warning is shown about no default SRs, set your local SR to be the default one (in the Graphical User Interface).

14. In the event of missing VM metadata backup information, the following procedure can be performed to restore VMs and disk mappings.

a. Restore VMs using available templates. Examples below show a real life situation with Graphical User Interface screenshots. Create a VM using a template (unselect Start VM automatically – this is unnecessary).

b. Delete default drives in the VM’s Storage tab. This action results in an empty VM with no storage attached. Reattaching VDIs can be a big challenge in large installations where many VMs exist and may have multiple VDIs attached. 

For example, two Debian Linux VMs were created on a XenServer host and subsequently the host was reinstalled, preserving the local storage. Having reintroduced the local storage, 4 VDIs are available – 2x4Gb and 2x512Mb – a root and a swap partition per VM.

c. Use the Graphical User Interface to create the VMs as described above and then attach available VDIs – see the screen shot below.

Note for XenServer hosts with one hard drive

The above procedures does not work on XenServer hosts with one local hard drive only because the system automatically assumes /dev/sda (on HP systems /dev/cciss/c0d0p0) as the Dom0 partition and /dev/sda3 (on HP systems /dev/cciss/c0d0p3) as the VM storage partition and cannot display the VM storage selection dialogue at all, rendering the above procedure impossible.

To circumvent this, you must delete the XenServer internal database and recreate it from the beginning, using the procedure above starting from step 12.

Stop XAPI service (to release the database file locks):

# service xapi stop

Delete the Xenserver database:

# rm /var/xapi/state.db

Restart XAPI service – this creates a new XenServer database. Bear in mind that this data base does not contain any storage/network/VM related data and that everything must be restored manually:

# xe-toolstack-restart

Follow the procedure above from step 12.