83

Question:

How should I use the new AWS EC2 classes (r3, i2) with my existing AMI without recreating the whole system setup?

The new EC2 classes support only HVM based virtualization but I have only PVM AMI images.

George Stocker
  • 57,289
  • 29
  • 176
  • 237
user3760692
  • 839
  • 1
  • 7
  • 4
  • 3
    Just tried this with CentOS but end up ins tep 10 with following error: /dev/xvdf does not have any corresponding BIOS drive. – Tobias Aug 27 '14 at 15:11
  • Further to what @ROMANARMY said, I reckon the answer *should* be split out into a proper answer. Then you can accept it so the question won't show up as unanswered. See http://stackoverflow.com/help/self-answer – Tim Oct 08 '14 at 08:28
  • @Tobias Zander did you ever get this working with CentOS? – dhempler Jan 02 '15 at 21:55
  • 1
    @dennis.hempler We didn't invest a lot more time and just created new instances – Tobias Jan 13 '15 at 11:46

2 Answers2

77

Answer:

  1. Start an Ubuntu HVM linux, any version, new

  2. Start an Ubuntu / with my existing AMI / PVM linux, and install grub packages on them: apt-get install grub-pc grub-pc-bin grub-legacy-ec2 grub-gfxpayload-lists

  3. Stop PVM linux

  4. Detach root (/dev/sda1) partition at PVM linux

  5. Attach PVM linux root partition to running HVM linux somewhere, e.g.: /dev/sdf

  6. On HVM linux: mkdir -p /mnt/xvdf && mount /dev/xvdf /mnt/xvdf

  7. rsync -avzXA /boot/ /mnt/xvdf/boot/

  8. mount -o bind /dev /mnt/xvdf/dev && mount -o bind /dev/pts /mnt/xvdf/dev/pts && mount -o bind /proc /mnt/xvdf/proc && mount -o bind /sys /mnt/xvdf/sys

  9. chroot /mnt/xvdf

  10. grub-install --no-floppy --recheck --force /dev/xvdf

  11. update-grub2

  12. exit chroot: CTRL+D

  13. stop HVM Linux

  14. detach /dev/sda1 original root AND detach /dev/sdf PVM root

  15. attach PVM root to HVM linux as /dev/sda1

  16. Start HVM linux, voilà!

  17. Create a new AMI image from the running HVM linux, it will be HVM virtualized.

Shagglez
  • 1,522
  • 3
  • 21
  • 38
divyenduz
  • 2,037
  • 19
  • 38
  • Worked perfectly...once I created the HVM instance on the exact same availability zone as the PVM volume. – cloudsurfin Dec 17 '14 at 23:03
  • 2
    I get a the following warning when running grub-install: `warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged.. /usr/sbin/grub-bios-setup: error: will not proceed with blocklists.` Then, everything looks good but it doesn't boot or give me any clue, why it is not booting. – Paulo Neves Feb 18 '15 at 00:52
  • 3
    This worked perfectly for me, thank you so much! I did get the scary grub warnings that Paulo Neves reported above, but my new instance booted fine anyway. – GaryO Feb 24 '15 at 15:29
  • I have an AMI Linux PVM instance and I am trying to migrate it to a new HVM AMI Linux instance. I tried to apply your solutions but I am stuck at step 10. Could someone please elaborate on how to do this for a new AMI Linux instance instead of Ubuntu? Thanks! – tolgamorf Mar 26 '15 at 13:01
  • I worked it out by manually transferring all files to the new instance. http://serverfault.com/a/678867/165528 – tolgamorf Mar 27 '15 at 21:23
  • Works like a charm! Maybe add step 5.5 "On HVM linux: sudo su -" as the mount command requires the "root" user and can't be done using sudo. – Anders Apr 13 '15 at 15:06
  • @tolgamorf. You would probably have to install grub first on AWS Linux: yum install grub -y – Anders Apr 13 '15 at 15:08
  • I'm having the same issue as tolgamorf. Installing grub first on AMS Linux helped, but didn't solve the problem. Instead of getting a message that grub didn't exist, I got this message (had to omit --force command because not supported): grep: /boot/grub/device.map: No such file or directory mv: cannot stat ‘/boot/grub/device.map’: No such file or directory Probing devices to guess BIOS drives. This may take a long time. Unknown partition table signature /dev/xvdf does not have any corresponding BIOS drive. – Jesse Barnum May 07 '15 at 15:23
  • Just upgraded my AMI thanks to this tutorial. Works perfectly at one exception: Docker. I had to completely uninstall and reinstall it to make it up and working again. – koleror Jul 17 '15 at 13:43
  • Does not work for me in step 5; I get message Error attaching volume: The instance configuration for this AWS Marketplace product is not supported. Please see http://aws.amazon.com/marketplace/pp?sku=8fvdn95s5ev33cprr62nq3q7t for more information about supported instance types, regions, and operating systems. – Mr. Developerdude Nov 16 '15 at 23:36
  • @PauloNeves - I am encountering the same issue. Were you able to resolve it? – curtp Jan 13 '16 at 23:34
  • @curtp: It was a long time ago so I dont really rememeber, but I have the idea it was something about the drive with our without letter that I was putting wrong and when i followed exactly it worked. – Paulo Neves Jan 14 '16 at 11:47
  • Just upgraded a Ubuntu PV to a Ubuntu HVM machine as per the procedure, confirming it still works perfectly well. – Sandro Gržičić Sep 29 '16 at 07:43
  • I confirm this works for Ubuntu 14.04 LTS. Some apps need extra kernel modules that depends on architeture. For instance, if you have docker installed, it may be necessary to run this command after everything on the new HVM instance: "sudo apt-get install linux-image-extra-$(uname -r)" – Marcelo Liberato Nov 25 '16 at 13:42
  • That's smart! Have a fresh HVM instance, mount the PV root volume, copy the HVM boot, then chroot and update grub. Simple! +1000 – jweyrich Feb 01 '17 at 12:40
  • Cannot find pkgs in setp2 in ubuntu 10.04 – petertc May 24 '17 at 12:35
  • 1
    Instance is unreachable when using a newer instance type (such as C5). There is no `xvd*` on these instances, there is only `nvme*` naming conventions. Replacing `xvd` with the correct `nvme` does not work. :( – Alex Feb 22 '18 at 03:28
  • I used this for a Rackspace XEN CentOS server to run it on a local VirtualBox. One issue - I had to clear the PV /boot/ directory completely. Not sure what it was but some cruft in there was preventing booting (could not mount disk). – jpgeek Jul 15 '20 at 02:02
3

The answer from @divyenduz works but needs some cleanup and clarification for modern (circa 2019) AWS EC2. Importantly, modern instance classes translate the device name differently.

Here are my modified steps.

For clarity, nodes are:

  • Original PVM node that you want to upgrade to HVM is "PVM01"
  • New HVM node that you will migrate PVM01's root disk to is "HVM01"

BEFORE PROCEEDING: Back up Original Node PVM01

  1. Install prerequisites on Node PVM01

    • install grub packages on PVM01

      apt-get install  grub-pc  grub-pc-bin grub-legacy-ec2 grub-gfxpayload-lists
      
  2. Stop node PVM01

    • Create snapshot of /dev/sda1
    • OR create AMI of entire node
  3. Create snapshot of original root volume on PVM01 and create new volume from this snapshot
    • Create snap of /dev/sda1 on PVM01
    • Create vol from this snap
    • Label each resource accordingly
      • Tag: Name; Value: pvm01:/dev/sda1:snapshot-for-conversion-to-hvm:2019-07-01
      • Etc
  4. create-instance: New Ubuntu HVM instance "HVM01". Any instance class should work; however, device name may be different, see notes, below
    • I used Bionic Ubuntu 18.04 HVM AMI ID ami-0a313d6098716f372 and instance class C5.XLARGE
  5. Attach PVM01 root partition (new volume from previous step) to new HVM01 /dev/sdf

    • NOTE: Older instance classes such as C3 will translate the volume name to /dev/xvdf
    • NOTE: Newer instance classes such as C5 will translate the volume name to /dev/nvme1
      • On newer instance classes, the root vol will be /dev/nvme0 -- ZERO
      • On newer instance classes, the attached, secondary vol will be /dev/nvme1 -- ONE
    • Make a note of the attached volume device name

    ssh PVM01 sudo fdisk -l

  6. On HVM01:

    # For xvdf, e.g. on C3.XLARGE
    DEVNAME=xvdf1
    # For nvme, e.g. on C5.XLARGE
    DEVNAME=nvme1n1
    mkdir -p /mnt/${DEVNAME} && mount /dev/${DEVNAME} /mnt/${DEVNAME}
    rsync -avzXA /boot/ /mnt/${DEVNAME}/boot/
    mount -o bind /dev /mnt/${DEVNAME}/dev && mount -o bind /dev/pts /mnt/${DEVNAME}/dev/pts && mount -o bind /proc /mnt/${DEVNAME}/proc && mount -o bind /sys /mnt/${DEVNAME}/sys
    chroot /mnt/${DEVNAME}
    grub-install --no-floppy --recheck --force /dev/${DEVNAME}
    update-grub2
    
  7. Exit chroot with CTRL+D

  8. Stop HVM01
  9. Detach both volumes
    • detach /dev/nvme0 PVM01 root
      • NOTE: This would be /dev/sda1 on a C3 instance class
    • detach /dev/${DEVNAME} (DEVNAME from above script)
  10. attach PVM01 root Volume to HVM01 as /dev/sda1
    • Once again, the /dev/sda1 name is in the console; this name will get translated to /dev/nvme0 or /dev/xvda1 depending on how modern the instance class is
  11. Start HVM01
  12. OPTIONAL: Create a new AMI image from the the now-running HVM01, it will be HVM virtualized.
JDS
  • 1,869
  • 1
  • 15
  • 17
  • This solution worked for me. Remember to recreate the `DEVNAME` variable after the `chroot` command. – Justin Lee Jul 29 '19 at 17:56
  • Something we ran into is that some of those older systems on PV instances might not have the enhanced networking feature enabled...we could get these to boot on HVM, but the network was unusable. Curious if there's an easy solution here? – Valerie R Aug 16 '21 at 20:37