Wix doesn't remove previous version of burn exe during major upgrade

Question

I have created a wix exe using burn bootstrapper. When I try to do a major upgrade on it, the new version gets installed. The features missing in the new upgrade are also removed from the existing folder structure. But in "Add or Remove Programs" I can still find both the installations. I have incremented the version from 1.0.0.0 to 1.0.1.0 in Bootstrapper project's Bundle.wxs file. I have also incremented the version in Setup project's Product element from 1.0.0.0 to 1.0.1.0. I didn't change the upgrade code in either of these files. I also created new GUID for Product ID in Setup project's Product.wxs. What am I doing wrong here? Why is the previous version still there in Add or Remove Programs? Please advice.

Below is the Bootstrapper project's Bundle.wxs.

<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi" xmlns:util="http://schemas.microsoft.com/wix/UtilExtension" xmlns:bal="http://schemas.microsoft.com/wix/BalExtension">
  <Bundle Name="Kube Installer" Version="1.0.0.0" Manufacturer="Zone24x7" UpgradeCode="DA4CFB28-4441-43BE-9E8E-0E3C92206324">
    <BootstrapperApplicationRef Id="ManagedBootstrapperApplicationHost" >
      <Payload SourceFile="..\CustomBA\BootstrapperCore.config"/>
      <Payload SourceFile="..\CustomBA\bin\Release\CustomBA.dll"/>
      <Payload SourceFile="..\CustomBA\bin\Release\GalaSoft.MvvmLight.WPF4.dll"/>
      <Payload SourceFile="C:\Program Files (x86)\WiX Toolset v3.8\SDK\Microsoft.Deployment.WindowsInstaller.dll"/>
    </BootstrapperApplicationRef>
    <WixVariable Id="WixMbaPrereqLicenseUrl" Value=""/>
    <WixVariable Id="WixMbaPrereqPackageId" Value=""/>
    <Chain>
      <MsiPackage SourceFile="..\KubeInstaller\bin\Release\KubeInstaller.msi" Id="KubeInstallationPackageId" Cache="yes" Visible="no"/>
    </Chain>

  </Bundle>
</Wix>

Below is the Setup project's Product.wxs.

<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">

  <Product Id="70F65E5D-F307-4155-8847-9E5A66024DE7" Name="Installer" Language="1033" Version="1.0.0.0" Manufacturer="Zone24x7" UpgradeCode="E90569B5-372E-45BB-B101-58E0B75AB5C7">
    <Package InstallerVersion="200" Compressed="yes" InstallScope="perMachine" Platform="x64"/>

    <Property Id="JAVA_CURRENT_VERSION64">
      <RegistrySearch Id="JRE_CURRENT_VERSION_REGSEARCH64" Root="HKLM" Key="SOFTWARE\JavaSoft\Java Runtime Environment" Name="CurrentVersion" Type="raw" Win64="yes" />
    </Property>

    <MajorUpgrade DowngradeErrorMessage="A newer version of [ProductName] is already installed." />
    <MediaTemplate />

    <Feature Id="ProductFeature" Title="Installer" Level="1" >
      <ComponentGroupRef Id="NewFilesGroup2"/>
    </Feature>
  </Product>

  <Fragment>
    <Directory Id="TARGETDIR" Name="SourceDir">
      <Directory Id="ProgramFilesFolder">
        <Directory Id="InstallDir" Name="Installer">
            <Directory Id="MyProgramDir2" Name="KKSDKService" />
        </Directory>
      </Directory>
    </Directory>
  </Fragment>


    <Fragment>
        <DirectoryRef Id="MyProgramDir2" />
    </Fragment>
    <Fragment>
        <ComponentGroup Id="NewFilesGroup2">
            <Component Id="cmpB40D9BC07A10F4C7CA1D590EA22BC096" Directory="MyProgramDir2" Guid="6CEF1581-D3B4-43EC-A561-8E35CFD9AB67">
                <File Id="filCD18463F625BD1E6DABFF218E37514D9" KeyPath="yes" Source="$(var.MyDir2)\BarcodeScannerPlugin.jar" />
            </Component>
            <Component Id="cmpCEA7123B4B5BD564DC3955797750BDBD" Directory="MyProgramDir2" Guid="0828F282-7F66-4550-9D8B-B9DD914F50BA">
                <File Id="fil1C3696A75F0BD878925FCA64111719CE" KeyPath="yes" Source="$(var.MyDir2)\KKSDK.jar" />
            </Component>
            <Component Id="cmp754951538EA86C396458E93F7C5EE8D8" Directory="MyProgramDir2" Guid="83532EF6-0AE6-4B9C-824E-7309E4E97BFA">
                <File Id="fil9EEABE714F6AF741BE3D5AA5B5209994" KeyPath="yes" Source="$(var.MyDir2)\SDKTestPlugin.jar" />
            </Component>
            <Component Id="cmp89BABCFC5524994A65C4664EA4FACE6C" Directory="MyProgramDir2" Guid="3C7A05CE-F786-4E23-80B4-2FCB9ABB314C">
                <File Id="fil65C3299403B5E2113A60782D09D56C54" KeyPath="yes" Source="$(var.MyDir2)\ProfileManagerPlugin.jar" />
            </Component>
            <Component Id="cmpCB38C422B08F1CA1B1E85B1EA87CC1E6" Directory="MyProgramDir2" Guid="66766CBB-A4CD-46C9-94F5-79410A8981C2">
                <File Id="filC9B025CB143BD34F87C0041C5D550955" KeyPath="yes" Source="$(var.MyDir2)\RFIDScannerPlugin.jar" />
            </Component>
            <Component Id="cmp0F9E9EA430641A0340428B2A9E45E672" Directory="MyProgramDir2" Guid="203631E3-581B-49C4-8D38-2B2D1CB9DDCD">
                <File Id="filCA701380AEB33971329784A6C3186684" KeyPath="yes" Source="$(var.MyDir2)\WebSocketServerPlugin.jar" />
            </Component>
        </ComponentGroup>
    </Fragment>

</Wix>

Below is the class which handles the installer related actions.

 public class MainViewModel : ViewModelBase
    {
        //constructor
        public MainViewModel(BootstrapperApplication bootstrapper)
        {

            this.IsThinking = false;

            this.Bootstrapper = bootstrapper;
            this.Bootstrapper.ApplyComplete += this.OnApplyComplete;
            this.Bootstrapper.DetectPackageComplete += this.OnDetectPackageComplete;
            this.Bootstrapper.PlanComplete += this.OnPlanComplete;

            this.Bootstrapper.CacheAcquireProgress += (sender, args) =>
            {
                this.cacheProgress = args.OverallPercentage;
                this.Progress = (this.cacheProgress + this.executeProgress) / 2;
            };
            this.Bootstrapper.ExecuteProgress += (sender, args) =>
            {
                this.executeProgress = args.OverallPercentage;
                this.Progress = (this.cacheProgress + this.executeProgress) / 2;
            };
        }

        #region Properties

        private bool installEnabled;
        public bool InstallEnabled
        {
            get { return installEnabled; }
            set
            {
                installEnabled = value;
                RaisePropertyChanged("InstallEnabled");
            }
        }

        private bool uninstallEnabled;
        public bool UninstallEnabled
        {
            get { return uninstallEnabled; }
            set
            {
                uninstallEnabled = value;
                RaisePropertyChanged("UninstallEnabled");
            }
        }

        private bool isThinking;
        public bool IsThinking
        {
            get { return isThinking; }
            set
            {
                isThinking = value;
                RaisePropertyChanged("IsThinking");
            }
        }

        private int progress;
        public int Progress
        {
            get { return progress; }
            set
            {
                this.progress = value;
                RaisePropertyChanged("Progress");
            }
        }

        private int cacheProgress;
        private int executeProgress;

        public BootstrapperApplication Bootstrapper { get; private set; }

        #endregion //Properties

        #region Methods

        public void InstallExecute()
        {
            IsThinking = true;
            Bootstrapper.Engine.Plan(LaunchAction.Install);
        }

        public void UninstallExecute()
        {
            IsThinking = true;
            Bootstrapper.Engine.Plan(LaunchAction.Uninstall);
        }

        public void ExitExecute()
        {
            CustomBA.BootstrapperDispatcher.InvokeShutdown();
        }

        /// <summary>
        /// Method that gets invoked when the Bootstrapper ApplyComplete event is fired.
        /// This is called after a bundle installation has completed. Make sure we updated the view.
        /// </summary>
        private void OnApplyComplete(object sender, ApplyCompleteEventArgs e)
        {
            IsThinking = false;
            InstallEnabled = false;
            UninstallEnabled = false;
            this.Progress = 100;
            ExitExecute();
        }

        /// <summary>
        /// Method that gets invoked when the Bootstrapper DetectPackageComplete event is fired.
        /// Checks the PackageId and sets the installation scenario. The PackageId is the ID
        /// specified in one of the package elements (msipackage, exepackage, msppackage,
        /// msupackage) in the WiX bundle.
        /// </summary>
        private void OnDetectPackageComplete(object sender, DetectPackageCompleteEventArgs e)
        {
            if (e.PackageId == "KubeInstallationPackageId")
            {
                if (e.State == PackageState.Absent)
                    InstallEnabled = true;

                else if (e.State == PackageState.Present)
                    UninstallEnabled = true;
            }
        }

        /// <summary>
        /// Method that gets invoked when the Bootstrapper PlanComplete event is fired.
        /// If the planning was successful, it instructs the Bootstrapper Engine to 
        /// install the packages.
        /// </summary>
        private void OnPlanComplete(object sender, PlanCompleteEventArgs e)
        {
            if (e.Status >= 0)
                Bootstrapper.Engine.Apply(System.IntPtr.Zero);
        }

        #endregion //Methods

        #region RelayCommands

        private RelayCommand installCommand;
        public RelayCommand InstallCommand
        {
            get
            {
                if (installCommand == null)
                    installCommand = new RelayCommand(() => InstallExecute(), () => InstallEnabled == true);

                return installCommand;
            }
        }

        private RelayCommand uninstallCommand;
        public RelayCommand UninstallCommand
        {
            get
            {
                if (uninstallCommand == null)
                    uninstallCommand = new RelayCommand(() => UninstallExecute(), () => UninstallEnabled == true);

                return uninstallCommand;
            }
        }

        private RelayCommand exitCommand;
        public RelayCommand ExitCommand
        {
            get
            {
                if (exitCommand == null)
                    exitCommand = new RelayCommand(() => ExitExecute());

                return exitCommand;
            }
        }

        #endregion //RelayCommands
    }

Log File 1 :

[0980:3888][2016-04-22T16:49:18]i001: Burn v3.10.2.2516, Windows v6.3 (Build 9600: Service Pack 0), path: C:\ProgramData\Package Cache\{71ae2d4e-5c00-442b-9ea1-00cec549800d}\KubePlatformSetup.exe
[0980:3888][2016-04-22T16:49:18]i003: This bundle is being run by a related bundle as type 'Upgrade'.
[0980:3888][2016-04-22T16:49:18]i009: Command Line: '-uninstall -quiet -burn.related.upgrade -burn.ancestors={f57e276b-2b99-4f55-9566-88f47c0a065c} -burn.embedded BurnPipe.{22EB703C-0051-46E7-8B68-BA209900F6F7} {B0B02376-A988-437A-92A4-6116CFBC5526} 13952'
[0980:3888][2016-04-22T16:49:18]i000: Setting string variable 'WixBundleLog' to value 'C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164918.log'
[0980:3888][2016-04-22T16:49:18]i000: Setting string variable 'WixBundleManufacturer' to value 'Zone24x7'
[0980:3888][2016-04-22T16:49:18]i000: Loading managed bootstrapper application.
[0980:3888][2016-04-22T16:49:18]i000: Creating BA thread to run asynchronously.
[0980:37B4][2016-04-22T16:49:18]i000: Launching custom TestBA UX
[0980:3888][2016-04-22T16:49:19]i100: Detect begin, 2 packages
[0980:3888][2016-04-22T16:49:19]i102: Detected related bundle: {f57e276b-2b99-4f55-9566-88f47c0a065c}, type: Upgrade, scope: PerMachine, version: 1.0.1.0, operation: None
[0980:3888][2016-04-22T16:49:19]i103: Detected related package: {8C442A83-F559-488C-8CC4-21B1626F4B8E}, scope: PerMachine, version: 1.0.1.0, language: 0 operation: Downgrade
[0980:3888][2016-04-22T16:49:19]i103: Detected related package: {8201DD23-40A5-418B-B016-4D29BE6F010B}, scope: PerMachine, version: 1.0.1.0, language: 0 operation: Downgrade
[0980:3888][2016-04-22T16:49:19]i101: Detected package: KubeUpdaterServiceInstallerId, state: Obsolete, cached: Complete
[0980:3888][2016-04-22T16:49:19]i101: Detected package: MosquittoInstallerId, state: Obsolete, cached: Complete
[0980:3888][2016-04-22T16:49:19]i199: Detect complete, result: 0x0
[0980:3888][2016-04-22T16:51:43]i500: Shutting down, exit code: 0x0
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleAction = 3
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleElevated = 1
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleInstalled = 1
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleLastUsedSource = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleLog = C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164918.log
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleManufacturer = Zone24x7
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleName = KubePlatformSetup
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleOriginalSource = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\KubePlatformSetup.exe
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleOriginalSourceFolder = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleProviderKey = {71ae2d4e-5c00-442b-9ea1-00cec549800d}
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleTag = 
[0980:3888][2016-04-22T16:51:43]i410: Variable: WixBundleVersion = 1.0.0.0
[0980:3888][2016-04-22T16:51:43]i007: Exit code: 0x0, restarting: No

Log File 2:

[3204:32B0][2016-04-22T16:48:38]i001: Burn v3.10.2.2516, Windows v6.3 (Build 9600: Service Pack 0), path: C:\Users\MACILA~1\AppData\Local\Temp\{E6470133-8EFE-49DE-8561-A335091D3464}\.cr\KubePlatformSetup.exe
[3204:32B0][2016-04-22T16:48:38]i009: Command Line: '-burn.clean.room=G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\KubePlatformSetup.exe'
[3204:32B0][2016-04-22T16:48:38]i000: Setting string variable 'WixBundleOriginalSource' to value 'G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\KubePlatformSetup.exe'
[3204:32B0][2016-04-22T16:48:38]i000: Setting string variable 'WixBundleOriginalSourceFolder' to value 'G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\'
[3204:32B0][2016-04-22T16:48:38]i000: Setting string variable 'WixBundleLog' to value 'C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838.log'
[3204:32B0][2016-04-22T16:48:38]i000: Setting string variable 'WixBundleName' to value 'KubePlatformSetup'
[3204:32B0][2016-04-22T16:48:38]i000: Setting string variable 'WixBundleManufacturer' to value 'Zone24x7'
[3204:32B0][2016-04-22T16:48:38]i000: Loading managed bootstrapper application.
[3204:32B0][2016-04-22T16:48:38]i000: Creating BA thread to run asynchronously.
[3204:3A5C][2016-04-22T16:48:38]i000: Launching custom TestBA UX
[3204:32B0][2016-04-22T16:48:39]i100: Detect begin, 2 packages
[3204:32B0][2016-04-22T16:48:39]i102: Detected related bundle: {71ae2d4e-5c00-442b-9ea1-00cec549800d}, type: Upgrade, scope: PerMachine, version: 1.0.0.0, operation: MajorUpgrade
[3204:32B0][2016-04-22T16:48:39]i103: Detected related package: {398C5992-4E70-4844-8063-48639892BACF}, scope: PerMachine, version: 1.0.0.0, language: 0 operation: MajorUpgrade
[3204:32B0][2016-04-22T16:48:39]i103: Detected related package: {392D3E7F-6FCB-4EFD-A40F-B2F7C2CB6D44}, scope: PerMachine, version: 1.0.0.0, language: 0 operation: MajorUpgrade
[3204:32B0][2016-04-22T16:48:39]i101: Detected package: KubeUpdaterServiceInstallerId, state: Absent, cached: None
[3204:32B0][2016-04-22T16:48:39]i101: Detected package: MosquittoInstallerId, state: Absent, cached: None
[3204:32B0][2016-04-22T16:48:39]i199: Detect complete, result: 0x0
[3204:32B0][2016-04-22T16:48:40]i200: Plan begin, 2 packages, action: Install
[3204:32B0][2016-04-22T16:48:40]i000: Setting string variable 'WixBundleRollbackLog_KubeUpdaterServiceInstallerId' to value 'C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_000_KubeUpdaterServiceInstallerId_rollback.log'
[3204:32B0][2016-04-22T16:48:40]i000: Setting string variable 'WixBundleLog_KubeUpdaterServiceInstallerId' to value 'C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_000_KubeUpdaterServiceInstallerId.log'
[3204:32B0][2016-04-22T16:48:40]i000: Setting string variable 'WixBundleRollbackLog_MosquittoInstallerId' to value 'C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_001_MosquittoInstallerId_rollback.log'
[3204:32B0][2016-04-22T16:48:40]i000: Setting string variable 'WixBundleLog_MosquittoInstallerId' to value 'C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_001_MosquittoInstallerId.log'
[3204:32B0][2016-04-22T16:48:40]i201: Planned package: KubeUpdaterServiceInstallerId, state: Absent, default requested: Present, ba requested: Present, execute: Install, rollback: Uninstall, cache: Yes, uncache: No, dependency: Register
[3204:32B0][2016-04-22T16:48:40]i201: Planned package: MosquittoInstallerId, state: Absent, default requested: Present, ba requested: Present, execute: Install, rollback: Uninstall, cache: Yes, uncache: No, dependency: Register
[3204:32B0][2016-04-22T16:48:40]i207: Planned related bundle: {71ae2d4e-5c00-442b-9ea1-00cec549800d}, type: Upgrade, default requested: Absent, ba requested: Absent, execute: Uninstall, rollback: Install, dependency: None
[3204:32B0][2016-04-22T16:48:40]i299: Plan complete, result: 0x0
[3204:32B0][2016-04-22T16:48:40]i300: Apply begin
[3204:32B0][2016-04-22T16:48:40]i010: Launching elevated engine process.
[3204:32B0][2016-04-22T16:48:40]i011: Launched elevated engine process.
[3204:32B0][2016-04-22T16:48:40]i012: Connected to elevated engine.
[3680:38D0][2016-04-22T16:48:40]i358: Pausing automatic updates.
[3680:38D0][2016-04-22T16:48:40]i359: Paused automatic updates.
[3680:38D0][2016-04-22T16:48:40]i360: Creating a system restore point.
[3680:38D0][2016-04-22T16:48:40]i361: Created a system restore point.
[3680:38D0][2016-04-22T16:48:40]i370: Session begin, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{f57e276b-2b99-4f55-9566-88f47c0a065c}, options: 0x7, disable resume: No
[3680:38D0][2016-04-22T16:48:40]i000: Caching bundle from: 'C:\Users\MACILA~1\AppData\Local\Temp\{89CCD5C8-EB48-4FB3-8FB5-32FC9EEAB83F}\.be\KubePlatformSetup.exe' to: 'C:\ProgramData\Package Cache\{f57e276b-2b99-4f55-9566-88f47c0a065c}\KubePlatformSetup.exe'
[3680:38D0][2016-04-22T16:48:40]i320: Registering bundle dependency provider: {f57e276b-2b99-4f55-9566-88f47c0a065c}, version: 1.0.1.0
[3680:38D0][2016-04-22T16:48:40]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{f57e276b-2b99-4f55-9566-88f47c0a065c}, resume: Active, restart initiated: No, disable resume: No
[3204:2B40][2016-04-22T16:48:40]i336: Acquiring container: WixAttachedContainer, copy from: G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\KubePlatformSetup.exe
[3204:2B40][2016-04-22T16:48:40]i000: Setting string variable 'WixBundleLastUsedSource' to value 'G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\'
[3680:350C][2016-04-22T16:48:40]i305: Verified acquired payload: KubeUpdaterServiceInstallerId at path: C:\ProgramData\Package Cache\.unverified\KubeUpdaterServiceInstallerId, moving to: C:\ProgramData\Package Cache\{8C442A83-F559-488C-8CC4-21B1626F4B8E}v1.0.1.0\KubeUpdaterServiceInstaller.msi.
[3680:350C][2016-04-22T16:48:40]i305: Verified acquired payload: cabDF68CF429512A78026D040F0FB13A50B at path: C:\ProgramData\Package Cache\.unverified\cabDF68CF429512A78026D040F0FB13A50B, moving to: C:\ProgramData\Package Cache\{8C442A83-F559-488C-8CC4-21B1626F4B8E}v1.0.1.0\cab1.cab.
[3680:350C][2016-04-22T16:48:40]i305: Verified acquired payload: MosquittoInstallerId at path: C:\ProgramData\Package Cache\.unverified\MosquittoInstallerId, moving to: C:\ProgramData\Package Cache\{8201DD23-40A5-418B-B016-4D29BE6F010B}v1.0.1.0\MosquittoInstaller.msi.
[3680:350C][2016-04-22T16:48:40]i305: Verified acquired payload: cab657E45B8C9A7EF24C9A74563C2FA6EB3 at path: C:\ProgramData\Package Cache\.unverified\cab657E45B8C9A7EF24C9A74563C2FA6EB3, moving to: C:\ProgramData\Package Cache\{8201DD23-40A5-418B-B016-4D29BE6F010B}v1.0.1.0\cab1.cab.
[3680:38D0][2016-04-22T16:48:40]i323: Registering package dependency provider: {8C442A83-F559-488C-8CC4-21B1626F4B8E}, version: 1.0.1.0, package: KubeUpdaterServiceInstallerId
[3680:38D0][2016-04-22T16:48:40]i301: Applying execute package: KubeUpdaterServiceInstallerId, action: Install, path: C:\ProgramData\Package Cache\{8C442A83-F559-488C-8CC4-21B1626F4B8E}v1.0.1.0\KubeUpdaterServiceInstaller.msi, arguments: ' ARPSYSTEMCOMPONENT="1" MSIFASTINSTALL="7" INSTALLLOCATION="G:\"'
[3204:32B0][2016-04-22T16:49:17]i319: Applied execute package: KubeUpdaterServiceInstallerId, result: 0x0, restart: None
[3680:38D0][2016-04-22T16:49:17]i325: Registering dependency: {f57e276b-2b99-4f55-9566-88f47c0a065c} on package provider: {8C442A83-F559-488C-8CC4-21B1626F4B8E}, package: KubeUpdaterServiceInstallerId
[3680:38D0][2016-04-22T16:49:17]i323: Registering package dependency provider: {8201DD23-40A5-418B-B016-4D29BE6F010B}, version: 1.0.1.0, package: MosquittoInstallerId
[3680:38D0][2016-04-22T16:49:17]i301: Applying execute package: MosquittoInstallerId, action: Install, path: C:\ProgramData\Package Cache\{8201DD23-40A5-418B-B016-4D29BE6F010B}v1.0.1.0\MosquittoInstaller.msi, arguments: ' ARPSYSTEMCOMPONENT="1" MSIFASTINSTALL="7" INSTALLLOCATION="G:\"'
[3204:32B0][2016-04-22T16:49:18]i319: Applied execute package: MosquittoInstallerId, result: 0x0, restart: None
[3680:38D0][2016-04-22T16:49:18]i325: Registering dependency: {f57e276b-2b99-4f55-9566-88f47c0a065c} on package provider: {8201DD23-40A5-418B-B016-4D29BE6F010B}, package: MosquittoInstallerId
[3680:38D0][2016-04-22T16:49:18]i301: Applying execute package: {71ae2d4e-5c00-442b-9ea1-00cec549800d}, action: Uninstall, path: C:\ProgramData\Package Cache\{71ae2d4e-5c00-442b-9ea1-00cec549800d}\KubePlatformSetup.exe, arguments: '"C:\ProgramData\Package Cache\{71ae2d4e-5c00-442b-9ea1-00cec549800d}\KubePlatformSetup.exe" -uninstall -quiet -burn.related.upgrade -burn.ancestors={f57e276b-2b99-4f55-9566-88f47c0a065c}'
[3204:32B0][2016-04-22T16:51:43]i319: Applied execute package: {71ae2d4e-5c00-442b-9ea1-00cec549800d}, result: 0x0, restart: None
[3680:38D0][2016-04-22T16:51:43]i372: Session end, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{f57e276b-2b99-4f55-9566-88f47c0a065c}, resume: ARP, restart: None, disable resume: No
[3680:38D0][2016-04-22T16:51:43]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{f57e276b-2b99-4f55-9566-88f47c0a065c}, resume: ARP, restart initiated: No, disable resume: No
[3204:32B0][2016-04-22T16:51:43]i399: Apply complete, result: 0x0, restart: None, ba requested restart:  No
[3204:32B0][2016-04-22T16:51:43]i500: Shutting down, exit code: 0x0
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleAction = 5
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleElevated = 1
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleLastUsedSource = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleLog = C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838.log
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleLog_KubeUpdaterServiceInstallerId = C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_000_KubeUpdaterServiceInstallerId.log
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleLog_MosquittoInstallerId = C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_001_MosquittoInstallerId.log
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleManufacturer = Zone24x7
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleName = KubePlatformSetup
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleOriginalSource = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\KubePlatformSetup.exe
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleOriginalSourceFolder = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleProviderKey = {f57e276b-2b99-4f55-9566-88f47c0a065c}
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleRollbackLog_KubeUpdaterServiceInstallerId = C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_000_KubeUpdaterServiceInstallerId_rollback.log
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleRollbackLog_MosquittoInstallerId = C:\Users\MACILA~1\AppData\Local\Temp\KubePlatformSetup_20160422164838_001_MosquittoInstallerId_rollback.log
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleSourceProcessFolder = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleSourceProcessPath = G:\Projects\Kube2\kube-installer\KubePlatformSetup\KubePlatformSetup\bin\Release\KubePlatformSetup.exe
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleTag = 
[3204:32B0][2016-04-22T16:51:43]i410: Variable: WixBundleVersion = 1.0.1.0
[3204:32B0][2016-04-22T16:51:43]i007: Exit code: 0x0, restarting: No

Answer 1

I found the solution atlast. I had to hook onto the DetectComplete method and call the plan method on the Engine. Below is how I did it.

void DetectComplete(object sender, DetectCompleteEventArgs e)
    {
        Bootstrapper.Engine.Log(LogLevel.Verbose,"fired! but does that give you any clue?! idiot!");
        if (LaunchAction.Uninstall == Bootstrapper.Command.Action)
        {
            Bootstrapper.Engine.Log(LogLevel.Verbose, "Invoking automatic plan for uninstall");
            Bootstrapper.Engine.Plan(LaunchAction.Uninstall);
        } 
    }

Answer 2

From the logs I'm seeing a few things.

In regards to the Upgrade, it looks like it's properly detecting all the packages and planning them correctly.

The two installer packages are being installed as a major upgrade and the related bundle (the one that is being upgrade) is being detected and run to uninstall it as well.

The issues all seem to be coming from your uninstall of the previous bundle.

The log you labeled as "Log File 1" is actually the log from when your upgrade re-ran the currently installed bootstrapper to remove it. The problem as I see it is right here:

[0980:3888][2016-04-22T16:49:19]i100: Detect begin, 2 packages
[0980:3888][2016-04-22T16:49:19]i102: Detected related bundle: {f57e276b-2b99-4f55-9566-88f47c0a065c}, type: Upgrade, scope: PerMachine, version: 1.0.1.0, operation: None
[0980:3888][2016-04-22T16:49:19]i103: Detected related package: {8C442A83-F559-488C-8CC4-21B1626F4B8E}, scope: PerMachine, version: 1.0.1.0, language: 0 operation: Downgrade
[0980:3888][2016-04-22T16:49:19]i103: Detected related package: {8201DD23-40A5-418B-B016-4D29BE6F010B}, scope: PerMachine, version: 1.0.1.0, language: 0 operation: Downgrade
[0980:3888][2016-04-22T16:49:19]i101: Detected package: KubeUpdaterServiceInstallerId, state: Obsolete, cached: Complete
[0980:3888][2016-04-22T16:49:19]i101: Detected package: MosquittoInstallerId, state: Obsolete, cached: Complete
[0980:3888][2016-04-22T16:49:19]i199: Detect complete, result: 0x0
[0980:3888][2016-04-22T16:51:43]i500: Shutting down, exit code: 0x0

The desired process for all burn bootstrappers is to do Detect phase, Plan phase, Apply phase.

For some reason your uninstall when being run by a related bundle is just stopping after the detect phase.

I also suspect your machine may be in a weird state right now. I think you have several installed, but hidden in the add remove programs list, installs for KubeUpdaterServiceInstaller and MosquittoInstaller.

The first thing I would do is stop installing anything on your build/development machine. When authoring installers you will make mistakes and cause machines to get in weird states with regards to knowing what it has installed and doesn't have installed on it. I would suggest using a virtual environment for all your installer testing. When you mess something up you can easily revert to a clean state and start testing again. If you are using a virtual environment then just ignore this paragraph you're doing it right but make sure you clean everything before testing changes and testing the upgrade.

In your MSI wix I would suggest you use a "*" as the Product Id so you don't have to change it every build.

<Product Id="*" Name="Installer" Language="1033" Version="1.0.0.0" Manufacturer="Zone24x7" UpgradeCode="E90569B5-372E-45BB-B101-58E0B75AB5C7">

and in your major upgrade element add AllowSameVersionUpgrades="yes"

<MajorUpgrade DowngradeErrorMessage="A newer version of [ProductName] is already installed." AllowSameVersionUpgrades="yes" />

So you don't have to update the version every time you want to test an upgrade just rebuild the msi installer.

For reference, here are my Bootstrapper upgrade logs for the bootstrapper application and the logs for when it removes the old install. Notice some differences

The Upgrade Install

When it re-runs the old installation to remove it

The upgrade log looks basically the same as yours which is good but the re-run to uninstall log has some differences. I'm not totally sure without basically seeing your whole codebase (and I don't have the time or will to do that) how to fix these issues but hopefully something in this answer will help you out.