This is a little rushed. I will check back if it makes sense to you.
You should be able to use the standard and built-in RemoveFile feature in MSI to remove log files - if you know the name of the files in question. Then you can add a CreateFolder entry to your installer which creates the logging directory in question and then use that directory property in your RemoveFile entry:
Sample from github:
<Component>
<File Source="WiX.chm" />
<Shortcut Id="WixChmShortcut" Directory="ShortcutFolder" Name="WiX Documentation"
Icon="WixSetupIcons.ico" IconIndex="0" Advertise="yes" />
<RemoveFile Id="RemoveWixChw" Name="WiX.chw" On="uninstall" />
</Component>
CreateFolder is a WiX element that you insert to indicate that an empty folder should be created as part of your installation:
<Directory Id="Mydir" Name="My Directory">
<Component Feature="MyFeature" >
<CreateFolder />
<File Id="NOTEPAD.EXE" Source="$(var.MyReleasePath)\Main Folder\NOTEPAD.EXE" />
</Component>
</Directory>
Some further thoughts:
I would not do logging in the installation folder - this folder is supposed to be read-only.
I would log elsewhere, somewhere in the UserProfile perhaps, or even the temp folder (and let it be cleaned by whatever mechanism the end users have for that).
I would also consider logging to the system's event log instead of to a file, if possible.