Finding and replacing symbols in list elements

Question

I need to take a list of items (that list is not static and will change every time the script is run) that may or may not have multiple certain symbols in the list elements, for example:

a = ['C:\\Temp\\Folder1',
 'C:\\Temp\\Folder2',
 'C:\\Temp\\Folder3',
 'C:\\Temp\\Folder4',
 '%ALLUSERSPROFILE%\\Temp',
 '%WINDIR%\\Temp']

and I need to change the item that would have the % symbol to output the following:

a = ['C:\\Temp\\Folder1',
 'C:\\Temp\\Folder2',
 'C:\\Temp\\Folder3',
 'C:\\Temp\\Folder4',
 '${allusersprofile}\\Temp',
 '${windir}\\Temp']

There could be more than one entry in the list as well so I need to make sure I hit every one that may exist. I've been wracking my brain trying to come up with something and looked at match-case, if/elif/else, and the suggestions are elegant but I'm not sure how to code something like this as it's not a simple 1:1 replacement. Any help is appreciated.

EDIT:

I've tried adding @shadowtalkers code to mine but can't get the desired output. I'm sure I'm missing something innocuous, but I don't know what. The last part of "for" where I'm adding the double-slash, is for API requirements and folder names. Here's a snippet of my current code:

# importing the module
import requests, json, os, re

# Pattern to recognize when preforming the Folder Exceptions
var_pattern = re.compile(r"%([^%]+)%")

# Function to swap out the '%' symbols in environment variables
def process_var_sub(match):
    var_name = match.group(1)
    return "${" + var_name.lower() + "}"

# Function to write Folder Exceptions
def write_folder_exception_list(excluded_folder_list, policy_name, section):
    if len(excluded_folder_list):
        for p in excluded_folder_list:
            var_pattern.sub(process_var_sub, p)
        for i in range(len(excluded_folder_list)):
             excluded_folder_list[i] = excluded_folder_list[i] + "\\"
        excluded_folder_list = {
            'name': policy_name + " "+section+" Exclusions",
            'description': "",
            'items': excluded_folder_list
        }
        payload = json.dumps(excluded_folder_list, indent=4)
        print(payload)
        url = baseurl + "directorylists"
        response = requests.request("POST", url, headers=headers, data=payload)
        print(response.text)
        del response
        del url
        del excluded_folder_list
        del payload
    else:
        return

EDIT 2:

Ok, it's a hack job, I know, but I got it to work. Any suggestions on cleaning it up?

def write_folder_exception_list(excluded_folder_list, policy_name, section):
    if len(excluded_folder_list):
        for i in range(len(excluded_folder_list)):
            excluded_folder_list_cleaned = [
                var_pattern.sub(process_var_sub, p)
                for p in excluded_folder_list
            ]
            excluded_folder_list = excluded_folder_list_cleaned
            excluded_folder_list[i] = excluded_folder_list[i] + "\\"
        excluded_folder_list = {
            'name': policy_name + " "+section+" Exclusions",
            'description': "",
            'items': excluded_folder_list_cleaned
        }
        payload = json.dumps(excluded_folder_list, indent=4)
        print(payload)

EDIT 3:

I was able to get it working (and made a few more optimizations to my code) thanks to shadowtalkers input. I'm sure it can be cleaned up and optimized more, but here are the snippets of the relative code if someone else has to deal with this in the future.

    # Pattern to recognize when performing the Folder Exceptions
    var_pattern = re.compile(r"%([^%]+)%")
    
    # Function to swap out the '%' symbols in environment variables from Apex One
    def process_var_sub(match):
        var_name = match.group(1)
        return "${" + var_name.lower() + "}"

    # Populate elements from the JSON File
    def populate_elements(policy_filename):
    with open(policy_filename) as jsonFile:  # Open the file as a JSON file
        data = json.load(jsonFile)  # Read the file
        policyname = (data['policy'][0]['policyName'])  # Get the name of the policy as seen in Apex One
        dictionary = (data['policy'][0]['managedSettings'])  # Read the managedSettings section (which holds the users custom data)
        settings = json.loads(dictionary)  # Read the settings as it's own JSON string
    return policyname, [settings]

    # Function to write Folder Exceptions using the API
    def write_folder_exception_list(excluded_folder_list, policy_name, section):
        if len(excluded_folder_list):
            for i in range(len(excluded_folder_list)):
                excluded_folder_list_cleaned = [
                    var_pattern.sub(process_var_sub, p)
                    for p in excluded_folder_list
                ]
                excluded_folder_list = excluded_folder_list_cleaned
                excluded_folder_list[i] = excluded_folder_list[i] + "\\"
            excluded_folder_list = {
                'name': policy_name + " "+section+" Exclusions",
                'description': "",
                'items': excluded_folder_list_cleaned
            }
            payload = json.dumps(excluded_folder_list, indent=4)
            url = baseurl + "directorylists"
            response = requests.request("POST", url, headers=headers, data=payload)
            print(response.text)
            del response
            del url
            del excluded_folder_list
            del payload
        else:
            return
    
    # Open JSON policy file(s) within the current working directory the script is run in
    for file in os.listdir():   # Get a list of files in the directory
    
        policy_filename = os.fsdecode(file)    # Pull the file name

        if policy_filename.endswith(".policy"):  # Make sure we're only working with the .policy files
    
                policyName, settings = populate_elements(policy_filename) # Call the function to populate the managed_settings information
    
                for settings in settings:    # Iterate the settings to populate fields
    
                    # RTS Exclusions
                    RTSExcludedFolders = (settings["managed_settings"][0]["settings"][1]["value"]['ExcludedFolder'])
                    RTSExcludedFiles = (settings["managed_settings"][0]["settings"][3]["value"]['ExcludedFile'])
                    RTSExcludedExtensions = (settings["managed_settings"][0]["settings"][5]["value"]['ExcludedExt'])
                    write_folder_exception_list(RTSExcludedFolders, policyName, "RTS Scan")
                    write_extensions_exception_list(RTSExcludedExtensions, policyName, "RTS Scan")
                    write_filename_exception_list(RTSExcludedFiles, policyName, "RTS Scan")

Answer 1

In general, you are looking to replace any pattern of the form %...%, which seems like a great job for regular expressions.

First, you need to match the desired pattern, and capture the text within. This is easy enough with the regex %([^%]+)%:

%         literal "%"
(         start a capture group
  [^%]    any character other than "%"
  +       repeat the previous character 1 or more times
)         end the capture group
%         literal "%"

See here to explore this regex further:

• https://regex101.com/r/Ps9FGA/1
• Replace a string located between (other question linked in comments)

---

Now we can use a nice Python feature and use a function to perform regex substitution on the captured result:

import re

var_pattern = re.compile(r"%([^%]+)%")

def process_var_sub(match):
    var_name = match.group(1)
    return "${" + var_name.lower() + "}"

paths_in = [
    r"C:\Temp\Folder1",
    r"C:\Temp\Folder2",
    r"C:\Temp\Folder3",
    r"C:\Temp\Folder4",
    r"%ALLUSERSPROFILE%\Temp",
    r"%WINDIR%\Temp",
]

paths_out = [
    var_pattern.sub(process_var_sub, p)
    for p in paths_in
]

assert paths_out == [
    r"C:\Temp\Folder1",
    r"C:\Temp\Folder2",
    r"C:\Temp\Folder3",
    r"C:\Temp\Folder4",
    r"${allusersprofile}\Temp",
    r"${windir}\Temp",
]

This feature is documented in https://docs.python.org/3/library/re.html#re.sub:

If repl is a function, it is called for every non-overlapping occurrence of pattern.

Finally, note the use of r"" to prevent Python from interpreting \ as a special character inside the quotes.