Dynamically create list/dict during for loop for conversion to JSON

Question

I am trying to build a list/dict that will be converted to JSON later on. I am trying to write the code that builds and populates the multiple levels of the JSON format I ultimately need. I am having an issue wrapping my head around this. Thank you for the help.

What I ultimately need -> Populate this list/dict:

dataset_permission_json = []

with this format:

{
   "projects":[
      {
         "project":"test-project-1",
         "datasets":[
            {
               "dataset":"testing1",
               "permissions":[
                  {
                     "role":"READER",
                     "google_group":"testing1@test.com"
                  }
               ]
            },
            {
               "dataset":"testing2",
               "permissions":[
                  {
                     "role":"OWNER",
                     "google_group":"testing2@test.com"
                  }
               ]
            },
            {
               "dataset":"testing3",
               "permissions":[
                  {
                     "role":"READER",
                     "google_group":"testing3@test.com"
                  }
               ]
            },
            {
               "dataset":"testing4",
               "permissions":[
                  {
                     "role":"WRITER",
                     "google_group":"testing4@test.com"
                  }
               ]
            }
         ]
      }
   ]
}

I have multiple for loops that successfully print out the information I am pulling from an external API but I to be able to enter that data into the list/dict. The dynamic values I am trying to input are:

'project' i.e. test-project-1
'dataset' i.e. testing1
'role' i.e. READER
'google_group' i.e. testing1@test.com

I have tried things like:

dataset_permission_json.update({'project': project})

but cannot figure out how not to overwrite the data during the multiple for loops.

for project in projects:
    
    print(project) ## Need to add this variable to 'projects'
    
    for bq_group in bq_groups:
        delegated_credentials = credentials.create_delegated(bq_group)
        http_auth = delegated_credentials.authorize(Http())
        list_datasets_in_project = bigquery_service.datasets().list(projectId=project).execute()
        datasets = list_datasets_in_project.get('datasets',[])
    
        print(dataset['datasetReference']['datasetId']) ##Add the dataset to 'datasets' under the project

        for dataset in datasets:
            get_dataset_permissions_result = bigquery_service.datasets().get(projectId=project, datasetId=dataset['datasetReference']['datasetId']).execute()
            dataset_permissions = get_dataset_permissions_result.get('access',[])
            ### ADD THE NEXT LEVEL 'permissions' level here?
            for dataset_permission in dataset_permissions:
                if 'groupByEmail' in dataset_permission:
                    if bq_group in dataset_permission['groupByEmail']:

                        print(dataset['datasetReference']['datasetId'] && dataset_permission['groupByEmail']) ##Add to each dataset

I appreciate the help.

EDIT: Updated Progress

Ok I have created the nested structure that I was looking for using StackOverflow

Things are great except for the last part. I am trying to append the role & group to each 'permission' nest, but after everything runs the data is only appended to the last 'permission' nest in the JSON structure. It seems like it is overwriting itself during the for loop. Thoughts?

Updated for loop:

for project in projects:

        for bq_group in bq_groups:
            delegated_credentials = credentials.create_delegated(bq_group)
            http_auth = delegated_credentials.authorize(Http())
            list_datasets_in_project = bigquery_service.datasets().list(projectId=project).execute()
            datasets = list_datasets_in_project.get('datasets',[])
    
            for dataset in datasets:
                get_dataset_permissions_result = bigquery_service.datasets().get(projectId=project, datasetId=dataset['datasetReference']['datasetId']).execute()
                dataset_permissions = get_dataset_permissions_result.get('access',[])
                for dataset_permission in dataset_permissions:
                    if 'groupByEmail' in dataset_permission:
                        if bq_group in dataset_permission['groupByEmail']:
                            dataset_permission_json['projects'][project]['datasets'][dataset['datasetReference']['datasetId']]['permissions']
                            permission = {'group': dataset_permission['groupByEmail'],'role': dataset_permission['role']}
                            dataset_permission_json['permissions'] = permission

UPDATE: Solved.

dataset_permission_json['projects'][project]['datasets'][dataset['datasetReference']['datasetId']]['permissions']
permission = {'group': dataset_permission['groupByEmail'],'role': dataset_permission['role']}
dataset_permission_json['projects'][project]['datasets'][dataset['datasetReference']['datasetId']]['permissions'] = permission