How to update yaml files from powershell without formatting change?

Question

I have an yaml file with below data.

    logJsonDataConfig: &logJsonDataConfig
      code: logJsonData
      path: /logs/
      ext: .json
    queueJsonDataConfig: &queueJsonDataConfig
      code: queueJsonData
      path: /queue/
      ext: .json
    gatewayGrpcConfig: &gatewayGrpcConfig
      code: gatewayGrpc
      driverName: tcp
      urlAddress: 0.0.0.0:5051
    excelGrpcConfig: &excelGrpcConfig
      code: excelGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5052
    adGrpcConfig: &adGrpcConfig
      code: adGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5053
    restfulWebServiceGrpcConfig: &restfulWebServiceGrpcConfig
      code: restfulWebServiceGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5054
    databaseGrpcConfig: &databaseGrpcConfig
      code: databaseGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5055
    serviceDeskPlusGrpcConfig: &serviceDeskPlusGrpcConfig
      code: serviceDeskPlusGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5056
    recordingAgentGrpcConfig: &recordingAgentGrpcConfig
      code: recordingAgentGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5057
    fileAndFolderGrpcConfig: &fileAndFolderGrpcConfig
      code: fileAndFolderGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5058
    pdfGrpcConfig: &pdfGrpcConfig
      code: pdfGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5059
    ocrGrpcConfig: &ocrGrpcConfig
      code: ocrGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5060
    ewsGrpcConfig: &ewsGrpcConfig
      code: ewsGrpc
      driverName: tcp
      urlAddress: 127.0.0.1:5061
    zapConfig: &zapConfig
      code: zap
      level: debug
      enableCaller: true
    logrusConfig: &logrusConfig
      code: logrus
      level: debug
      enableCaller: false
    logConfig: *zapConfig
    excelMicroserviceConfig: &excelMicroserviceConfig
      code: excelMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    activeDirectoryMicroserviceConfig: &activeDirectoryMicroserviceConfig
      code: activeDirectoryMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    restfulWebServiceMicroserviceConfig: &restfulWebServiceMicroserviceConfig
      code: restfulWebServiceMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    recordingAgentMicroserviceConfig: &recordingAgentMicroserviceConfig
      code: recordingAgentMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    databaseMicroserviceConfig: &databaseMicroserviceConfig
      code: databaseMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    serviceDeskPlusMicroserviceConfig: &serviceDeskPlusMicroserviceConfig
      code: serviceDeskPlusMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    fileAndFolderMicroserviceConfig: &fileAndFolderMicroserviceConfig
      code: fileAndFolderMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    pdfMicroserviceConfig: &pdfMicroserviceConfig
      code: pdfMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    ocrMicroserviceConfig: &ocrMicroserviceConfig
      code: ocrMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    ewsMicroserviceConfig: &ewsMicroserviceConfig
      code: ewsMicroservice
      logDataConfig: &logDataConfig
        code: logData
        dataStoreConfig: *logJsonDataConfig
    useCaseConfig:
      logJson:
        code: logJson
      queue:
        code: queue
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *queueJsonDataConfig
      excel:
        code: excel
        microserviceConfig: *excelMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      activeDirectory:
        code: activeDirectory
        microserviceConfig: *activeDirectoryMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      restfulWebService:
        code: restfulWebService
        microserviceConfig: *restfulWebServiceMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      database:
        code: database
        microserviceConfig: *databaseMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      serviceDeskPlus:
        code: serviceDeskPlus
        microserviceConfig: *serviceDeskPlusMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      recordingAgent:
        code: recordingAgent
        microserviceConfig: *recordingAgentMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStroreConfig: *logJsonDataConfig
      fileAndFolder:
        code: fileAndFolder
        microserviceConfig: *fileAndFolderMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      pdf:
        code: pdf
        microserviceConfig: *pdfMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      ocr:
        code: ocr
        microserviceConfig: *ocrMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      ews:
        code: ews
        microserviceConfig: *ewsMicroserviceConfig
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      loop:
        code: loop
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig
      ifelse:
        code: ifElse
        logDataConfig: &logDataConfig
          code: logData
          dataStoreConfig: *logJsonDataConfig

You can see, the sections is not in (section:) format, instead in (section: &section).

I tried using powershell PSYaml module and am able to add/update a new section to it.

My code:

$MyDir = [System.IO.Path]::GetDirectoryName($myInvocation.MyCommand.Definition)
Import-Module $($MyDir + "\\modules\\PSYaml")
$text = ConvertFrom-Yaml -Path $($MyDir + "\\input.yml")
    
if ($text.automCenterConfig) {
    echo "Object exists, only update required"
}
else {
    #copy contents of one object and create a new one from that
    $text.automCenterConfig = $text.logJsonDataConfig.psobject.copy()
}
    
echo "---Updating content---"
$text.automCenterConfig.code = "value1"
$text.automCenterConfig.path = "value2"
$text.automCenterConfig.ext = "value3"
    
echo "Convert object to yaml and write to file"
$output = Convertto-yaml $text
$output | Out-File $($MyDir + "\\output.yml")

The output.yml generated:

---

  logJsonDataConfig: 
    code: 'value1' 
    path: 'value2' 
    ext: 'value3' 
  queueJsonDataConfig: 
    code: 'queueJsonData' 
    path: '/queue/' 
    ext: '.json' 
  gatewayGrpcConfig: 
    code: 'gatewayGrpc' 
    driverName: 'tcp' 
    urlAddress: '0.0.0.0:5051' 
  excelGrpcConfig: 
    code: 'excelGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5052' 
  adGrpcConfig: 
    code: 'adGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5053' 
  restfulWebServiceGrpcConfig: 
    code: 'restfulWebServiceGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5054' 
  databaseGrpcConfig: 
    code: 'databaseGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5055' 
  serviceDeskPlusGrpcConfig: 
    code: 'serviceDeskPlusGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5056' 
  recordingAgentGrpcConfig: 
    code: 'recordingAgentGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5057' 
  fileAndFolderGrpcConfig: 
    code: 'fileAndFolderGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5058' 
  pdfGrpcConfig: 
    code: 'pdfGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5059' 
  ocrGrpcConfig: 
    code: 'ocrGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5060' 
  ewsGrpcConfig: 
    code: 'ewsGrpc' 
    driverName: 'tcp' 
    urlAddress: '127.0.0.1:5061' 
  zapConfig: 
    code: 'zap' 
    level: 'debug' 
    enableCaller: true 
  logrusConfig: 
    code: 'logrus' 
    level: 'debug' 
    enableCaller:  
  logConfig: 
    code: 'zap' 
    level: 'debug' 
    enableCaller: true 
  excelMicroserviceConfig: 
    code: 'excelMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  activeDirectoryMicroserviceConfig: 
    code: 'activeDirectoryMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  restfulWebServiceMicroserviceConfig: 
    code: 'restfulWebServiceMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  recordingAgentMicroserviceConfig: 
    code: 'recordingAgentMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  databaseMicroserviceConfig: 
    code: 'databaseMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  serviceDeskPlusMicroserviceConfig: 
    code: 'serviceDeskPlusMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  fileAndFolderMicroserviceConfig: 
    code: 'fileAndFolderMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  pdfMicroserviceConfig: 
    code: 'pdfMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  ocrMicroserviceConfig: 
    code: 'ocrMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  ewsMicroserviceConfig: 
    code: 'ewsMicroservice' 
    logDataConfig: 
      code: 'logData' 
      dataStoreConfig: 
        code: 'logJsonData' 
        path: '/logs/' 
        ext: '.json' 
  useCaseConfig: 
    logJson: 
      code: 'logJson' 
    queue: 
      code: 'queue' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'queueJsonData' 
          path: '/queue/' 
          ext: '.json' 
    excel: 
      code: 'excel' 
      microserviceConfig: 
        code: 'excelMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    activeDirectory: 
      code: 'activeDirectory' 
      microserviceConfig: 
        code: 'activeDirectoryMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    restfulWebService: 
      code: 'restfulWebService' 
      microserviceConfig: 
        code: 'restfulWebServiceMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    database: 
      code: 'database' 
      microserviceConfig: 
        code: 'databaseMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    serviceDeskPlus: 
      code: 'serviceDeskPlus' 
      microserviceConfig: 
        code: 'serviceDeskPlusMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    recordingAgent: 
      code: 'recordingAgent' 
      microserviceConfig: 
        code: 'recordingAgentMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStroreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    fileAndFolder: 
      code: 'fileAndFolder' 
      microserviceConfig: 
        code: 'fileAndFolderMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    pdf: 
      code: 'pdf' 
      microserviceConfig: 
        code: 'pdfMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    ocr: 
      code: 'ocr' 
      microserviceConfig: 
        code: 'ocrMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    ews: 
      code: 'ews' 
      microserviceConfig: 
        code: 'ewsMicroservice' 
        logDataConfig: 
          code: 'logData' 
          dataStoreConfig: 
            code: 'logJsonData' 
            path: '/logs/' 
            ext: '.json' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    loop: 
      code: 'loop' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
    ifelse: 
      code: 'ifElse' 
      logDataConfig: 
        code: 'logData' 
        dataStoreConfig: 
          code: 'logJsonData' 
          path: '/logs/' 
          ext: '.json' 
  automCenterConfig: 
    code: 'value1' 
    path: 'value2' 
    ext: 'value3'

The issues here are, It added --- on top of the yaml and kept the complete document as sub fields to it.

I wrote code to create automCenterConfig from first object. The object, instead of copying, it modified both the contents. logJsonData and automCenterConfig, both are changing if I change automCenterConfig.

Mainly, it removed the & part after section heading. changed (logJsonDataConfig: &logJsonDataConfig) to (logJsonDataConfig:)

Can anyone please suggest, how to fix these.

Source code: github_link

Best would be to modify the Convertto-yaml function in the module to add a parameter that would allow you repeat the section with "& $sectionname" ... something like that. — Hazrelle, Oct 22 '21 at 09:20
I think you'll want `ConvertTo-Yaml -NestingLevel 1` instead of `ConvertTo-Yaml` — Mathias R. Jessen, Oct 22 '21 at 09:25
The --- is removed but the spacin g is like that only as sub elements. — uday kiran, Oct 22 '21 at 11:50
For background on your problem, see [I want to load a YAML file, possibly edit the data, and then dump it again. How can I preserve formatting?](https://stackoverflow.com/q/60891174/347964). In this case, the anchor names will be most definitely lost since PSYaml serializes above the event stream level. Moreover, it doesn't seem to support anchors when writing out YAML, which is why you get duplicate objects instead of aliases. You need to use a tool that allows you to modify YAML at the event level to do what you want, PSYaml doesn't afaik. — flyx, Oct 22 '21 at 18:32

TamusJRoyce · Answer 1 · 2023-05-03T16:09:28.793

From Josep - A Brief introduction to YAML in Powershell:

module	mappings	sequences
PSYaml	OrderedDictionary	Array
powershell-yaml	Hastable	List
powershell-yaml with -Ordered	OrderedDictionary	List
powershell native objects	Hastable	Array

Use powershell-yaml with -Ordered:

$yml = ConvertFrom-YAML $content -Ordered

Note: I haven't figured out formatting of spacing when converting back...converts 2 spaces to 3 spaces in the resulting json format

How to update yaml files from powershell without formatting change?

1 Answers1