1

I need to integrate a JSON file which contains paths of the different objects in a PS script that generates and compares the hash files of source and destination. The paths in the JSON file are written in the format that I have stated below. I want to use the paths in that manner and pipe them into Get-FileHash in PowerShell. I can't figure out how to integrate my current PowerShell script with the JSON file that contains the information (File Name, full path) etc.

I have two scripts that I have tested and they work fine. One generates the MD5 hashes of two directories (source and destination) and stores them in a csv file. The other compares the MD5 hashes from the two CSV files and generates a new one, showing the result(whether a file is absent from source or destination).

Now, I need to integrate these scripts in another one, which is basically a Powershell Installer. The installer saves the configurations (path, ports, new files to be made etc etc) in a JSON format. In my original scripts, the user would type the path of the source and destination which needed to be compared. However, I now need to take the path from the JSON configuration files. For example, the JSON file below is of the similar nature as the one I have.

{
"destinatiopath": "C:\\Destination\\Mobile Phones\\"
"sourcepath": "C:\\Source\\Mobile Phones\\"
"OnePlus" : {
"files": [
{
"source": "6T",
"destination: "Model\\6T",
]
}
"Samsung" : {
"files": [
{
"source": "S20",
"destination": "Galaxy\\S20",
}
]

This is just a snippet of the JSON code. It's supposed to have the destination and source files. So for instance if the destination path is: C:\\Destination\\Mobile Phones\\ and the source path is C:\\Source\\Mobile Phones\\ and OnePlus has 6T as source and Model\\6T as destination, that means that the Powershell Installer will have the full path C:\\Destination\\Mobile Phones\\Model\\6T as the destination, and C:\\Source\\Mobile Phones\\6T as the source. The same will happen for Samsung and others.

For now, the MD5 hash comparison PS script just generates the CSV files in the two desired directories and compares them. However, I need to check the source and destination of each object in this case. I can't figure out how I can integrate it here. I'm pasting my MD5 hash generation code below.

Generating hash

#$p is the path. In this case, I'm running the script twice in order to get the hashes of both source and destination. 
#$csv is the path where the csv will be exported. 
Get-ChildItem $p -Recurse | ForEach-Object{ Get-FileHash $_.FullName -Algorithm MD5 -ErrorAction SilentlyContinue} | Select-Object Hash,
    @{
        Name = "FileName";
        Expression = { [string]::Join("\", ($_.Path -split "\\" | Select-Object -Skip ($number))) }
    } | Export-Csv -Path $csv
Aithorusa
  • 113
  • 2
  • 10
  • 1
    I've read your post three times now, I still have no idea what your question is. Please add a meaningful example of what you have (paths? hashes? CSV files?) and the expected output for the given case. – Tomalak Feb 10 '21 at 09:22
  • @Tomalak I have updated the post. I hope that makes it more clear now. Before, I could just take the full path of the desired destination and source folders using `Read-Host` and pipe that into `Get-FileHash`. However, I now need to read the file's path from that JSON file that I have written above. I can't figure out how to do that. – Aithorusa Feb 10 '21 at 09:48

1 Answers1

4

I want to use the paths in that manner and pipe them into Get-FileHash in PowerShell.

As the first step I would reorganize the JSON to be easier to handle. This will make a big difference on the rest of the script.

{
    "source": "C:\\Source\\Mobile Phones",
    "destination": "C:\\Destination\\Mobile Phones",
    "phones": [
        {
            "name": "OnePlus",
            "source": "6T",
            "destination": "Model\\6T"
        },
        {
            "name": "Samsung",
            "source": "S20",
            "destination": "Galaxy\\S20"
        }
    ]
}

Now it's very easy to get all the paths no matter how many "phone" entries there are. You don't even really need an intermediary CSV file.

$config = Get-Content config.json -Encoding UTF8 -Raw | ConvertFrom-Json

$config.phones | ForEach-Object {
    $source_path = Join-Path $config.source $_.source
    $destination_path = Join-Path $config.destination $_.destination

    $source_hashes = Get-ChildItem $source_path -File -Recurse | Get-FileHash -Algorithm MD5
    $destination_hashes = Get-ChildItem $destination_path -File -Recurse | Get-FileHash -Algorithm MD5

    # the combination of relative path and file hash needs to be unique, so let's combine them
    $source_relative = $source_hashes | ForEach-Object {
        [pscustomobject]@{
            Path = $_.Path
            PathHash = $_.Path.Replace($source_path, "") + '|' + $_.Hash
        }
    }
    $destination_relative = $destination_hashes | ForEach-Object {
        [pscustomobject]@{
            Path = $_.Path
            PathHash = $_.Path.Replace($destination_path, "") + '|' + $_.Hash
        }
    }

    # Compare-Object finds the difference between both lists
    $diff = Compare-Object $source_relative $destination_relative -Property PathHash, Path

    Write-Host $diff

    $diff | ForEach-Object {
        # work with $_.Path and $_.SideIndicator
    }
}
Tomalak
  • 332,285
  • 67
  • 532
  • 628
  • Unfortunately I cannot change the JSON file so I will have to do this for all "phones". When I read the JSON file, it displays the output like this: `OnePlus : @{files=System.Object[]}` `Samsung : @{files=System.Object[]}` I may be incorrect here but I'm guessing I need to make a hash table for these values in JSON so that I can read them individually and then maybe use that to read individual values such as OnePlus source and destination? – Aithorusa Feb 11 '21 at 10:30
  • @Aithorusa If the JSON structure is fixed, you can use my answer to [Iterating through a JSON file PowerShell](https://stackoverflow.com/questions/33520699/iterating-through-a-json-file-powershell) to be able to use `ForEach-Object` anyway. Give it a try. Of course this would loop through *all* keys (`destinatiopath`, `sourcepath`, `OnePlus`, `Samsung`), you would have to filter that to get only the device-related keys. – Tomalak Feb 11 '21 at 10:49
  • I see. The hard part for me is reading the PSCustomObject. However, your answer to that post helps me out a lot. Thank you very much. – Aithorusa Feb 11 '21 at 11:00
  • @Aithorusa If there's any general take-away from that, it's: Avoid creating your own objects that put data into keys (like here with `OnePlus` and `Samsung` - those are names, they should live under a `name` key). Dealing with this kind of object typically creates more trouble than it's worth, not only with PowerShell. If you look at it, `sourcePath` and `destinationPath` are the same type of thing. `sourcePath` and `Samsung` are not. Whenever you find yourself doing this, stop & restructure your object. – Tomalak Feb 11 '21 at 11:05
  • Iterating through the JSON file with objects living under `name` would make things a lot easier. However, I am not allowed to change the JSON file and have to make do with this. Manually going through each `phone` such as `OnePlus` and `Samsung` can be done and I can do `Join-Path` as well. But I was looking for a more "automated" way of doing it with `ForEach-Object` instead. I guess I cannot use `ForEach-Object` to go through all phones and will need to do it for each phone individually but as it stands, I think I can loop through the `files` and read `source` and `destination`. – Aithorusa Feb 11 '21 at 11:21
  • @Aithorusa I don't follow? My other answer shows you *exactly* what to do to loop through all the keys with `ForEach-Object`. – Tomalak Feb 11 '21 at 11:29
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/228615/discussion-between-aithorusa-and-tomalak). – Aithorusa Feb 12 '21 at 08:56