Powershell Get-Content specific content inside text

Question

I receive a text file with a multiple lists like shown below (edit: more accurate example dataset included)

# SYSTEM X
# SINGULAR
192.168.1.3
# SUB-SYSTEM V
192.168.1.4
192.168.1.5
192.168.1.6
# SYSTEM Y
# MANDATORY
192.168.1.7
192.168.1.8
192.168.1.9
192.168.1.7
192.168.1.8
192.168.1.9

Each "SYSTEM comment" means its a new set after it. I want to read each block of content separately so each set should be assigned to an object discarding the embedded comments. I just need the IPs. Something like:

$ipX = get-content -path [file.txt] [set X]
$ipY = get-content -path [file.txt] [set Y]
$ipZ = get-content -path [file.txt] [set Z]

But I'm not sure how to actually assign these sets separately. Help please.

You will have to read the file into a temp variable and then "walk through" the resulting array to find where you need to partition it into the separate sets. — Jeff Zeitlin, Oct 30 '20 at 15:24
Please show us an **accurate** example file, or are all IP addresses really embedded in square brackets? — Theo, Oct 30 '20 at 15:34
Theo, I have included a better example of the dataset I receive. It usually has about 150 IPs for about 5 systems. — markfree, Oct 31 '20 at 00:32

marsze · Accepted Answer · 2020-10-30T15:39:22.753

Here's one possible solution. The result will be a hashtable, each key containing any array of ips for the set:

$result = @{}
get-content file.txt | foreach {
    if ($_ -match "#\s*SET\s+(\w+)") {
        $result[($key = $matches.1)] = @()
    }
    elseif ($_ -notlike "#*") {
        $result[$key] += $_
    }
}

Contents of $result:

Name                           Value                                                                                                                                                                                  
----                           -----                                                                                                                                                                                  
Y                              {[ip], [ip], [more ips]}                                                                                                                                                               
Z                              {[ip], [ip], [more ips]}                                                                                                                                                               
X                              {[ip], [ip], [more ips]}

After trying this solution for a while I wasn't able to make it work. It says index is null at ` $result[$key] += $_` — markfree, Nov 03 '20 at 20:06
Using my example above, I switched to something like: `$sys_name = "SYSTEM"` (...) `-match "^#\s*\w*\W*$sys_name\s+(\w+)"` to account for "-SYSTEM". — markfree, Nov 03 '20 at 20:42
@markfree I guess it's just a matter of finding the right regex here. Use an online regex tool, to check if your regex matches what you expect. — marsze, Nov 03 '20 at 20:44

score 0 · Answer 2 · answered Oct 30 '20 at 16:04

Here's another approach. We will take advantage of Foreach-Object's -End block to [PSCustomObject] the final one.

Get-Content $file | Foreach-Object {
    if($_ -match 'SET (.+?)'){
        if($ht){[PSCustomObject]$ht}
        $ht = [ordered]@{Set = $Matches.1}
    }
    if($_ -match '^[^#]'){
        $ht["IPs"] += $_
    }
} -End {if($ht){[PSCustomObject]$ht}}

Output

Set IPs               
--- ---               
X   [ip][ip][more ips]
Y   [ip][ip][more ips]
Z   [ip][ip][more ips]

If you want to also ensure $ht is empty to start with you could use the -Begin block.

Get-Content $file | Foreach-Object -Begin{$ht=$null}{
    if($_ -match 'SET (.+?)'){
        if($ht){[PSCustomObject]$ht}
        $ht = [ordered]@{Set = $Matches.1}
    }
    if($_ -match '^[^#]'){
        $ht["IPs"] += $_
    }
} -End {if($ht){[PSCustomObject]$ht}}

AdminOfThings · Answer 3 · 2020-10-30T17:19:59.537

You can use Select-String to extract a specific section of text:

# Update $section to be the set you want to target
$section = 'Set Y'
Get-Content a.txt -Raw |
    Select-String -Pattern "# $section.*\r?\n(?s)(.*?)(?=\r?\n# Set|$)" | Foreach-Object 
        {$_.Matches.Groups[1].Value}

Using Get-Content with -Raw reads in the file as a single string making multi-line matching easier. With PowerShell 7, Select-String includes a -Raw switch making this process a bit simpler.

This outputs capture group 1 results, which match the (.*?). If you want to capture between comments rather than between Set <something> and Set <something>, you can edit the -Pattern value at the end to only be # rather than # Set.

Regex Breakdown:

# matches the characters # literally
$section substitutes your variable value matches the value literally provided there are no regex characters in the string
.* matches any character (except for line terminators)
\r matches a carriage return
? Quantifier — Matches between zero and one times, as many times as possible, giving back as needed (greedy)
\n matches a line-feed (newline) character
(?s) modifier: single line. Dot matches newline characters
1st Capturing Group (.*?)
.*? matches any characters lazily
Positive Lookahead (?=\r?\n# Set)
\r? matches a carriage return zero or more times
\n matches a line-feed (newline) character
# Set matches the characters # Set literally
$ matches the end of the string

score 0 · Answer 4 · answered Oct 31 '20 at 10:21

If I understand the question with the new example correctly, you want to parse out the file and create single variables of that each holding an array ip IP addresses.

If that is the case, you could do:

# loop through the file line-by-line
$result = switch -Regex -File 'D:\Test\thefile.txt' {
    '#\sSYSTEM\s(\w+)' {
        # start a new object, output the earlier object if available
        if ($obj) { $obj }
        $obj = [PsCustomObject]@{ 'System' = $Matches[1]; 'Ip' = @() }
    }
    '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}' {
        # looks like an IPv4 address. Add it to the Ip property array of the object
        $obj.Ip += $_
    }
    default {}
}

Now you have an array ob objects in $result:

System Ip                                                     
------ --                                                     
Y      {192.168.1.7, 192.168.1.8, 192.168.1.9, 192.168.1.7...}
X      {192.168.1.3, 192.168.1.4, 192.168.1.5, 192.168.1.6}

To make separate variables of that is as easy as:

$ipX = ($result | Where-Object { $_.System -eq 'X' }).Ip
$ipY = ($result | Where-Object { $_.System -eq 'Y' }).Ip
$ipZ = ($result | Where-Object { $_.System -eq 'Z' }).Ip

^{Your example has duplicate IP addresses. If you don't want these do

$ipX = ($result | Where-Object { $_.System -eq 'X' }).Ip | Select-Object -Unique (same for the others)}

The duplicate IPs were lazy coping of mine for the example. In reality they are/must be unique. I really appreciate your insight. I'm studing it. — markfree, Nov 03 '20 at 20:48

Powershell Get-Content specific content inside text

4 Answers4