how do you select just one property from an array of "results" from the Azure command 'az group list in powershell'?

Question

I am trying to select just one property from an array of "result" (objects?) which come back from the Azure command az group list in powershell?

I know this sounds trivial, but here's where it gets strange, and I hope there is a simple explanation.

If I run the Azure command az group list -o table (after I have succesfully logged in using az login) I get the following typical response

PS src> az group list -o table             
Name              Location    Status
----------------  ----------  ---------
group0            westeurope  Succeeded
group1            westeurope  Succeeded
group2            uksouth     Succeeded
group3            westeurope  Succeeded
group4            westeurope  Succeeded
group5            westeurope  Succeeded
group6            westeurope  Succeeded
group7            uksouth     Succeeded
group8            westeurope  Succeeded
group9            westeurope  Succeeded

however, if I try to select just the Name property by doing

az group list | select -p name

Then i get about 2 screens full of empty lines, with nothing displayed. So the question is, what's wrong with the command above? And how should I fix it?

I tried the following experiments to dig into the exact types of objects being returned and get some results that I don't understand. I'm hoping this will make sense to someone with more Azure and powershell experience.

Here's the steps to reproduce the problem very easily, assuming you have an azure account.

1. start powershell, e.g. on mac terminal, type pwsh
2. log in to azure az login
3. type az group list -o table

observe that the list comes back and is formatted correctly.

4. type az group list | select -p name

observe a few screens full of blank lines. no text.

5. scratch your head and wonder whats just happened? (grin)

THE PLOT THICKENS

az group list on it's own returns a few screens full of this

[
 ... lots of these ...
 {
    "id": "/subscriptions/this-is-super-wierd-i-cant-select-name-prop/resourceGroups/spikes",
    "location": "westeurope",
    "managedBy": null,
    "name": "spikes",
    "properties": {
      "provisioningState": "Succeeded"
    },
    "tags": {},
    "type": null
  }
]

however, (az group list).getType() returns

PS src> (az group list).getType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

and lastly, hopefully the last 2 pieces of the puzzle

PS src> (az group list)[0].getType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String                                   System.Object

so the return types from az group list appear to be an array of objects or maybe it's an array of object[], my powershell is scratchy here. So to double check, I query for the first 10 elements of that array by doing...(az group list)[0..10] and that returns bizarely 10 strings!. Ok, I know it's supposed to be 10 strings, only because it's a computer and if that's what it is, then, that's what it really is. I just dont understand why.

[
  {
    "id": "/subscriptions/12345678-1234-1234-1234-123456789012/resourceGroups/somegroup",
    "location": "westeurope",
    "managedBy": null,
    "name": "admin",
    "properties": {
      "provisioningState": "Succeeded"
    },
    "tags": {},
    "type": null

So all of this, to cut a long story short, is I'm wanting to know, how do you select just one property from the result of an azure query? In my case, I simply want to display the names of all my resource groups.

This az group list | select -p name should work, but does not, and I'd like to know how to do it properly, and in the process find out why it didn't work, and we can all learn something about Azure and powershell in the process, and life can be great!

Love you all, Alan

Answer 1

Let's work through this. When we specify -o table e.g.:

az group list -o table

We say to Azure PowerShell CLI take the JSON content you get, and format it into a nice table for us. Typically, we don't want to work with RAW JSON, and we don't want to work with formatted tables either. Working with string arrays in PowerShell are also not a nice thing to use. In PowerShell, we want to work with "nice" easy objects. So, let's look at the other ways to get our information. Let's take your example and simply save it to a variable that we can look at:

$GroupList = az group list

Then if we look as the type:

PS C:\> $GroupList.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

Indeed, we have an array of objects. They must be an array of our groups... right?... well no. It's not what you think. If we look at the size of the array and return the first few elements of the array we will see what's going on:

PS C:\> $GroupList.Count
125
PS C:\Temp> $GroupList[0..2]
[
  {
    "id": "/subscriptions/12345678-1234-1234-1234-123456789012/resourceGroups/somegroup",

Well, that's not what we wanted. The array is far to large... and looking at the content reveals that what we actually have is an array of each line of the JSON text. This means that running:

az group list | select -p name

or:

$GroupList | select -p name

Is saying, loop through the array, and only output the "Name" property. Since the "Name" property does not exist on a string of text, it simply outputs a blank line. Since there are a few hundred lines of text, we get a few hundred lines of blanks.

So why does PowerShell take the input and break it into an array of strings separated by new lines? Isn't this kinda hard to use? Isn't this not a "great" way to handle a JSON formatted text? Why don't we just get one giant string? Isn't that easier to handle and parse? What's the one reason for this oddness?

Well with PowerShell, the need to support pipelines drives decisions on how we output objects:

"The primary purpose ... is to provide a way to ensure that the result of a pipeline execution is always an indexable collection." Quote

This is why we get an array of objects outputted (See: @mklement0 answer here for more in depth discussion) to support pipeline operations. And if we look at how text files are read and written to, we can highlight exactly why we end up with this specific cough Programmer cough convenience cough... I mean weirdness.

To set things up, we can pipe the output directly to a text file:

az group list | Out-File -FilePath List.json

Woah, wait a second, why did that just work? (In situations like this, I like to say that PowerShell does Magic!), Don't we have to mess with looping through arrays, appending strings terminated with newlines to get one giant contiguous block of text that ends with an EOF, and exactly matches our desired text file?

A simplified reason behind what really happens? Well, out of programmer convenience, Out-File takes the array of strings, iterates through it, and does a simple File.WriteLine() for each string (not bad for a 3 line for loop!). Hence, we just generated a nice JSON formatted text file complete with newlines without breaking a sweat. Reading it back in:

PS C:\> $ListFromFile = Get-Content .\List.json
PS C:\> $ListFromFile.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array


PS C:\> $ListFromFile.Count
125

Does the reverse. It takes the file, does a File.ReadLine(), appends the strings to an array, and returns it. That's why we end up with an object array that contains strings.

Now, what do we really want? Well we know from the start, we don't want to work with one giant string, and we especially don't want to work with strings in an object array, what we want to work with is a nice native PSCustomObject that we can access. That's what PowerShell works best with, that's what we work best with. So, we simply have to convert our (big air quotes) "text input", which we know is formatted as JSON, and convert it into an object:

$List = $GroupList | ConvertFrom-Json

And looking at the count and properties:

PS C:\> $List.Count
10
PS C:\> $List.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array


PS C:\> $List[0].GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    PSCustomObject                           System.Object

We see the count now matches how many groups we have, and the types are no longer an array of strings, but actual objects. So... Now, we can start sorting and selecting:

PS C:\> $List | select -Property Name

Name            
----
group0
group1
group2
group3
group4
group5
group6
group7
group8
group9

And we get the output that we really wanted.

Answer 2

wow, so much text for something as simple as:

az group list --query '[].name' -o tsv

In the query, we use [] because the output of a list command is an array. Next, we select the single column that we want, name. If we wanted to select more than one column, we would use [name, BaseType]. All "list" commands in the Azure CLI return an array, but if use any other CLI command which returns a single result, we would simply use --query 'name'.

By default, the output of a Azure CLI Command is json. We use --output or -o to change the format, and we select tsv because it provides the cleanest output.