3

I'm attempting to create multiple CSV files with no headers using Export-Csv within a PowerShell script. I've read a few posts about using the Select-Object -Skip 1 trick, but I haven't gotten it to successfully work on my code. Here's how I tried to call it:

$DataSet.Tables[0] | Select-Object -Skip 1 |
    Export-Csv -NoTypeInformation -Path "C:\$($csvname[$i])_$timer.csv"

Here's my full, functioning code:

function Run-Query {
    Param([string[]]$queries,[string[]]$csvname)
    Begin {
        $SQLServer = 'myserver'
        $Database = 'mydatabase'
        $SqlConnection = New-Object System.Data.SqlClient.SqlConnection
        $SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
        $timer = (Get-Date -f yyyy-MM-dd)
    } # End Begin
    Process {
        # Loop through each query
        for ($i = 0; $i -lt $queries.Count; $i++) {
            $SqlCmd = New-Object System.Data.SqlClient.SqlCommand

            # Use the current index ($i) to get the query
            $SqlCmd.CommandText = $queries[$i]
            $SqlCmd.Connection = $SqlConnection
            $SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
            $SqlAdapter.SelectCommand = $SqlCmd
            $DataSet = New-Object System.Data.DataSet
            $SqlAdapter.Fill($DataSet)

            # Use the current index ($i) to get the sheetname for the CSV
            $DataSet.Tables[0] | Export-Csv -NoTypeInformation -Path "C:\$($csvname[$i])_$timer.csv"
        }
    } # End Process
    End {
        $SqlConnection.Close()
    }
} # End function run-query.

#Entery Query
$queries = @()
$queries += @'
SELECT * FROM table2
'@
$queries += @'
SELECT * FROM table1
'@

#List of CSV file names.
$csvname = @()
$csvname += 'file1'
$csvname += 'file2'
Run-Query -queries $queries -csvname $csvname
Ansgar Wiechers
  • 193,178
  • 25
  • 254
  • 328
Joe
  • 33
  • 1
  • 1
  • 4
  • The `Select-Object -skip 1` "trick" is for _importing_ CSVs as text files rather than as objects. You can't export a CSV without the header information. Why do you want to? The definition of the CSV format includes the headers. – Jeff Zeitlin Aug 30 '17 at 15:20
  • It's required for the 3rd party that I'm transferring the file to. There's no way to get around this with powershell? – Joe Aug 30 '17 at 15:21
  • @Joe Your 3rd party requires a comma separated file without headers? How do they know what any of the fields mean? – Maximilian Burszley Aug 30 '17 at 15:33
  • Possible duplicate of [Powershell export-csv with no headers?](https://stackoverflow.com/questions/26389952/powershell-export-csv-with-no-headers) or [Powershell Export-Csv with no header row](https://stackoverflow.com/q/29246005/150605) – Lance U. Matthews Aug 30 '17 at 16:24
  • Your only choices are to either manually compose your comma-separated lines and write them out as a text file, or write it out to a tempfile using Export-CSV, then read the tempfile as a text file (e.g., with `Get-Content`), dropping the first line, and writing it back out without that first line. Or see the questions that @BACON linked to. – Jeff Zeitlin Aug 30 '17 at 17:01
  • @JeffZeitlin, I know your comment is old. But I just want to ask you (and anyone else who might know) anyway. Why did you say the definition of the CSV format includes the headers? I can see RFC4180 ( https://tools.ietf.org/html/rfc4180 ) treats header thing as optional. – Alexander Ites May 08 '20 at 11:24

2 Answers2

6

You're misunderstanding how CSVs work in PowerShell. Assume you have a CSV file with the following content:

foo,bar,baz
A,1,2
B,3,4

Importing that CSV via Import-Csv will give you 2 objects with the properties foo, bar, and baz filled with the values from one data row each. Using JSON notation the list of objects would look like this:

[
    {
        "foo": "A",
        "bar": "1",
        "baz": "2"
    },
    {
        "foo": "B",
        "bar": "3",
        "baz": "4"
    }
]

Export-Csv does basically the same, just in reverse. I takes objects as input and writes the values of their properties to the fields of the CSV. The fields are determined by the properties of the first object Export-Csv receives. If one of these properties is missing in a subsequent object a NULL value is written to the respective field. If a subsequent object has additional properties that the first object didn't have they are ignored.

Using Select-Object -Skip 1 with either Import-Csv or Export-Csv is useless, because normally you don't want to skip any of the input or output objects (otherwise you'd lose data).

There are, however, two other cmdlets similar to Import-Csv and Export-Csv that read and write strings rather than files: ConvertFrom-Csv and ConvertTo-Csv. Import-Csv is basically a combination of Get-Content and ConvertFrom-Csv, and Export-Csv a combination of ConvertTo-Csv and Set-Content.

So, where does Select-Object -Skip 1 come into play here? When you convert your object input to CSV via ConvertTo-Csv and skip the first row of the string output, you effectively remove the header line from the output text, something you couldn't do with Export-Csv.

Demonstration:

PS C:\> $data = @()
PS C:\> $data += [PSCustomObject]@{"foo"="A"; "bar"="1"; "baz"="2"}
PS C:\> $data += [PSCustomObject]@{"foo"="B"; "bar"="3"; "baz"="4"}
PS C:\> $data | Format-Table -Auto

foo bar baz
--- --- ---
A   1   2
B   3   4

PS C:\> $data | ConvertTo-Csv -NoType
"foo","bar","baz"
"A","1","2"
"B","3","4"
PS C:\> $data | ConvertTo-Csv -NoType | Select-Object -Skip 1
"A","1","2"
"B","3","4"

Pipe that output into Set-Content and you have the headerless CSV files you want.

$DataSet.Tables[0] |
    ConvertTo-Csv -NoType |
    Select-Object -Skip 1 |
    Set-Content "C:\$($csvname[$i])_$timer.csv"
Ansgar Wiechers
  • 193,178
  • 25
  • 254
  • 328
0

Here's one way to output CSV input objects as tab-separated data without headers:

function ConvertTo-Tsv {
  param(
    [parameter(ValueFromPipeline=$true)]
    [Object[]] $Object
  )
  process {
    foreach ( $objectItem in $Object ) {
      ($objectItem.PSObject.Properties | Select-Object -ExpandProperty Value) -join "`t"
    }
  }
}

You could then do either of the following:

Import-Csv "inputfile.csv" | ConvertTo-Tsv

or

$data = Import-Csv "inputfile.csv"
ConvertTo-Tsv $data
Bill_Stewart
  • 22,916
  • 4
  • 51
  • 62
  • I added "Import-Csv "$($csvname[$i])_$timer.csv" | ConvertTo-Tsv" right under my "Export-Csv" line. This drastically increased the execution time of the script, but didn't seem to have any affect on the headers. – Joe Aug 30 '17 at 15:51
  • The function provided by this answer is a good solution, but the examples for how to use it are misleading. The source of the data is an SQL database, not another CSV file so there's no need to use `Import-Csv`. Instead, replace `Export-Csv` with `ConvertTo-Csv` like so: `$DataSet.Tables[0] | ConvertTo-Tsv | Set-Content -Path "C:\$($csvname[$i])_$timer.csv"` – Lance U. Matthews Aug 30 '17 at 16:11
  • Correct, my example is generic (referring to CSV) -- it's assumed that the user is responsible for passing the correct objects into the pipeline for the function. – Bill_Stewart Aug 30 '17 at 16:21