106

In PowerShell, what's the difference between Out-File and Set-Content? Or Add-Content and Out-File -append?

I've found if I use both against the same file, the text is fully mojibaked.

(A minor second question: > is an alias for Out-File, right?)

Andrew D. Bond
  • 902
  • 1
  • 11
  • 11
Colonel Panic
  • 132,665
  • 89
  • 401
  • 465

7 Answers7

112

Here's a summary of what I've deduced, after a few months experience with PowerShell, and some scientific experimentation. I never found any of this in the documentation :(

[Update: Much of this now appears to be better documented.]

Read and write locking

While Out-File is running, another application can read the log file.

While Set-Content is running, other applications cannot read the log file. Thus never use Set-Content to log long running commands.

Encoding

Out-File saves in the Unicode (UTF-16LE) encoding by default (though this can be specified), whereas Set-Content defaults to ASCII (US-ASCII) in PowerShell 3+ (this may also be specified). In earlier PowerShell versions, Set-Content wrote content in the Default (ANSI) encoding.

Editor's note: PowerShell as of version 5.1 still defaults to the culture-specific Default ("ANSI") encoding, despite what the documentation claims. If ASCII were the default, non-ASCII characters such as ü would be converted to literal ?, but that is not the case: 'ü' | Set-Content tmp.txt; (Get-Content tmp.txt) -eq '?' yields $False.

PS > $null | out-file outed.txt
PS > $null | set-content set.txt
PS > md5sum *
f3b25701fe362ec84616a93a45ce9998 *outed.txt
d41d8cd98f00b204e9800998ecf8427e *set.txt

This means the defaults of two commands are incompatible, and mixing them will corrupt text, so always specify an encoding.

Formatting

As Bartek explained, Out-File saves the fancy formatting of the output, as seen in the terminal. So in a folder with two files, the command dir | out-file out.txt creates a file with 11 lines.

Whereas Set-Content saves a simpler representation. In that folder with two files, the command dir | set-content sc.txt creates a file with two lines. To emulate the output in the terminal:

PS > dir | ForEach-Object {$_.ToString()}
out.txt
sc.txt

I believe this formatting has a consequence for line breaks, but I can't describe it yet.

File creation

Set-Content doesn't reliably create an empty file when Out-File would:

In an empty folder, the command dir | out-file out.txt creates a file, while dir | set-content sc.txt does not.

Pipeline Variable

Set-Content takes the filename from the pipeline; allowing you to set a number of files' contents to some fixed value.

Out-File takes the data as from the pipeline; updating a single file's content.

Parameters

Set-Content includes the following additional parameters:

  • Exclude
  • Filter
  • Include
  • PassThru
  • Stream
  • UseTransaction

Out-File includes the following additional parameters:

  • Append
  • NoClobber
  • Width

For more information about what those parameters are, please refer to help; e.g. get-help out-file -parameter append.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Colonel Panic
  • 132,665
  • 89
  • 401
  • 465
  • 4
    `Set-Content` default encoding: translated to `(Get-Culture).Textinfo.ANSICodePage` (Windows 8.1, Powershell 4.0, CurrentCulture `cs-CZ`, CurrentUICulture `en-GB`, ANSICodePage `1250`, OEMCodePage `852`, tested using `'řž'`string with different codes in above code pages). – JosefZ Mar 23 '16 at 00:25
  • 1
    Also note that `Out-File` has problems with long lines in certain situations. For example: `$x = [pscustomobject]@{A=('a' * 500); B=('b' * 500)}; $x | Out-File -Path myfile.txt`. – Bacon Bits Feb 11 '18 at 01:49
20

Out-File has the behavior of overwriting the output path unless the -NoClobber and/or the -Append flag is set. Add-Content will append content if the output path already exists by default (if it can). Both will create the file if one doesn't already exist.

Another interesting difference is that Add-Content will create an ASCII encoded file by default and Out-File will create a little endian unicode encoded file by default.

> is an alias syntactic sugar for Out-File. It's Out-File with some pre-defined parameter settings.

Andy Arismendi
  • 50,577
  • 16
  • 107
  • 124
  • Thanks, knowing the encoding differences is useful. You're not quite right, if you do `echo "" > $null | Add-Content abc.txt` it _doesn't_ create the file `abc.txt`, whereas Out-File would. – Colonel Panic May 22 '12 at 11:13
  • @MattHickford That's kinda an odd edge case example. That code pipes to $null so `Add-Content` doesn't receive anything. If `Add-Content` doesn't receive anything why should it create a file? On the other hand the same question could be asked of Out-File. – Andy Arismendi May 22 '12 at 15:03
  • The difference matters to me `gci $folder | Out-File log.txt ; cat log.txt` works whereas `gci $folder | Add-Content log.txt ; cat log.txt` blows up – Colonel Panic May 22 '12 at 15:27
  • @MattHickford I'd probably make sure the file exists before trying to process it. Probably a good habit for all languages. – Andy Arismendi May 22 '12 at 15:51
  • Another difference is that while `Set-Content` is being used, the file is unavailable to other applications. – Colonel Panic Jun 06 '12 at 11:31
10

Well, I would disagree... :)

  1. Out-File has -Append (-NoClober is there to avoid overwriting) that will Add-Content. But this is not the same beast.
  2. command | Add-Content will use .ToString() method on input. Out-File will use default formatting.

so:

ls | Add-Content test.txt

and

ls | Out-File test.txt

will give you totally different results.

And no, '>' is not alias, it's redirection operator (same as in other shells). And has very serious limitation... It will cut lines same way they are displayed. Out-File has -Width parameter that helps you avoid this. Also, with redirection operators you can't decide what encoding to use.

HTH Bartek

BartekB
  • 8,492
  • 32
  • 33
  • 3
    It is an alias in the meaning that `>` and Out-File are the same thing. They call the same code. From Bruce Payette's PowerShell in Action Second Edition (Kindle Locations 4646): `In fact, myScript > file.txt is just “syntactic sugar” for myScript | out-file -path file.txt In some cases, you’ll want to use Out-File directly because it gives you more control over the way the output is written.` – Andy Arismendi May 18 '12 at 18:57
  • 1
    Good point about default formatting (Out-File) vs ToString (Add-Content) – Andy Arismendi May 18 '12 at 19:10
  • My point was: even though both do generally the same, alias has it's meaning in PowerShell... so I wouldn't use this term to describe relation between those too.. ;) Alias replaces command, in this case it should make syntax: ls | > file.txt possible. Obviously, that won't work... – BartekB May 18 '12 at 20:30
  • Right, that's easily discovered via `Get-Alias`. I used the term in the more general sense. – Andy Arismendi May 18 '12 at 20:33
  • FYI `ls | Out-File` is the exact table you see from `ls` whereas `ls | Set-Content` is a simple list of files. – Colonel Panic May 19 '12 at 13:58
  • "Out-File will use default formatting." What exactly is default formatting? – dan-gph Jun 02 '14 at 07:50
  • 1
    Default formatting is the way given object is presented in console. Most of core cmdlets/ object types have formatting metadata that informs PowerShell how to display them in user-friendly fashion. In other words: pipeing results of command to Out-File can be used to save output of command to the file, without loosing formatting done by PowerShell. – BartekB Jun 02 '14 at 09:39
  • 2
    Yes, I think it's an important distinction that `>` is not the exact equivalent for `out-file`. If you set `$PSDefaultParameterValues["Out-File:Encoding"] = "UTF8"`, it will be ignored by `>`. – wisbucky Oct 24 '17 at 09:33
4

Set-Content supports -Encoding Byte, while Out-File does not.

So when you want to write binary data or result of Text.Encoding#GetBytes() to a file, you should use Set-Content.

SATO Yusuke
  • 1,600
  • 15
  • 39
1

Wanted to add about difference on encoding:

Windows with PowerShell 5.1:

  • Out-File - Default encoding is utf-16le
  • Set-Content - Default encoding is us-ascii

Linux with PowerShell 7.1:

  • Out-File - Default encoding is us-ascii
  • Set-Content - Default encoding is us-ascii
JagWireZ
  • 23
  • 6
0

Out-file -append or >> can actually mix two encodings in the same file. Even if the file is originally ASCII or ANSI, it will add Unicode by default to the bottom of it. Add-content will check the encoding and match it before appending. Btw, export-csv defaults to ASCII (no accents), and set-content/add-content to ANSI.

phuclv
  • 37,963
  • 15
  • 156
  • 475
js2010
  • 23,033
  • 6
  • 64
  • 66
0

TL;DR, use Set-Content as it's more consistent over Out-File.

  1. Set-Content behavior is the same over different powershell versions
    Out-File as @JagWireZ says produces different encodings for the default settings, even on the same OS(Windows) the docs for powershell 5.1 and powershell 7.3 state that the encoding changed from unicode to utf8NoBOM

  2. Some issues like Malformed XML arise from using Out-File, that could of course be fixed by setting the desired encoding, however it's likely to forget to set the encoding and end up with issues.

weshouman
  • 632
  • 9
  • 17