Is there a "split" equivalent in Powershell?

Question

I am looking for a PowerShell equivalent to "split" *NIX command, such as seen here : http://www.computerhope.com/unix/usplit.htm

split outputs fixed-size pieces of input INPUT to files named PREFIXaa, PREFIXab, ...

This is NOT referring to .split() like for strings. This is to take a LARGE array from pipe and then be stored into X number of files of each with the same number of lines.

In my use case, the content getting piped is list of over 1Million files...

Get-ChildItem $rootPath -Recurse | select -ExpandProperty FullName | foreach{ $_.Trim()} | {...means of splitting file here...}

Sort is a blocking cmdlet. It's going to sabotage your pipeline memory efficiency because all of the input will have to accumulate there before anything continues on down the pipeline. — mjolinor, Mar 30 '16 at 22:24
Is your intent to split each file into X smaller files? Or to split the content of all files combined into X smaller files? So files a.txt, b.txt, c.txt each have 20 lines, so 60 lines total... Do you want to split it into 5 files with 12 lines each, or split each file into 5 files, and end up with 15 files that have 4 lines each? Or are you looking to just get the paths, and split the array of paths into 5 files, each with the same number of lines? — TheMadTechnician, Mar 31 '16 at 00:12

score 1 · Answer 1 · edited May 23 '17 at 10:28

I don't think it exists a CmdLet doing exactly what you want. but you can quickly build a function doing that.

It's a kind of duplicate of How can I split a text file using PowerShell? and you will find more scripts solutions if you google "powershell split a text file into smaller files"

Here is a peace of code to begin, my advice is to use the .NET class System.IO.StreamReader to handle more efficiently big files.

$sourcefilename = "D:\temp\theFiletosplit.txt"
$desFolderPathSplitFile = "D:\temp\TFTS"
$maxsize = 2 # The number of lines per file 
$filenumber = 0
$linecount = 0

$reader = new-object System.IO.StreamReader($sourcefilename)

while(($line = $reader.ReadLine()) -ne $null)
{
  Add-Content $desFolderPathSplitFile$filenumber.txt $line
  $linecount ++
  If ($linecount -eq $maxsize)
  {
    $filenumber++
    $linecount = 0
  }
}
$reader.Close()
$reader.Dispose()

Is there a "split" equivalent in Powershell?

1 Answers1