46

I've stated before I'm not very good at scripting so what I have below copies files from a share directory to mine where I change their permissions. But then I wanted to remove the first line in each of the CSV files — the headings — and then I wanted to get this to run every hour.

cd /users
scp dehpc14_Disk_Quota_Report.csv /h/u544835
scp dehpc14_User_Disk_Usage.csv /h/u544835
cd /h/u544835
chmod 755 dehpc14_Disk_Quota_Report.csv
chmod 755 dehpc14_User_Disk_Usage.csv

* insert delete csv first line here *

Can anyone help me with deleting the first line of the CSV file and getting this to run automatically every hour, for example, or direct me to something I might understand to get cracking with it?

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
Zenaphor
  • 761
  • 5
  • 13
  • 21

4 Answers4

108

You can delete the first line of a file using sed:

sed -i '' 1d file.csv

If you just want to get the contents of the file without the first line, and without modifying the file, remove the -i '' flag:

sed 1d file.csv

Here 1d is the command to execute:

  • 1 => line where to act
  • d => delete

So 1d means 'delete line 1'

If you want to get the first line, you can use sed too:

sed -n 1p file.csv

Here p stands for 'print' or

sed 1q file.csv

(see William Pursell's comment)

Carlos Campderrós
  • 22,354
  • 11
  • 51
  • 57
  • Thanks very much for both the answer and the explanation – Zenaphor Mar 09 '12 at 11:29
  • 2
    The 'sed 1p file.csv' example will print the first line twice, and every other line once. To print only the first line, you need 'sed -n 1p' (although it is more common to do 'sed 1q' and quit after printing one line.) – William Pursell Mar 09 '12 at 17:33
  • 2
    Note that this answer only applies to GNU `sed`; it does not work with BSD `sed`, also found on Mac OS X. BSD `sed` requires a suffix after the `-i`; if you do not want a suffix, use `-i ''` (with space). But the manual page explicitly recommends against that. See [Unix script to delete the first line on a Mac](http://stackoverflow.com/questions/29482817/) for the details. Other Unix systems (AIX, HP-UX, Solaris, etc) may not have the `-i` option at all; it is not mandated by the POSIX specification for [`sed`](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html) – Jonathan Leffler Apr 07 '15 at 03:27
  • @RKR this shouldn't take long (maybe if you have a huge file). What's your filesize and time to complete? Using exactly which command? – Carlos Campderrós Jan 20 '17 at 14:13
  • This is the command I used. `sed 1d file.csv` .It took more than 20-30 minutes for a 3 GB file. – RKR Jan 21 '17 at 07:29
  • 4
    For folks getting the ERROR `1d: no such file or directory`, do this instead `sed -i '1d' filename.csv` – kev Nov 21 '19 at 07:28
18

I tried to delete the first row from a CSV, cd into the location and then the answer to the question didn't work for me, was getting

sed: can't read 1d: No such file or directory

So, I've used the following stated here by @amit

sed -i '1d' filename.csv
Tiago Martins Peres
  • 14,289
  • 18
  • 86
  • 145
3

To create a new CSV file without the header, you can do the following:

sed 1d file_with_header.csv > file_without_header.csv
Nic Scozzaro
  • 6,651
  • 3
  • 42
  • 46
1

While looking for this answer I also found this other thread. There they say use tail, which doesn't actually make an modifications which is what the OP wanted. You could over course copy to a temporary file. Additionally, if you were going to stream the files into another tool, you could use 'tail' in a pipe and never have to write temporary files or write to disk.

How can I remove the first line of a text file using bash/sed script?

Community
  • 1
  • 1
Andrew Mellinger
  • 319
  • 3
  • 14