0

Can someone point me what I may be missing here? How can put a file (with 100/1000 or lines in an array and whether it's possible. The lines will have spaces/special characters etc.

To mimic what I need is, here's an example. At command line, I can do this to create an array and it works.

[userMe@fastServer goga]$ aaa=("1 my name is koba" "2 you name is shenzi")

[userMe@fastServer goga]$ echo ${aaa[0]}
1 my name is koba

[userMe@fastServer goga]$ echo ${aaa[1]}
2 your name is shenzi

Now, What I want is: If I put those 2 lines in a file /tmp/ks.txt i.e.

[userMe@fastServer goga]$ cat /tmp/ks.txt
1 my name is koba
2 you name is shenzi

Then my questions are:

  1. What command can I run to populate the array? I tried this but it didn't work, it's NOT taking the full first line in [0] and full 2nd line in [1] index.

    aaa=($(cat /tmp/ks.txt | sed "s/^/\"/;s/$/\"/")); echo ${aaa[0]} and ${arr[1]}

  2. How BASH array will work / their limit, when the file has 100000/more of lines in it.

I want to use Arrays (instead of using files) while doing grep command to find bunch of words in these lines. Thought using arrays will be faster than file operations. Please advise otherwise.

AKS
  • 16,482
  • 43
  • 166
  • 258
  • 3
    See [Bash FAQ 001](http://mywiki.wooledge.org/BashFAQ/001) ("How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?") and [Bash FAQ 005](http://mywiki.wooledge.org/BashFAQ/005#Loading_lines_from_a_file_or_stream) ("Loading lines from a file or stream" into an array). – Etan Reisner Sep 10 '15 at 16:05
  • 1
    If you are trying to find *any* of the words in the array in files with grep (all at once) then using a file to store the words one-per-line and using `-Fwf wordfile` is probably the best approach. – Etan Reisner Sep 10 '15 at 16:07
  • Thanks Etan. Actually I found the missing piece. All I need to do was add an extra "|" (after I thought carefully enough) to do "tr '\012' ' '" using eval. I know eval is not that safe but it's working. BTW, I have a grep operation running in a loop which is taking 1 minute to parse Whole SVN RedBook for finding concordance. I want that script to complete under 3 seconds, is that possible using Arrays or some other form? – AKS Sep 10 '15 at 16:09
  • 1
    Using `grep` *properly* is almost certainly going to be faster than anything you can cobble together in `bash`. – chepner Sep 10 '15 at 16:59
  • 1
    Use mapfile instead of a loop. It is *much* faster and no quoting is required. However, looping over a bash array will be an order of magnitude slower than just using grep. – rici Sep 10 '15 at 22:19
  • I agree rici. I have to update BASH to 4.x – AKS Sep 15 '15 at 18:15

1 Answers1

-1

After carefully thinking, it seems like I missed tr to replace the new line character and using eval (which is not that safe but it worked).

$ eval aaa=($(cat /tmp/ks.txt | sed "s/^/\"/;s/$/\"/" | tr '\012' ' '))
$ echo ${aaa[0]}
1 my name is koba
$
AKS
  • 16,482
  • 43
  • 166
  • 258