-2


do you have any experience with such a long dictionaries in one line 20.000? I am working on a module that has such. First I imported this dictionary with json, but I feel like it's better to import as least as possible. The code is not meant to be ever opened by the users and is working well for now. Do you have any experience with line that has 20.000 characters? can it cause any problems in the future? Thanks

Jsindl
  • 105
  • 1
  • 7
  • 4
    20k characters long I presume? That's nothing, The real question is what are you trying to achieve, what's the purpose and if you can store it in other way that may be more suitable(database for instance). – Alexander Ejbekov Nov 07 '18 at 12:44
  • 1
    are you getting any errors? it's not exactly clear what you're asking, are you able to provide a minimal working example? – vencaslac Nov 07 '18 at 12:45
  • Do you mean a={key:'val...20,000...ue'} or a={k1:val1, k2:val2, k19999:val19999}? The second is ok, the first seems like an invitation to refactor. I have used tmp = [line.strip() for line in open(inFile)] for files with far more the 20000 lines with no problem. – Paul Smith Nov 07 '18 at 12:48
  • I am not getting any error and yes it is the second one. {1:1,2:2,3:3....20000:20000}.. it's just seems a bit suspicous but it's working. I just wanted to know if there's a trick that in some cases it wouldn't work – Jsindl Nov 07 '18 at 13:44

1 Answers1

1

One practical problem you could run into is LINE_MAX. On *nix systems, that's the maximum length of a line which is guaranteed to be supported by all the usual utilities like grep and so on. The value promised by POSIX is 2048 bytes, but some systems have a larger value like 4096.

So, if you do decide to have 20,000 characters on a single line, you can't expect the usual utility programs to be able to operate on that file.

Ref: https://www.gnu.org/software/libc/manual/html_node/Utility-Minimums.html

John Zwinck
  • 239,568
  • 38
  • 324
  • 436
  • I don't think the GNU utils still have this limitation. I just tried grep on a file with 2GB lines, and it happily finds patterns in the middle of these lines. – Sven Marnach Nov 07 '18 at 13:36
  • 1
    At least for grep, this is [documented](https://www.gnu.org/software/grep/manual/grep.html): "Though grep expects to do the matching on text, it has no limits on input line length other than available memory, and it can match arbitrary characters within a line." – Sven Marnach Nov 07 '18 at 13:39