-1

I have a .tsv consisting of a key and two values. I want to create two dictionaries, one mapping from the key to the first value for each line in the file, and the other mapping from the key to the second value. I can do this:

v1Dict = {key: v1 for (key,v1,v2) in (line.split("\t") for line in (open (myinputfile)))}
v2Dict = {key: v2 for (key,v1,v2) in (line.split("\t") for line in (open (myinputfile)))}

But that's obviously inefficient as it reads/parses the entire file twice.

Alternately, I can do this:

v1Dict = {}
v2Dict = {}
for (key,v1,v2) in (line.split("\t") for line in (open (myinputfile))):
    v1Dict[key]=v1
    v2Dict[key]=v2

Is that the "best" way to do it?

PurpleVermont
  • 1,179
  • 4
  • 18
  • 46
  • 1
    you need to extend this i am afraid. you need to make sure the file is closed. and the file system io overhead should be larger than current computational overhead. – Jason Hu Mar 31 '15 at 16:50

1 Answers1

0

You can use with:

v1Dict = {}
v2Dict = {}
with open(myinputfile) as f:
    for(key, v1, v2) in [line.split("\t") for line in f]:
        v1Dict[key] = v1
        v2Dict[key] = v2

Alternatively, you could make this into a one-liner:

f = open(myinputfile);v1Dict,v2Dict=[{key: v1 for (key,v1,v2) in (line.split("\t") for line in f)}, {key: v2 for (key,v1,v2) in (line.split("\t") for line in f)}]

Look here for more information on using with.

Community
  • 1
  • 1
A.J. Uppal
  • 19,117
  • 6
  • 45
  • 76
  • I think you want to replace `(open(myinputfile))` with `f` if you use `with`. What does the `with` gain me over using `in`? – PurpleVermont Mar 31 '15 at 16:33
  • @PurpleVermont `with` is more pythonic, it opens the file once, and automatically closes it at the end. – A.J. Uppal Mar 31 '15 at 16:33
  • I assumed that the `in` also did that when using it in a comprehension like that. – PurpleVermont Mar 31 '15 at 16:34
  • Your list comprehension also opens it once, but you do not close the file at the end which may lead to unnecessary issues. You asked for which method is the best, and from a perspective of memory usage and "pythonicness", so to speak, `with` is better. – A.J. Uppal Mar 31 '15 at 16:35
  • The with statement should be the other way with "open(filename) as f" – skar Mar 31 '15 at 16:36
  • I don't think you can compare it with "in". "With" statement handles events like race condition and closing the file in case a failure occurs. When opening a file its considered the best practice in Python. `https://docs.python.org/2/reference/compound_stmts.html#the-with-statement` – skar Mar 31 '15 at 16:39
  • I have a number of lines in my program like `fooList = [str.lower(line.rstrip()) for line in open(fooFile)]` -- do I need to either use with or close the file later? – PurpleVermont Mar 31 '15 at 16:41
  • I would suggest using `with`, but you can also open it as you did above and close it later. – A.J. Uppal Mar 31 '15 at 16:43
  • Also, if this answer helped you, would you mind accepting/upvoting it? Thanks :) – A.J. Uppal Mar 31 '15 at 16:43
  • Thanks for the reminder to use `with` but the key point of my question is really whether I have to do the iteration explicitly like that or whether there's a clever way to create both dictionaries at once with a comprehension-type approach – PurpleVermont Mar 31 '15 at 16:43
  • Just curious why do you need this? Who is generating the tsv? – skar Mar 31 '15 at 16:44
  • 1
    No you cannot create both dictionaries with a comprehension, you can only create one object with those by accessing an `open`, because if you try to create two, it `open`s the file each time – A.J. Uppal Mar 31 '15 at 16:44
  • you can do `v1Dict[key], v2Dict[key] = v1, v2` if it helps – skar Mar 31 '15 at 16:47
  • Yes, @kartik, true, and you can also make it into a pseudo-one-liner, as I demonstrated in my edit above :) – A.J. Uppal Mar 31 '15 at 16:48
  • @kartik, if this answer helped you, would you mind upvoting it? Thanks :) – A.J. Uppal Mar 31 '15 at 16:48
  • @A.J. You want to address it to PurpleVermont :) – skar Mar 31 '15 at 16:55
  • The one-liner splits the line twice though, so that is not efficient – PurpleVermont Mar 31 '15 at 17:45
  • @A.J. I would upvote (and ultimately accept if no better answer comes along in the next 24 hours or so) it if the answer (rather than the comments) explains that you can't do it with a single comprehension, and acknowledges that the one-liner(and any dual comprehension) is inefficient because it parses the file twice (and loses the advantages of with. Based on what I've gleaned from the comments the answer should be -- you can't do it with a single comprehension, using multiples is wasteful, my way was approximately the right way to do it, but I should use "with" and explain why briefly.... – PurpleVermont Mar 31 '15 at 20:06
  • Your answer focuses on using "with" which was kind of incidental to my question. It's useful, but it should be put into context. – PurpleVermont Mar 31 '15 at 20:07