0

I have the below code:

data['ShortLongFlag'] = data['End DateTime'].apply(lambda x:
                                                   -1 if (x.month == 3 and x.date() in shortlongdates == True) else (1 if (x.month == 10 and x.date() in shortlongdates == True) else 0))

enter image description here

What I'm trying to do is the following:

Create a new column in my dataframe which is populated with either -1, 0 or 1 based on the following conditions:

  • -1 if the month of the value in my datetime column is equal to 3 and the date is in my list of dates called "shortlongdates"
  • 1 if the month of the value in my datetime column is equal to 10 and the date is in my list of dates called "shortlongdates"
  • 0 otherwise

Right now all the values are being outputted as 0 in the new column... why?

SlowlyLearning
  • 155
  • 6
  • 2
    There's really no reason why you wouldn't create a proper function here rather than trying to compress things into a `lambda` – roganjosh Sep 28 '21 at 17:13
  • Could you maybe share some data? – Cimbali Sep 28 '21 at 17:13
  • @roganjosh do you have any suggestions? Or can you explain why the lambda doesn't work? It doesn't feel that compressed to me – SlowlyLearning Sep 28 '21 at 17:17
  • 1
    lambdas are functions. See https://stackoverflow.com/questions/16501/what-is-a-lambda-function/16509#16509 The problem is that we have to scroll like mad to read your function – 2e0byo Sep 28 '21 at 17:18
  • I've removed the extra spaces at the end of the scroll and also included a screenshot. – SlowlyLearning Sep 28 '21 at 17:22
  • 1
    @SlowlyLearning the problem isn't that we don't know how to scroll ;) but that putting everything on one line is inherently unreadable. – 2e0byo Sep 28 '21 at 17:23

2 Answers2

2

The reason for this problem is chaining comparison operators

Comparisons can be chained arbitrarily, e.g., x < y <= z is equivalent to x < y and y <= z, except that y is evaluated only once (but in both cases z is not evaluated at all when x < y is found to be false).

Comparisons, including membership tests and identity tests have same precedence.

i.e.

x.month == 3 and x.date() in shortlongdates == True

is same as

x.month == 3 and x.date() in shortlongdates and shortlongdates == True

Note that it can be written as x.month == 3 and x.date() in shortlongdates, or use brackets. However as already stated in comments this lambda is better written as regular function.

def replace_value(x):
    if x.date() in shortlongdates:
        return {3:-1, 10:1}.get(x.month, 0)
    return 0

I will leave up to you to convert this back to [simpler] lambda if you insist.

buran
  • 13,682
  • 10
  • 36
  • 61
  • Whilst I like your dict simplification at the end it might be worth pointing out how it works – 2e0byo Sep 28 '21 at 21:25
0

So we have this unreadable thing:

lambda x: -1 if (x.month == 3 and x.date() in shortlongdates == True) else (1 if (x.month == 10 and x.date() in shortlongdates == True) else 0)

Let's write it out as a standard function:

def filterfn(x):
    if x.month == 3 and x.date() in shortlongdates:
        return -1
    elif x.month == 10 and x.date() in shortlongdates:
        return 1
    else:
        return 0

That removing some stranger tests (redundant == True) and brackets.

Is x.date() really a callable? Should it be x.date? Without seeing your data and listofdates it's impossible to say how this might fail, but at least we can see what it's doing.

2e0byo
  • 5,305
  • 1
  • 6
  • 26
  • Just in case OP is unsure, after defining `filterfn`, as above, it can be used as follows: `data['ShortLongFlag'] = data['End DateTime'].apply(filterfn)` i.e., where you had the lambda previously, you now just say `filterfn`. – 4D45 Sep 28 '21 at 18:36
  • No sense evaluating `x.date() in shortlongdates` twice; if it's false, you can return 0 immediately before even examining `x.month`. – chepner Sep 28 '21 at 19:21
  • 1
    @chepner this answer is *a very long way* from ideal. I'll probably delete it. It merely presents what the lambda was *supposed* to do as a fn. I deliberately avoided simplifying it, waiting for the OP to get back with the more info. buran has the right (simplified) answer below – 2e0byo Sep 28 '21 at 21:23