9

I have a table where every person has a record for every day of the year. I used this function to achieve a running total based on the daily balance column

CALCULATE(
SUM(Leave[Daily Balance]),
FILTER(
   ALLEXCEPT(Leave, Leave[Employee Id]),
   Leave[Date] <= EARLIER(Leave[Date])
))

but I need the running total to restart from 1 if Type = Working AND the running total of Daily Balance is less than zero AND the Type of the previous row is not equal to Working. Below is a screen shot from Excel. The required function column is what I need to get to.

enter image description here

RADO
  • 7,733
  • 3
  • 19
  • 33
LynseyC
  • 93
  • 1
  • 5
  • 1
    On the row for November 5, Person 1, suppose our test data had a blank in type. Would the 'required function' return a 1 or a 2 on November 6? – Ryan B. Feb 03 '20 at 14:42
  • It would return a 2 for November 6. The "reset" wouldn't happen because November 5 would be 1 (not a negative number). Thanks for your detailed post. I am reviewing today – LynseyC Feb 05 '20 at 10:59

6 Answers6

5

Overview

This is a challenging thing to ask PowerBI to do, so a tidy approach may be difficult to find.

The biggest issue is that PowerBI’s data model does not support the concept of a running tally – at least not the way we do it in Excel. In Excel, a column can reference values that occur in the ‘previous row’ of that same column and then be adjusted by some ‘daily change’ listed in a different column.

PowerBI can only imitate this by adding up all the daily changes over some subset of rows. We take the date value in our current row and create a filtered table where all dates are less than this current row’s date, and then sum up all the daily changes from that subset. This may seem to be a subtle difference, but it is quite significant:

This means that there’s no way to ‘override’ our running total. The only math that’s being done is happening on the column containing daily changes – the column containing ‘running total’ is only a result – it is never used in any subsequent row’s calculation.

We must abandon the concept of ‘reset’ and instead imagine making a column that contains an ‘adjustment’ value. Our adjustment will be a value that can be included so that when the described conditions are met, the total of daily balances and adjustments will sum to 1.

If we look at the calculated running given by OP, we see that the value of our running total on a ‘non-working’ day just prior a ‘working’ day gives us that needed amount that, if reversed, would sum to zero and cause the running total on each following working day to increase by one. This is our desired behavior (with one problem to be described later on).

Result

enter image description here

Most Recent Date Prior to Work = 

CALCULATE(
Max(Leave[Date]),
FILTER(
   ALLEXCEPT(Leave, Leave[Id]),
   Leave[Date] = EARLIER(Leave[Date]) -1 && Leave[Type] <> "Working" && Earlier(Leave[Type]) = "Working"
))

It helps to know the difference between row and filter contexts and how EARLIER operates to follow this calculation. In this scenario, you can think of "EARLIER" as meaning 'this reference points to the value in the current row" and otherwise a reference points to the whole table returned by "ALLEXCEPT(Leave, Leave[Id])." In this way, we find the places where the current row has type "Working" and the prior day's row has some other type.

Most Recent Date Prior to Work Complete = 

CALCULATE(
Max(Leave[Most Recent Date Prior to Work]),
FILTER(
   ALLEXCEPT(Leave, Leave[Id]),
   Leave[Date] <= EARLIER(Leave[Date])
))

This calculation imitates a 'fill down' kind of operation. It says, "When looking at all the rows whose date is before the date on THIS row, return the biggest value in 'Most Recent Date Prior to Work."

Daily Balance Adjustment = 

CALCULATE(
SUM(Leave[Running Daily Balance]),
FILTER(
   ALLEXCEPT(Leave, Leave[Id]),
   Leave[Date] = EARLIER(Leave[Most Recent Date Prior to Work Complete])
))

Now that every row has a field explaining where to go to find the daily balance to use as our adjustment, we can just go look it up from the table.

Adjusted Daily Balance = Leave[Running Daily Balance] - Leave[Daily Balance Adjustment]

And finally we apply the adjustment to our running total for the final result.

The Issue

This approach fails to address that the count should not reset unless the running daily balance is below zero. I have been proved wrong before, but I would say that this can't be accomplished in DAX alone because it creates a circular dependency. Essentially, you make a requirement: use the aggregated value to determine what should be included in the aggregation.

So that's as far I can bring you. Hope it helps.

Ryan B.
  • 3,575
  • 2
  • 20
  • 26
4

I think I have it!

Here's the result, building upon the solution I posted earlier: (The data has been modified to show off more "work / no work" behaviors and use cases)

RESULT

enter image description here

DETAILS

(1) Drop "Adjusted Running Daily Balance" and "Daily Balance Adjustment" Colums. We'll get the same outcome with one less step in just a moment.

(2) Create the following column (RDB = "running daily balance")...

Grouped RDB = 

CALCULATE(
SUM(Leave[Daily Balance]),
FILTER(
   ALLEXCEPT(Leave, Leave[Id], Leave[Most Recent Date Prior to Work Complete]),
   Leave[Date] <= EARLIER(Leave[Date]) 
))

Having created the "Most Recent Date Prior to Work Complete," we have actually the piece needed to do our 'reset' that I claimed was impossible before. By filtering on this field, we have the opportunity to start each slice at '1'

(3) We still have the same problem tho, we can't look at the result in our column and use it to decide what to do later in that same column. But we CAN build a new adjustment column that will hold that info! And we already have a reference to 'Most Recent Date Prior to Work' -- that's the last day in the previous group... the row with the information we need!

Grouped RDB Adjustment = 

VAR CalculatedAdjustment =
CALCULATE(
SUM(Leave[Grouped RDB]),
FILTER(
   ALLEXCEPT(Leave, Leave[Id]),
   Leave[Date] IN SELECTCOLUMNS(
        FILTER(
            Leave,
            Leave[Most Recent Date Prior to Work] <> BLANK() &&
            Leave[id] = EARLIER(Leave[Id])), "MRDPtW", Leave[Most Recent Date Prior to Work]) &&
   Leave[Most Recent Date Prior to Work Complete] < EARLIER(Leave[Most Recent Date Prior to Work Complete]) &&
   Leave[Most Recent Date Prior to Work Complete] <> Blank()
))

RETURN if (CalculatedAdjustment > 0, CalculatedAdjustment, 0)

So we look at the last day in Each prior group and if the total sum of those adjustments has a positive value we apply it and if it's negative we leave it alone instead. Also, if our person's first few days are non-working days, we don't want that initial negative bit in our adjustment at all so it get's filtered away too.

(4) This last step will bring the adjustment into the final result. Sum up the two new columns and we should finally have our Adjusted Running Daily Balance. Voila!

Adjusted Running Daily Balance = Leave[Grouped RDB] + Leave[Grouped RDB Adjustment]

We built a lot of extra columns along the way to this result which usually isn't my favorite thing to do. But, this was a tricky one.

Ryan B.
  • 3,575
  • 2
  • 20
  • 26
  • Hi @Ryan B. This works perfectly for over 200 people in my organisation but one is not working. I've tried changing the code myself but I can't get anything to resolve the problem. I think it is because they have worked a long time and then worked just one day before having more time off. I've linked to an image to show the problem. Thanks [Image](https://drive.google.com/file/d/1lO9aTijjNLuStDQTmAasJck7SKesbRuP/view?usp=sharing) – LynseyC Feb 05 '20 at 23:36
  • I have modified the "Grouped RDB Adjustment" measure so that it should pass large accruals of leave across multiple "work / no work" cycles. – Ryan B. Feb 06 '20 at 16:51
  • 2
    Hi, thanks for all the effort, much appreciated. Unfortunately the modification didn't resolve the problem. However if I removed the last condition in the filter "Leave[Most Recent Date Prior to Work Complete] <> Blank()" it resolved the problem but then it broke the original people's calcs again :-( – LynseyC Feb 07 '20 at 15:07
  • Shoot. Well, I hope you can find something that works. – Ryan B. Feb 07 '20 at 15:26
3

Hope next time you will paste a csv or code which generates sample data instead of picture. :)

Let me just suggest you do your calculations in PowerQuery instead. I tried to split code for few steps to improve readability. This may look a bit more complex, however works well. Just paste it in advanced editor and then replace source with your source data. Best of luck!

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjDUMzDSMzIwtFTSUQpILSrOz1MwBDLL84uyM/PSlWJ1gGqMsKuBSBrjkzQhwnRTItSYEaHGHJ9DLPBJWhI23dAAjwGGOAIRIokj9OCmxwIA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [date = _t, name = _t, #"type" = _t]),
    SetTypes = Table.TransformColumnTypes(Source,{{"date", type date}, {"name", type text}, {"type", type text}}),
    TempColumn1 = Table.AddColumn(SetTypes, "LastOtherType", (row)=>List.Max(Table.SelectRows(SetTypes, each ([name] = row[name] and [type] <> row[type] and [date] <= row[date]))[date], row[date]), type date) //Here for each row we select all rows of other type with earlier date, and take max that date. Thus we know when was previous change from one type to another
 //Here for each row we select all rows of other type with earlier date, and take max that date. Thus we know when was previous change from one type to another
,
    TempColumn2 = Table.AddColumn(TempColumn1, "Count", (row)=>
(if row[type]="working" then 1 else -1) * 
Table.RowCount(
Table.SelectRows(SetTypes, each ([name] = row[name] and [type] = row[type] and [date] <= row[date] and [date] > row[LastOtherType])) /* select all rows between type change (see prev step) and current row */
), /*and count them*/
Int64.Type) // finally multiply -1 if they are not working type
,
    FinalColumn = Table.AddColumn(TempColumn2, "FinalFormula", (row)=> 
(if row[type] = "working" then row[Count] else /* for working days use Count, for others take prev max Count and add current Count, which is negative for non-working*/
Table.LastN(Table.SelectRows(TempColumn2, each [name] = row[name] and [type] = "working" and [LastOtherType] <= row[LastOtherType]),1)[Count]{0}
+ row[Count])
, Int64.Type),
    RemovedTempColumns = Table.RemoveColumns(FinalColumn,{"LastOtherType", "Count"})
in
    RemovedTempColumns
Eugene
  • 1,234
  • 1
  • 11
  • 19
  • I'm not certain this covers every scenario, but it seems like the right approach. – Mike Honey Feb 05 '20 at 04:21
  • I can only get this to work if the first type for each person is Working. Also as with the DAX examples it restarts the numbering for a Working movement when the cumulative total for the previous row is a positive number. I guess my picture was misleading as it only contained this scenario. I should have included a time when the type changed to working but the previous row total was a positive. – LynseyC Feb 05 '20 at 13:05
  • @LynseyC well, this code is not perfect and complete solution, of course, but rather an example of methods which may be used. Just modify if for your scenario. – Eugene Feb 07 '20 at 11:14
  • @LynseyC also, one of advantages doing this math in PowerQuery rather than DAX is an easy way to keep temp columns off data model. – Eugene Feb 07 '20 at 11:31
2

Took a while, but I was able to come up with a workaround. Assuming, the balance value for blanks is always -1 and the value is 1 for "Working" and that data is available for all dates without gap, something like the below calculation could work:

Running Total = 
    VAR Employee = Leave[Employee ID]
    VAR Date1 = Leave[Date]
    VAR Prev_Blank = CALCULATE(MAX(Leave[Date]),
                        FILTER(Leave,Leave[Date] < Date1),
                        FILTER(Leave,Leave[Employee ID]=Employee),
                        FILTER(Leave,Leave[Type]=BLANK()))  
    VAR Day_count_Working = CALCULATE(COUNT(Leave[Date]),
                        FILTER(Leave,Leave[Date] > Prev_Blank),
                        FILTER(Leave,Leave[Date] <= Date1),
                        FILTER(Leave,Leave[Employee ID]=Employee),
                        FILTER(Leave,Leave[Type]="Working")) 
    VAR Day_count = CALCULATE(COUNT(Leave[Date]),
                        FILTER(Leave,Leave[Date] >= Prev_Blank),
                        FILTER(Leave,Leave[Date] <= Date1),
                        FILTER(Leave,Leave[Employee ID]=Employee)) 
RETURN (IF(Day_count_Working=BLANK(),Day_count,Day_count-1)-Day_count_Working)*-1 + Day_count_Working

Keep in mind, this might not be a finished product as I worked with a small sample, but this should get you started. Hope this helps.

CR7SMS
  • 2,520
  • 1
  • 5
  • 13
  • Thanks @CR7SMS. It restarts the running total when the type = Working but the running total when the type is blank is not working. For 7 Nov it reduces to 3 but then from 8-14 Nov it returns -2. Can you help with amending the code to get the running total to work when the type is blank? Thanks – LynseyC Jan 30 '20 at 11:04
  • Hi Lynsey, I tried a different calculation. I have added it as another answer since the calculation was a bit long. But hopefully the new calculation works. – CR7SMS Jan 31 '20 at 23:34
  • @CR7SMS please avoid adding more than one answers to a single question. It confuses other users that may search for a similar problem/solution and it's not nice. Instead, you should add whatever you may come up as a solution to one answer and split every different aspect to sections. – Christos Lytras Feb 08 '20 at 15:40
2

The calculation is a bit lengthy, but it seems to be working in the sample data I am using. Give this a try:

Running Total = 
    VAR Employee = Leave[Employee ID]
    VAR Date1 = Leave[Date]
    VAR Prev_Blank = CALCULATE(MAX(Leave[Date]),
                        FILTER(Leave,Leave[Date] < Date1),
                        FILTER(Leave,Leave[Employee ID]=Employee),
                        FILTER(Leave,Leave[Type]=BLANK()))  
    VAR Prev_Working = CALCULATE(MAX(Leave[Date]),
                        FILTER(Leave,Leave[Date] < Date1),
                        FILTER(Leave,Leave[Employee ID]=Employee),
                        FILTER(Leave,Leave[Type]="Working"))    
    VAR Prev_Blank1 = CALCULATE(MAX(Leave[Date]),
                        FILTER(Leave,Leave[Date] < Prev_Working),
                        FILTER(Leave,Leave[Employee ID]=Employee),
                        FILTER(Leave,Leave[Type]=BLANK()))  
    VAR Prev_type = CALCULATE(MAX(Leave[Type]),
                        FILTER(Leave,Leave[Date] = Date1-1),
                        FILTER(Leave,Leave[Employee ID]=Employee))
    VAR Prev_Blank2 = IF(Leave[Type]="Working" && (Prev_Blank1=BLANK() || Prev_type=BLANK()),Date1-1,Prev_Blank1)    
    VAR Day_count_Working = CALCULATE(COUNT(Leave[Date]),
                        FILTER(Leave,Leave[Date] > Prev_Blank2),
                        FILTER(Leave,Leave[Date] <= Date1),
                        FILTER(Leave,Leave[Employee ID]=Employee),
                        FILTER(Leave,Leave[Type]="Working")) 
    VAR Day_count = CALCULATE(COUNT(Leave[Date]),
                        FILTER(Leave,Leave[Date] >= Prev_Blank2),
                        FILTER(Leave,Leave[Date] <= Date1),
                        FILTER(Leave,Leave[Employee ID]=Employee)) 
RETURN (IF(Day_count_Working=BLANK(),Day_count,Day_count-1)-Day_count_Working)*-1 + Day_count_Working

I have used a bunch of variables here. You maybe able to come up with a shorter version. Basically the idea is to find the previous first occurrence of "Working" to find where to start the calculation from. This is calculated in the variable "Prev_Blank2". Once we know the starting point (it starts with 1 here), then we can simply count the number of days with "Working" or blank() in between Prev_Blank2 and the date of the current record. Using these days, we can return the final value for running total.

Hopefully this does the trick ;)

CR7SMS
  • 2,520
  • 1
  • 5
  • 13
1

This is not only a running total with a condition, but also a nested/clustered one, as the logic has to applied on the ID-level. For large tables, M is better at it than DAX, as it doesn't use as much RAM. (I've blogged about that here: Link to Blogpost

The following function adapts that logic to the current case and has to be applied on ID-level: (Required column names are: "Type", "Daily Allowance", "Adjustments")

(MyTable as table) => let SelectJustWhatsNeeded = Table.SelectColumns(MyTable,{"Type", "Daily Allowance", "Adjustments"}), ReplaceNulls = Table.ReplaceValue(SelectJustWhatsNeeded,null,0,Replacer.ReplaceValue,{"Adjustments"}), #"Merged Columns" = Table.CombineColumns(ReplaceNulls,{"Daily Allowance", "Adjustments"}, List.Sum,"Amount"), TransformToList = List.Buffer(Table.ToRecords(#"Merged Columns")), ConditionalRunningTotal = List.Skip(List.Generate( () => [Type = TransformToList{0}[Type], Result = 0, Counter = 0], each [Counter] <= List.Count(TransformToList), each [ Result = if TransformToList{[Counter]}[Type] = "working" and [Result] < 0 and [Type] <> "working" then TransformToList{[Counter]}[Amount] else TransformToList{[Counter]}[Amount] + [Result] , Type = TransformToList{[Counter]}[Type], Counter = [Counter] + 1 ], each [Result] )), Custom1 = Table.FromColumns( Table.ToColumns(MyTable) & {ConditionalRunningTotal}, Table.ColumnNames(MyTable) & {"Result"} ) in Custom1

ImkeF
  • 1,388
  • 3
  • 12
  • 11