How to subtract specific elements in a list using functional programming in Mathematica?

Question

I have a list of dates and values in the format:

{{{dateInfo1},value1},{{dateInfo2},value2},...,{{dateInfoN},valueN}}

With some actual dates and values:

{{{1971, 1, 31, 0, 0, 0.}, 1.0118}, {{1971, 2, 28, 0, 0, 0}, 1.0075},
 ..., {{2010, 5, 31, 0, 0, 0.}, 1.0403}}

For those curious, it's a list of US versus CAD $ values pulled from the FRED database.

I would like to simply subtract value1 from value 2, and then create a new list with data in the form of:

 {{{dateInfo1},0},{{dateInfo2},change1},...,{{dateInfoN},changeN-1}}

(with change1 being value2-value1)

I know there must be a relatively easy way to do this using functional programming, as opposed to Do or While with index variables and counting and all that nonsense. The method I'm trying to accomplish has to be relatively robust, because I am automatically pulling in datasets from sources that have the same formatting, but different time intervals. Replotting is then much easier if I don't have to specify the ListPlot date intervals (which would happen if I stripped the dateInfo from the list).

I am familiar with the Documentation Center and non-programming Mathematica capabilities. I have been learning programming with Mathematica, and really want to extend that ability into functional programming, but have found most of the resources on the topic a bit too difficult. I feel like I am at that hump in the learning curve where it's about to click into place, but right now I am struggling. At the very least if you have a good source on functional programming I would be more than happy to look into those! Any help is much appreciated! Sorry if it's TMI, but I'm sure many of you have felt the same way.

You may have noticed that all the provided solutions are just a little more awkward when you try to keep that leading 0, which is AFAICT mathematically meaningless. I think the universe is trying to tell you something there.... — Pillsy, Jun 24 '10 at 14:38
I understand that the leading 0 doesn't make as much sense mathematically, but in terms of presentation, it makes more sense. For example, if using 1971 as the base year in the analysis, then the change for Jan to Feb should be Feb-Jan. It then makes the presentation more logical if that value is displayed as the change in Feb, especially since I will be plotting the data. — Alec, Jul 09 '10 at 19:02

score 7 · Accepted Answer · edited May 23 '17 at 12:01

You have a list of {date,value} pairs so if you Transpose that you'll have a list of two lists -- the first a list of dates and the second a list of corresponding values. You can then take the Differences of the values, Prepend 0, and then Transpose again to get back to a list of pairs.

In code,

data = {{{1971,1,31,0,0,0}, 1.0118}, 
        {{1971,2,28,0,0,0}, 1.0075}, 
        {{2010,5,31,0,0,0}, 1.0403}}
{dates, values} = Transpose[data];
diffs = Prepend[Differences[values], 0];
answer = Transpose[{dates, diffs}]

which returns:

{{{1971,1,31,0,0,0}, 0}, 
 {{1971,2,28,0,0,0}, -0.0043}, 
 {{2010,5,31,0,0,0}, 0.0328}}

To wrap that up into a single function, with thanks to Janus for the idea:

taildiffs[data_]:= 
  Transpose @ {#1, Prepend[Differences[#2], 0]}& @@ Transpose@data

Note that the ... #1 ... #2 ... & construct is a pure function:

http://reference.wolfram.com/mathematica/ref/Function.html

The f@x syntax is simply shorthand for f[x].

Finally, f@@list is shorthand for Apply[f, list]:

http://reference.wolfram.com/mathematica/ref/Apply.html

So taildiffs as defined above is just a terse (perhaps cryptic) version of this:

Apply[Transpose[Function[{x,y}, {x, Prepend[Differences[y],0]}], Transpose[data]]

So, to make sure that I am making progress in understanding the Functional Notation, the singe function works like this: 1) data is Transposed into two lists, #1 and #2. 2) Those two lists are applied in the following manner; #1 is left alone, and #2 is Differenced and Prepended with 0. 3) #1 and the modified #2 are then Transposed back into a single list. Is that correct? — Alec, Jul 09 '10 at 19:22
Exactly. Well said. You should know more generally that, for example, foo[#2,#1]& is a pure function of two arguments that calls foo on those two arguments (in reverse order). Other fancy syntax there is the @ and @@. Let me append something to the answer to explain those... — dreeves, Jul 09 '10 at 20:55

score 3 · Answer 2 · edited Jun 24 '10 at 12:57

Except for the fact that you want the initial 0, you are looking for Differences. To leave the dates alone, transpose and apply to the second part only, like so:

TailDifferences[data_]:=
  Transpose@Apply[{#1,{0}~Join~Differences[#2]}&,Transpose[data]]

Applying this to your data yields something like this:

data={{{dateInfo1},value1},{{dateInfo2},value2},{{dateInfo3},value3}};
TailDifferences[data]

{{{dateInfo1},0},{{dateInfo2},-value1+value2},{{dateInfo3},-value2+value3}}

score 2 · Answer 3 · edited Jun 24 '10 at 03:13

2

data = {{{dateInfo1}, value1}, {{dateInfo2}, value2}, {{dateInfo3}, value3}}

Map[{#[[2,1]], #[[2,2]] - #[[1,2]]}&, {Take[data, Length[data] - 1], Rest[data]}]

gives

{{{dateInfo2}, -value1 + value2}, {{dateInfo3}, -value2 + value3}}

This first element in your result list

{{dateInfo1},0}

doesn't really fit into the sequence, so you can manually prepend it to the list

edited Jun 24 '10 at 03:13

dreeves

26,430
45
154
229

answered Jun 24 '10 at 00:45

Max

19,654
13
84
122

score 2 · Answer 4 · answered Jun 24 '10 at 01:33

I recommend using Reap and Sow for this:

In[13]:= lis= {{{1971,1,31,0,0,0.},1.0118},{{1971,2,28,0,0,0},1.0075},{{2010,5,31,0,0,0.},1.0403}};

In[14]:= First@Last@Reap[
   (* set first previous to first value to get 0 *)
   Module[{prev = lis[[1, 2]]},
    Scan[
     (
       (* First[#] = date, Last[#] = value *)
       Sow[{First[#], Last[#] - prev}];
       (* set new previous to this value *)
       prev = Last[#]
       ) &,
     lis]]
   ]

Out[14]= {{{1971, 1, 31, 0, 0, 0.}, 0.},
  {{1971, 2, 28, 0, 0, 0}, -0.0043},
  {{2010, 5, 31, 0, 0, 0.}, 0.0328}}

The output of Reap is a little complicated if you aren't familiar with it, but Reap and Sow basically give you a way to "sow" things into lists and then "reap" them after the evaluation. Reap and Sow are much more efficient than using AppendTo with a list, for example.

HTH!

score 1 · Answer 5 · answered Dec 24 '11 at 09:08

This operation could also be done with Part and Set:

data = {{{1971,1,31,0,0,0}, 1.0118}, 
        {{1971,2,28,0,0,0}, 1.0075}, 
        {{2010,5,31,0,0,0}, 1.0403}};

Module[{a = data},
  a[[2 ;;, 2]] = Differences[a[[All, 2]]];
  a[[1, 2]] = 0;
  a
]

{{{1971, 1, 31, 0, 0, 0}, 0},
 {{1971, 2, 28, 0, 0, 0}, -0.0043},
 {{2010, 5, 31, 0, 0, 0}, 0.0328}}

How to subtract specific elements in a list using functional programming in Mathematica?

5 Answers5

Linked