Execute Groovy Script to transform date in Nifi

Question

I'm trying to transform my JSON dates using Nifi. They are imported in this format:

import groovy.json.JsonSlurper
import groovy.json.JsonBuilder

def ff = session.get()
if(!ff)return
ff = session.write(ff, {rawIn, rawOut->

    // transform streams into reader and writer
    rawIn.withReader("UTF-8"){reader->
        rawOut.withWriter("UTF-8"){writer->

            //parse reader into Map
            def json = new JsonSlurper().parse(reader)

            // set my variable and define what format it is in
            json.date = new Date(json.date as Long).format('HH:mm yyyy-MM-dd')

            // Reformat it
            json.date = DateFormat.parse("yyyy-MM-dd HH:mm", json.date)

            //write changed object to writer
            new JsonBuilder(json).writeTo(writer)
        }
    }
} as StreamCallback)
session.transfer(ff, REL_SUCCESS)

The incoming flowfile has this body:

[{"date":"09:00 2019-05-29","data":460.0,"name":"login"},{"date":"10:00 2019-05-29","data":548.0,"name":"login"},{"date":"11:00 2019-05-14","data":0.0,"name":"login"},{"date":"00:00 2019-06-15","data":0.0,"name":"login"}]

I want this output:

[{"date":"2019-05-29 09:00","data":460.0,"name":"login"},{"date":"2019-05-29 10:00","data":548.0,"name":"login"},{"date":"2019-05-14 11:00","data":0.0,"name":"login"},{"date":"2019-06-15 00:00","data":0.0,"name":"login"}]

The error I get is this:

Can anyone please help me understand where I am going wrong?

The "incoming" has no `timeStamp` keys. Did you mean, that the example is the output you like to see? — cfrick, Aug 02 '19 at 10:08
That is the input, I wanted it the same but with date as the transformed date. Should I change it to be json.date = new Date(json.date.....? — J.Zil, Aug 02 '19 at 10:10
In your code, you are refering to `json.timestamp` and in the example ingress there is no `timestamp`. Please provide the minimal failing case and the errors you get. — cfrick, Aug 03 '19 at 17:40
@cfrick Hello. I updated my question with your feedback. It picks up on the dates now but it doesn’t transform it into the required date format. — J.Zil, Aug 04 '19 at 07:12
I am not yet convinced that this is describing your actual problem. Your _incoming_ file contains `"date":"09:00 2019-05-29"`, but you are trying to cast that string to a long and then format it _again_ back to the input format `new Date(json.date as Long).format('HH:mm yyyy-MM-dd')`? I have no idea, what's the point of this. — cfrick, Aug 04 '19 at 12:26
@cfrick I’m new to this and trying to reformat the date to be in the format yyyy-MM-dd HH:mm:ss.SSS form it’s original format — J.Zil, Aug 04 '19 at 13:21
Then i'd say, this is a duplicate of https://stackoverflow.com/questions/3817862/groovy-string-to-date — cfrick, Aug 04 '19 at 13:38
@cfrick Appreciate that, but that’s for a variable as opposed to all dates in a json body — J.Zil, Aug 04 '19 at 15:50
@cfrick I'm sorry I'm still stuck and looking for help. I've added in the code you suggested, added in expected output and the exact error — J.Zil, Aug 05 '19 at 09:10

score 1 · Answer 1 · answered Aug 05 '19 at 10:20

1

The input is a list of the objects in question. The incoming date is a String -- not a Long.

So the first error is to use json.date as it implies json*.date (which gives a list of all date).

Next casting the date to Long, create a new Date and then format it is the wrong way around.

So to change the format of all the date something like this is needed:

json.each{
    it.date = Date.parse('HH:mm yyyy-MM-dd', it.date).format('yyyy-MM-dd HH:mm')
}

answered Aug 05 '19 at 10:20

cfrick

35,203
6
56
68

Thank you very much for this. It does do the transform correctly, however it seems to only do one of the json tags and stops. For example the output is this: [{"date":"2019-05-29 09:00","data":460.0,"name":"blmoblogin"}] – J.Zil Aug 05 '19 at 10:34
1

This is also what your error above suggest. You just get one element in the list. Above code does not change the shape of the container. – cfrick Aug 05 '19 at 10:39
Is there a way to loop through and do all? – J.Zil Aug 05 '19 at 10:40
1

This _is_ doing all. There is no more input. How do you determine your _input_ is like you have described it? There is no logical reason other than the input is a single element list. The JsonSlurper can only all-or-nothing. – cfrick Aug 05 '19 at 10:46

Execute Groovy Script to transform date in Nifi

1 Answers1