0

I am trying to replace a value with a variable assigned in flowfile.

In my flowfile, I have assigned flowID to flow_id variable.

In UpdateRecord processor, I try to update a column named /flow which has INFLOW and OUTFLOW I have following as ${field.value:replaceAll('INFLOW',$flow_id)}

Flowfile before UpdateRecord:

id,flow,flow_id
1,INFLOW,IN
2,OUTFLOW,OUT
3,INFLOW,IN

After the conversion flowfile should be:

id,flow,flow_id
1,IN,IN
2,OUT,OUT
3,IN,IN

But it fails with an error

unexpected token

Edit : After answer and comments

Now I have following settings:

${field.value:replace('INFLOW',flow_id)}

Due to unexpected token flow_id

Same error for Literal Value and Record path value Replacement strategies.

Sachith Muhandiram
  • 2,819
  • 10
  • 45
  • 94

2 Answers2

3

This Answer is for Attribute substitution. OP actually wants to replace the column values from the other column values.

From the documentation, you can find the detailed usage of the variables and parameters.

  • For the variables, see here
  • For the parameters, see here

When you use a Variable such as

  • Processor-specific attributes
  • FlowFile properties
  • FlowFile attributes
  • From Variable Registry:
    • User defined properties (custom properties)
    • System properties
    • Operating System environment variables

You can refer the variable such as:

${name of variable}

When you use a parameter from the parameter context, you can refer the parameter such as:

#{name of parameter}

In your case, you want to use a variable and so it should be:

${field.value:replaceAll('INFLOW',${flow_id})}

I have tested with the csv data,

index,flow
1,INFLOW
2,OUTFLOW
3,INFLOW

and the attribute flow_id with the value 'flowId'. Now, I have set the UpdateRecord processor with the options as follows:

Record Reader                  CSVReader
Record Writer                  CSVRecordSetWriter
Replacement Value Strategy     Literal Value
/flow                          ${field.value:replace('INFLOW', ${flow_id})}

The output csv is as expected.

index,flow
1,flowID
2,OUTFLOW
3,flowID
Lamanus
  • 12,898
  • 4
  • 21
  • 47
  • 1
    This answer is confusing variables with parameters. Expression language which can reference variables, flow file attributes, system props, and env vars uses ${ } . Parameters which is a new feature and only references parameters, uses #{ }. – Bryan Bende Nov 21 '19 at 15:02
  • `Unexpected Tocken ` – Sachith Muhandiram Nov 22 '19 at 01:24
  • If you want to use field.value or variable, the option of UpdateRecord should be `Literal Value`. Or use replace not replaceAll. – Lamanus Nov 22 '19 at 01:42
  • The problem is still going on, please update your question in more detail, I will do my best. – Lamanus Nov 22 '19 at 02:29
  • @Lamanus **failed to process standard flowfile record will route to null** , as I have showed, `flow_id` is another column in the flowfile. not a predefined value. – Sachith Muhandiram Nov 27 '19 at 05:21
  • If `flow_id` is a different column value, you cannot do it in this way. I will think more about this. – Lamanus Nov 27 '19 at 06:01
  • @Lamanus exactly, this is what I want to do. I will replace current value with another column data and then have to perform many actions on that. Another question which is related to this is : https://stackoverflow.com/questions/59047115/add-two-columns-together-using-apache-nifi – Sachith Muhandiram Nov 27 '19 at 06:08
  • Yeap, but your example just replace the column flow_id to flow, that is easy. If there are complex conditions, then you have to split each record and do some logic, merge again that is the only way I think. – Lamanus Nov 27 '19 at 06:33
1

Set up the Nifi Workflow as shown below :

enter image description here

Set your input directory in the GetFile Processor. In the first UpdateRecord Processor concatenate the flow and flow_id columns and assign it into flow. Do this by changing the configurations of the First UpdateRecord processor as shown below.

enter image description here

The next UpdateRecord Processor replaces INFLOW and OUTFLOW accordingly. Set the configurations of the second UpdateRecord Processor as shown below.

enter image description here

Because of the initial concating " and , is added in the flow column. I have used the other two processors to remove them.

The third UpdateRecord Processor must have a new property named /flow and have the value ${field.value:replaceFirst(${field.value:substringAfter(',')},'')}. The rest of the configurations remain as the Second UpdateRecord Processor.

The Fourth UpdateRecord Processor must have a new property named /flow and have the value ${field.value:replaceFirst('"',''):replaceFirst(',','')}. The rest of the configurations remain as the Second UpdateRecord Processor.

Note: This answer was based on the concepts used in this question.

Himsara Gallege
  • 934
  • 1
  • 8
  • 24