Stack Overflow
Stack Overflow

Questions tagged [flowfile]

29 questions
1
vote
1 answer

How to make a Nifi processors event driven

For example, if there is a pipeline made of 3 processors P1, P2, P3. When P2 produces an output flowfile, then after exactly 5 minutes I want processor P3 to work. I cant use a fixed CRON job because the P2 processor can run at anytime. Nifi version…
prathik vijaykumar
  • 355
  • 3
  • 12
1
vote
0 answers

NiFi: Filter flow file based on attribute

I have a workflow that creates two different flowfiles. I want to filter one specific flowfile based on a condition and send it to the next processor. My work so far def ff=session.get() if(!ff)return def flowFiles=session.get(2) if(!flowFiles ||…
Parthi
  • 76
  • 5
1
vote
0 answers

Apache Nifi Flowfile to Elasticsearch bulk api

I'm trying to prepare flowfile to make bulk call with versioning to elasticsearch using standard InvokeHttp processor, since I failed to make it work with PutElasticsearch* processors Elasticsearch bulk endpoint accept ndjson data format that looks…
sid
  • 35
  • 8
1
vote
1 answer

Generate exactly 1 Flowfile

I'm using the GenerateFlowFile processor in Apache Nifi - When I activate it, I want the processor to create exactly 1 Flowfile. Right now I use the REST API via Python to change the state to RUNNING, wait 0.5 seconds and change the state to…
Cribber
  • 2,513
  • 2
  • 21
  • 60
0
votes
1 answer

How can I extract value by using Extract Text Processor?

I need to extract a property by using an extract text processor I have a Replace Text processor that has the property "Input_String": {"columnId":"19","value":"Helloworld","name":"Test"}, "Message_Type":"ADT_A01" I am trying to extract…
Chinnu
  • 1
  • 2
0
votes
0 answers

Nifi GenerateFlowFile with Attribute Parameter

is it possible to generate a FlowFile with Attribute Parameters, which get filled on a UpdateAttribute Processor or EvaluateJsonPath Processor? For Example I have a FlowFile Content: ${database} …
BenjaminR
  • 23
  • 4
0
votes
0 answers

Merge/combine/join 2 JSON FlowFile into 1

I have 1 JSON FlowFile A (taken from 1 invoke HTTP processor) and 1 Json FlowFile B (also from 1 invoke HTTP processor). The problem I need to solve is to merge/combine/join them into 1 FlowFile C, whose schema is the sum of 2 Flow files A and B. I…
Achilles_VN
  • 31
  • 2
0
votes
1 answer

Connect one NiFi cluster to another over a unidirectional network / data diode / restricted network

I need to connect two Apache NiFi instances / cluster in a restricted network environment. Basically the whole exchange must work over a unidirectional network / data diode, that only supports TCP packages from client to server and ACK packages from…
Endzeit
  • 4,810
  • 5
  • 29
  • 52
0
votes
1 answer

How can I get the Nifi ExecuteSQL processor COUNT(*) Result and use it in PutSQL processor?

I have a table (T1) that is inserted frequently. I want to check whether the insertions successful. I have another table (T2) to store the row counts of this table. I used ExecuteSQL processor to send SELECT COUNT(*) query against the T1. I want to…
Eren Irmak
  • 17
  • 6
0
votes
0 answers

Apache Nifi: Monitor Files in Queue for longer duration

I have an ETL pipeline setup on Apache Nifi. It has more than 1300 processors. I need to get into each process group manually to check whether there are any files stuck in a queue. Is there any way that these queues can be monitored and notified…
Jaga Priyan
  • 3
  • 5
0
votes
1 answer

how to merge few flowfile into one

I need one flowfile on end of queue my config now,
zolwik
  • 1
0
votes
1 answer

ApacheNifi - Is there any way to send one flowfile at a time? (one for each minute)

Good Night! Is there any way to send one flowfile at a time? (one for each minute) illustration for the accumulated files : I have 138 flowfiles and I want to send 1 flowfile per minute. Is there one processor or mode to execute 1 per…
Rodrigo Hajime Himauari
  • 57
  • 4
0
votes
0 answers

Apache NiFi - Loop over flowfile's attribute value

I'm fairly new to either Stack Overflow and Apache NiFi, but I can't find any solution to my problem. My incoming Flowfile has an integer value N in an attribute, let's call it myAttribute. What I'm trying to do, is to generate N FlowFiles looping…
Andrea Bozzano
  • 1
  • 2
0
votes
1 answer

In Nifi, how do I create a JSON flow file from a list of tags and a file containing all of the records, one record per line?

I have a semicolon-delimited file containing 1000 records, one record per line, with each record containing several fields delimited by semicolons. I also have a list of tags for the fields in the records. How do I combine these into a JSON flow…
PyNerd
  • 645
  • 2
  • 9
  • 19
0
votes
1 answer

Get Filename with current date using regex

I have a situation where I have to get a new file every day at a specific time which is saved automatically on a specific folder on a remote server using GETSFTP Processor. for example today there will be a file created on a remote server named…
Muneer
  • 29
  • 1
  • 9
1
2