How to pass result of one chained input, into next chained input

Question

I am writing a watcher, to first fetch destination.ip field (using aggregation), and then I have to use 1st input result for terms query value (in 3rd input)

POST _watcher/watch/_execute
{
  "watch": {
    "trigger": {
      "schedule": {
        "interval": "10s"
      }
    },
    "input": {
      "chain": {
        "inputs": [
          {
            "first": {
              "search": {
                "request": {
                  "indices": [
                    "test-index"
                  ],
                  "body": {
                    "size": 0,
                    "aggs": {
                      "destination_ip_aggs": {
                        "terms": {
                          "field": "destination.ip",
                          "size": 2
                        }
                      }
                    }
                  }
                }
              }
            }
          },
          {
            "second": {
              "transform": {
                "script": {
                  "lang": "painless",
                  "source": """
                  List ips = new ArrayList();
        for(def bucket: ctx.payload.first.aggregations.destination_ip_aggs.buckets) {
          ips.add(bucket.key)
        }
    return ips;
    """
                }
              }
            }
          },
          {
            "third": {
              "search": {
                "request": {
                  "indices": [
                    "test-index"
                  ],
                  "body": {
                    "query": {
                      "terms": {
                        "source.ip": [
                          "{{ctx.payload.second._value}}"
                        ]
                      }
                    }
                  }
                }
              }
            }
          }
        ]
      }
    },
    "actions": {
      "log_error": {
        "logging": {
          "text": "{{ctx.payload.second._value}}"
        }
      }
    }
  }
}

The watcher output is like

"chain" : {
          "first" : {
            "type" : "search",
            "status" : "success",
            "payload" : {
              "_shards" : {
                "total" : 1,
                "failed" : 0,
                "successful" : 1,
                "skipped" : 0
              },
              "hits" : {
                "hits" : [ ],
                "total" : 2,
                "max_score" : null
              },
              "took" : 1,
              "timed_out" : false,
              "aggregations" : {
                "destination_ip_aggs" : {
                  "doc_count_error_upper_bound" : 0,
                  "sum_other_doc_count" : 0,
                  "buckets" : [
                    {
                      "doc_count" : 1,
                      "key" : "10.20.70.200"
                    },
                    {
                      "doc_count" : 1,
                      "key" : "10.20.70.210"
                    }
                  ]
                }
              }
            },
            "search" : {
              "request" : {
                "search_type" : "query_then_fetch",
                "indices" : [
                  "test-index"
                ],
                "rest_total_hits_as_int" : true,
                "body" : {
                  "size" : 0,
                  "aggs" : {
                    "destination_ip_aggs" : {
                      "terms" : {
                        "field" : "destination.ip",
                        "size" : 2
                      }
                    }
                  }
                }
              }
            }
          },
          "second" : {
            "type" : "transform",
            "status" : "success",
            "payload" : {
              "_value" : [
                "10.20.70.200",
                "10.20.70.210"
              ]
            }
          },
          "third" : {
            "type" : "search",
            "status" : "failure",
            "error" : {
              "root_cause" : [
                {
                  "type" : "query_shard_exception",
                  "reason" : "failed to create query: '{0=10.20.70.200, 1=10.20.70.210}' is not an IP string literal.",
                  "index_uuid" : "cH54bWHPTa2V2i9SskRPhw",
                  "index" : "test-index"
                }
              ],
              "type" : "search_phase_execution_exception",
              "reason" : "all shards failed",
              "phase" : "query",
              "grouped" : true,
              "failed_shards" : [
                {
                  "shard" : 0,
                  "index" : "test-index",
                  "node" : "5sTXsEnqRFabKkXTrUiPBA",
                  "reason" : {
                    "type" : "query_shard_exception",
                    "reason" : "failed to create query: '{0=10.20.70.200, 1=10.20.70.210}' is not an IP string literal.",
                    "index_uuid" : "cH54bWHPTa2V2i9SskRPhw",
                    "index" : "test-index",
                    "caused_by" : {
                      "type" : "illegal_argument_exception",
                      "reason" : "'{0=10.20.70.200, 1=10.20.70.210}' is not an IP string literal."
                    }
                  }
                }
              ]
            },
            "search" : {
              "request" : {
                "search_type" : "query_then_fetch",
                "indices" : [
                  "test-index"
                ],
                "rest_total_hits_as_int" : true,
                "body" : {
                  "query" : {
                    "terms" : {
                      "source.ip" : [
                        "{0=10.20.70.200, 1=10.20.70.210}"
                      ]
                    }
                  }
                }
              }
            }
          }
        }

How can I use the second input value, in the third input (for terms query)? I tried to transform the data again, but still not able to get it in the correct list format.

Can anyone please help me resolve this ?

Musab Dogan · Answer 1 · 2023-03-20T14:23:08.330

0

UPDATE: The output {0=10.20.70.200, 1=10.20.70.210} need to be converted into ["10.20.70.200", "10.20.70.210"]. I found a way to convert the output like the following "10.20.70.200, 10.20.70.210", you can use the mustache language and you can find a way to fit your scenario.

"source.ip": [
  "{{#join}}ctx.payload.second._value{{/join}}"
]

Check this link:

{{#join}}field_name{{/join}}

The problem is the query looking for the field as IP but the value is 0=10.20.70.200, and it's not a valid IP address.

'{0=10.20.70.200, 1=10.20.70.210}' is not an IP string literal.

Please update the third part like the following and test it.

"third": {
  "search": {
    "request": {
      "indices": [
        "test-index"
      ],
      "body": {
        "query": {
          "terms": {
            "source.ip": [
              "{{ctx.payload.second._value.0.0}}"
            ]
          }
        }
      }
    }
  }
}

Here is a similar problem link

edited Mar 20 '23 at 14:23

answered Mar 20 '23 at 12:11

Musab Dogan

1,811
1
6
8

Thanks for the answer. If I use `"{{ctx.payload.second._value.0}}"`, then I get only 1st `destination.ip`, I need a list of all the `destination.ip`, i.e all the bucket values, so that I can pass the list in terms query value. Is there any way to do that? – ESCoder Mar 20 '23 at 12:16
You're welcome. Is that worked? + in the first chain the "size": 2. So, you only need 2 of them and you can use "{{ctx.payload.second._value.0}}, {{ctx.payload.second._value.1}}"? – Musab Dogan Mar 20 '23 at 12:51
no, there are many values in the bucket, size=2 was just for testing purposes. Is there a way to get the complete list, and use it as a value of terms query? And even though the second input payload, shows results as this ``` "_value" : [ "10.20.70.200", "10.20.70.210" ] ```, not sure why the _value is storing in key-value format – ESCoder Mar 20 '23 at 12:56
You can use the `mustache` language and convert the data. I updated the answer please check that. I found out how to convert an array to a comma-delimited array, but it's not fit your case. I believe you can find an answer by using the mustache lang. – Musab Dogan Mar 20 '23 at 14:22
this does not suits my case, since I cannot pass comma delimited array as a terms query value – ESCoder Mar 21 '23 at 09:38

How to pass result of one chained input, into next chained input

1 Answers1