3

I need to filter an array in my AWS Step Functions state. This seems like something I should easily be able to achieve with JsonPath but I am struggling for some reason.

The state I want to process looks like this:

{
  "items": [
    {
      "id": "A"
    },
    {
      "id": "B"
    },
    {
      "id": "C"
    }
  ]
}

I want to filter this array by removing entries for which id is not in a specified whitelist.

To do this, I define a Pass state in the following way:

"ApplyFilter": {
  "Type": "Pass",
  "ResultPath": "$.items",
  "InputPath": "$.items.[?(@.id in ['A'])]",
  "Next": "MapDeployments"
}

This makes use of the JsonPath in operator.

Unfortunately when I execute the state machine I receive an error:

{
  "error": "States.Runtime",
  "cause": "An error occurred while executing the state 'ApplyFilter' (entered at the event id #8). Invalid path '$.items.[?(@.id in ['A'])]' : com.jayway.jsonpath.InvalidPathException: com.jayway.jsonpath.InvalidPathException: Space not allowed in path"
}

However, I don't understand what is incorrect with the syntax. When I test here everything works correctly.

What is wrong with what I have done? Is there another way of achieving this sort of filter using JsonPath?

Douglas Figueroa
  • 675
  • 6
  • 17
timxyz
  • 399
  • 2
  • 15
  • 1
    AWS might use an older release of the Jayway JasonPath dependency that does not support the `in` operator, but that's actually unlikely since this feature is around for quite some time. Maybe it's the Pass filter. Could you try to reverse the logic and use the `nin` operator instead? – wp78de Feb 09 '21 at 22:48
  • @wp78de Yes `nin` doesn't work either. However `==` does. I think you are right, and that AWS are using a very old version of JsonPath for some reason. From what I can see `in` and `nin` have been available since version 2.1.0 released in 2015 – timxyz Feb 10 '21 at 10:10
  • 1
    I combed the AWS GitHub repo and I think the current implementation is using Jayway JasonPath 2.0(pre)? – wp78de Feb 11 '21 at 18:20

1 Answers1

0

According to the official AWS docs for Step Functions,

The following in paths are not supported @ .. , : ? *

https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-paths.html

yoodan
  • 65
  • 2
  • 5
    This only applies to "reference paths" and `InputPath` is not a reference path. For instance `$.items.[?(@.id == 'A')]` works fine. – timxyz Feb 16 '21 at 09:16