I would like to have a default value for a parameter passed into a step function
e.g.,
"Parameters": {
"foo.$": "$.foo" OR "bar" if "$.foo" not specified
}
is there a way to do this natively with JSONPath or do I have to use a choice + pass state?
I'd even settle for using choice/pass if there were a way to not break when a parameter is not specified in the input.
If I don't include "foo": "" in the input, I will get an error like "JSONPath ... could not be found in the input."
There's actually a much simpler approach, it's still using extra steps; but it's better IMHO than a choice+pass. The concept is to define your defaults as a Pass state. This is required because we're going to merge the defaults with the input.
I've demonstrated that you can set more than just one default at once that reveals the benefits of not using Choice.
Note I always use $$.Execution.Input instead of relying on the input to the step function when I want to refer to the called args. This allows me to move steps around without fear of another state wiping them out.
{
"Define Defaults": {
"Type": "Pass",
"Next": "Apply Defaults",
"ResultPath": "$.inputDefaults",
"Parameters": {
"foo": "bar",
"x": -1,
"baz": null
}
},
"Apply Defaults": {
"Type": "Pass",
"Next": "Start Work",
"ResultPath": "$.withDefaults",
"OutputPath": "$.withDefaults.args",
"Parameters": {
"args.$": "States.JsonMerge($.inputDefaults, $$.Execution.Input, false)"
}
}
}
This will then provide you with a yielded value:
{
"foo": "bar",
"x": -1,
"baz": null
}
I'm solving this using a combination of the Choice and Pass states. Given that a state machine at least gets an empty input object, you can check if it has members present using the IsPresent comparison operator in the Choice state. If your variable of desire is not present, you can route to a Pass state to inject a default fallback object.
{
"keyThatMightNotExist": {
"options": {
"foo": "bar",
"baz": false
},
"id": 1234
}
}
{
"Comment": "An example for how to deal with empty input and setting defaults.",
"StartAt": "Choice State: looking for input",
"States": {
"Choice State: looking for input": {
"Type": "Choice",
"Choices": [
{
Checking for existence and if so, also validate a child member:
"And": [
{
"Variable": "$.keyThatMightNotExist",
"IsPresent": true
},
{
"Variable": "$.keyThatMightNotExist.id",
"IsNull": false
}
],
If the key variable is present and its child node "id" is true, skip the next state and hop over to the "State that works with $.keyThatMightNotExist"
"Next": "State that works with $.keyThatMightNotExist"
}
],
"Default": "LoadDefaults"
},
The following is where we inject our defaults:
"LoadDefaults": {
"Type": "Pass",
"Result": {
"id": 0,
"text": "not applicable"
},
"ResultPath": "$.keyThatMightNotExist",
"Next": "State that works with $.keyThatMightNotExist"
},
At this point, there is an object to work with. Either from actual input, or by using the defaults:
"State that works with $.keyThatMightNotExist": {
"Type": "Succeed"
}
}
}
For further information, refer to the AWS Step Functions Developer Guide, choice-state-example
We had a similar issue that we resolved in a way that may be helpful dependant on what your SFN task will be doing. In our case it was a Lambda so we could handle the default behaviour within.
You can set a parameter to the "$" value that will take all the inputs provided to the SFN.
"Parameters": {
"sfn_input.$": "$"
}
This "sfn_input" parameter will now be exposed to the stage and contain ALL inputs to the SFN. In our case, we could handle its optional presence in code.
This obviously only works if your task can eval the existence of the value. There could even be a concept of an "initialise" Lambda added at the start of your SFN to specifically handle this and add default values that pass back out to the SFN.
This can be achieved by creating a Pass State in the starting of step function with the combination of "Result" and "ResultPath".
"Result" of Pass State will be the value of foo"ResultPath" : "$.foo" in Pass State, so it's going to add foo variable in your pass input.State Definition for the same will look something like this:
"Hello": {
"Type": "Pass",
"Result": "Added from Hello state",
"ResultPath":"$.foo",
"Next": "OtherStates"
}
An example would be: Input to Hello state:
{
"bar" : "From input"
}
Output of Hello state:
{
"bar" : "From input",
"foo" : "Added from Hello state"
}
In this case it's going to add value of foo as "Added from Hello state". So to avoid breaking things because of no foo present in input, you will have to define this Pass State as the very first state or atleast before the state where you are going to use it.
P.S. This is applicable for the case when you just have to add a single default variable as you mentioned. For adding multiple default variable, I would recommend to create a Task State and achieve creating default variable using a lambda function.
I was able to solve this problem by creating a cloudwatch rule that inserts json everytime the stepfunction runs.
If you go to Cloudwatch > Create Rule > Targets > Stepfunction > Configure Input > Constant (JSON text)
You can insert your default json there, that json is inserted everytime your stepfunction runs.
JSONPath can only query your JSON, and you won't be able to add data on the fly, the default value would need to be present in your input. After many tests I was unable to find a "clean" way of doing this, but here's some methods to achieve this.
Method 1:
Since you mentioned you'll even settle for a way to not break if a parameter is not present in the input, this is actually quite easy to do, however it won't allow us to send a default value (except for an empty array) if the values are missing. To do this we can filter out specific values on our leaf nodes by specifying multiple children (['' (, '')]). For example the JSONPath "$.['Bar','Foo']" will filter out only the leaf nodes containing the nodes Bar and Foo, however it will not return an error if either or both are missing. In the case of all the children specified being missing it will return an empty array []. Here's an example State Machine to illustrate this.
{
"StartAt": "ExampleLeafNodes",
"States": {
"ExampleLeafNodes": {
"Type": "Parallel",
"Branches": [
{
"StartAt": "State1-BarPresent",
"States": {
"State1-BarPresent": {
"Type": "Pass",
"Parameters": {
"Bar": "Baz"
},
"Next": "State2-BarPresent"
},
"State2-BarPresent": {
"Type": "Pass",
"Parameters": {
"Foo.$": "$.['Bar','AnyOtherField']"
},
"End": true
}
}
},
{
"StartAt": "State1-BarNotPresent",
"States": {
"State1-BarNotPresent": {
"Type": "Pass",
"Parameters": {
"Field1": "Value1"
},
"Next": "State2-BarNotPresent"
},
"State2-BarNotPresent": {
"Type": "Pass",
"Parameters": {
"Foo.$": "$.['Bar','AnyOtherField']"
},
"End": true
}
}
}
],
"End": true
}
}
}
Method 2:
Depending on how much control you have over the structure of the input and if you don't need to reference any other variables from the input you might be able to do this in a single State using InputPath and Parameters. The idea behind this is to get your Input to the State in the form:
"OutputArray": [
{
"Bar": {
"Value": [
"Baz"
]
}
},
{
"Foo": {
"Value": [
"DefaultValueFoo"
]
}
}
]
The first element of the array should be the value which might be missing, if the value is missing the default value (foo) will be the first element. As the first element will always be present we can use the JSONPath "$['OutputArray'][0]..Value" to get the Value of the first element. After that in Parameters we can use the JSONPath "$[0][0]" to extract the exact value. Here's an example State Machine for this example:
{
"StartAt": "State1",
"States": {
"State1": {
"Type": "Pass",
"Parameters": {
"OutputArray": [
{
"Bar": {
"Value": [
"Baz"
]
}
},
{
"Foo": {
"Value": [
"DefaultValueFoo"
]
}
}
]
},
"Next": "State2"
},
"State2": {
"Type": "Pass",
"InputPath": "$['OutputArray'][0]..Value",
"Parameters": {
"Foo.$": "$[0][0]"
},
"End": true
}
}
}
