How to pipe wget results to subsequent curl post?

Question

I am in Windows 10 attempting to get Confluent CE up via docker. See their instructions here.

The trouble is I believe these are specifically for MAC OS, and Windows requires slightly different syntax for the following command:

wget https://github.com/confluentinc/kafka-connect-datagen/raw/master/config/connector_pageviews_cos.config
curl -X POST -H "Content-Type: application/json" --data @connector_pageviews_cos.config http://localhost:8083/connectors

I think I am supposed to be piping the wget results to the curl. How to do that in Windows 10 ?

The powershell exception:

At line:1 char:57
+ ... ntent-Type: application/json" --data @connector_pageviews_cos.config ...
+                                          ~~~~~~~~~~~~~~~~~~~~~~~~
The splatting operator '@' cannot be used to reference variables in an expression. '@connector_pageviews_cos' can be
used only as an argument to a command. To reference variables in an expression use '$connector_pageviews_cos'.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : SplattingNotPermitted

Thanks!

codewario · Accepted Answer · 2019-10-31T12:29:11.710

Both curl and wget are aliased in Powershell 5.1 to Invoke-WebRequest, although it seems that curl.exe is available under System32 (and is available in place of an alias in Powershell Core). But, it's not too difficult to translate both calls to the Powershell-native Invoke-WebRequest cmdlet.

Your wget call becomes:

Invoke-WebRequest -Uri https://github.com/confluentinc/kafka-connect-datagen/raw/master/config/connector_pageviews_cos.config -Outfile connector_pageviews_cos.config

And your curl call becomes:

Invoke-WebRequest -Uri http://localhost:8083/connectors -Method POST -ContentType 'application/json' -Body (Get-Content -Raw connector_pageviews_cos.config)

If desired, you can combine both calls into a single line, as the -Body parameter accepts pipeline input:

Invoke-WebRequest -Uri https://github.com/confluentinc/kafka-connect-datagen/raw/master/config/connector_pageviews_cos.config |
  Invoke-WebRequest -Uri http://localhost:8083/connectors -Method POST -ContentType 'application/json'

Omit the -OutFile parameter on the first Invoke-WebRequest, which outputs the payload contents to the pipeline. You then pipe that output into the second Invoke-WebRequest while providing other positional and named arguments as needed.

More information on Invoke-WebRequest is available here, and you would also do well to check out the convenient-to-use Invoke-RestMethod which makes working with RESTful APIs much more comfortable.

As a note, the @ operator is used for a concept called splatting in Powershell, which explains the error you were receiving. Read the link above for more information, and elsewhere I've also written an answer on how to use splatting to pass arguments to cmdlets and commands.

Yes. I did start to gather as much about the Invoke-WebRequest, and also read about this "splatting". It appears that the results of these commands would leave that config file behind. Is there a way to pipe the out of such from first cmd to become the input of the second? (Just curious!) The results being that I would end up with the post and no residual file to clean up. Thanks! I will try this out... — Adam Cox, Oct 31 '19 at 04:28
Thanks! It's a good answer. Moved me forward slightly in the struggle to get a test Confluent environment running via Docker in a Windows 10 OS... — Adam Cox, Oct 31 '19 at 19:31

How to pipe wget results to subsequent curl post?

1 Answers1