52

I have a script that uses SCP to pull a file from a remote Linux host on AWS. After running the same code nightly for about 6 months without issue, it started failing today with protocol error: filename does not match request. I reproduced the issue on some simpler filenames below:

$ scp -i $IDENT $HOST_AND_DIR/"foobar" .
# the file is copied successfully

$ scp -i $IDENT $HOST_AND_DIR/"'foobar'" .
protocol error: filename does not match request
# used to work, i swear...

$ scp -i $IDENT $HOST_AND_DIR/"'foobarbaz'" .
scp: /home/user_redacted/foobarbaz: No such file or directory
# less surprising...

The reason for my single quotes was that I was grabbing a file with spaces in the name originally. To deal with the spaces, I had done $HOST_AND_DIR/"'foo bar'" for many months, but starting today, it would only accept $HOST_AND_DIR/"foo\ bar". So, my issue is fixed, but I'm still curious about what's going on.

I Googled the error message, but I don't see any real mentions of it, which surprises me.

Both hosts involved have OpenSSL 1.0.2g in the output of ssh -v localhost, and bash --version says GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu) Any ideas?

dcc310
  • 1,038
  • 1
  • 9
  • 13
  • This isn't a helpful comment but I've just started getting this today for something I've had running hourly for a month. Only happened after installing updates on ubuntu. – JBond Feb 08 '19 at 19:20
  • 1
    I posted another question about this feature here: https://unix.stackexchange.com/questions/499958/why-does-scps-strict-filename-checking-reject-quoted-last-component-but-not-oth – Reinstate Monica Feb 11 '19 at 16:09

2 Answers2

102

I ended up having a look through the source code and found the commit where this error is thrown:

GitHub Commit

remote->local directory copies satisfy the wildcard specified by the user.

This checking provides some protection against a malicious server sending unexpected filenames, but it comes at a risk of rejecting wanted files due to differences between client and server wildcard expansion rules.

For this reason, this also adds a new -T flag to disable the check.

They have added a new flag -T that will ignore this new check they've added so it is backwards compatible. However, I suppose we should look and find out why the filenames we're using are flagged as restricted.

JBond
  • 3,062
  • 5
  • 27
  • 31
  • very interesting! I was pretty sure I hadn't updated any software that would also update openssh stuff, but I just learned about `/var/log/apt/history.log*` and it seems like openssh was updated this morning by `/usr/bin/unattended-upgrade`. The last time it was automatically upgraded before that was November 2018. I didn't know that my (very vanilla, uncustomized) system was being automatically updated at all. I don't see anything saying that this auto-updating is the default on EC2 Ubuntu, but I guess it must be. – dcc310 Feb 08 '19 at 21:07
  • 3
    We encountered the same problem and -T fixed it for us as well. The funny thing is that we're using absolute paths without any wildcards in both src and dst. I'm tempted to say it's a bug in the commit referenced above. In our environment the scp command is being called by a PHP program which is executed by a cron job. – mmalone Feb 11 '19 at 23:25
  • 5
    this comes in handy when you are doing `scp -T "foo@bar:'/long filename/with annoying/spaces in it/that you dont want/to escape'"` – David Dombrowsky Mar 07 '19 at 19:46
  • 2
    Adding `alias scp='scp -T'` to your `.bashrc` will do this automatically. – Richard Smith Sep 20 '19 at 12:03
  • 11
    `that will ignore this new check they've added so it is backwards compatible` That's not how backwards-compatibility works... – SineSwiper Oct 17 '19 at 14:10
  • In the middle of 2021 this is actual. For modern scp this is the answer. But for older i.e. CentOS 7.8, there is no -T option. Then user may try to use `sftp` instead of `scp` OR use `sshpass + scp`. The last construction is used successfully in Teamcity between CentOS-based agent and Windows SSH server. – Rapekas Oct 12 '21 at 13:33
  • This bug is reported at https://bugzilla.mindrot.org/show_bug.cgi?id=2966 – leoly Oct 24 '21 at 04:57
  • The original exploit is documented [here](https://www.exploit-db.com/exploits/46516). – pedz Jun 17 '22 at 14:57
2

In my case, I had [] characters in the filename that needed to be escaped using one of the options listed here. for example:

scp USERNAME@IP_ADDR:"/tmp/foo\[bar\].txt" /tmp
eta32carinae
  • 473
  • 4
  • 12