16

I've been trying to move files from my android device to my osx machine using something similar to: adb shell tar -c directory_to_copy | tar -x. It seems that the remote tar is working but somewhere on the way the file gets corrupted. After some playing around I've found:

  • It seems the adb shell command translates LF to CRLF:

    % adb shell 'cd /mnt/sdcard;echo hi>a.bin'
% adb shell 'cd /mnt/sdcard;cat a.bin' | hexdump -C
00000000  68 69 0d 0a                                       |hi..|
00000004
% adb pull /mnt/sdcard/a.bin
0 KB/s (3 bytes in 0.457s)
% hexdump -C a.bin
00000000  68 69 0a                                          |hi.|
00000003

  • It looks either the server or the daemon are causing that and not the client (see the len=4):

    % ADB_TRACE=1 adb shell 'cd /mnt/sdcard;cat a.bin'
[... snip ...]
system/core/adb/commandline.c::read_and_dump():read_and_dump(): post adb_read(fd=3): len=4
[... snip ...]

I would guess that the daemon is doing that sort of translation in the shell command for windows users.

My questions are:

  1. wtf? (what does that and for what purpose?)
  2. is there any way to tell it (adbd?) to not do that?
  3. can anyone think of any creative way to circumvent that (I thought about base64 encoding the data, but I would prefer to avoid that overhead. Also, creating a local file is not an option since my filesystem is quite full)

Thanks!

m0she
  • 444
  • 1
  • 4
  • 9

3 Answers3

20

Adb isn't doing this on purpose, but it's not smart enough to refrain from allocating a terminal when you give it a command to run; the terminal on the android side is what's cooking the bytes. Instead of 

adb shell 'cd /mnt/sdcard;cat a.bin' | hexdump -C

try doing

adb shell 'stty raw; cd /mnt/sdcard;cat a.bin' | hexdump -C

This instructs the terminal device not to mangle the bytes at all, but to pass them through.

Glenn Willen
  • 420
  • 4
  • 10
15

Use adb exec-out <command> instead of adb shell.

Example adb exec-out cat /data/myfile.txt > localfile.txt

Fabian Zeindl
  • 5,860
  • 8
  • 54
  • 78
  • Do I need to install something or do I need some specific ADB version to run this "exec-out"? When I try to run "adb exec-out..." it is listing all adb commands options, same when I use some random command like "adb foobar...", which means it does not exists. – Kyore Mar 04 '17 at 14:26
  • My version is Android Debug Bridge version 1.0.36 – Fabian Zeindl Mar 05 '17 at 09:10
  • More info about the `adb exec-out` command: https://android.googlesource.com/platform/system/core/+/5d9d434efadf1c535c7fea634d5306e18c68ef1f – tjanez Aug 08 '17 at 09:24
  • I suspect that exec-out requires an adb on the phone that is new enough to support it, and my recover ROM is not. – mcr May 14 '18 at 22:28
  • But `adb exec-out` doesn't seem to support stdin... :( There's `adb exec-in`, but you can't do communicate bidirectionally, which is needed for e.g. `rsync` – WGH Jun 28 '18 at 14:08
  • Try `adb raw exec:'command'` - it seems to be a bidirectional `adb exec-in/out` – Luke-Jr Mar 13 '22 at 20:59
  • In my case `adb shell` works faster than `adb exec-out`. – pmor Nov 11 '22 at 09:14
11

The "stty" trick mentioned in the first answer does not work in general.

For piping the output to the hexdump command on the host side it might be ok. However, for tar (as is mentioned in the subject of the original question) and for many other commands that can accept binary stream input, this does not work. As Fabian Zeindl has pointed out correctly, using 'adb exec-out ...' is the right way to go.

Some examples for transferring the output of tar over ADB:

  1. Here it is assumed that the tar executable is found in your path on Android:

    • adb exec-out 'cd /sdcard; tar -cf - DCIM/' > DCIM.tar
    • adb exec-out 'cd /sdcard; tar -cf - DCIM/' | tar -tvf -

  2. Some more complex examples involving compression and the use of busybox:

    • adb exec-out 'cd /sdcard; /system/xbin/extras/busybox tar -czf - DCIM/' > DCIM.tgz
    • adb exec-out 'cd /sdcard; GZIP="-9" /system/xbin/extras/busybox tar -czf - DCIM/' | tar -tvzf -
    • adb exec-out 'cd /sdcard; BGZIP2="-9" /system/xbin/extras/busybox tar -cjf - DCIM/' > DCIM.tar.bz2
F.M.
  • 321
  • 3
  • 4
  • How can I get the tar executable? – Shayan Jan 29 '20 at 15:49
  • 7Zip is unable to open the DCIM tar file made this way on windows. Maybe the android tar software is broken somehow or exec-out modifies the binary content in windows – beppe9000 Mar 17 '20 at 09:13
  • 2
    @beppe9000 Probably you were using Powershell, which deliberately converts redirected output in UTF-16 (see https://stackoverflow.com/questions/40098771/changing-powershells-default-output-encoding-to-utf-8). If you use the new Powershell Core, or the classic "cmd", it works. – sigi_tm Feb 20 '21 at 17:01