114

Attempting to pull a single file using

adb pull /data/data/com.corp.appName/files/myFile.txt myFile.txt

fails with

failed to copy '/data/data/com.corp.appName/files/myFile.txt myFile.txt' to 'myFile.txt': Permission denied

despite that USB debugging is enabled on the device.

We can go around the problem through the archaic route

adb shell
run-as com.corp.appName
cat files/myFile.txt > myFile.txt

but this is unwieldy for more than one file.

How can I pull the directory /data/data/com.corp.appName/files to my MacBook?

Doing this either directly or through a transit in `/storage/sdcard0/myDir (from where I can continue with Android File Transfer) is fine.

Additional Comment

It may be that just running

adb backup  -f myFiles com.corp.appName

will generate the files I am looking for. In that case I am looking for a way to untar/unzip the resulting backup!

Calaf
  • 10,113
  • 15
  • 57
  • 120
  • 5
    Unless the device is rooted, you simply can't do that. – 323go Mar 21 '13 at 21:24
  • I hope you're wrong, but just in case, please move your comment to be an answer. Could you add some kind of reference to back up why you think so? I would like to save all the junk files that the buggy versions of my program have saved to make sure down the road that my closer-to-release program will gracefully handle all kinds of input files. – Calaf Mar 21 '13 at 21:27
  • I'm going to cite experience (20+ devices), and of course http://developer.android.com/guide/topics/data/data-storage.html#filesInternal tells you that Internal Storage is private to the app. If you're just collecting debug data, write it to external storage or make it `WORLD_READABLE`. – 323go Mar 21 '13 at 21:33
  • 1
    Does that mean that one could chmod the directory from world:--x to world:r-x long enough to be able to fetch the files? – Calaf Mar 21 '13 at 21:45
  • I think it might be helpful to know if you are trying to copy files from an App you are responsible for, or whether this is someone elses App. If it is your App, then you can simply write the required data to an external storage medium. If it is someone elses App, you are going to require an interface of some kind (such as a ContentProvider) or root. – Knossos Mar 21 '13 at 21:55
  • It is indeed my app. It is still midway through development. To write, I opened the files using 'context.getApplicationContext().openFileOutput( myFile, Context.MODE_PRIVATE )'. But I can now see that this is a bad idea, and perhaps that is why so many apps store their files at the root of the filesystem rather than in the directory officially sanctioned by Android. – Calaf Mar 21 '13 at 22:05
  • What I meant is don't use MODE_PRIVATE, but rather MODE_WORLD_READABLE, during development. Or better yet, use external storage during development. – 323go Mar 21 '13 at 22:11
  • 2
    @Knossos Notwithstanding that it's my app, is it not legitimate for users to be able to save a copy of the private data that an app has stored on their devices? After all, the data presumably belong to the device owner, not the app author. – Calaf Mar 22 '13 at 15:32
  • 1
    The problem is how do you differentiate between the owner, and a malicious App/program, trying to interface with your Apps private data. (which could have a users personal data in) – Knossos Mar 22 '13 at 19:38
  • I have written a bash script for syncing your app's files https://gist.github.com/peterchaula/fedfea3f503a97d2370b5500e1593f90 – Peter Chaula Sep 21 '19 at 12:02

14 Answers14

124

adb backup will write an Android-specific archive:

adb backup  -f myAndroidBackup.ab  com.corp.appName

This archive can be converted to tar format using:

dd if=myAndroidBackup.ab bs=4K iflag=skip_bytes skip=24 | openssl zlib -d > myAndroidBackup.tar

Reference:

http://nelenkov.blogspot.ca/2012/06/unpacking-android-backups.html

Search for "Update" at that link.


Alternatively, use Android backup extractor to extract files from the Android backup (.ab) file.

Community
  • 1
  • 1
Calaf
  • 10,113
  • 15
  • 57
  • 120
  • 5
    `bs=24 skip=1` is much faster than `bs=1 skip=24` (on my system, 38.6 MB/s vs 1.7 MB/s) :) – netvope Nov 03 '13 at 07:04
  • http://blog.shvetsov.com/2013/02/access-android-app-data-without-root.html very helpful article...! – Thomas Gatt Nov 24 '13 at 09:22
  • 23
    Python alternative for where openssl has not been compiled with zlib: `dd if=data.ab bs=1 skip=24 | python -c "import zlib,sys;sys.stdout.write(zlib.decompress(sys.stdin.read()))" | tar -xvf -` (from above source) – OJFord Feb 25 '14 at 19:36
  • 7
    I've never had any success with the `dd` command on my MacBook at work. Instead, I use `java -jar ./abe.jar unpack data.ab data.tar`. You can download the abe utility from http://sourceforge.net/projects/adbextractor/files/?source=navbar – Someone Somewhere Jun 05 '14 at 00:25
  • 5
    Also, I noticed that I've had to force-close the app whose data I want to backup otherwise the backup is empty ! (Android 4.4.2) Which is the same effect as when the manifest shows `android:allowBackup="false"` – Someone Somewhere Jun 05 '14 at 00:27
  • 3
    Even better, directly extract the files into the local directory: `dd if=myAndroidBackup.ab bs=24 skip=1 | openssl zlib -d | tar -x` – mreichelt Nov 05 '14 at 11:30
  • Backed up Game data with apk. Nougat Oneplus 2. adb backup "-apk com.nekki.shadowfight" -f "c:\myapk\samsung2.ab" – Samin Apr 27 '17 at 05:49
  • I also had to use @OJFord's solution of piping to python. I am using Windows Subsystem for Linux (WSL) for Bash support on Windows 10. – Mavaddat Javid Dec 31 '17 at 21:00
  • 4
    I don't know why we use `dd` as much as we do. Here `tail -c+25 myAndroidBackup.ab | ...` is fine. – Dzamo Norton May 16 '19 at 10:51
61

I had the same problem but solved it running following:

$ adb shell
$ run-as {app-package-name}
$ cd /data/data/{app-package-name}
$ chmod 777 {file}
$ cp {file} /mnt/sdcard/

After this you can run

$ adb pull /mnt/sdcard/{file}
Trunst
  • 854
  • 7
  • 8
  • 2
    thanks man! you also need to use "cp -r" to make it a recursive copy. https://stackoverflow.com/questions/39961621/no-such-file-or-directory-when-cp-but-not-with-ls – 朱西西 Oct 03 '17 at 05:55
44

Here is what worked for me:

adb -d shell "run-as com.example.test cat /data/data/com.example.test/databases/data.db" > data.db

I'm printing the database directly into local file.

Mattias Isegran Bergander
  • 11,811
  • 2
  • 41
  • 49
  • 3
    Exactly this command caused my db file to be corrupted. But I made it to work with the following steps: ------ `adb shell` ------ inside the shell run `run-as com.example.test` ------ inside the shell run `cat ...db > /mnt/sdcard/file.db` ------ `adb pull /mnt/sdcard/file.db .` ------ end (sorry, line breaks are not working) – Kirill Oficerov May 23 '16 at 18:34
  • @KirillOficerov it would be good if those commands can be written in one line – emen Oct 08 '16 at 23:25
  • Great when you need 1 file. Been using this for database. Facing the case now where I need multiple files. – Dave Thomas Oct 12 '16 at 16:26
  • For some reason using `adb shell` didn't work for me, but following [this answer](https://stackoverflow.com/a/36531095/2413303) and using `adb exec-out` it worked! – EpicPandaForce Oct 05 '17 at 11:31
  • It returns "adb.exe: no devices found" – Ulysses Alves Mar 30 '21 at 17:28
  • 1
    @UlyssesAlves what does `adb devices` show? you connected over usb? did you enable developer options, and usb debugging? – nhed Jun 21 '21 at 15:06
23

On MacOSX, by combining the answers from Calaf and Ollie Ford, the following worked for me.

On the command line (be sure adb is in your path, mine was at ~/Library/Android/sdk/platform-tools/adb) and with your android device plugged in and in USB debugging mode, run:

 adb backup -f backup com.mypackage.myapp

Your android device will ask you for permission to backup your data. Select "BACKUP MY DATA"

Wait a few moments.

The file backup will appear in the directory where you ran adb.

Now run:

dd if=backup bs=1 skip=24 | python -c "import zlib,sys;sys.stdout.write(zlib.decompress(sys.stdin.read()))" > backup.tar

Now you'll you have a backup.tar file you can untar like this:

 tar xvf backup.tar

And see all the files stored by your application.

  • Your Mac version works! NOTE: both `adb backup` and the `dd` tar conversion steps provide no status while working and can take minutes if you have a lot of data. Be patient! – MechEthan May 23 '16 at 17:29
  • 2
    I am getting: "zlib.error: Error -3 while decompressing data: incorrect header check" My backup is asking for a password to encrypt though. This python method doesn't ask for password to decrypt backup. The abe.jar in the comments in above example asks for password, but I did not get all of the files after untaring using that method. – Dave Thomas Oct 12 '16 at 16:23
  • My second attempt with abe.jar on an encrypted backup worked. – Dave Thomas Oct 12 '16 at 22:35
10

Newer versions of Android Studio include the Device File Explorer which I've found to be a handy GUI method of downloading files from my development Nexus 7.

You Must make sure you have enabled USB Debugging on the device

  1. Click View > Tool Windows > Device File Explorer or click the Device File Explorer button in the tool window bar to open the Device File Explorer.
  2. Select a device from the drop down list.
  3. Interact with the device content in the file explorer window. Right-click on a file or directory to create a new file or directory, save the selected file or directory to your machine, upload, delete, or synchronize. Double-click a file to open it in Android Studio.

    Android Studio saves files you open this way in a temporary directory outside of your project. If you make modifications to a file you opened using the Device File Explorer, and would like to save your changes back to the device, you must manually upload the modified version of the file to the device.

file explorer

Full Documentation

Jon
  • 9,156
  • 9
  • 56
  • 73
6

You may use this shell script below. It is able to pull files from app cache as well, not like the adb backup tool:

#!/bin/sh

if [ -z "$1" ]; then 
    echo "Sorry script requires an argument for the file you want to pull."
    exit 1
fi

adb shell "run-as com.corp.appName cat '/data/data/com.corp.appNamepp/$1' > '/sdcard/$1'"
adb pull "/sdcard/$1"
adb shell "rm '/sdcard/$1'"

Then you can use it like this:

./pull.sh files/myFile.txt
./pull.sh cache/someCachedData.txt
Dave Thomas
  • 3,667
  • 2
  • 33
  • 41
Tamas
  • 3,254
  • 4
  • 29
  • 51
  • 11
    The best part is when you don't give arguments and then `rm /sdcard/` substitutes, lucky it's not `-rf`. – TWiStErRob Oct 19 '14 at 16:18
  • 3
    You can just cat directly to a file on the host: "adb shell run-as com.whatever.Fnord cat /data/data/com.whatever.Fnord/databases/logging.db > /tmp/foo" (where /tmp/foo is a file on the host machine, not the android device) – James Moore Jun 24 '16 at 20:53
  • TBH I don't understand the fuss about this. The (original) script would never have deleted the directory (missing -rf switch), unless it was empty. I haven't checked but even if the directory is empty then I doubt that the script deletes the directory as it's a mount point. The script looked dangerous, but it never was... – Tamas May 04 '18 at 06:23
6

If you are using a Mac machine and a Samsung phone, this is what you have to do (since run-as doesn't work on Samsung and zlib doesn't work on Mac)

  1. Take a backup of your app's data directory adb backup -f /Users/username/Desktop/data.ab com.example

  2. You will be asked for a password to encrypt in your Phone, don't enter any. Just tap on "Back up my data". See How to take BackUp?

  3. Once successfully backed up, you will see data.ab file in your Desktop. Now we need to convert this to tar format.

  4. Use Android Backup Extractor for this. Download | SourceCode

  5. Download it and you will see abe.jar file. Add this to your PATH variable.

  6. Execute this to generate the tar file: java -jar abe.jar unpack /Users/username/Desktop/data.ab /Users/username/Desktop/data.tar

  7. Extract the data.tar file to access all the files

Henry
  • 17,490
  • 7
  • 63
  • 98
  • 2
    Followed this exactly and the `data.tar` just extracts to a `data.tar.cpgz` @henry – user-44651 Feb 13 '18 at 15:58
  • I've read 5 other topics articles on this thread "how to get database from my app installed on a device" and this is the only solution that is working for me. My phone is not rooted, most commands from `adb` is not working such as `run-as` `adb shell, cd \data\data ls` etc., I don't want to install Android studio (I'm using Visual Studio Xamarin), and this solution is the most painless for me. – mihkov Apr 30 '21 at 20:34
2

After setting the right permissions by adding the following code:

File myFile = ...;
myFile.setReadable(true, false); // readable, not only for the owner

adb pull works as desired.

see File.setReadable()

18446744073709551615
  • 16,368
  • 4
  • 94
  • 127
  • 1
    It's all right to change the permissions while debugging, but at some point near release doing so would defeat the protection offered by default (to prevent other apps from accessing/messing up with the files). Hence a way for the developer to access the files even though they are just user-readable/writable is more desirable. – Calaf Aug 01 '13 at 18:41
2

This answer is based on my experience with other answers, and comments in the answers. My hope is I can help someone in a similar situation.

I am doing this on OSX via terminal.

Previously Vinicius Avellar's answer worked great for me. I was only ever most of the time needing the database from the device from a debug application.

Today I had a use case where I needed multiple private files. I ended up with two solutions that worked good for this case.

  1. Use the accepted answer along with Someone Somewhere's OSX specific comments. Create a backup and use the 3rd party solution, sourceforge.net/projects/adbextractor/files/?source=navbar to unpack into a tar. I'll write more about my experience with this solution at the bottom of this answer. Scroll down if this is what you are looking for.

  2. A faster solution which I settled with. I created a script for pulling multiple files similar to Tamas' answer. I am able to do it this way because my app is a debug app and I have access to run-as on my device. If you don't have access to run-as this method won't work for you on OSX.

Here is my script for pulling multiple private files that I'll share with you, the reader, who is also investigating this awesome question ;) :

#!/bin/bash
#
# Strict mode: http://redsymbol.net/articles/unofficial-bash-strict-mode/
set -euo pipefail
IFS=$'\n\t'

# 
# Usage: script -f fileToPull -p packageName
# 

# This script is for pulling private files from an Android device
# using run-as. Note: not all devices have run-as access, and
# application must be a debug version for run-as to work.
# 
# If run-as is deactivated on your device use one of the
# alternative methods here:
# http://stackoverflow.com/questions/15558353/how-can-one-pull-the-private-data-of-ones-own-android-app
# 
# If you have encrypted backup files use:
# sourceforge.net/projects/adbextractor/files/?source=navbar 
# From comments in the accepted answer in the above SO question
# 
# If your files aren't encrypted use the accepted answer 
# ( see comments and other answers for OSX compatibility )
# 
# This script is open to expansions to allow selecting 
# device used. Currently first selected device from
# adb shell will be used.

#Check we have one connected device
adb devices -l | grep -e 'device\b' > /dev/null

if [ $? -gt 0 ]; then
    echo "No device connected to adb."
    exit 1
fi

# Set filename or directory to pull from device
# Set package name we will run as
while getopts f:p: opt; do
    case $opt in
        f)
            fileToPull=$OPTARG
            ;;
        p)
            packageName=$OPTARG
            ;;
    esac
done;

# Block file arg from being blank
if [ -z "$fileToPull" ]; then
    echo "Please specify file or folder to pull with -f argument"
    exit 1
fi

# Block package name arg from being blank
if [ -z "$packageName" ]; then
    echo "Please specify package name to run as when pulling file"
    exit 1
fi

# Check package exists
adb shell pm list packages | grep "$packageName" > /dev/null
if [ $? -gt 0 ]; then
    echo "Package name $packageName does not exist on device"
    exit 1
fi

# Check file exists and has permission with run-as
fileCheck=`adb shell "run-as $packageName ls $fileToPull"`
if [[ $fileCheck =~ "Permission denied" ]] || [[ $fileCheck =~ "No such file or directory" ]]; then
    echo "Error: $fileCheck"
    echo "With file -> $fileToPull"
    exit 1
fi

# Function to pull private file
#
# param 1 = package name
# param 2 = file to pull
# param 3 = output file
function pull_private_file () {

    mkdir -p `dirname $3`

    echo -e "\033[0;35m***" >&2
    echo -e "\033[0;36m Coping file $2 -> $3" >&2
    echo -e "\033[0;35m***\033[0m" >&2

    adb shell "run-as $1 cat $2" > $3
}

# Check if a file is a directory
# 
# param 1 = directory to check
function is_file_dir() {

    adb shell "if [ -d \"$1\" ]; then echo TRUE; fi"
}

# Check if a file is a symbolic link
# 
# param 1 = directory to check
function is_file_symlink() {

    adb shell "if [ -L \"$1\" ]; then echo TRUE; fi"
}

# recursively pull files from device connected to adb
# 
# param 1 = package name
# param 2 = file to pull
# param 3 = output file
function recurse_pull_private_files() {

    is_dir=`is_file_dir "$2"`
    is_symlink=`is_file_symlink "$2"`

    if [ -n "$is_dir" ]; then

        files=`adb shell "run-as $1 ls \"$2\""`

        # Handle the case where directory is a symbolic link
        if [ -n "$is_symlink" ]; then
            correctPath=`adb shell "run-as $1 ls -l \"$2\"" | sed 's/.*-> //' | tr -d '\r'`
            files=`adb shell "run-as $1 ls \"$correctPath\""`
        fi

        for i in $files; do

            # Android adds nasty carriage return that screws with bash vars
            # This removes it. Otherwise weird behavior happens
            fileName=`echo "$i" | tr -d '\r'` 

            nextFile="$2/$fileName"
            nextOutput="$3/$fileName"
            recurse_pull_private_files "$1" "$nextFile" "$nextOutput"
        done
    else

        pull_private_file "$1" "$2" "$3"
    fi
}

recurse_pull_private_files "$packageName" "$fileToPull" "`basename "$fileToPull"`"

Gist: https://gist.github.com/davethomas11/6c88f92c6221ffe6bc26de7335107dd4


Back to method 1, decrypting a backup using Android Backup Extractor

Here are the steps I took on my Mac, and issues I came across:

First I queued up a backup ( and set a password to encrypt my backup, my device required it ):

adb backup -f myAndroidBackup.ab  com.corp.appName

Second I downloaded just abe.jar from here: https://sourceforge.net/projects/adbextractor/files/abe.jar/download

Next I ran:

java -jar ./abe.jar unpack myAndroidBackup.ab myAndroidBackup.tar

At this point I got an error message. Because my archive is encrypted, java gave me an error that I needed to install some security policy libraries.

  • So I went to http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html and downloaded the security policy jars I needed. Now in my case the install instructions told me the wrong location to put the jar files. It says that the proper location is <java-home>/lib/security. I put them there first and still got the error message. So I investigated and on my Mac with Java 1.8 the correct place to put them was: <java-home>/jre/lib/security. I made sure to backup the original policy jars, and put them there. Vola I was able to enter a password with abe.jar and decrypt to a tar file.

Lastly I just ran ( after running previous command again )

tar xvf myAndroidBackup.tar

Now it is important to note that if you can just run-as and cat, it is much much faster. One, you only get the files you want and not the entire application. Two, the more files ( + encryption for me ) makes it slower to transfer. So knowing to do this way is important if you don't have run-as on OSX, but the script should be first goto for a debug application.

Mind you I just wrote it today and tested it a few times, so please notify me of any bugs!

Dave Thomas
  • 3,667
  • 2
  • 33
  • 41
  • 1
    Thanks to your response with method #1 I've been able to get and explore a package backup from a debug targeted app installed on a production non rooted device, but the backup contains only the manifest file. With method #2 I downloaded all app's data, but binaries (i.e. SQLite files) are unreadable. I add a solution based on you script that worked for me. – Paolone Mar 05 '17 at 11:44
2

Similar to Tamas's answer, here is a one-liner for Mac OS X to fetch all of the files for app with your.app.id from your device and save them to (in this case) ~/Desktop/your.app.id:

(
    id=your.app.id &&
    dest=~/Desktop &&
    adb shell "run-as $id cp -r /data/data/$id /sdcard" &&
    adb -d pull "/sdcard/$id" "$dest" &&
    if [ -n "$id" ]; then adb shell "rm -rf /sdcard/$id"; fi
)
  • Exclude the -d to pull from emulator
  • Doesn't stomp your session variables
  • You can paste the whole block into Terminal.app (or remove newlines if desired)
Zack Morris
  • 4,727
  • 2
  • 55
  • 83
1

Starting form Dave Thomas script I've been able to write my own solution to overcome 2 problems:

  1. my backup was containing only the manifest file
  2. binary files got with Dave Thomas where unreadable

This is my script, that copies app data to sdcard and then pull it

#Check we have one connected device
adb devices -l | grep -e 'device\b' > /dev/null

if [ $? -gt 0 ]; then
    echo "No device connected to adb."
    exit 1
fi

# Set filename or directory to pull from device
# Set package name we will run as
while getopts f:p: opt; do
    case $opt in
        f)
            fileToPull=$OPTARG
            ;;
        p)
            packageName=$OPTARG
            ;;
    esac
done;

# Block package name arg from being blank
if [ -z "$packageName" ]; then
    echo "Please specify package name to run as when pulling file"
    exit 1
fi

# Check package exists
adb shell pm list packages | grep "$packageName" > /dev/null
if [ $? -gt 0 ]; then
    echo "Package name $packageName does not exist on device"
    exit 1
fi

    adb shell "run-as $packageName cp -r /data/data/$packageName/ /sdcard/$packageName"
    adb pull /sdcard/$packageName
    adb shell rm -rf /sdcard/$packageName
Paolone
  • 475
  • 1
  • 7
  • 16
0

Backed up Game data with apk. Nougat Oneplus 2.

**adb backup "-apk com.nekki.shadowfight" -f "c:\myapk\samsung2.ab"**
Samin
  • 577
  • 6
  • 6
  • ```adb backup -f myapp.ab -apk com.myapp # backup on one device``` ```adb restore myapp.ab # restore to the same or any other device``` – Steven Linn Jul 08 '19 at 16:31
0

Does that mean that one could chmod the directory from world:--x to world:r-x long enough to be able to fetch the files?

Yes, exactly. Weirdly enough, you also need the file to have the x bit set. (at least on Android 2.3)

chmod 755 all the way down worked to copy a file (but you should revert permissions afterwards, if you plan to continue using the device).

serv-inc
  • 35,772
  • 9
  • 166
  • 188
0

you can do:

adb pull /storage/emulated/0/Android/data//

  • 6
    Don't get me wrong but, have you read the question properly? OP is asking for pulling the 'private' data **NOT** 'public' – Neeraj May 22 '18 at 16:01