Backing up and restoring data on Android devices directly via USB (Howto)

Motivation

I was looking for a simple way to backup data on Android devices directly to a device running GNU/Linux connected over a USB cable (in my case, a desktop computer).

Is this really so unique that it’s worth writing a new article about it? Well, in my case, I did not want to buffer the data on any “intermediate” devices such as storage cards connected via microSD or USB-OTG. Also, I did not want to use any proprietary tools or formats. Instead, I wanted to store my backups in “oldschool” formats such as dd-images or tar archives. I did not find a comprehensive howto for that, so I decided to write this article.

Disclaimer


All data and information provided in this article is for informational purposes only. The author makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this article and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.

In no event the author we be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from loss of data or profits arising out of, or in connection with, the use of this article.

Overview

This article describes two different approaches. Both have their pros and cons:

  • Block-level: Doing 1:1 block-level backups (above the file system) is an imaging approach that corresponds to doing dd-style backups.
  • Filesystem-level: Doing filesystem-level backups (on the file system) corresponds to tar-style backups.

An important factor when doing backups is also performance. Filesystem-level backups are usually faster on block devices which are only filled up to a small degree. However, due to the file system overhead they have a lower “raw” throughput rate — especially when backing up data on flash media such as microSD cards. Here, typical filesystems such as ext4 or f2fs operating with a 4K block size are a major bottleneck as these media often have horrible 4k write/read performance.

The following instructions for applying these approaches assume that you already have a “liberated” Android device which can boot into TWRP (a free Android recovery) or CWM. I am using the example of a Nexus-S running Replicant 4.2.0004 and TWRP 2.8.7.0, but the approaches also work with most other Android distributions and recoveries.

Getting familiar with the block devices on your Android device

First of all, you should know which block device you actually want to backup. The internal flash on Android devices is usually partitioned in about 15-25 partitions, depending on the device. To get a first overview, you can try the following (I am using adb shell on the desktop):

$ adb shell cat /proc/partitions

major minor #blocks name

31 0 2048 mtdblock0
31 1 1280 mtdblock1
31 2 8192 mtdblock2
31 3 8192 mtdblock3
31 4 480768 mtdblock4
31 5 13824 mtdblock5
31 6 6912 mtdblock6
179 0 15552512 mmcblk0
179 1 524288 mmcblk0p1
179 2 1048576 mmcblk0p2
179 3 13978607 mmcblk0p3
179 16 1024 mmcblk0boot1
179 8 1024 mmcblk0boot0

To find out what the partitions are about you can inspect the directory /dev/block/platform//by-Name/ which contains symlinks to the actual partitions. In my case, the Nexus-S has two flash chips and I am listing the partitions of the one where the userdata-partition resides on:

$ adb shell ls -l /dev/block/platform/s3c-sdhci.0/by-name

lrwxrwxrwx root root 2016-11-02 19:51 media -> /dev/block/mmcblk0p3
lrwxrwxrwx root root 2016-11-02 19:51 system -> /dev/block/mmcblk0p1
lrwxrwxrwx root root 2016-11-02 19:51 userdata -> /dev/block/mmcblk0p2

Please note that unlike the Nexus-S, most newer Android devices only have a single eMMC flash chip and don't use MTD devices anymore.

Block-level approach

Block-level backups take up a lot of space (without compression) and extracting single files is cumbersome (especially when talking about encrypted data partitions or backups of the whole flash). On the other hand, "just" restoring a full backup is easy.

Backing up a single partition

Now that you know which block devices you want to backup, you can directly create a 1:1 image via adb pull as you would normally do by using dd. In our case:

$ adb pull /dev/block/platform/s3c-sdhci.0/by-name/userdata

7942 KB/s (1073741824 bytes in 132.027s)

On your workstation, you will obtain a file named userdata which contains the whole partition/filesystem as an image. If you didn't enable encryption on your Android device, you can directly mount this file as loopback device and access its contents:

$ mount userdata /mnt

Restoring a single partition


BE CAREFUL! RESTORING THE WRONG IMAGE OR WRITING TO THE WRONG BLOCK DEVICE CAN RUIN YOUR DATA OR EVEN BRICK YOUR ANDROID DEVICE!

To restore your backup, you can simply use adb push. In my case:

$ adb push userdata /dev/block/platform/s3c-sdhci.0/by-name/userdata

failed to copy 'userdata' to '/dev/block/platform/s3c-sdhci.0/by-name/userdata': No space left on device

Alternative: Operating on the whole block device

Instead of backing up just a single partition, it is also possible to backup the whole flash device including all partitions. Example:

$ adb pull /dev/block/mmcblk0

Remarks:

  • On some devices, not all partitions are readable and, thus, cannot be backed up.
  • Please be careful with restores!
  • Accessing files inside this image is not straight-forward (but doable).

Filesystem-level approach

Filesystem-level backups only work for single partitions, take up as much space as the files on the particular filesystem you backup and it is easy to access individual files in them. I am using a combination of adb, netcat and tar to create and restore these backups.

Backing up your data

First, connect to your device via an adb shell:

$ adb shell

Then, change to the directory from where you want to create your backup. If your device was not automatically mounted, you have to do it first:

# mount /dev/block/platform/s3c-sdhci.0/by-name /data

Now change to this directory:

# cd /data

Now, start the netcat process:

# tar -cvp . | busybox nc -lp 5555

On the receiver side (desktop), set up adb port forwarding:

$ adb forward tcp:4444 tcp:5555

Then, start the process to receive the tar file:

$ nc -w 10 localhost 4444 > userdata.tar

You should see your files being packed up on the Android side:

./
./lost+found/
./dontpanic/
./misc/
./misc/adb/
./misc/audit/
./misc/audit/audit.log
...

Now wait for the process to exit.

Restoring your data

Again, on your receiver side (Android device), mount /data if it was not mounted yet and change in there:

# mount /dev/block/platform/s3c-sdhci.0/by-name /data
# cd /data

Now, start the tar extraction process:

# busybox nc -lp 5555 | tar -xpvf -

On the sender side (desktop), again, set up adb port forwarding:

$ adb forward tcp:4444 tcp:5555

And send the tar file:

$ cat userdata.tar | nc -q 2 localhost 4444

Now you should be able to see your previously backed up files getting restored...

References

For the filesystem-level part of the article I used and adapted the following sources:

  • http://www.screenage.de/blog/2007/12/30/using-netcat-and-tar-for-network-file-transfer/
  • http://stackoverflow.com/questions/15278587/pipe-into-adb-shell