Posts Tagged ‘Android’
Extracting files from a nandroid backup using unyaffs
I recently upgraded my G1 phone to the latest Cyanogen build (5.x). Since the upgrade instructions recommend wiping user data, I made a “nandroid” backup first, using the handy Amon_RA recovery image. I’ve gotten pretty familiar with the Android filesystem layout, and was confident I could restore anything I really missed (such as my wpa_supplicant.conf with all of my WiFi credentials).
It wasn’t until I finished with the upgrade that I realized the backup wasn’t trivial to work with. It’s a raw yaffs2 flash image, which can’t be simply mounted on a loop device. After messing around for a bit with the nandsim module, mtd-utils and the yaffs2 kernel module, I realized there was a much simpler way: the unassuming unyaffs. It says that it can only extract images created by mkyaffs2image, but apparently the images in the nandroid backup are created this way (or otherwise compatible with unyaffs).
So I downloaded and built unyaffs:
svn checkout http://unyaffs.googlecode.com/svn/trunk/ unyaffs
cd unyaffs
gcc -o unyaffs unyaffs.c
and then ran it on the backup image:
mkdir g1data && cd g1data # unyaffs extracts into the current directory
~/src/android/unyaffs/unyaffs /media/G1data/nandroid/HT839GZ23983/BCDS-20100529-1311/data.img
At which point I could restore files one by one, e.g.:
adb push /tmp/g1data/misc/wifi/wpa_supplicant.conf /data/misc/wifi/
After toggling WiFi off and then back on, all of my credentials were restored. I was able to restore preferences for various applications in the same way.
Cognitive time travel through reminder lists
Credit: guinnessgurl/pamelaadam
List-keeping is generally regarded as boring administrative work, something only important to compulsive organizers. When a writer wants to portray a character as meticulous and dull, they need only brand them as a list-maker, with eyes bespectacled from years of squinting over their lists.
The reality of list-keeping is much more exciting: reminder lists are a mechanism for cognitive time travel. They allow us to transport information from the time when it occurs to us, to a time in the future when it will actually be useful. Like a wormhole, they connect distant points in spacetime (though unfortunately only in one direction, as in the Stargate universe).
Throughout my day, I will remember things I need to do, though not right away: an article which looks interesting, or someone I need to remember to call. Putting these items on a list frees my mind to keep going with whatever I’m doing, knowing that the idea is not lost. A common scenario for me is that I’m riding the tube, reading RSS feeds offline on my Android phone using NewsRob, and come across something I want to explore further. There is as yet no wireless service on the tube, so I can’t do anything but read, but I can send myself an email using K-9 which will be delivered later. At the other end of the wormhole, when I’m back online, I receive the email (usually at my computer) and pick up where I left off.
Traveling through time in your head may not be as exciting as flitting about in a TARDIS, but it is much more accessible, and genuinely rewarding.
Fix broken Android permissions by re-installing apps
I run the CyanogenMod derivative of Android on my G1, and somehow managed to get it into a state where it had withdrawn the security permissions from my installed applications. I think this happened when I attempted to upgrade the 1GB microSD to a 4GB one, but the phone failed to boot.
I first noticed the problem when trying to refresh in NewsRob would hang the application, and adb logcat showed:
W/dalvikvm( 540): threadid=3: thread exiting with uncaught exception (group=0x4001e170)
E/NewsRob ( 540): Caught the following exception:
E/NewsRob ( 540): java.lang.SecurityException: Neither user 10039 nor current process has android.permission.WAKE_LOCK.
NewsRob clearly had had this permission before, to prevent the phone from sleeping during a sync. The Manage Applications screen still showed that it did (“System tools: prevent phone from sleeping”). Watching adb logcat while the phone was booting showed what was going on, and that many other applications had the same problem:
W/PackageManager( 138): Not granting permission android.permission.INTERNET to package com.newsrob because it was previously installed without
W/PackageManager( 138): Not granting permission android.permission.WAKE_LOCK to package com.newsrob because it was previously installed without
W/PackageManager( 138): Not granting permission android.permission.ACCESS_NETWORK_STATE to package com.newsrob because it was previously installed without
W/PackageManager( 138): Not granting permission android.permission.VIBRATE to package com.newsrob because it was previously installed without
W/PackageManager( 138): Not granting permission android.permission.WRITE_EXTERNAL_STORAGE to package com.newsrob because it was previously installed without
/data/system/packages.xml, which seems to record which applications are installed and which permissions they have, showed:
<package name="com.newsrob" codePath="/data/app/com.newsrob.apk" system="false" ts="1264200476000" version="353" userId="10039" installer="com.google.android.feedback">
<sigs count="1">
<cert index="25" key="..." />
</sigs>
<perms />
</package>
i.e. the permissions block was empty. It should have looked more like this:
<perms>
<item name="android.permission.VIBRATE" />
<item name="android.permission.WRITE_EXTERNAL_STORAGE" />
<item name="android.permission.ACCESS_NETWORK_STATE" />
<item name="android.permission.WAKE_LOCK" />
<item name="android.permission.INTERNET" />
</perms>
I tried manually hacking it, and also moving and replacing the .apk file on the phone, but packages.xml always returned to this state. Maybe it’s not the master copy of that data.
What finally fixed it for me was to re-install the applications using the package manager, by running:
cd /data/app
for app in *.apk; do pm install -r $app; done
I hadn’t known about the pm command until then, and discovered it by accident when invoking an adb command told me about it. The phone chugged along for quite a while, but eventually re-installed all of the applications, and the problem was fixed.
Web searches showed that I was not the only person to find themselves in this predicament, and did not reveal an obvious solution, so I’m documenting mine here.
Android emancipation
This afternoon, I decided to take the plunge and replace the OS on my G1. I had been having problems with the factory software for a while (including extreme slowness and a recurring crash in the calendar sync code) which finally outweighed the risk and inconvenience of flashing it. It looked like I was going to have to wipe it, and if I have to suffer that inconvenience anyway, I might as well go for broke.
“Rooting” the phone to enable the use of an alternative OS was very easy, thanks to the excellent Recovery Flasher application. I installed it by pointing the phone’s browser at the .apk download, though in retrospect it should be even easier to use adb. Recovery Flasher installs an alternative system recovery application, used when you hold down the magic buttons while the phone is starting up. The one which comes with the phone only allowed me to perform a few simple operations, while Recovery Flasher offers more features and doesn’t limit me to OS images blessed by Google.
Next, I downloaded the latest version of the highly recommended Cyanogen build. Thanks to Recovery Flasher, I didn’t even need to rename the file, as it lets me select any .zip file from the SD card to install. This is particularly convenient when experimenting with multiple OS images. On the first try, it failed to verify the .zip, so I copied it again, and it worked. I think the microSD card which was included with the phone is a bit flaky.
I decided to try without wiping my settings, to see if I could preserve them. The Cyanogen installation instructions warned that the first boot would take a long time, but after several minutes, I started to worry. adb logcat gave me some bad news:
I/SystemServer( 631): Starting Content Manager. I/SystemServer( 631): Starting System Content Providers. I/SystemServer( 631): Starting Battery Service. I/SystemServer( 631): Starting Hardware Service. W/HAL ( 631): load: module=/system/lib/hw/lights.trout.so error=Cannot find library W/HAL ( 631): load: module=/system/lib/hw/lights.trout.so error=Cannot find library E/ActivityThread( 631): Failed to find provider info for settings W/dalvikvm( 631): threadid=31: thread exiting with uncaught exception (group=0x40019a68) E/AndroidRuntime( 631): Uncaught handler: thread PowerManagerService exiting due to uncaught exception E/AndroidRuntime( 631): *** EXCEPTION IN SYSTEM PROCESS. System will crash. E/AndroidRuntime( 631): java.lang.NullPointerException E/AndroidRuntime( 631): at android.content.ContentQueryMap.(ContentQueryMap.java:65) E/AndroidRuntime( 631): at com.android.server.PowerManagerService.initInThread(PowerManagerService.java:414) E/AndroidRuntime( 631): at com.android.server.PowerManagerService$1.onLooperPrepared(PowerManagerService.java:374) E/AndroidRuntime( 631): at android.os.HandlerThread.run(HandlerThread.java:59) E/AndroidRuntime( 631): Crash logging skipped, no checkin service I/Process ( 631): Sending signal. PID: 631 SIG: 9
It was stuck in an infinite loop due to this crash. I searched Google for the error and found no help, and after some experimentation (including clearing out the dalvik-cache directory), I resigned myself to wiping my settings. I copied off the /data directory from the phone and rebooted, and the crash disappeared. Since I have a copy, I can restore the settings individually as I need them (and may be able to identify which one triggered the crash).
adb push makes it easy to copy files back onto the device from a backup. It was easy to restore my list of WiFi networks and keys: just restore /data/misc/wifi/wpa_supplicant.conf. It looks like Bluetooth pairings are stored in /data/misc/hcid, and I’ve copied that as well, though I haven’t tested whether it worked.
Restoring applications was also straightforward: adb push some.app.apk /data/app
I didn’t bother trying to figure out how to restore my Google account settings, Twidroid settings or Twitta settings. I just re-entered my account details. Twitta doesn’t seem to be working, though. It gets a timeout trying to contact Twitter and verify my credentials (but I can ping twitter.com fine).
The phone is much snappier now, though whether it’s due to the fresh start or the Cyanogen build, I’m not sure.
We'll see | Matt Zimmerman 