Dealing with Asset Compression in Android Apps

When developing an Android app, any data file, image or XML file (that is, any Resource or Asset) you use is bundled into your application package (APK) for distribution. The Android Asset Packaging Tool, or aapt, is responsible for creating this bundle, which you can think of as a ZIP file with a particular layout that the Android OS can understand. When your application is installed, whether in development mode or by an end user, this APK file is simply dropped into a special location on the device’s (or emulator’s) filesystem.

As part of preparing your APK, aapt selectively compresses various assets to save space on the device. The way aapt determines which assets need compression is by their file extension. The following code, taken from Package.cpp in the aapt source code, sheds some light on which types of files are not compressed by default:

/* these formats are already compressed, or don't compress well */
static const char* kNoCompressExt[] = {
    ".jpg", ".jpeg", ".png", ".gif",
    ".wav", ".mp2", ".mp3", ".ogg", ".aac",
    ".mpg", ".mpeg", ".mid", ".midi", ".smf", ".jet",
    ".rtttl", ".imy", ".xmf", ".mp4", ".m4a",
    ".m4v", ".3gp", ".3gpp", ".3g2", ".3gpp2",
    ".amr", ".awb", ".wma", ".wmv"
};

The only way (that I’ve discovered, as of this writing) to control this behavior is by using the -0 (zero) flag to aapt on the command line. This flag, passed without any accompanying argument, will tell aapt to disable compression for all types of assets. Typically you will not want to use this exact option, because for most assets, compression is a desirable thing. Sometimes, however, you will have a specific type of asset (say, a database), that you do not want to apply compression to. In this case, you have two options.

First, you can give your asset file an extension in the list above. While this does not necessarily make sense, it can be an easy workaround if you don’t want to deal with aapt on the command line. The other option is to pass a specific extension to the -0 flag, such as -0 db, to disable compression for assets with that extension. You can pass the -0 flag multiple times, and each time with a separate extension, if you need more than one type to be uncompressed.

Currently, there is no way to pass these extra flags to aapt when using the ADT within Eclipse, so if you don’t want to sacrifice the ease of use of the GUI tools, you will have to go with the first option, and rename your file’s extension.

So, Why Disable Compression Anyway?

For most types of assets, you shouldn’t need to be concerned with how they are packaged. Given the list of extensions above, Android will do the right thing for a large subset of the files you’ll use. If you have a type of asset that is already compressed by nature of the file type, but doesn’t have one of the sanctioned extensions, you can typically ignore that inconsistency and just allow aapt to try to compress it. It will not be very much, if at all, smaller in the final APK, but the performance hit should be relatively minimal unless you just have tons of these files.

This begs the question, why even bother disabling compression? The not-so-obvious answer is that prior to Android 2.3, any compressed asset file with an uncompressed size of over 1 MB cannot be read from the APK. This could come into play if your asset needed to be copied out of the APK and into the app’s writable files area, for example to provide a pre-populated database in your app. When you try to use the various AssetManager or Resources classes’ methods to get an InputStream to the file, it will throw an exception and display a LogCat message like this:

DEBUG/asset(725): Data exceeds UNCOMPRESS_DATA_MAX (1662976 vs 1048576)

The only way around this type of issue is to disable compression for assets that exceed the 1 MB limit. If your file format supports it, you can choose to split the asset into several smaller files that have an uncompressed size of less than 1 MB each, but this can be a real annoyance.

Hopefully the knowledge in this post can help you avoid the headaches I went through to learn it!

Update March 1, 2011: The limit on the uncompressed size of compressed assets was removed in Android 2.3. So someday in the future, when you don’t have to worry about Android versions lower than 2.3, you can avoid this heartache.

This entry was posted in android. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

10 Comments

  1. Dmitry
    Posted May 21, 2010 at 11:29 am | Permalink

    hello!
    very usefull information, but there is one thing i can’t understand: what gives asset compression, it’s seems bad through article, does “assets” files becomes unreadeble via getAssets().open(“read_asset.txt”); and others way of handling?

  2. Dmitry
    Posted May 21, 2010 at 11:42 am | Permalink

    sorry, don’t just read =)

  3. Posted May 21, 2010 at 11:49 am | Permalink

    Hi Dmitry. If I understand your question correctly, you’re asking about why you would need to bypass compression.

    For most files it shouldn’t matter if they are compressed, but files larger than a hard-set operating system limit (1MB) can’t be uncompressed by the AssetManager. Thus, for such assets you would have to prevent them from being compressed.

    Hope this helps clarify.

  4. csm101
    Posted June 24, 2010 at 12:58 pm | Permalink

    I have an Ant build script for my Android project. I’m using aapt to package various resource files. I want the resources files to be uncompressed. I ‘ve tried the ‘-0′ flag but the files still get compressed! But when I build through Eclipse, the resource files are uncompressed. Why doesn’t this flag work?

    Here is the Ant target that uses aapt:

    Packaging resources and assets…

  5. Posted June 24, 2010 at 8:18 pm | Permalink

    It looks as though some of your comment was eaten by WordPress. Can you post the relevant pieces again?

  6. jscoobysnack
    Posted November 3, 2010 at 4:04 pm | Permalink

    Hi Brian,
    I am having the same issue of assets still being compressed. I added this to my Build.xml before the opening “setup task” tag.

    Generating R.java / Manifest.java from the resources Joe style…

    The comment makes me think this only affects Resources and not files in the Assets folder. I tried to add the arg -0 to this section:

    Packaging resources

    but I get errors that aaptexec does not support args. Any ideas?

  7. jscoobysnack
    Posted November 3, 2010 at 4:24 pm | Permalink

    Sorry. Here is a link to the code:

    I added this to my Build.xml before the opening “setup task” tag.
    http://warpedimage.com/codesample/code.txt

    The comment makes me think this only affects Resources and not files in the Assets folder. I tried to add the arg -0 to this section:
    http://warpedimage.com/codesample/code2.txt

  8. tomgamer
    Posted February 28, 2011 at 6:26 pm | Permalink

    This seemed to work for me. On my Moto CLIQ and my (and wifes) Motorola Droid 2 and several other phones, along with the emulators, this seemed to work just fine. But when I published, the complaints came streaming in.
    Debugging on a friend LG Ally (android 2.1), I found that the inputstream.read was throwing an exception.
    Maybe it was my code? Not sure, since it did work fine on some phones. But reducing the file size below the limit resolved the problem with the LG Ally.

  9. Posted March 1, 2011 at 10:40 am | Permalink

    @tomgamer, I just went and verified the current status of this issue, and made a small update to the post. It appears that this limitation on compressed asset size was removed in Android 2.3.

  10. Shibbs
    Posted November 15, 2011 at 4:11 pm | Permalink

    Thanks Brian – We were shipping 6 MB of DB & some device were able to show the contents but some where not. This post really helped us in figuring out what was the real problem. We went with the option-1 that you mentioned & renamed our db as ‘bible.db.png’ & it worked. Just that the apk bloated back from 2.3 to close 6 MB, but I guess we can take that trade off.
    Kudos!

9 Trackbacks

  • By Error when copying to File.DirDefaultExternal on January 23, 2011 at 1:43 pm

    [...] at this link for the reason for the error Dealing with Asset Compression in Android Apps. "any compressed asset file with an uncompressed size of over 1 MB cannot be read from the [...]

  • [...] 中文社区中对此有这样的一篇广为转载的文章,《Asset限制文件大小UNCOMPRESS_DATA_MAX为1MB》,其中的结论是: 目前解决的方法只有将文件放入到sdcard,但这样除了microsd的读取IO效率和耗电量解决不是很好,同时sd卡移除时可能存在问题,解决的方法如果数据量大的化通过sqlite是一种解决方法,同时openFileOutput方式读取也是不错的选择,看来android操作系统中最安全控制的严格,但犯了一个不小的错误。 字体文件显然没办法塞到sqlite里面,要求所有的用户都有SD卡也不太友好。又翻到一篇相关的英文文章,Dealing with Asset Compression in Android Apps。从中可以看出,Android 2.3之前的版本只对压缩过的资源文件的原始大小做了限制,这种限制可能是出于计算性能的考虑。对于没有压缩的文件,是没有大小限制的。哪些文件不会压缩?图片、音频、视频,包括这些后缀: “.jpg”, “.jpeg”, “.png”, “.gif”, “.wav”, “.mp2″, “.mp3″, “.ogg”, “.aac”, “.mpg”, “.mpeg”, “.mid”, “.midi”, “.smf”, “.jet”, “.rtttl”, “.imy”, “.xmf”, “.mp4″, “.m4a”, “.m4v”, “.3gp”, “.3gpp”, “.3g2″, “.3gpp2″, “.amr”, “.awb”, “.wma”, “.wmv” 从Android的源代码中可以知道,Android的打包工具是通过文件的后缀名来判断文件是否需要压缩的。如果文件的后缀名是上面这些后缀,文件就不会被压缩,大小也就没有限制;如果文件的后缀不是上面这些,就对其进行压缩,并且在Android 2.2上有1MB的大小限制。字体文件后缀通常是.ttf,属于要被压缩的,在Android 2.2之前的版本中就有了体积限制。不过,看到这里,解决方案也出来了,把字体文件的.ttf后缀改成上述任意一个后缀,然后修改相应的代码就可以。好猥琐的解决方案啊~~ [...]

  • By Shipping a Database with your App | on October 23, 2011 at 4:40 pm

    [...] need to change its file extension (for example to .png). See this post for a detailed explanation: http://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps/ . Which again, increases your APK size even [...]

  • [...] | Ponystyle Industries. Entrada publicada en la categoría Programación, con las etiquetas android, assets, [...]

  • By Android, SQLite and a few Problems | banani.de on March 30, 2012 at 1:07 pm

    [...] prepopulated database in your asset folder is larger than 1MB. The whole topic is described in this post. The easiest solution to this problem is to rename your database’s extension to an extension [...]

  • By Android pre-populated Database | cmanios on May 14, 2012 at 2:12 am

    [...] Every file that is stored in Assets directory is compressed. Prior to Android 2.3, any compressed asset file with an uncompressed size of over 1 MB cannot be read from the APK. The limit on the uncompressed size of compressed assets was removed in Android 2.3. So someday in the future, when you don’t have to worry about Android versions lower than 2.3, you can avoid this heartache. (source: Brian Hardy-Dealing with Asset Compression in Android Apps [...]

  • [...] take a look at this blog post by Brian Hardy. All “compressible files” are compressed even xml files. Xml [...]

  • [...] by renaming the file extension to one which is recognised as "already" compressed e.g. ".jpg" (see http://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps/ for more details) else you get exceptions like [...]

Post a Comment

You must be logged in to post a comment.