A very long story about my optimization of Cocos2d-x Android apps in size and performance

The back story:

As I wrote before in the Chirp Chirp topic, the situation with internal memory on Android is bad. On most devices, it’s not bad, it’s just VERY bad.

The HTC desire is a very good example here. It contains 512 MB of Read-Only Memory (ROM) and 576 MB of Random Access Memory (RAM). Even though ROM means “read only”, it is where the Android OS is installed. It’s also the location where the user interface is installed, and the apps.

Getting rid of Android to increase the available memory for new apps is obviously not an option. The user interface (the UI on top of Android, like HTC Sense) can not be fully removed without rooting the device. So that leaves a lot less memory available of those 512 MB. But hold on, it gets even worse: very useful apps like GMail (4.46 MB), Google Maps (11.30 MB), Android Market (6.05 MB) and the Youtube app (1.70 MB) are all installed right there. They cannot be moved to the SD card, so that’s more memory that’s not available for additional apps, like Cocos2d-x games.

In the end it means that on the HTC Desire, there’s about 50 MB available on average. But there’s more: it’s not possible to use all of that, because it must be shared with stuff like personal data (sms, email, etc) and most importantly, at least 30 MB of free internal memory is ALWAYS required when the vendor comes up with a ROM upgrade to install a more recent or otherwise updated version of Android.

So, on the HTC Desire, there’s only an average of 20 MB available for actual user apps. I really hope this speech gets the point accross, because, well, 20 MB is not much :slight_smile: My point is: if you’re a developer looking into developing apps for Android, you’ll have to limit the installed file size of your apps, because otherwise the user will simply not install it, or they’ll remove your app much faster when they need space to install new apps, making you less money when you’re dependent on advertising, which seems to be one of the few methods to actually make some money on Android.

But… then there’s Cocos2d-x. Cocos2d-x is an awesome library. No doubt about that. But on my system, built using NDK-r4, it’s about 2.47 MB for just cocos2d.so alone. That’s without CocosDenshion, Chipmunk or Box2D!

Finding a way out:

Many people would just say: hey, just enable the “Move to SD” option and let the user install your app onto the SD card. Well, you could. But it’s only useful for your assets (images, sounds, etc). The actual app will not shrink in size, so when I made a nice little game using Cocos2d-x, I found it to be at least about 4 MB in size. That’s without the optimized ARMv7 code. With it, the size would literally double up, even though it’s an absolute requirement for some apps/games because floating point calculations will be MUCH faster with it. Box2D, for example, will get a huge speedup, which you cannot ignore as a developer looking into bringing high performance apps to the user. Fortunately, I didn’t use Box2D for that nice little game.

The biggest problem with “Move to SD” is that each version of Android up to 2.2 will install native libraries (like the Cocos2d-x libraries) into /data/data/appname/lib/libxxx.so , so that’s why the “Move to SD” option is useless: the native libraries will stay on the internal memory, not the SD card. Only with Android 2.3 and up, native libraries can be installed to the SD card, according to the following url (as told by Dianne Hackborn, who is an Android framework engineer): http://osdir.com/ml/android-ndk/2011-04/msg00415.html

Is there a way out of distributing Cocos2d-x apps for Android without taking away at least 4 MB of internal memory or more, from the limited amount of ~20 MB user app memory? Fortunately, yes, there is :slight_smile:

The solutions:

  • Move to SD

As mentioned before, this is only part of the solution as it will only install java code and assets to the SD card, leaving the rest of the (native) code inside the internal memory. Yes, it should be enabled, but no, it’s not the solution for fixing the issues with limited internal memory on most Android devices and making your users happier with your small sized apps.

  • “APP_OPTIM := release” @ Application.mk

Now we’re getting somewhere. With this option put into Application.mk, your native libraries will lose debugging symbols and all that, but it will be much smaller. cocos2d.so (2.47 MB when built straight from Git) turns into a smaller library of 1.70 MB with this option enabled.

  • Turning Cocos2d-x into a static library

This doesn’t have as much of an effect as the APP_OPTIM option, but it’s good for another 0.20 - 0.30 MB off the library size. That’s almost 1/3 of one Megabyte indeed. You can make Cocos2d-x go static by performing the following rituals:

# remove ‘cocos2d’ and other modules that should be static from Application.mk
# in Android.mk, add LOCAL_STATIC_LIBRARIES := libcocos2d (just before the include $(BUILD_SHARED_LIBRARY))
# change $(BUILD_SHARED_LIBRARY) to $(BUILD_STATIC_LIBRARY)
# in the Tests or Helloworld demo, or your own app, you should link with the following libraries:
libz, libpng, libxml2, libjpeg, libskia

Those additional libraries you’ll have to link to are normally linked into the .so file, but that won’t happen with a static library. Keep in mind that you’ll probably have to swizzle the inclusion order of these libraries to get rid of linkage errors.

The result for all of this is a static Cocos2d-x library that can be linked into your own app’s native shared library, so you’ll end up with just one library that needs to be loaded from the Java side of the code.

  • GCC Visibility feature

Last, but most definitely not least, is the GCC visibility attribute. It’s a bit like __declspec for MSVC, but for GCC instead. Using this feature you can improve loading times for your native libraries and cut down on the exported symbol table, which can get as large as 5-6 MB when there’s 30,000 symbols. The compiler can also optimize your code a little better when it’s enabled.

Here’s some more information on how to use the feature, it’s quite easy:
http://gcc.gnu.org/wiki/Visibility
http://goldenhammersoftware.blogspot.com/2010/12/android-ndk-and-storage-size.html

(the person from the last article talks about stripping away debugging information, but as far as I know that’s not necessary once you’re using the release optimization in Application.mk)

The conclusion:

With all of these options it’s easily possible to make a Cocos2d-x library about 2.3 times smaller compared to what you’d get from Git. I don’t know if the 3rd party libs (the binary versions distributed with Cocos2d-x) were compiled without these options, but if they were, the results could be even better.

So here’s the part where I started testing these wonderful optimizations. I modified the makefiles for my app, included the static Cocos2d-x libraries, all combined into one shared object, and there I was: expecting to reap the huge benefits…

…well, no. Far from it. The actual APK file (the one you’d offer for downloading on the Market) had decreased in file size with about 1/3 less of the original size. That’s great, but my initial goal was to make the app use less internal memory on the actual device, in which I did not succeed :slight_smile: Why? Dunno! The installation size (and I did move it to the SD card) was actually just as big as before without all the optimizations!

So I guess we’ll have to wait for Android 2.3, which does install native libraries to the SD card.

Thank you for reading this topic, and please, please do not spend six hours on it like I did :wink: But if you do, please do post your results. Maybe I didn’t do things the right way? Who knows…

ps. The APP_OPTIM option is quite good though. You (the Cocos2d-x team) might want to consider adding it to Git. It does a nice job on cutting at least part of the fat away!

5 Likes

Thanks for your share!
Maybe we must consider to change .so to .a static library as default, and add APP_OPTIM to Application.mk

1 Like

The static library approach might be bad for installation size (yeah, I’m really worried about that :stuck_out_tongue: ), but APP_OPTIM is awesome :slight_smile:

Also, when I tried the GCC visibility feature, I only tried to hide everything with the exception of all the JNI C-functions because those functions should be visible to Java. What I didn’t try, however, is to use the visibility attribute like __declspec as in the example on the GCC wiki page. It might do less in terms of decreasing the file size without turning everything into one big happy static library, but it also might decrease the size just enough to make things a little better for the shared library approach while keeping the install size as small as possible.

More!

All righty, I went ever deeper to figure out how to make cocos2d.so smaller in size :slight_smile:
The following libraries are used by Cocos2d-x: libxml2, libpng, libz, libjpeg and libskia.

Skia

Skia isn’t really big in what it’s used for with Cocos2d-x, it compiles to only about 0,01 Megabyte. Not worth the time to optimize. Leave it alone. It generates nice and sexy fonts.

JPEG library

This is an interesting one. libjpeg is quite large, so it’s a great candidate to look at when optimizing for installation size. Also, there’s the knowledge of image quality. JPEG images are great for saving and loading photorealistic images, but it’s horrible for cartoony images, images with gradients, clean images without much detail or few colors, and so on.

Now, most Cocos2d-x games aren’t using any real photo’s or photorealistic images at all, so as a game developer you’re most probably using Portable Network Graphics because this format supports images without any compression artifacts.

I know I will never use JPEG for an app, so for me, it’s a great choice to disable loading of JPEG images altogether. Fortunately, that’s very easy to do with Cocos2d-x, just a few lines in the texture classes. Add some comments and return false or null and you’re done.

The results: well, cutting libjpeg away makes cocos2d.so go from 1,70 MB (with ‘APP_OPTIM := release’ only) to 1,60 MB. Not even that much, but still, well worth the very small effort if you aren’t using the functionality anyway.

Z library

This one is needed for libpng. Don’t remove.

PNG library

Again, this library is needed if you’d like to load images at all. Can’t be removed.

XML library

Ahh, this is a great one. libxml2 from the Cocos2d-x distribution is about 3,02 MB in size, which is… well, really big. If you’re not afraid to do some dirty work by downloading libxml2, spending some time to dig out a bunch of Android.mk magic (use the Android OS version as an example) and carefully craft a well fitting xmlversion.h to disable support for stuff like DOM tree manipulation (DOM is used by Cocos2d-x, but the manip functions aren’t), push parsing interfaces, xmlPattern, ftp/http, validation, legacy stuff, canonicalization, catalogs, docbook, xpath, xpointer, xinclude, iconv, icu, debugging, memory debugging, runtime debugging, unicode (not sure if that’s good for China, Japan, etc. Probably not, need to experiment with it), regexp, automata, formal expressions, schemas, schematron, modules and zlib support. Well, if you disable all of that, you’ll end up with a static libxml2 library that’s about 2,32 MB.

Together with the removal of libjpeg, a cocos2d.so linked with this optimized libxml2.lib comes down to about 1,53 MB in size.

It would be possible to cut another 0,01 MB away by disabling the SAX1 functions for libxml, but Cocos2d-x is using the xmlSAXUserParseMemory function so that’s not possible (even though Cocos2d-x seems to do SAX2 instead of SAX1). xmlCreateMemoryParserCtxt is probably the function to use here (as xmlSAXUserParseMemory does that as well), but like I said before… it’s not worth the time to optimize for only 0,01 MB.

The end (?)

As it is now, my local version of cocos2d.so went down from 2,47 MB to 1,53 MB.

I’m really happy, because the APK for my little game also went down to 0,9 MB (libraries are zipped into the APK), meaning that the physiological barrier to download my game on a capped connection to the web just went down as well.

More users, more happy users because of the smaller installation size = awesome!

I hope this information will be useful to other developers!

3 Likes

Thanks for your awesome work! Finally you success on optimize it!
I modify the title a little, and add your article into sticky FAQ in the forum http://www.cocos2d-x.org/boards/6/topics/567 It will be VERY helpful to other developers!

Excellent post!
Here the real link for Dianne Hackborn’s post: http://groups.google.com/group/android-ndk/browse_thread/thread/94986c853be60709/5bf27165b1482237

Thanks for taking the time to write this.

very very useful;, thanks!

Thanks :slight_smile:

And oh, silly me… I forgot about the most obvious optimization of all :smiley:

I used to use the Crystax version of the NDK (like said, r4). However, it does come with a slight overhead. Now if you’re using Crystax as well, please consider moving to the official NDK. I just did that, and what do you know… the size of cocos2d.so went down even more, to a nice size of about 0,99 MB (!).

With this new smaller cocos2d.so, the size of my little app’s APK is… 0,75 MB. I guess I’m done with it now :slight_smile:

I`ve got some problem turning cocos2d-x into a static library as described above.
My test app builds and works well until I try to use some “bridge” features (touchscreen or keypad for example). In this case the app crashes due to “No implementation found for native”.
Something like that:

01-26 18:26:20.220: W/dalvikvm(22860): No implementation found for native Lorg/cocos2dx/lib/Cocos2dxRenderer;.nativeTouchesBegin (IFF)V
01-26 18:26:20.260: W/dalvikvm(22860): threadid=11: thread exiting with uncaught exception (group=0x4001d648)
01-26 18:26:20.260: E/AndroidRuntime(22860): FATAL EXCEPTION: GLThread 12
01-26 18:26:20.260: E/AndroidRuntime(22860): java.lang.UnsatisfiedLinkError: nativeTouchesBegin
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.nativeTouchesBegin(Native Method)
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.handleActionDown(Cocos2dxRenderer.java:74)
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxGLSurfaceView$7.run(Cocos2dxGLSurfaceView.java:302)

Any suggestions?

P.S. I`m really sorry for my English (

Dan Zurz wrote:

I`ve got some problem turning cocos2d-x into a static library as described above.
My test app builds and works well until I try to use some “bridge” features (touchscreen or keypad for example). In this case the app crashes due to “No implementation found for native”.
Something like that:
>
01-26 18:26:20.220: W/dalvikvm(22860): No implementation found for native Lorg/cocos2dx/lib/Cocos2dxRenderer;.nativeTouchesBegin (IFF)V
01-26 18:26:20.260: W/dalvikvm(22860): threadid=11: thread exiting with uncaught exception (group=0x4001d648)
01-26 18:26:20.260: E/AndroidRuntime(22860): FATAL EXCEPTION: GLThread 12
01-26 18:26:20.260: E/AndroidRuntime(22860): java.lang.UnsatisfiedLinkError: nativeTouchesBegin
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.nativeTouchesBegin(Native Method)
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.handleActionDown(Cocos2dxRenderer.java:74)
01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxGLSurfaceView$7.run(Cocos2dxGLSurfaceView.java:302)
>
Any suggestions?
>
P.S. I`m really sorry for my English (

I got this problem too, have you solved it?
and could you tell me how to solve it,thanks.

Michelle Sangster wrote:

Dan Zurz wrote:
> I`ve got some problem turning cocos2d-x into a static library as described above.
> My test app builds and works well until I try to use some “bridge” features (touchscreen or keypad for example). In this case the app crashes due to “No implementation found for native”.
> Something like that:
>
> 01-26 18:26:20.220: W/dalvikvm(22860): No implementation found for native Lorg/cocos2dx/lib/Cocos2dxRenderer;.nativeTouchesBegin (IFF)V
> 01-26 18:26:20.260: W/dalvikvm(22860): threadid=11: thread exiting with uncaught exception (group=0x4001d648)
> 01-26 18:26:20.260: E/AndroidRuntime(22860): FATAL EXCEPTION: GLThread 12
> 01-26 18:26:20.260: E/AndroidRuntime(22860): java.lang.UnsatisfiedLinkError: nativeTouchesBegin
> 01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.nativeTouchesBegin(Native Method)
> 01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxRenderer.handleActionDown(Cocos2dxRenderer.java:74)
> 01-26 18:26:20.260: E/AndroidRuntime(22860): at org.cocos2dx.lib.Cocos2dxGLSurfaceView$7.run(Cocos2dxGLSurfaceView.java:302)
>
> Any suggestions?
>
> P.S. I`m really sorry for my English (
>
I got this problem too, have you solved it?
and could you tell me how to solve it,thanks.

I got it solved.
It is because the native fuction you defined is call in the java side and because it is not accessed in the c*+ side so the compiler won’t link it to the so.
you should pretend to access them at some place of the c*+ side to force it link to so.

You can link the .a by using LOCAL_WHOLE_STATIC_LIBRARIES in Android.mk, then it will not remove the functions that are not used in c++ codes.

very very useful

Hi,
Thank for great info,can you please write code for disable libs.
What things need to write in Android.mk to make it work.
Because now iPhone app size is 60 MB and on Android apk is 107 MB.
If there is any other ways are then please tell so that we can also decrease the size of apk.

Thank you.

Hi,

I want to achieve the same success story, but I’m not sure what files are to be changed to remove the undesired libraries. I tried to comment out their references on few .cpp and .mk files, but got too many errors. So any help would be very much appreciated.

Thanks!

Can you post example?

Nice post…

I really appreciate all your posts, thanks MAN, You’re AMAZING!!

Great post!

Memory size largely depends on your texture, I’ll look into them if I want to reduce the memory usage of my game. Things like switching around texture format, reduce quality ETC.

On this page: https://developer.android.com/topic/performance/reduce-apk-size.html
they talk about reducing the apk size by avoiding to extract native libraries.
Apparently this could be done by setting the android:extractNativeLibs flag to false in the manifest flag.

Has anyone experienced this?
Do you know about possible issues when using this flag?

I am also interested in knowing the same. :slight_smile: