do Cocos2dxBitmap.java methods have to be thread safe ?

Hello,

The subject is a bit abrupt, here is the story: we all know CCLabelTTF creation/modification is slow, especially on Android (it’s even said in the documentation).

On Android, a big part of the problem is that the methods to do so allocate new objects everytime (I’d bet the Paint and byte[] are the main culprits), which results on the garbage collector being called a lot. When testing (with my app on a Nexus 7), a call to the garbage collector triples the time spent for CCLabelTTF::updateTexture(). On a label-heavy scene, the GC is called for more than half the labels, which results on a performance bottleneck.

Here is the question : when modifying Cocos2dxBitmap methods to have shared objects that are not reallocated at every call, is it safe to assume they are not going to be called concurrently ? This would save me quite a bit of hassle, since I can just put objects as static class objects and instantiate them on the first call.

P.S. : I am currently running cocos2dx-2.2.0, but I checked the develop branch, cocos2dx-3 still use the exact same code for Cocos2dxBitmap

My concern would be in the 4.x Android series. If you look at the bugs developers have filed against Google, you’ll find quiet a few concurrency issues withing GLSurfaceView and Bitmaps / Surfaces.

I would expect problems if these aren’t implemented in a thread safe fashion, but I’m just extrapolating. Testing under heavy load is probably the best way to find out how well it holds up.

The worst offender is 4.3.x for unexpected threads.

BTW, full disclosure — I quit using TTF labels because they’re so slow on Android. We moved to Bitmaps and are sooo happy.

I wish I could go back and replace my native Android TTF code with Bitmap fonts =)

Thanks for your answer.

Since then, I modified it to implement those optimizations and more, and the performance is great! I tested it on 2.3.7 (via a simulator), 4.3, 4.4 and had no issue at all.

Though you are probably right, it may be possible to create code that would break those optimizations (mine don’t, but I see how you could do that). The only thing I’m not sure about is if code breaking those optimizations (i.e. being called from another thread), would work. I have had problems with TTF labels not being created from main thread in the past.

In my case, moving to Bitmaps is not an option, since I have user-generated text and don’t want to restrict the character-set (loading time and memory are also issues if you have big character-set, that’s the reason I moved to TTF labels only in the first place, along with quality issues).

If someone is interested, I can post those optimizations. It’s on 2.2.0. I checked the develop branch, but unfortunately the code isn’t exactly the same as I thought, and I cannot test the code to be sure it won’t break, so I won’t do a pull request right now.

Hi Francois,

would love to know what optimizations you did inside Cocos2dxBitmap. even just some pointers without going too deep would be interesting. I have a feeling that performance issue like this will never get the attention of cocos2dx dev, so it’s pretty much on our own.

Mo

Hi mosonson,

Sorry, I don’t check cocos2d-x forum that often.

The main problem is that the GC is runned every single time you change a LabelTTF. On my test device, the GC takes twice as much time as the Label creation, which is huge.
I tried to limit the allocations, by making most of the methods variables static.

It was not enough, as Bitmap needs to be recreated every time because they have different size (unless you have 4.4, which have an optimization). The solution is to simply retain the Bitmaps and release them all at once. A subtlety is that you need to use SoftReference to avoid blowing up the available memory (the GC can release SoftReference when it really needs it).

Since most of the initialization of LabelTTF (in my case, may not adapt to your situation), I switch to an “init” mode which retains Bitmaps. When leaving the “init” mode, the Bitmaps are released. If you init all your labels when you created a scene, switch in “init” mode at the start of the scene creation, then leave the “init” mode at the next frame. That way, in most cases you’ll only have one GC run.

Here is the full modified file :

/****************************************************************************
Copyright (c) 2010-2011 cocos2d-x.org

http://www.cocos2d-x.org

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
 ****************************************************************************/
package org.cocos2dx.lib;

import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.ArrayList;
import java.util.LinkedList;

import android.content.Context;
import android.graphics.Bitmap;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.graphics.Paint.Align;
import android.graphics.Paint.FontMetricsInt;
import android.graphics.Rect;
import android.graphics.Typeface;
import android.text.TextPaint;
import android.text.TextUtils;
import java.lang.Math;
import java.lang.ref.SoftReference;

import android.util.Log;
import android.annotation.TargetApi;

public class Cocos2dxBitmap {
	// ===========================================================
	// Constants
	// ===========================================================

	/* The values are the same as cocos2dx/platform/CCImage.h. */
	private static final int HORIZONTALALIGN_LEFT = 1;
	private static final int HORIZONTALALIGN_RIGHT = 2;
	private static final int HORIZONTALALIGN_CENTER = 3;
	private static final int VERTICALALIGN_TOP = 1;
	private static final int VERTICALALIGN_BOTTOM = 2;
	private static final int VERTICALALIGN_CENTER = 3;

	// ===========================================================
	// Fields
	// ===========================================================

	private static Context sContext;
	
	//Avoid instantiation of re-used objects
	private static Paint paint = null;
	private static Canvas canvas = null;
	private static int pixelsSize = 0;
	private static byte[] pixels = null;
	private static ByteBuffer buf = null;
	private static FontMetricsInt fm = null;
	
	//Since bitmaps have to be recreated, keep them in memory to avoid GC runs during scene initialization
	private static boolean isInInitialization = true;
	private static ArrayList<SoftReference<Bitmap>> bitmaps = null;
	
	public static boolean isInInitialization()
	{
		return isInInitialization;
	}
	
	//Return true if some bitmaps were cleared, otherwise further trimming is necessary
	public static boolean forceClearBitmaps()
	{
		boolean notReleasedBitmap = false;
		if(isInInitialization && bitmaps != null && bitmaps.size() > 1)
		{
			for(int i = 0; i < bitmaps.size() - 1; i++)
			{
				//Since it's a soft reference, it may already be released
				if(bitmaps.get(i).get() != null)
				{
					notReleasedBitmap = true;
				}
			}
			//This is equal to a removeRange (protected method, can't be used)
			//Keep last bitmap as it may be still in use
			bitmaps.subList(0, bitmaps.size() - 1).clear();
		}
		return notReleasedBitmap;
	}
	
	public static void setIsInInitialization(boolean value)
	{
		if(value != isInInitialization)
		{
			//Remove bitmaps so that they can be garbage collected
			if(isInInitialization && bitmaps != null)
				bitmaps.clear();
			isInInitialization = value;
		}
	}

	// ===========================================================
	// Constructors
	// ===========================================================

	// ===========================================================
	// Getter & Setter
	// ===========================================================

	public static void setContext(final Context pContext) {
		Cocos2dxBitmap.sContext = pContext;
	}

	// ===========================================================
	// Methods for/from SuperClass/Interfaces
	// ===========================================================

	// ===========================================================
	// Methods
	// ===========================================================

	private static native void nativeInitBitmapDC(final int pWidth,
			final int pHeight, final byte[] pPixels);

	/**
	 * @param pWidth
	 *            the width to draw, it can be 0
	 * @param pHeight
	 *            the height to draw, it can be 0
	 */
	public static void createTextBitmap(String pString, final String pFontName,
			final int pFontSize, final int pAlignment, final int pWidth,
			final int pHeight) {
		
		//
		createTextBitmapShadowStroke( pString, pFontName, pFontSize, 1.0f, 1.0f, 1.0f,   	// text font and color
									  pAlignment, pWidth, pHeight,							// alignment and size
									  false, 0.0f, 0.0f, 0.0f,								// no shadow
									  false, 1.0f, 1.0f, 1.0f, 1.0f);						// no stroke
									 
	}
	
	@TargetApi(19)
	private static Bitmap createBitmap(TextProperty textProperty, float bitmapPaddingX, float bitmapPaddingY, int bitmapTotalHeight)
	{
		Bitmap bitmap = null;
		if(android.os.Build.VERSION.SDK_INT >= 19 && bitmaps != null && bitmaps.size() > 0 && bitmaps.get(bitmaps.size()-1).get() != null
				&& bitmaps.get(bitmaps.size()-1).get().getAllocationByteCount() > 
					(textProperty.mMaxWidth + (int)bitmapPaddingX) * (bitmapTotalHeight + (int)bitmapPaddingY) * 4) //The config ARGB_8888 stores each pixel on 4 bytes
		{ //try to reuse last bitmap (the larger one), there is a check on allocation size to avoid the exception
			try
		    {
		    	bitmap = bitmaps.get(bitmaps.size()-1).get();
		    	bitmap.reconfigure(textProperty.mMaxWidth + (int)bitmapPaddingX,
					bitmapTotalHeight + (int)bitmapPaddingY, Bitmap.Config.ARGB_8888);
		    	bitmap.eraseColor(Color.TRANSPARENT);
		    }
		    catch(Exception e)
		    {
				bitmap = Bitmap.createBitmap(textProperty.mMaxWidth + (int)bitmapPaddingX,
						bitmapTotalHeight + (int)bitmapPaddingY, Bitmap.Config.ARGB_8888);
		    }
		}
		else 
		{
			bitmap = Bitmap.createBitmap(textProperty.mMaxWidth + (int)bitmapPaddingX,
					bitmapTotalHeight + (int)bitmapPaddingY, Bitmap.Config.ARGB_8888);
		}
		return bitmap;
	}

	public static void createTextBitmapShadowStroke(String pString,  final String pFontName, final int pFontSize,
													final float fontTintR, final float fontTintG, final float fontTintB,
													final int pAlignment, final int pWidth, final int pHeight, final boolean shadow,
													final float shadowDX, final float shadowDY, final float shadowBlur, final boolean stroke,
													final float strokeR, final float strokeG, final float strokeB, final float strokeSize) {

		
		final int horizontalAlignment = pAlignment & 0x0F;
		final int verticalAlignment   = (pAlignment >> 4) & 0x0F;

		pString = Cocos2dxBitmap.refactorString(pString);
		Cocos2dxBitmap.newPaint(pFontName, pFontSize, horizontalAlignment);
		
		// set the paint color
		paint.setARGB(255, (int)(255.0 * fontTintR), (int)(255.0 * fontTintG), (int)(255.0 * fontTintB));

		fm = paint.getFontMetricsInt();
		
		final TextProperty textProperty = Cocos2dxBitmap.computeTextProperty(pString, pWidth, pHeight, paint);
		final int bitmapTotalHeight = (pHeight == 0 ? textProperty.mTotalHeight: pHeight);
		
		// padding needed when using shadows (not used otherwise)
		float bitmapPaddingX   = 0.0f;
		float bitmapPaddingY   = 0.0f;
		float renderTextDeltaX = 0.0f;
		float renderTextDeltaY = 0.0f;
		
		if ( shadow ) {

			int shadowColor = 0xff7d7d7d;
			paint.setShadowLayer(shadowBlur, shadowDX, shadowDY, shadowColor);
	
			bitmapPaddingX = Math.abs(shadowDX);
			bitmapPaddingY = Math.abs(shadowDY);
					
			if ( shadowDX < 0.0 )
			{
				renderTextDeltaX = bitmapPaddingX;
			}
			
			if ( shadowDY < 0.0 )
			{
				renderTextDeltaY = 	bitmapPaddingY;
			}
		}
		Bitmap bitmap = createBitmap(textProperty, bitmapPaddingX, bitmapPaddingY, bitmapTotalHeight);
		if(bitmaps == null)
		{
			bitmaps = new ArrayList<SoftReference<Bitmap>>();
		}
		if(isInInitialization && !bitmaps.contains(bitmap))
		{
			bitmaps.add(new SoftReference<Bitmap>(bitmap));
		}
		if(canvas == null)
		{
			canvas = new Canvas();
		}
		canvas.setBitmap(bitmap);

		/* Draw string. */
		int x = 0;
		int y = Cocos2dxBitmap.computeY(pHeight, textProperty.mTotalHeight, verticalAlignment);
		
		final String[] lines = textProperty.mLines;
		
		for (final String line : lines) {
			
			x = Cocos2dxBitmap.computeX(line, textProperty.mMaxWidth, horizontalAlignment);
			canvas.drawText(line, x + renderTextDeltaX, y + renderTextDeltaY, paint);
			y += textProperty.mHeightPerLine;
			
		}
		 
		// draw again with stroke on if needed 
		if ( stroke ) {
			
			Cocos2dxBitmap.newPaint(pFontName, pFontSize, horizontalAlignment);
			paint.setStyle(Paint.Style.STROKE);
			paint.setStrokeWidth(strokeSize * 0.5f);
			paint.setARGB(255, (int)strokeR * 255, (int)strokeG * 255, (int)strokeB * 255);
			
			x = 0;
			y = Cocos2dxBitmap.computeY(pHeight, textProperty.mTotalHeight, verticalAlignment);
			final String[] lines2 = textProperty.mLines;
			
			for (final String line : lines2) {
				
				x = Cocos2dxBitmap.computeX(line, textProperty.mMaxWidth, horizontalAlignment);
				canvas.drawText(line, x + renderTextDeltaX, y + renderTextDeltaY, paint);
				y += textProperty.mHeightPerLine;
				
			}
			
		}
		
		Cocos2dxBitmap.initNativeObject(bitmap);
	}

	private static void newPaint(final String pFontName, final int pFontSize,
			final int pHorizontalAlignment) {
		if(paint == null)
			paint = new Paint();
		
		paint.setColor(Color.WHITE);
		paint.setTextSize(pFontSize); 
		paint.setAntiAlias(true);

		/* Set type face for paint, now it support .ttf file. */
		if (pFontName.endsWith(".ttf")) {
			try {
				final Typeface typeFace = Cocos2dxTypefaces.get(
						Cocos2dxBitmap.sContext, pFontName);
				paint.setTypeface(typeFace);
			} catch (final Exception e) {
				Log.e("Cocos2dxBitmap", "error to create ttf type face: "
						+ pFontName);

				/* The file may not find, use system font. */
				paint.setTypeface(Typeface.create(pFontName, Typeface.NORMAL));
			}
		} else {
			paint.setTypeface(Typeface.create(pFontName, Typeface.NORMAL));
		}

		switch (pHorizontalAlignment) {
		case HORIZONTALALIGN_CENTER:
			paint.setTextAlign(Align.CENTER);
			break;
		case HORIZONTALALIGN_RIGHT:
			paint.setTextAlign(Align.RIGHT);
			break;
		case HORIZONTALALIGN_LEFT:
		default:
			paint.setTextAlign(Align.LEFT);
			break;
		}
	}
	
	private static TextProperty computeTextProperty(final String pString,
			final int pWidth, final int pHeight, final Paint pPaint) {
		final int h = (int) Math.ceil(fm.bottom - fm.top);
		int maxContentWidth = 0;

		final String[] lines = Cocos2dxBitmap.splitString(pString, pWidth,
				pHeight, pPaint);

		if (pWidth != 0) {
			maxContentWidth = pWidth;
		} else {
			/* Compute the max width. */
			int temp = 0;
			for (final String line : lines) {
				temp = (int) Math.ceil(pPaint.measureText(line, 0,
						line.length()));
				if (temp > maxContentWidth) {
					maxContentWidth = temp;
				}
			}
		}

		return new TextProperty(maxContentWidth, h, lines);
	}

	private static int computeX(final String pText, final int pMaxWidth,
			final int pHorizontalAlignment) {
		int ret = 0;

		switch (pHorizontalAlignment) {
		case HORIZONTALALIGN_CENTER:
			ret = pMaxWidth / 2;
			break;
		case HORIZONTALALIGN_RIGHT:
			ret = pMaxWidth;
			break;
		case HORIZONTALALIGN_LEFT:
		default:
			break;
		}

		return ret;
	}

	private static int computeY(final int pConstrainHeight, 
			final int pTotalHeight,
			final int pVerticalAlignment) {
		int y = -fm.top;

		if (pConstrainHeight > pTotalHeight) {
			switch (pVerticalAlignment) {
			case VERTICALALIGN_TOP:
				y = -fm.top;
				break;
			case VERTICALALIGN_CENTER:
				y = -fm.top + (pConstrainHeight - pTotalHeight)
						/ 2;
				break;
			case VERTICALALIGN_BOTTOM:
				y = -fm.top + (pConstrainHeight - pTotalHeight);
				break;
			default:
				break;
			}
		}

		return y;
	}

	/*
	 * If maxWidth or maxHeight is not 0, split the string to fix the maxWidth
	 * and maxHeight.
	 */
	private static String[] splitString(final String pString,
			final int pMaxWidth, final int pMaxHeight, final Paint pPaint) {
		final String[] lines = pString.split("\\n");
		String[] ret = null;
		final int heightPerLine = (int) Math.ceil(fm.bottom - fm.top);
		final int maxLines = pMaxHeight / heightPerLine;

		if (pMaxWidth != 0) {
			final LinkedList<String> strList = new LinkedList<String>();
			for (final String line : lines) {
				/*
				 * The width of line is exceed maxWidth, should divide it into
				 * two or more lines.
				 */
				final int lineWidth = (int) Math.ceil(pPaint
						.measureText(line));
				if (lineWidth > pMaxWidth) {
					strList.addAll(Cocos2dxBitmap.divideStringWithMaxWidth(
							line, pMaxWidth, pPaint));
				} else {
					strList.add(line);
				}

				/* Should not exceed the max height. */
				if (maxLines > 0 && strList.size() >= maxLines) {
					break;
				}
			}

			/* Remove exceeding lines. */
			if (maxLines > 0 && strList.size() > maxLines) {
				while (strList.size() > maxLines) {
					strList.removeLast();
				}
			}

			ret = new String[strList.size()];
			strList.toArray(ret);
		} else if (pMaxHeight != 0 && lines.length > maxLines) {
			/* Remove exceeding lines. */
			final LinkedList<String> strList = new LinkedList<String>();
			for (int i = 0; i < maxLines; i++) {
				strList.add(lines[i]);
			}
			ret = new String[strList.size()];
			strList.toArray(ret);
		} else {
			ret = lines;
		}

		return ret;
	}

	private static LinkedList<String> divideStringWithMaxWidth(
			final String pString, final int pMaxWidth, final Paint pPaint) {
		final int charLength = pString.length();
		int start = 0;
		int tempWidth = 0;
		final LinkedList<String> strList = new LinkedList<String>();

		/* Break a String into String[] by the width & should wrap the word. */
		for (int i = 1; i <= charLength; ++i) {
			tempWidth = (int) Math.ceil(pPaint.measureText(pString, start,
					i));
			if (tempWidth >= pMaxWidth) {
				final int lastIndexOfSpace = pString.substring(0, i)
						.lastIndexOf(" ");

				if (lastIndexOfSpace != -1 && lastIndexOfSpace > start) {
					/* Should wrap the word. */
					strList.add(pString.substring(start, lastIndexOfSpace));
					i = lastIndexOfSpace + 1; // skip space
				} else {
					/* Should not exceed the width. */
					if (tempWidth > pMaxWidth) {
						strList.add(pString.substring(start, i - 1));
						/* Compute from previous char. */
						--i;
					} else {
						strList.add(pString.substring(start, i));
					}
				}

				/* Remove spaces at the beginning of a new line. */
				while (i < charLength && pString.charAt(i) == ' ') {
					++i;
				}

				start = i;
			}
		}

		/* Add the last chars. */
		if (start < charLength) {
			strList.add(pString.substring(start));
		}

		return strList;
	}

	private static String refactorString(final String pString) {
		/* Avoid error when content is "". */
		if (pString.compareTo("") == 0) {
			return " ";
		}

		/*
		 * If the font of "\n" is "" or "\n", insert " " in front of it. For
		 * example: "\nabc" -> " \nabc" "\nabc\n\n" -> " \nabc\n \n".
		 */
		final StringBuilder strBuilder = new StringBuilder(pString);
		int start = 0;
		int index = strBuilder.indexOf("\n");
		while (index != -1) {
			if (index == 0 || strBuilder.charAt(index - 1) == '\n') {
				strBuilder.insert(start, " ");
				start = index + 2;
			} else {
				start = index + 1;
			}

			if (start > strBuilder.length() || index == strBuilder.length()) {
				break;
			}

			index = strBuilder.indexOf("\n", start);
		}

		return strBuilder.toString();
	}

	private static void initNativeObject(final Bitmap pBitmap) {
		final byte[] pixels = Cocos2dxBitmap.getPixels(pBitmap);
		if (pixels == null) {
			return;
		}

		Cocos2dxBitmap.nativeInitBitmapDC(pBitmap.getWidth(),
				pBitmap.getHeight(), pixels);
	}

	private static byte[] getPixels(final Bitmap pBitmap) {
		if (pBitmap != null) {
			if(pixelsSize < pBitmap.getWidth()* pBitmap.getHeight() * 4)
			{
				pixelsSize = pBitmap.getWidth()* pBitmap.getHeight() * 4;
				pixels = new byte[pixelsSize];
				buf = ByteBuffer.wrap(pixels);
				buf.order(ByteOrder.nativeOrder());
			}
			else
			{
				buf.rewind();
			}
			pBitmap.copyPixelsToBuffer(buf);
			return pixels;
		}

		return null;
	}

	private static int getFontSizeAccordingHeight(int height) {
		Paint paint = new Paint();
		Rect bounds = new Rect();

		paint.setTypeface(Typeface.DEFAULT);
		int incr_text_size = 1;
		boolean found_desired_size = false;

		while (!found_desired_size) {

			paint.setTextSize(incr_text_size);
			String text = "SghMNy";
			paint.getTextBounds(text, 0, text.length(), bounds);

			incr_text_size++;

			if (height - bounds.height() <= 2) {
				found_desired_size = true;
			}
			Log.d("font size", "incr size:" + incr_text_size);
		}
		return incr_text_size;
	}

	private static String getStringWithEllipsis(String pString, float width,
			float fontSize) {
		if (TextUtils.isEmpty(pString)) {
			return "";
		}

		TextPaint paint = new TextPaint();
		paint.setTypeface(Typeface.DEFAULT);
		paint.setTextSize(fontSize);

		return TextUtils.ellipsize(pString, paint, width,
				TextUtils.TruncateAt.END).toString();
	}

	// ===========================================================
	// Inner and Anonymous Classes
	// ===========================================================

	private static class TextProperty {
		/** The max width of lines. */
		private final int mMaxWidth;
		/** The height of all lines. */
		private final int mTotalHeight;
		private final int mHeightPerLine;
		private final String[] mLines;

		TextProperty(final int pMaxWidth, final int pHeightPerLine,
				final String[] pLines) {
			this.mMaxWidth = pMaxWidth;
			this.mHeightPerLine = pHeightPerLine;
			this.mTotalHeight = pHeightPerLine * pLines.length;
			this.mLines = pLines;
		}
	}
}