Pixel perfect touch detection

cocos2d-x C++
#1

Hello community!

I’m trying to do a pixel detection function, after failing with a pixel perfect collision detection with masks. I thought it would be easier to handle one point but im also stuck.

I am creating a RenderFrame, all black and placing a sprite on it. Then im trying to read the color of the point I’m touching, and if it’s not black, it means I’ve touched my sprite. Everything seems perfect but I am getting different color data, I mean, it’s not black nor mySprite’s color. I don’t really know what is happening but I am pretty sure that the buffer is loading information from somewhere else than the RenderFrame. Here goes the code:

bool CollisionDetection::isTheSpriteBeingTouched(Sprite* sprite1,
                                          Touch* touch,
                                            bool pp,
                                              RenderTexture* rt)
{
   
   auto size = Director::getInstance()->getWinSize();
   
   rt->beginWithClear( 0, 0, 0, 0);
   
   sprite1->visit();
   
   auto position = touch->getLocation();
   auto x = position.x * (CC_CONTENT_SCALE_FACTOR());
  auto y = position.y * (CC_CONTENT_SCALE_FACTOR());
   

   auto buffer = (Color4B *)malloc(sizeof(Color4B));
   
   // Get color values of intersection area
   glReadPixels(x, y, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
   
   rt->end();

   // Read buffer
   for(unsigned int i = 0; i < 1; i++) {
      CCLOG("the color values: %d, %d, %d and the a: %d ",buffer->r,buffer->g,buffer->b, buffer->a);
      // Here we check if a single pixel has both RED and BLUE pixels
      if (buffer->r > 0 || buffer->g > 0 || buffer->b > 0) {
         CCLOG("FOUND");
      }else{
         CCLOG("NOT FOUND");
      }
   }
   
   free(buffer);
   
   return false;
}

And here is how I declare my RenderTexture:

collisionDetectionRenderTexture_ =
   RenderTexture::create(winSize.width, winSize.height, Texture2D::PixelFormat::RGBA4444);

   collisionDetectionRenderTexture_->setVisible(true);
   collisionDetectionRenderTexture_->ignoreAnchorPointForPosition(false);
   this->addChild(collisionDetectionRenderTexture_, 5000);
   collisionDetectionRenderTexture_->setPosition(Point(winSize.width * .5f, winSize.height * .5f));

Any help is highly appreciated!

#2

RenderTexture just send commands to render queue, which means it is not rendered into frame buffer when you invoke glReadBuffer().

#3

@zhangxm Hello, thanks for the fast reply! But I am not invoking glReadBuffer() so I dont know what you mean. :;qst

#4

@lucas0 - Google is your friend here: https://www.opengl.org/sdk/docs/man2/xhtml/glReadBuffer.xml

@zhangxm has given you a function to take advantage of.

#5

It is the same you use glReadPixels().

#6

Hello~

I think this is whats going on

  • if you are using Cocos2d-x 3.x, Rendertexture no longer binds the framebuffer immediately, so your [quote=“lucas0, post:1, topic:14712”]
    glReadPixels(x, y, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
    [/quote]
    will reads the current frame buffer (the screen), and you will get a different color

what you could do is

  1. make a node, implement a “draw” function, the draw function should does nothing other than glreadpixel and returns the value. you then visit this node inside the rt->begin() and rt->end()

  2. you could modify the RenderTexture class and change onbegin() onEnd() from protected to public. This allows you to manually call those function which will immediately change the framebuffer to the rendertexture, allows your glreadpixel to read from the correct buffer. the drawback is you can not do normal sprite->visit() inside this

Synchronous glReadPixel in cocos2d-x-v3
#7

Thank you for the answers. I’ve moved away from this question by now. Im doing bounding box recognition and its pretty ok. But the first alternative you gave seems interesting. Actually, with the new updates, a solution using shaders will be my first try when I come back to this problem. Thanks again!

#8

@Wuhao, can you explain further how to do option 1? In Cocos v2.2 I used glReadPixels to calculate the transparency of a RenderTexure with the following code:

float TWRenderTexture::getPercentageTransparent()
{
if ( _pixelFormat != Texture2D::PixelFormat::RGBA8888 ) return 0.0;

Size s = _texture->getContentSizeInPixels(); // [texture_ contentSizeInPixels];
int tx = s.width;
int ty = s.height;

int bitsPerPixel                = 4 * 8;
int bytesPerPixel               = bitsPerPixel / 8;
int bytesPerRow					= bytesPerPixel * tx;
unsigned long myDataLength				= bytesPerRow * ty;

int numberOfPixels              = tx * ty;
float numberOfTransparent       = 0;

GLubyte *buffer	= (GLubyte *)malloc(sizeof(GLubyte)*myDataLength);

if( ! (buffer) ) {
	log("cocos2d: CCRenderTexture#getUIImageFromBuffer: not enough memory");
    free(buffer);
	return -1.0f;
}

begin();
glPixelStorei(GL_PACK_ALIGNMENT, 1);
glReadPixels(0,0,tx,ty,GL_RGBA,GL_UNSIGNED_BYTE, buffer);
end();

int x,y;
for(y = 0; y < ty; y++) {
    // just want the last byte (alpha) for each pixel
	for(x = 0; x < tx; x++) {
        GLubyte a = buffer[(y * 4 * tx + ((x * 4)+3))];
        if(a == 0) {
            numberOfTransparent++;
        }
	}
}

    free(buffer);

    return (numberOfTransparent/numberOfPixels);
}

But with v3.x it no longer works. At other places in my code, I do visit sprites, so sounds like option 2 would not work, but option 1 might get me what I want.

Any further help you can provide would be greatly appreciated. Thank you.

#9

with the new renderer, draw function will submit a command to the render que
when the node tree has finished visiting, the render que will then execute the commands

so if your code reads the pixel data on the logical layer, it will not pick up anything, as the render que has not even start to render anything yet.

so the solution is to read it while the render que has rendered the stuff you want.

so one of the solution is to extend cocos2d::Node
implemet its draw function,
the draw function should submit a custom command to the render que, basically, it works like a callback
for the callback from the custom command, you read the pixels

cheers

#10

Thanks. That was enough info to get me pointed in the right direction. Since I was already subclassing RenderTexture, I opted to duplicate RenderTexture’s saveToFile order of operations to get the data from the image after it’s drawn.

#11

This works, but this solution is asynchronous meaning that if I want to test a pixel perfect touch inside a onTouchBegan, I don’t know what to return (whether true or false) to acknowledge the touch since I will only be able to read the pixel in a later stage.

Is there any way to do this synchronously?