Big window, small framebuffer?

Hey all. I’m creating a pixel art game (Songbringer) with a screen res of 420x240 that gets scaled up to whatever the window size is (example 2560x1800). I want to reduce the workload of the GPU and I also want the game to look more like a true pixel art game. Is there a way to get cocos2d-x to render everything to a framebuffer that is only 420x240, then scale it up to the window size after rendering?

See, here’s how the shadows look in the game now (they use a custom blur shader which you can tell is running at the higher resolution):

And here is how I want them to look:

I’ve already tried the obvious stuff like:

  • Changing the size of the window or the frame zoom factor with GLViewImpl::createWithRect()
  • Changing the design resolution or design res policy

The next things I will try:

  • Exploring Camera::setDefaultViewport
  • Experimenting with experimental::FrameBuffer

If anyone has any clues at all, please share!

Thanks.

That’s on my todo list as well mostly for WinPhone/WinRT since it renders to a smaller frame buffer (roughly 1/4 resolution), but then is scaled up with the hardware scalar ( https://github.com/MSOpenTech/angle/issues/42 ).

I’d probably start with the CameraFrameBuffer test. If you figure it out let me know.

My only concern would be touch/mouse event locations for UI and gameplay driven by touch (pan, select, etc). With game controllers and keyboard you’ll be fine. Maybe the touch virtual d-pad (and similar) will be solved by rendering those to a separate camera.

1 Like

After a quick test using code from CameraFramebufferTest in a new project I found that it should work for you, but performance testing would need to confirm there’s any benefit (there should be a large benefit ideally due to much reduced pixel count, except for the final render of the sprite with the frame buffer texture).

I needed to set design resolution to same as frame buffer or it won’t be pixel perfect since it will texture filter out pixels or stretch them due to UV mapping.

glview->setDesignResolutionSize(420, 240, ResolutionPolicy::SHOW_ALL);
auto fboSize = iSize(420,240);
auto fbo = cocos2d::experimental::FrameBuffer::create(1, fboSize.w, fboSize.h);

also in a quick test I needed to flip the sprite used for rendering the frame buffer to screen

auto sprite = Sprite::createWithTexture(fbo->getRenderTarget()->getTexture());
sprite->setFlippedY(true);
1 Like

Thanks, @stevetranby !! Yes, it totally works and because the game uses custom shaders the performance boost is significant. Instead of running a fragment shader for 4.6 million pixels, it does for only 100 thousand.

Big win! Especially for the GPU.

Today’s live stream and Youtube vid will cover the resultant code. I basically just copied CameraFramebufferTest including the RenderTarget and RenderTargetDepthStencil. I wonder if either of those are unnecessary.

I’m guessing touch positions will be fine as with this technique you technically have both full screen and fbo elements. One simply selects which camera to render to using Node::setCameraMask.

Will have to test on Windows too (and Linux eventually).

It looks like it is possible to create a FrameBuffer without a RenderTargetDepthStencil but only if this minor bug is fixed in cocos2d-x.

Confirmed that it works okay on Windows (also with significant performance improvements).

1 Like

@natweiss Hey could you link me to the vid with your discussion of getting this working, I’m doing something very similar with shadow mapping so it would be great to get some tips.

Nevermind, think I found it: https://www.youtube.com/watch?v=yttShH4N31I

1 Like

Great. Was going to suggest that vid.

1 Like

I noticed you can share the fbo, rt and even rtTexture throughout different scenes. Only the sprite and camera nodes have to be unique to make the sceneGraph happy.

2 Likes

@stevetranby What to do, if I want to zoom in, to have FrameBuffer sprite fullscreen?
My designResolution is 960x720, the size if the framebuffer is 320x240 (for shader reasons). After that I want to show the framebuffer sprite fullscreen without doing frameBufferSprite->Scale() because I hope, I will get a better performance if it is achieved with camera or viewport? Thank you!

@natweiss Are you sure about the performance improvements? I have done the same, doing a Hqx shader on my frame buffer sprite with 320x240 but to display it afterwards fullscreen, I have to scale the frame buffer sprite and my FPS are getting low -> on iPad Air about 40 FPS, without scaling of frame buffer sprite 60 FPS.

I did more research on this and for my understanding, the fragment shader has to do more work, if the sprite was scaled. Can anybody confirm this?

Are you doing any postprocessing effects on the scaled image? I think you should do those on the 320x240 frame buffer.
The shader rendering the scaled buffer (aka the final frame rendered) should just be texturing a quad so it’s not performance heavy.

Hi @bolin:

my problem is maybe not the shader. I have a new approach, trying to render the frame buffer to a render texture and doing the required scaling on the render texture and not on the frame buffer sprite.

But the FPS are only 40 FPS. Maybe I am doing something wrong there. Could you kindly have a look? All I want to achieve is, apply shader on FBO and render it to a render texture which I will upscale to have a fullscreen display. Thank you!

// on "init" you need to initialize your instance

// add overall layer
mGameLayer = LayerColor::create(Color4B(128, 128, 128, 255));
this->addChild(mGameLayer, 0, 0);


float lFboScale = 1.0f;
auto lWinSize = Director::getInstance()->getWinSizeInPixels();
Size lRTexSize = Size(320 * lFboScale, 240 * lFboScale);

// create FrameBuffer
auto lFrameBufferSize = Size(320, 240);
auto lFrameBuffer = experimental::FrameBuffer::create(1, lFrameBufferSize.width, lFrameBufferSize.height);
lFrameBuffer->setClearColor(Color4F(0, 255, 0, 255));
auto lRenderTarget = experimental::RenderTarget::create(lFrameBufferSize.width, lFrameBufferSize.height, Texture2D::PixelFormat::RGB5A1);
lFrameBuffer->attachRenderTarget(lRenderTarget);
lFrameBuffer->getRenderTarget()->getTexture()->setAliasTexParameters();	// otherwise texture will be mushy after resize
auto lRenderTargetDepthStencil = experimental::RenderTargetDepthStencil::create(lFrameBufferSize.width, lFrameBufferSize.height);
lFrameBuffer->attachDepthStencilTarget(lRenderTargetDepthStencil);

// create texture from FrameBuffer
auto lFrameBufferSprite = Sprite::createWithTexture(lFrameBuffer->getRenderTarget()->getTexture());
lFrameBufferSprite->setPosition(lRTexSize.width / 2, lRTexSize.height / 2);
//lFrameBufferSprite->setPosition(lWinSize.width / 2, lWinSize.height / 2);
lFrameBufferSprite->setScale(lFboScale);// lWinSize.height / 240.0f); // zoom into to have full height off screen
lFrameBufferSprite->setFlippedY(true);
mFrameBufferSprite = lFrameBufferSprite;
mFrameBufferSprite->retain();
//mGameLayer->addChild(mFrameBufferSprite);

// create camera and set new viewport to zoom in and show picture with maximal height
auto camera = Camera::create();
camera->setCameraFlag(CameraFlag::USER1);
camera->setDepth(-1);
camera->setFrameBufferObject(lFrameBuffer);
float viewPortRectWidthRatio = static_cast<float>(lWinSize.width) / static_cast<float>(960);
float viewPortRectHeightRatio = static_cast<float>(lWinSize.height) / static_cast<float>(720);
float viewPortW = ceilf(viewPortRectWidthRatio * (float)1000.0) / (float)1000.0;
float viewPortH = ceilf(viewPortRectHeightRatio * (float)1000.0) / (float)1000.0;
experimental::Viewport lViewPort = experimental::Viewport(0.0, 0, viewPortW, viewPortH);
camera->setViewport(lViewPort);
this->addChild(camera);

// add sprite and set camera mask to it
sprite2 = Sprite::create("cut3x.png");
int fboWidth = lFrameBufferSprite->getTexture()->getContentSizeInPixels().width * 3;
int fboHeight = lFrameBufferSprite->getTexture()->getContentSizeInPixels().height * 3;
sprite2->setPosition(fboWidth / 2, fboHeight / 2);
sprite2->getTexture()->setAliasTexParameters();
sprite2->setCameraMask((unsigned short)CameraFlag::USER1);
mGameLayer->addChild(sprite2);


// create render texture
Size iGameLayerSize = lWinSize;
mRTex = RenderTexture::create(lRTexSize.width, lRTexSize.height, Texture2D::PixelFormat::RGB5A1);
mRTex->setPosition(iGameLayerSize.width / 2, iGameLayerSize.height / 2);
//mRTex->retain();
//mRTex->getSprite()->setCameraMask((unsigned short)CameraFlag::USER1);
mRTex->getSprite()->setScale(lWinSize.height / lRTexSize.height);
mGameLayer->addChild(mRTex);


// apply shader to FrameBuffer sprite
//InitHq4xShader(mFrameBufferSprite);

//lFrameBufferSprite->setPosition(lRTexSprite->getPositionX() + 160, lRTexSprite->getPositionY() + 120);

//this->setScale(3.0);

return true;

}

void HelloWorld::draw(cocos2d::Renderer* renderer, const cocos2d::Mat4 &transform, uint32_t flags)
{ 
	//CCConsole::printSceneGraphBoot(0);

	mRTex->begin();
	
	mFrameBufferSprite->visit();
	
	mRTex->end();
}
1 Like

Yes, I think this is the correct way of doing it.

Couldn’t you render the scene directly on the RenderTexture and avoid using the lFrameBuffer and lFrameBufferSprite? Then you just render the RenderTexture’s sprite and scale it as much as you want.

The funny thing is, it is getting slow as soon as I set frame buffer to the camera:

camera->setFrameBufferObject(lFrameBuffer);

It is even slow if the frame buffer has only a size of 1x1

After a little bit more research, the root cause seems to be ApplyViewPort of Camera class.

I still do not get it. Is it such an expensive step?

It seems so :slight_smile: You could try debugging the cocos2d-x code and try to find the root cause.

I still don’t understand why you are using FrameBuffer instead of using RenderTexture. RenderTexture will handle all the frame buffer stuff for you and after rendering to the RenderTexture you can just use the RenderTexture sprite and add it to your scene.

Ok, I will give it a try. Framebuffer was really nice because my scene was rendered (resized) to 320x240 pixels instead of original resolution (960x720). What would be the best way to achieve this with use of RenderTexture?

I think you just have to create a 320x240 RenderTexture.
RenderTexture create method receives width, height as well as pixel format.

static RenderTexture * create(int w, int h, Texture2D::PixelFormat format);

hi bolin,

i have created a new topic for render texture discussion because this one is more related to the frame buffer.