Amazing!
So, can I doing my work based on your vr branch now?
@songchengjiang rebased.
Please, clone this repo:
https://github.com/ricardoquesada/cocos2d-x/tree/vr
If needed, we can change the API and add/remove more abstractions, etc.
For example, we might need to add something to support the different “Eye” transformations from the different SDKs.
But so far, I like the design: it works with existing code… I would say it works in 99.9% of the games.
Let me know if you have any question. I’ll keep working on this. I’ll add distortion and some abstractions.
Fix: resolution is automatically detected… it looks good in any phone/desktop computer.
Todo: add distortion.
iPhone 5S:
iPhone 6+:
Mac:
Wow, it is cool!
@ricardo I have finished the VR rendering of cardboard.
But, I have a problem!
In VRCardboard::render(Scene* scene, Renderer* renderer), we need change viewport for per-eye. like following code:
cbapi_beforeDrawFrame();
glEnable(GL_SCISSOR_TEST);
glDepthMask(true);
for (unsigned short i = 0; i < CB_EYE_NUM; ++i){
auto vp = _eyes.eyeParams[i].viewport;
glScissor(vp.x, vp.y, vp.width, vp.height);
glViewport(vp.x, vp.y, vp.width, vp.height);
glClearColor(0.125f, 0.125f, 0.125f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
const float eyeOffset = ( i ? -0.5f : 0.5f ) * _hmd.device.interLensDistance;
scene->render(renderer, Vec3(eyeOffset,0,0));
}
glDepthMask(false);
glDisable(GL_SCISSOR_TEST);
cbapi_afterDrawFrame();
Unfortunately, I found Camera::apply() would change the viewport by using Camera::applyViewport().
For now, in order to make VR rendering correctly, I commented out Camera::applyViewport().
How to fix it? Any ideas?
@songchengjiang I confirm that Camera::setDefaultViewport(vp) works.
stutus:
- little refactor: Instead of using two framebuffers, I’m using a single one, similar to Cardboard. That means that I have to change the viewport before switching eyes (that’s why I can confirm that
Camera::setDefaultViewport()works). - started distortion code: it is based on google’s code. 60% complete. I think I’ll have it working by tomorrow.
1 Like
Distortion added… camera’s values are hardcoded… doesn’t look good yet… a few bug fixes here and there and done.
2 Likes
@ricardo I have finished the VR rendering integration of GearVR and Deepoon.
Code is here:
https://github.com/songchengjiang/cocos2d-x/blob/riq_vr/cocos/vr/CCVRGearVR.cpp
https://github.com/songchengjiang/cocos2d-x/blob/riq_vr/cocos/vr/CCVRDeepoon.cpp
Now, I am working on the integration of Oculus on windows. it will be finished tomorrow.
not using hardcoded values anymore… only a few bugs remaining.
@songchengjiang
please, send me a Pull Request with your changes… or just merge them directly into cocos2d-x. thanks.
@ricardo I have finshed VR rendering of Oculus.
I will send you a PR later.
By the way, could you finish the abstraction of headtracking this week-end?
I made a few changes:
// OLD
void Scene::render(Renderer* renderer, const Vec3& eyeOffset);
// NEW
void Scene::render(Renderer* renderer, const Mat4& eyeTransform);
The differences is that the eyeTransform includes the Head Tracking transform and the eye offset.
So, if you want to apply the head transform, you should do something like this:
Mat4 leftTransform;
Mat4::createTranslation(-eyeOffset, 0, 0, &leftTransform); // left offset transform
leftTransform *= _headTracker->getLocalRotation(); // multiply by the head tracking transform
Mat4 rightTransform;
Mat4::createTranslation(+eyeOffset, 0, 0, &rightTransform); // same thing for right eye
rightTransform *= _headTracker->getLocalRotation();
_fb->applyFBO();
Camera::setDefaultViewport(_leftEye.viewport);
scene->render(renderer, leftTransform); // render left eye
Camera::setDefaultViewport(_rightEye.viewport);
scene->render(renderer, rightTransform); // render right eye
_fb->restoreFBO();
I also made some changes in the VRProtocol:
-
VRProtocolrenamed toVRIRenderer -
NEW Protocol :
VRIHeadTracker
which has these 2 methods:
Vec3 getLocalPosition()Mat4 getLocalRotation()
If we need it to return a Quaternion instead of a Mat4, we can do that as well.
Let me know if you need help with it.
I renamed a few files as well.
This is my commit: https://github.com/ricardoquesada/cocos2d-x/commit/a33faafa1a2b6cffdc32d87f3c1c321f46f94789
My “generic” head tracking code is not working now, so getLocalRotation is returning Mat4::IDENTITY for the moment.
a few bugs here and there but built-in VR is ready.
- distortion working Ok on all platforms
- head tracking working Ok only on iOS
ha. That video is not correct. It is changing the camera like if it were a “view” and not a “camera”.
Fixed. I just needed to add a getInversed() in the eyeTransform matrix.!
@ricardo This is super fantastic! Thank you for this amazing progress.
Nice SX-64 in the background.
ha ha. Thanks 
@songchengjiang
Important: the eyeTransform that is passed to Scene::render() should be in “view” coordinates.
Scene will invert it and use it as if it were in “model” coordinates. So, you need to pass the eye transform matrix in “eye coordinates”.
Yes, I noticed that, I have almost completed the integration of headtrackers(GearVR/Deepoon/Cardboard/Oculus). I will optimize the code today, and then, I will send a PR!
1 Like