i have a question, it seems that distortion logic is optimized by headset manufacturer in the device sdk , so maybe the generic logic can’t support all the hardware. am i right?
to support the same visual feeling for user producer (game studio) and customer , it is much better use the same logic in the engine (or sdk) and the device. am i right?
correct. The generic code is useful only for quick-n-dirty testing… let’s say that you want to test VR on a windows phone and no SDKs are available, then you can use our generic implementation.
Users should use the official SDKs: cardboard SDK, oculus sdk, etc…
@ricardo For now, I have found the following BUGS that we need resolve:
The ProjectionMatrix error, different VR SKD define specific ProjectionMatrix Information for VR rendering, we need replace the camera’s ProjectionMatrix by the SKD’s ProjectionMatrix, otherwise, we will see the rendering error when we using headtracking.
a) passing SKD’s ProjectionMatrix to scene
replace Scene::render(Renderer* renderer, const Mat4& eyeTransform)
by Scene::render(Renderer* renderer, const Mat4& eyeTransform, const Mat4& eyeProjection)
and replace director->loadMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_PROJECTION, Camera::_visitingCamera->getViewProjectionMatrix());
by director->loadMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_PROJECTION, eyeProjection * Camera::_visitingCamera->getViewMatrix());
b)VRIRenderer provide a interface like virtual VRIPerspective getSuggestedPerspective() const = 0;
and for default camera, we can doing some changes in Camera::initDefault()
float zeye = Director::getInstance()->getZEye();
Vec3 eye, center, up;
if (Director::getInstance()->getOpenGLView() && Director::getInstance()->getOpenGLView()->isVREnabled()){
auto perspective = Director::getInstance()->getOpenGLView()->getVRRenderer()->getSuggestedPerspective();
initPerspective(perspective.fov, perspective.width / perspective.height, perspective.zNear, zeye + perspective.height / 2.0f);
}else{
//initialization like before
}
for user camera, developers need calling Director::getInstance()->getOpenGLView()->getVRRenderer()->getSuggestedPerspective() to get the perspective information and using it to create camera by Camera::createPerspective.
would you like a) or b), or other ideas?
Culling error, part of the texts was culled. I haven’t found the cause of the problem, do you have any ideas?
Coordinate mapping error of touch event, normally, when enter the VR mode, we should not receive the touch event(because phone in VR devices, we cannot touch the screen), how can we solve this problem or have other alternatives?
I think it is a bug in the “UI Scroll list” component. we should review that code… I think it is using the camera for the culling and perhaps it is not getting the values correctly, or perhaps we introduced a bug in the camera.
For example Cardboard SDK uses touches (actually just one touch)… the touch can be in any place.
So, I think the user should disable / enable the touches when VR is enabled/disabled. We shouldn’t automate that because it varies from SDK to SDK
hey, after merging your patch in my “vr” branch, Android does not compile. It gives me this error:
Android NDK: /Users/riq/progs/cocos2d-x/tests/cpp-tests/proj.android/../../../external/recast/Android.mk: Cannot find module with tag 'gearvr/prebuild' in import path
Android NDK: Are you sure your NDK_MODULE_PATH variable is properly defined ?
Android NDK: The following directories were searched:
Android NDK:
make: Entering directory `/Users/riq/progs/cocos2d-x/tests/cpp-tests/proj.android'
/Users/riq/progs/cocos2d-x/tests/cpp-tests/proj.android/../../../cocos/Android.mk:332: *** Android NDK: Aborting. . Stop.
make: Leaving directory `/Users/riq/progs/cocos2d-x/tests/cpp-tests/proj.android'
Error running command, return code: 2.
The gearvr, cardboardsdk, and the other sdk are optional features, and cocos2d-x should compile and run without them. Once we have the package manager working, users should install the different SDKs using the package manager.
In the meantime, could you remove the gearvr dependency? Thanks!
@songchengjiang
Another thing that we need to discuss is the workflow.
For example, I tried to compile your VR patch, but it fails because it requires the “deepoon/include/deepoon_sdk_native.h” file.
Of course, the solution is to avoid compiling the “CCVRDeepon*” files. Or to tell users to install Deepon and put it in the path. But those two solutions are not good from a UX point of view.
So either:
the “CCVRDeepon*” (and the other SDK*) should not be part of cocos2d-x
or users should not be forced to install the different VR SDKs in order to compile cocos2d-x.
Another way to look at it is:
it should work with cocos2d-x precompiled libraries.
One possible solution is to put the different CCVRDeepon* / GRVear / Oculus* files outside cocos2d-x, and tell users to include those files in their games…
Any other solution?
perfect. where should we place the GearVR, Deepoon, Cardboard SDKs ?
Right now they are in cocos/vr, but they should be moved somewhere else. what is the best place? the package manager ?
I’m almost finished the integration of GVR SDK(Google’s new VR SDK for daydream and cardboard), cardboard has been test OK on Android platform. I will test daydream on nexus 6P(the only phone supports daydream at this time) next step and send a PR to @ricardo after testing is completed.
@songchengjiang Does supporting VR mean we will finally have gamepad support for platforms that support VR?
Yes, In finally, I think we should support gamepad on all platforms. But now the first thing we should support the VR rendering and headtracking.
@songchengjiang
Can you tell me how to enable each VR SDK we support? Gear, Deepoon, Oculus, Cardboard, etc.
We are going to use cocos package manager to manage the VR SDKs. If we want to enable GearVR, we just doing like this in command line:
-> CMP install GearVR
-> installed GearVR SDK…
-> replaced AndroidManifest.xml
->…
Then, we just add two lines in the AppDelegate as Riq said:
// In the AppDelegate, just add these two lines:
auto vrImpl = new VRGearRenderer;
glview->setVR(vrImpl);
yes, we should document the whole process, including how to use the package manager. Although I’m not sure if that is ready… and I haven’t used it, so I don’t know the answer
