Cocos2dx 3.17
Hello thanks for viewing! so cocos2dx’s experimental built in VideoPlayer only works for Android/IOS (to my knowledge). I need a universal solution that works on Windows,Linux and Mac as well so i looked into ffmpeg C library and successfully integrated it.
The problem i have now is what would be the recommended or suggested way to display raw data? is it the Cocos2dx Renderer? if so could someone give me some pointers on how to display raw data using the Cocos2dx Renderer?
@slackmoehrle thanks for the link. It does not have a linux solution sadly. would it possible for you to share that old ffmpeg code : D. or i was looking for something in cocos2dx that allows me to render my umcompressed raw data from ffmpeg. any linux solution ideas?
ffmpeg should work on Linux 
@slackmoehrle 
so i managed to get ffmpeg to play nice but i needed a way to display the video so i used SDL2. but… when the video is load into SDL2 it is loaded in a separate window. Now my only issue is trying to find a way to get the video inside the same window as my cocos2dx application. any ideas? (cocos2dx does not allow a window Handle or anything of the sort for me to hook into i think…)
I haven’t used SDL in like 10 years. Iirc it is OpenGL based? We used to have a
CustomCommand Class you could use but I think it is deprecated now
@zhangxm what are good ways to add custom OpenGL into Cocos2d-x easily?
Yes it is openGL based. I’ve seen the CustomCommand class I’ll look into it. If zhangxm has some ideas I’ll take those as well. But, thank you for your constant help @slackmoehrle I really do appreciate you being the guardian angel of these forums.
Thanks. You could use draw() and put your code there but I don’t think it is the appropriate place.
@ManaInfested are you want to integrate video into the game or just want to display a video? VideoPlayer plays video in another view that different from game view. If you want this feature, then i suggest you uses local video module if possible.
If you want to put the video in the same view with game view, that make video as part of game, then you should draw the raw data as you said above. And custom command is what you need.
@slackmoehrle @zhangxm thank you both i found the custom command you both spoke of. I am havning problem utilizing it, I found the Api reference is there any cpp-tests or other examples of custom command? this whole linux video player issue is just giving me a really hard time…
I think there is but I can’t look until tomorrow. I’m without a laptop visiting friends for the day
ok. I’ll be on the lookout until tomorrow.
In the mean time there are other posts here that may be helpful.
ok so this is what i have so far but after i call glTexImage2D. nothing is updated on the screen at all. using opengl is a entirely new area for me so I have been looking over trying to figure out what I am doing wrong to not have anything updated on the screen
void MainScene::onDraw(const Mat4 &transform, uint32_t flags)
{
const char * input_filename = "/home/testuser/Desktop/projecttest/Classes/testvid.mp4";
AVFormatContext *pFormatCtx = nullptr;
avformat_open_input(&pFormatCtx,input_filename,nullptr,nullptr);
avformat_find_stream_info(pFormatCtx, nullptr);
av_dump_format(pFormatCtx, 0, input_filename, 0);
AVCodec *pCodec = nullptr;
AVCodecParameters *pCodecParameters = nullptr;
int video_stream_index = -1;
for(unsigned int i = 0; i < pFormatCtx->nb_streams; i++){
AVCodecParameters * LocalCodecParameters = nullptr;
LocalCodecParameters = pFormatCtx->streams[i]->codecpar;
std::cout << "AVStream->time_base before open coded " << pFormatCtx->streams[i]->time_base.num << pFormatCtx->streams[i]->time_base.den << std::endl;
std::cout << "AVStream->r_frame_rate before open coded " << pFormatCtx->streams[i]->r_frame_rate.num << pFormatCtx->streams[i]->r_frame_rate.den << std::endl;
std::cout << "AVStream->start_time " << pFormatCtx->streams[i]->start_time << std::endl;
std::cout << "AVStream->duration " << pFormatCtx->streams[i]->duration << std::endl;
std::cout << "finding the proper decoder (CODEC)" << std::endl;
AVCodec *pLocalCodec = nullptr;
pLocalCodec = avcodec_find_decoder(LocalCodecParameters->codec_id);
if(pLocalCodec == nullptr) {
logging("ERROR unsupported codec!");
}
if (LocalCodecParameters->codec_type == AVMEDIA_TYPE_VIDEO) {
if (video_stream_index == -1) {
video_stream_index = i;
pCodec = pLocalCodec;
pCodecParameters = LocalCodecParameters;
}
std::cout << "Video Codec: resolution" << LocalCodecParameters->width << " x " << LocalCodecParameters->height << std::endl;
} else if (LocalCodecParameters->codec_type == AVMEDIA_TYPE_AUDIO) {
std::cout << "Audio Codec: " << LocalCodecParameters->channels << " channels " << " sample rate " << LocalCodecParameters->sample_rate << std::endl;
}
std::cout << "Codec " << pLocalCodec->name << " ID " << pLocalCodec->id << " bit_rate " << pCodecParameters->bit_rate << std::endl;
}
AVCodecContext * pCodecContext = avcodec_alloc_context3(pCodec);
if (!pCodecContext)
{
logging("failed to allocated memory for AVCodecContext");
}
if (avcodec_parameters_to_context(pCodecContext, pCodecParameters) < 0)
{
logging("failed to copy codec params to codec context");
}
if (avcodec_open2(pCodecContext, pCodec, nullptr) < 0)
{
logging("failed to open codec through avcodec_open2");
}
AVFrame *pFrame = av_frame_alloc();
if (!pFrame)
{
logging("failed to allocated memory for AVFrame");
}
AVPacket *pPacket = av_packet_alloc();
if (!pPacket)
{
logging("failed to allocated memory for AVPacket");
}
int response = 0;
int how_many_packets_to_process = 8;
while (av_read_frame(pFormatCtx, pPacket) >= 0)
{
if (pPacket->stream_index == video_stream_index) {
logging("AVPacket->pts %" PRId64, pPacket->pts);
// Supply raw packet data as input to a decoder
response = avcodec_send_packet(pCodecContext, pPacket);
if (response < 0) {
//logging("Error while sending a packet to the decoder: %s", av_err2str(response));
}
while (response >= 0)
{
// Return decoded output data (into a frame) from a decoder
// https://ffmpeg.org/doxygen/trunk/group__lavc__decoding.html#ga11e6542c4e66d3028668788a1a74217c
response = avcodec_receive_frame(pCodecContext, pFrame);
if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
break;
} else if (response < 0) {
//logging("Error while receiving a frame from the decoder: %s", av_err2str(response));
}
if (response >= 0) {
logging(
"Frame %d (type=%c, size=%d bytes) pts %d key_frame %d [DTS %d]",
pCodecContext->frame_number,
av_get_picture_type_char(pFrame->pict_type),
pFrame->pkt_size,
pFrame->pts,
pFrame->key_frame,
pFrame->coded_picture_number
);
glTexImage2D(GL_TEXTURE_2D,
0,
3,
pFrame->width,
pFrame->height,
0,
GL_RGB,
GL_UNSIGNED_BYTE,
pFrame->data[0]);
}
}
if (response < 0)
break;
if (--how_many_packets_to_process <= 0) break;
}
av_packet_unref(pPacket);
}
logging("releasing all the resources");
avformat_close_input(&pFormatCtx);
avformat_free_context(pFormatCtx);
av_packet_free(&pPacket);
av_frame_free(&pFrame);
avcodec_free_context(&pCodecContext);
}
void MainScene::draw(Renderer *renderer, const Mat4 &transform, uint32_t flags)
{
//renderer = Director::getInstance()->getRenderer();
RenderTexture * VideoTest = RenderTexture::create(1280, 720);
VideoTest->setPosition(960,540);
_customCommand.init(VideoTest->getGlobalZOrder(), transform, flags);
_customCommand.func = CC_CALLBACK_0(MainScene::onDraw, this, transform, flags);
renderer->addCommand(&_customCommand);
}
Try to format the code in your post by adding 3 backtick ` characters before and after each code section, since it’s a bit hard to read your code.
Now, the onDraw should be called once per frame, yet you’re trying to play the entire video in one onDraw call. You should be reading each frame in a different loop (perhaps called from update() instead), storing that frame somewhere, and then drawing it in onDraw().
So, onDraw should only contain this (or whatever is required to draw the frame, but nothing else):
glTexImage2D(GL_TEXTURE_2D,
0,
3,
pFrame->width,
pFrame->height,
0,
GL_RGB,
GL_UNSIGNED_BYTE,
pFrame->data[0]);
Anything related to video packet decoding needs to be processed separately in it’s own update method.
going to follow up on everything you just said right now, just wanted to say thank for telling me about " ``` " i did not know that :).