Why websocket JSB can only be a one-off in the maximum receive 4096 bytes of data?

Why websocket JSB can only be a one-off in the maximum receive 4096 bytes of data, how do I to break this limit?

How did you test it?
The texture size is limited to 2048x2048. So in the WebsocketTest, you could not create a ttf label with very long text.

James Chen wrote:

How did you test it?
The texture size is limited to 2048x2048. So in the WebsocketTest, you could not create a ttf label with very long text.

I just received very long text from server and print the length of data which came from onMessage callback.

How to reproduce this issue?

James Chen wrote:

How to reproduce this issue?

You can connect to ws://echo.websocket.org and send some very long text which size more than 4096 bytes.

garfield ho wrote:

James Chen wrote:
> How to reproduce this issue?
>
You can connect to ws://echo.websocket.org and send some very long text which size more than 4096 bytes.

Hi,Can you reproduce this issue?

I came across this issue also. There are a couple of ways to fix it.

It’s because the libwebsockets library has a default receive buffer size of 4096 bytes. In WebSocket.cpp, the onSocketCallback method gets called multiple times with LWS_CALLBACK_CLIENT_RECEIVE, in chunks of 4096. This forwards the message data straight to the JavaScript callback via the sendMessageToUIThread method. You could probably write some logic in your websocket.onmessage handler that deals with this, but that’s probably not the best solution.

Either you can alter the per protocol receive buffer size, which can be set in the WebSocket::init method, where the _wsProtocols member is setup.
ie. After line 326 in WebSocket.cpp:

_wsProtocols[i].rx_buffer_size = 1024 * 64; // 64;

Or you can add code to the onSocketCallback method in the LWS_CALLBACK_CLIENT_RECEIVE case, that calls libwebsockets_remaining_packet_payload to determine if any more data is expected. If it is, buffer the data just passed and wait for the next callback. When libwebsockets_remaining_packet_payload returns 0, you can pass all the received data to the JavaScript onmessage handler.

I’ve done the second option and have attached my patched WebSocket.cpp and .h files in case anyone is interested. They are patched from the 2.1.4 release version.

Hope that helps,

Steve

Steve Longhurst wrote:

I came across this issue also. There are a couple of ways to fix it.
>
It’s because the libwebsockets library has a default receive buffer size of 4096 bytes. In WebSocket.cpp, the onSocketCallback method gets called multiple times with LWS_CALLBACK_CLIENT_RECEIVE, in chunks of 4096. This forwards the message data straight to the JavaScript callback via the sendMessageToUIThread method. You could probably write some logic in your websocket.onmessage handler that deals with this, but that’s probably not the best solution.
>
Either you can alter the per protocol receive buffer size, which can be set in the WebSocket::init method, where the _wsProtocols member is setup.
ie. After line 326 in WebSocket.cpp:
>
_wsProtocols[i].rx_buffer_size = 1024 * 64; // 64;
>
Or you can add code to the onSocketCallback method in the LWS_CALLBACK_CLIENT_RECEIVE case, that calls libwebsockets_remaining_packet_payload to determine if any more data is expected. If it is, buffer the data just passed and wait for the next callback. When libwebsockets_remaining_packet_payload returns 0, you can pass all the received data to the JavaScript onmessage handler.
>
I’ve done the second option and have attached my patched WebSocket.cpp and .h files in case anyone is interested. They are patched from the 2.1.4 release version.
>
Hope that helps,
>
Steve

Thanks,Steve, I’ll test your codes,and merge it to our develop branch.

Steve Longhurst wrote:

I came across this issue also. There are a couple of ways to fix it.
>
It’s because the libwebsockets library has a default receive buffer size of 4096 bytes. In WebSocket.cpp, the onSocketCallback method gets called multiple times with LWS_CALLBACK_CLIENT_RECEIVE, in chunks of 4096. This forwards the message data straight to the JavaScript callback via the sendMessageToUIThread method. You could probably write some logic in your websocket.onmessage handler that deals with this, but that’s probably not the best solution.
>
Either you can alter the per protocol receive buffer size, which can be set in the WebSocket::init method, where the _wsProtocols member is setup.
ie. After line 326 in WebSocket.cpp:
>
_wsProtocols[i].rx_buffer_size = 1024 * 64; // 64;
>
Or you can add code to the onSocketCallback method in the LWS_CALLBACK_CLIENT_RECEIVE case, that calls libwebsockets_remaining_packet_payload to determine if any more data is expected. If it is, buffer the data just passed and wait for the next callback. When libwebsockets_remaining_packet_payload returns 0, you can pass all the received data to the JavaScript onmessage handler.
>
I’ve done the second option and have attached my patched WebSocket.cpp and .h files in case anyone is interested. They are patched from the 2.1.4 release version.
>
Hope that helps,
>
Steve

Thank you ,Steve!

Steve Longhurst wrote:

I came across this issue also. There are a couple of ways to fix it.
>
It’s because the libwebsockets library has a default receive buffer size of 4096 bytes. In WebSocket.cpp, the onSocketCallback method gets called multiple times with LWS_CALLBACK_CLIENT_RECEIVE, in chunks of 4096. This forwards the message data straight to the JavaScript callback via the sendMessageToUIThread method. You could probably write some logic in your websocket.onmessage handler that deals with this, but that’s probably not the best solution.
>
Either you can alter the per protocol receive buffer size, which can be set in the WebSocket::init method, where the _wsProtocols member is setup.
ie. After line 326 in WebSocket.cpp:
>
_wsProtocols[i].rx_buffer_size = 1024 * 64; // 64;
>
Or you can add code to the onSocketCallback method in the LWS_CALLBACK_CLIENT_RECEIVE case, that calls libwebsockets_remaining_packet_payload to determine if any more data is expected. If it is, buffer the data just passed and wait for the next callback. When libwebsockets_remaining_packet_payload returns 0, you can pass all the received data to the JavaScript onmessage handler.
>
I’ve done the second option and have attached my patched WebSocket.cpp and .h files in case anyone is interested. They are patched from the 2.1.4 release version.
>
Hope that helps,
>
Steve

Hello,Steve.I found a bug in your Websocket.ccp at line 661 “data~~>len = len;".It should be "data~~>len=_current_data_len;”.

garfield ho wrote:

Hello,Steve.I found a bug in your Websocket.ccp at line 661 “data~~>len = len;".It should be "data~~>len=_current_data_len;”.

Yep, I think it should be _current_data_len.

James Chen wrote:

garfield ho wrote:
> Hello,Steve.I found a bug in your Websocket.ccp at line 661 “data~~>len = len;".It should be "data~~>len=_current_data_len;”.
>
Yep, I think it should be _current_data_len.

Yes, you’re right, sorry I missed that. Since I was using only text data, which gets turned into a JavaScript string later in the path using c_string_to_jsval, presumably using the null terminator, it didn’t effect my code.

Cheers,

Steve

Did you notice that when we send a text which larger than 4096 bytes, libwebsockets will throw an error like :

[1376896997:9541] ERR: Unable to spill ext 5008 vs 4118
Cocos2d: libwebsocket_write error...
Cocos2d: Partial write LWS_CALLBACK_CLIENT_WRITEABLE

The only way I found to resolve this issue is to set _wsProtocols[i].rx_buffer_size bigger. Any other solutions?

garfield ho wrote:

Why websocket JSB can only be a one-off in the maximum receive 4096 bytes of data, how do I to break this limit?

Sending large buffers should be handled carefully with ‘fragmented buffers’ functions.

http://webcache.googleusercontent.com/search?q=cache:xfTgVVTIt78J:ml.libwebsockets.org/pipermail/libwebsockets/2013-March/000393.html+&cd=5&hl=ko&ct=clnk&gl=kr

Please check ‘/* check for the “send a big file by hand” example case */’.

  1. send first fragment with LWS_WRITE_NO_FIN to tell it is not the final fragment.
  2. wait until LWS_CALLBACK*….*WRITABLE
  3. send remaining fragment with appropriate flag (with LWS_WRITE_NO_FIN or without)

Hi Nako Sung,
Thanks for your reply, do you think we should encapsulate that in WebSocket class? Does Developer need to take care of whether the message is a big file?

Nako Sung wrote:

garfield ho wrote:
> Why websocket JSB can only be a one-off in the maximum receive 4096 bytes of data, how do I to break this limit?
>
Sending large buffers should be handled carefully with ‘fragmented buffers’ functions.
>
http://webcache.googleusercontent.com/search?q=cache:xfTgVVTIt78J:ml.libwebsockets.org/pipermail/libwebsockets/2013-March/000393.html+&cd=5&hl=ko&ct=clnk&gl=kr
>
Please check ‘/* check for the “send a big file by hand” example case */’.
>

  1. send first fragment with LWS_WRITE_NO_FIN to tell it is not the final fragment.
  2. wait until LWS_CALLBACK*….*WRITABLE
  3. send remaining fragment with appropriate flag (with LWS_WRITE_NO_FIN or without)

I think cocos 2d x should provide WebSocket as transparent as HTML5’s. It isn’t desirable that the behaviors are different among platforms.
Really large files should be dealt with AssetsManager api, so the relatively large (~100kb) can be handled with ‘encapsulated interface’ nicely.

I commited my fix for fragmented recv/send into my old PR. (sorry, I’m not familiar with git. I failed to cherry-pick commit among branches… damn…)