UIEditBox - Lag of up to 5 seconds on iOS

cocos2d-x C++
#21

@zhangxm

Okay, I’ve worked a bit more on the reproduction project and I can indeed get the behavior with the new project. It’s random in my main project and it’s also random in the repro project. I wasn’t able to eliminate the randomness, so it might be you have to start the app 10 times and it does not happen and then it does happen.
It’s very important that you restart the app after every try, as the biggest lag only happens once. Afterwards the lags get way way smaller.

Link to new repro project:

I edited UIEditBoxImplIOS to add a log for how long it takes to do the openKeyboard call:

void EditBoxImplIOS::nativeOpenKeyboard()
{
    CFTimeInterval startTime = CACurrentMediaTime();
    
    [_systemControl setVisible:YES];
    [_systemControl openKeyboard];
    
    CFTimeInterval elapsedTime = CACurrentMediaTime() - startTime;
    NSLog(@"This took %f seconds", elapsedTime);
}

Here’s the result of five subsequent tries (restart and first tap on the edit box):

2017-04-01 13:35:12.536643 EditBoxIssue-mobile[655:156416] This took 0.935062 seconds
2017-04-01 13:36:59.281055 EditBoxIssue-mobile[661:157419] This took 1.641139 seconds
2017-04-01 13:38:58.581670 EditBoxIssue-mobile[665:157752] This took 20.313935 seconds
2017-04-01 13:40:09.509423 EditBoxIssue-mobile[670:158587] This took 3.671407 seconds
2017-04-01 14:04:56.089067 EditBoxIssue-mobile[239:5534] This took 0.149004 seconds

On the third try it took 20 (!) seconds until the keyboard opened. Unfortunately, I didn’t video tape it and I didn’t get a delay of that length afterwards…but I started video taping and got the ~3.6 seconds delay on video.

Even though not 20 seconds, I believe that every delay greater than ~200 ms is already unbearable and very noticeable for the user. Hence also the ~1-3 seconds that I measured should never happen.

In my main project there’s much more pressure on the CPU due to way more draw calls and sprites…but it seems like even this very minimal demo project is sufficient to get the error a reasonable amount of time.

In case you can not reproduce it on your iPhone, maybe you can also try it with an iPad. It feels like it’s easier to get it on iPad even though I also get it on iPhone.

#22

Just curious, what device are you testing on?
Thanks for the test project, hopefully it helps someone figure this out.
Unfortunately, I probably won’t look a this right now, as it’s not an issue we get support emails about.

#23

I’m testing on an iPad Pro 9.7’’ and an iPhone 6S.

@stevetranby How do you mean about the support emails?
Do you mean with “we” the cocos2d-x team or you personally with your games?

Some new findings:
Interestingly, I’m also seeing similar issues on Android with the latest version of the UIEditBox (from GitHub). The keyboard either does not open at all, or it does open after a huge delay.
With a version prior to 3.14.1, this did not happen on Android at least.

#24

Interesting that those are such high-perf devices with this issue.

Although if this issue doesn’t exist prior to 3.14.1 then only one of the games I’ve worked on has this issue since they haven’t been updated to beyond that version yet

Support emails from games I’ve been involved with and have access to that we get for example we have a lot of audio issues (or actually game “lag” issues that are [almost] always due to the AudioEngine not being updated yet) as well as just game play stuff. Since our games have only one or two places with keyboard shown and as you mention its both random and doesn’t happen all the time and apparently don’t think people care enough to write in about it.
(this is only one data point and anecdotal, however)

I’m not on the cocos2d-x team, just a community member who’s used the engine for a handful of games (couple own, few for clients).

#25

Yes, the lag happens on all kind of devices - and even in a bare test project with just one single EditBox on the screen.

Well, I have not seen the Android issue prior to 3.14.1, the iOS issue has definitely existed prior to it, but I can’t tell exactly when it started since I only use the EditBox since ~3.12 maybe. I also didn’t do extensive testing with it before I realized the massive lags randomly.

I see about the support emails, makes sense. I’m just afraid people will just delete the app if they should enter their name and there’s a 5 second freeze. I sure would, since I’d assume it crashed. Hence, I assume those people to be lost before they’re even as engaged to write a support email. Alternatively, as you pointed out, they only get a minor lag and it’s not a big issue for them.

Okay, I was quite sure you we were not in the dev team - nonetheless you’re one of the most active members here and one with the deepest knowledge - so it could have been possible :wink:

@zhangxm

I found the source of my issue on Android, you won’t see it on the test project, but when using the UIEditBox delegate:

void AlertLayer::editBoxTextChanged(ui::EditBox *editBox, const std::string &text)
{
    if (nullptr != _editBox)
    {
        std::string textToUse = std::string(_editBox->getText());
        if (!_editBoxAllowWhiteSpace)
        {
            Utils::replaceAll(textToUse, " ", "");
        }
        CCLOG("Text changed");
        
        _editBox->setText(textToUse.c_str());
    }
}

I used the EditBox delegate to disallow the user entering white spaces in certain situations. Since I had a weird suspicion that this delegate function might get called every single frame on android…I’ve put a log there. Here’s my console output, bingo:

04-02 01:48:49.215 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.245 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.245 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.245 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.285 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.285 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.285 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.315 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.325 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.325 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.355 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.355 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed
04-02 01:48:49.355 15453-15482/com.forestringgames.TowerDuel D/cocos2d-x debug info: Text changed

It’s even called multiple times per frame when it’s just on the screen, doing nothing. I didn’t even tap it to open the keyboard. Just added it to the scene and the chaos begins to happen.

I’m slowly starting to believe that this UIEditBox might be so full of bugs that there’s no way, we’ll ever be able to use it - but hey, there’s still hope :wink:

1 Like
#26

I’m guessing you need the full features of UIEditBox, but have you tried TextFieldTTF? It may have the same problems, however, since it still requires opening up the platform’s keyboard (non-desktop).

The multi-frame call definitely seems like a culprit, so good find.

You may want to try filtering at the platform level?
CCUIEditBoxIOS.mm
Cocos2dxEditBoxHelper.java

You could even at least try logging in the .java helper to check if the many per-frame is on the android or cocos2d side.

#27

Quick google search to try and help:

Might need to add an after event or move the “cocos2d’s changed event” into afterTextChanged instead of onTextChanged? This is probably only necessary if you need to change & set the text.
https://stackoverflow.com/questions/476848/android-textwatcher-aftertextchanged-vs-textwatcher-ontextchanged .

https://developer.android.com/reference/android/text/TextWatcher.html .

Hope that helps.

#28

First of all, I’ve ugly-fixed the lags by just checking if the string changed. I still don’t know the reason why this even happens:

@Override
public void onTextChanged(CharSequence s, int start, int before, int count) {
    //The optimization can't be turn on due to unknown keyboard hide in some custom keyboard
//                        mFrameLayout.setEnableForceDoLayout(false);
    // Note that we must to copy a string to prevent string content is modified
    // on UI thread while 's.toString' is invoked at the same time.
    final String text = new String(s.toString());
    mCocos2dxActivity.runOnGLThread(new Runnable() {
        @Override
        public void run() {
            if (false == editBox.getText().toString().equals(text))
            {
                Log.d(TAG, "Old Text:" + editBox.getText() + "New:" + text);
                Cocos2dxEditBoxHelper.__editBoxEditingChanged(index, text);
            }
        }
    });
}

Thanks for the findings, I’ve tried both of these (afterTextChanged and onTextChanged) but the same code that works on iOS to prevent the user from entering whitespace does not work on android. The user can just keep on typing as many white spaces as they like :wink:

I actually wanted to use some of the features of UIEditBox, e.g. the user being able to copy/edit wherever they want which is pretty cool. Unfortunately, it doesn’t seem like this will be possible. Also you apparently can not disable that the native iOS/Android TextBox is being swapped out back and forth with a Cocso2d-x label when editing starts or ends. It makes a lot of sense to do this, in case something will be moved over the TextBox, since Cocos2d-x can not draw over and under the EditBox…but for our use case it’s just unnecessary. I’m also seeing a lot of flickering between the Cocos2d-x label and the native label. Often they’re even displayed at the same time looking really ugly. This screenshot is from a test in which we did not do any changes of the text via the delegate, so this is not the source of the error:

IMG_0376.PNG1536×2048 770 KB

I think I’ll take your advice and try to explore the TextFieldTTF for now…it’s very sad that this more advanced UI element is in such an unusable state and that we have to cut back features now (or write our own thingy).

EDIT: I just found out that my fix for Android does not work and in fact just breaks the delegate completely. It’s never called and does not function. The EditBox text was already changed there and therefore the check I’m doing is just not working - sorry about the confusion.
I quickly tried something with beforeTextChanged but also no luck. Anyway, this seems to be one of the definitely fixable issues, spending enough time to think it through. The iOS crazy lag and the label-flickering are much worse and should be fixed first if possible.

#29

Interesting. Thanks for your continued log.

One final thought for now:

One suggestion I have based on that screenshot is you could look at writing some small amount of platform glue code to show a native modal view instead. UIView (and UIViewController’s present as modal method for launching).

This requires code per platform, which is not ideal, but that’s technically how EditBox was built anyway.

Just a thought, if you’re needs are mainly this single (or a few) concept, “Submit Feedback”, then it might be quicker to do this instead of fixing/workaround EditBox while still giving you the features you want.

Also, submit an issue on github if you haven’t already. It may never get looked at, worked on, and may be closed, but it’s the best way to get it in front of the dev team or community contributors.

#30

You’re welcome, I hope it helped :wink:

I also thought about code per platform, but I really wanted to avoid that…also I wanted to avoid to break the natural feel of the game with native UI. I will try standard TextFieldTTF / UITextField and similar things now…, they’re not as feature rich as the EditBox, but maybe they’ll do for now.

I submitted an issue, great idea - thanks!

#31

@zhangxm Did you have any time to check out my demo project? Are you able to reproduce the bug/bugs that you see in my video and screenshots?

#32

Sorry, i am busy for v3.15 and creator c++/lua support.

#33

These issue on Android platform should be fixed at here:

please give it a try.

thanks.

#34

Try to add some code in your AppController.mm,


- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {   

// ... some code has been omitted.

 // IMPORTANT: Setting the GLView should be done after creating the RootViewController
    cocos2d::GLViewImpl *glview = cocos2d::GLViewImpl::createWithEAGLView(eaglView);
    cocos2d::Director::getInstance()->setOpenGLView(glview);
    
// the following code is needed to speed up the lag issue
    UITextField *lagFreeField = [[UITextField alloc] init];
    [window addSubview:lagFreeField];
    [lagFreeField becomeFirstResponder];
    [lagFreeField resignFirstResponder];
    [lagFreeField removeFromSuperview];
//=================================

   app->run();
    return YES;
}

Refer:

After apply this trick, the lag time would be reduced to 20ms, I think this is a accept value with my iPhone 7.

#35

Interesting comment on that stackoverflow about making sure to test release build, on device, while not being plugged in and especially not debugging.

#36

This seems to indeed fix it on iOS. Also @owen did some more fixes on the EditBox on Android that are in the latest version on GitHub that also make the EditBox usable on Android.

Now there’s only a keyboard-open-lag left on Android. Also I have to do more testing on iOS with different devices and release builds.

1 Like