Finally, I give up using LuaJava.
I write a test class in java, used android API to create a dialog,
then I just use cocos JniHelper::getStaticMethodInfo to find the class which I build in java,
then write a class in c++, use CCLuaEngine lua_register to register the class you will called in lua,
and then haha\~ many thens, so complicate, called the method in your lua code, it works.