CCFileUtils::sharedFileUtils()->getFileData and Binary file problem


#1

Hi wondering if anyone can help me. I seem to be having a strange issue where when opening an SQLite file I’m only getting the first part of the file. I’ve trie using the same code with a plain text file and it’s opening it fine. Is there something that I’m missing or have set wrong?

 unsigned long size = 0;
    string fileName = "dummy.sql";

    const char * path = fileUtils->fullPathFromRelativePath(fileName.c_str());
    unsigned char* smth = fileUtils->getFileData(path, "r", &size);

    CCLog("Data is : %s",smth);
    CCLog("Size: %lu\n\n",size);
    fflush(stdout);

If I look into the logs I just get ‘Data is : SQLite format 3’ for the (rather than the whole file) but the Size is displayed correctly. What I’m trying to do is to then write the file to the writablePath so I can update it.

Any ideas would be appreciated!


#2

I’m having a similar problem, but when reading from a json file.
FileUtils is returning just a small portion of the file (i’m using json as a sql replacement), thus making rapidjson fail to parse the file. The file is only 66 lines long and weights 1,9kb, but i’m suspecting FileUtils just doesn’t allocate much space. I’m trying to find a way to fix or work around this, and by searching the forums I see it’s a common problem. If someone finds the solution, please share with us!

Here is the code i’m using and the file is attached:

unsigned long filesize = 0;
    unsigned char* fileData = NULL;
    std::string content, fullPath;

    fullPath = CCFileUtils::sharedFileUtils()->fullPathFromRelativePath(s_LanguagesPath);

    fileData = CCFileUtils::sharedFileUtils()->getFileData(fullPath.c_str(), "r", &filesize);
    content.append((char*)fileData);
    CCLog(content.c_str());
    if(languagesDocument.Parse<0>(content.c_str()).HasParseError()) {
        CCLog(languagesDocument.GetParseError());
    }

#3

Guilherme Maia wrote:

I’m having a similar problem, but when reading from a json file.
FileUtils is returning just a small portion of the file (i’m using json as a sql replacement), thus making rapidjson fail to parse the file. The file is only 66 lines long and weights 1,9kb, but i’m suspecting FileUtils just doesn’t allocate much space. I’m trying to find a way to fix or work around this, and by searching the forums I see it’s a common problem. If someone finds the solution, please share with us!
>
Here is the code i’m using and the file is attached:
[…]

Try replacing content.append fileData ); with content.assign, fileSize)


#4

Lance Gray wrote:

Guilherme Maia wrote:
> I’m having a similar problem, but when reading from a json file.
> FileUtils is returning just a small portion of the file (i’m using json as a sql replacement), thus making rapidjson fail to parse the file. The file is only 66 lines long and weights 1,9kb, but i’m suspecting FileUtils just doesn’t allocate much space. I’m trying to find a way to fix or work around this, and by searching the forums I see it’s a common problem. If someone finds the solution, please share with us!
>
> Here is the code i’m using and the file is attached:
> […]
>
Try replacing content.append fileData ); with content.assign, fileSize)

I’ve tried this before, but still the same problem. Here’s what CCLog reads from the file data pointer (content from my languages.json):

01-22 02:41:56.628: D/cocos2d-x debug info(3227): {
01-22 02:41:56.628: D/cocos2d-x debug info(3227):   "en" : {
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "score" : "Score",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "timeLeftTotal" : "Time Left",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "timeLeftRune" : "Rune Time Left",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "totalTime" : "Total Time",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "finish" : "Finished!",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "gameFinished" : "Game Finished!",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "next" : "Next",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "restart" : "Restart",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "clean" : "Clean",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "accuracy" : "Accuracy",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "failLoadImg" : "Failed to load the following Images",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "exitQuestion" : "Do you really want to exit?",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "exitTitle" : "Exit",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "arcadeTitle" : "Arcade Mode",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "puzzleTitle" : "Puzzle Mode",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "optionsTitle" : "Options",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "yes" : "Yes",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "no" : "No",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "batteryCritical" : "Battery level is critical. Please recharge your device."
01-22 02:41:56.628: D/cocos2d-x debug info(3227):   },
01-22 02:41:56.628: D/cocos2d-x debug info(3227):   "pt" : {
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "score" : "Pontos",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "timeLeftTotal" : "Tempo Restante",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "timeLeftRune" : "Tempo da Runa",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "totalTime" : "Tempo",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "finish" : "Fim!",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "gameFinished" : "Fim de Jogo!",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "next" : "Próximo",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "restart" : "Reiniciar",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "clean" : "Limpar",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "accuracy" : "Precisão",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "failLoadImg" : "Falha ao carregar as seguintes Imagens",
01-22 02:41:56.628: D/cocos2d-x debug info(3227):       "exitQuestion" : "Deseja realmente sair?",
01-22 02:41:56.628: D/cocos2d-x debug info(3227): Loaded Languages file.

I’ve certified there’s no special character after “exitQuestion” : “Deseja realmente sair?”, that could somehow cause an EOF. I tried to test my guess by loading a simple txt file full of “a”, “b”, “c”s and the result is the same, only a small portion of the file is read (file is attached):

01-22 02:48:59.761: D/cocos2d-x debug info(3625): Initialized Scene
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
01-22 02:48:59.777: D/cocos2d-x debug info(3625): bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
01-22 02:48:59.777: D/cocos2d-x debug info(3625): Loaded Languages file.

I’m suspecting pBuffer = new unsigned char[FileInfo.uncompressed_size]; inside CCFileUtilsCommon_cpp.h is initializing the buffer with the wrong size, maybe the compressed instead of the uncompressed…


#5

Kinda weird. I once had this problem but reinterpret_cast<> worked for me.


#6

I’ve just tried it with another file, this time a .rtf file and it’s not reaching the end of this file either!

Is this a bug or have I missed a parameter somewhere?!


#7

James K wrote:

I’ve just tried it with another file, this time a .rtf file and it’s not reaching the end of this file either!
>
Is this a bug or have I missed a parameter somewhere?!

I updated to latest version of cocos2d-x (cocos2d-2.1beta3-x-2.1.0) but the problem continues. The variable fileSize stores the correct file size, but not all data is retrieved by getFileData…

As I’m loading from a JSON, I decided to use a c++ string to store it while this problem is not solved, but it’s extremely counter-intuitive and will slow-down updates, but now it works for me…


#8

Guilherme Maia wrote:

James K wrote:
> I’ve just tried it with another file, this time a .rtf file and it’s not reaching the end of this file either!
>
> Is this a bug or have I missed a parameter somewhere?!
>
I updated to latest version of cocos2d-x (cocos2d-2.1beta3-x-2.1.0) but the problem continues. The variable fileSize stores the correct file size, but not all data is retrieved by getFileData…
>
As I’m loading from a JSON, I decided to use a c*+ string to store it while this problem is not solved, but it’s extremely counter-intuitive and will slow-down updates, but now it works for me…
Do you have any example code as to how your using the C*+ string?

I’m still struggling with this, I’ve found I can read in large text files fine, it’s just reading in the sqlite file that’s causing the issue. I’m running on iOS at the minute (though Android is the same). I suspect it’s because the sqlite file is not plain text that the issue is occuring, but I don’t know if there’s anything I can change - I’ve set the read type to binary. I’ve tried upgrading to the latest Beta version (cocos2d-2.1beta3-x-2.1.1) of cocos2dx as well, with no change.

 CCFileUtils* fileUtils = CCFileUtils::sharedFileUtils();
    unsigned long tmpSize;

    const char* fileName = "pronouns.sql";
    string path = fileUtils->fullPathForFilename(fileName);
    string writablePath = fileUtils->getWriteablePath();
    writablePath.append(fileName);

    unsigned char* sqlData = fileUtils->getFileData(path.c_str(), "rb", &tmpSize);

    CCLog("Data is : '%s'",sqlData);

    FILE *fp = fopen(writablePath.c_str(), "w");

    if (! fp)
    {
        CCLOG("can not create file %s", path.c_str());
        return;
    }

    string data = std::string(reinterpret_cast(sqlData));

    fputs(data.c_str(), fp);
    fclose(fp);

    CC_SAFE_DELETE_ARRAY(sqlData);

#9

James K wrote:

Guilherme Maia wrote:
> James K wrote:
> > I’ve just tried it with another file, this time a .rtf file and it’s not reaching the end of this file either!
> >
> > Is this a bug or have I missed a parameter somewhere?!
>
> I updated to latest version of cocos2d-x (cocos2d-2.1beta3-x-2.1.0) but the problem continues. The variable fileSize stores the correct file size, but not all data is retrieved by getFileData…
>
> As I’m loading from a JSON, I decided to use a c*+ string to store it while this problem is not solved, but it’s extremely counter-intuitive and will slow-down updates, but now it works for me…
>
Do you have any example code as to how your using the C*+ string?
>
I’m still struggling with this, I’ve found I can read in large text files fine, it’s just reading in the sqlite file that’s causing the issue. I’m running on iOS at the minute (though Android is the same). I suspect it’s because the sqlite file is not plain text that the issue is occuring, but I don’t know if there’s anything I can change - I’ve set the read type to binary. I’ve tried upgrading to the latest Beta version (cocos2d-2.1beta3-x-2.1.1) of cocos2dx as well, with no change.
>
[…]

My problem is slightly different from yours as I’m basically trying to load a text file holding a JSON object, not binary data, but here’s how i’m using it:
My JSON Object

static const char s_LanguagesJson[] = "{\
    \"en\" : {\
        \"ln\" : \"English\",\
        \"language\" : \"Language\",\
        \"score\" : \"Score\",\
        \"timeLeftTotal\" : \"Time Left\",\
        \"timeLeftRune\" : \"Rune Time Left\",\
        \"totalTime\" : \"Total Time\",\
        \"finish\" : \"Finished!\",\
        \"gameFinished\" : \"Game Finished!\",\
        \"next\" : \"Next\",\
        \"restart\" : \"Restart\",\
        \"clean\" : \"Clean\",\
        \"accuracy\" : \"Accuracy\",\
        \"exitQuestion\" : \"Do you really want to exit?\",\
        \"exitTitle\" : \"Exit\",\
        \"arcadeTitle\" : \"Arcade Mode\",\
        \"writerTitle\" : \"Writer Mode\",\
        \"optionsTitle\" : \"Options\",\
        \"arcadeInfo\" : \"In Arcade Mode you have 60s to draw as many Runes as you can!\",\
        \"writerInfo\" : \"In Writer Mode you must draw the runes with maximum accuracy!\",\
        \"writerPassRuneInfo\" : \"Required Accuracy to change to next Rune\",\
        \"yes\" : \"Yes\",\
        \"no\" : \"No\",\
        \"musicVolume\" : \"Music Volume\",\
        \"sfxVolume\" : \"SFX Volume\",\
        \"creditsTitle\" : \"Credits\",\
        \"creditsInfo\" : \"Rune Maker\\nMr. Studios\\n\\nGraphics Designers\\nGyokusho (Gustavo Campedelli)\\nTakeshi (Ricardo Garcia)\\n\\nProgrammer\\nChaoseiro (Guilherme Maia)\\n\\nSound Designer\\nDip (Gabriel de Paula)\\n\\nTester\\nReddoRY (Gabriel Kiesshau)\\n\\nThanks for Playing!\",\
        \"lastHighscore\" : \"Highscore\",\
        \"finalScore\" : \"Final Score\",\
        \"replay\" : \"Play Again\",\
        \"pauseTitle\" : \"Paused\",\
        \"resume\" : \"Resume\",\
        \"quitMenu\" : \"Quit to Menu\"\
    }\
}";

My Class which return data based on the loaded JSON:

/* 
 * File:   LanguageClass.cpp
 * Author: guilherme
 * 
 * Created on January 23, 2013, 6:18 PM
 */

#include "LanguageClass.h"
#include "cocos2d.h"
#include "ResourcesPaths.h"
USING_NS_CC;
using namespace LikeASir;
using namespace rapidjson;

Document LanguageClass::languagesDocument;
const char* LanguageClass::s_chosenLanguage = "pt";
LanguageClass* LanguageClass::s_languageClass = 0;

void LanguageClass::getLanguages(std::vector& langs) const {
    langs.clear();
    for(Value::ConstMemberIterator it = languagesDocument.MemberBegin(); it != languagesDocument.MemberEnd(); ++it) {
        if(DEBUG) CCLog("Pushed %s", it->name.GetString());
        langs.push_back(it->name.GetString());
    }
}

void LanguageClass::setLanguage(const char* newLang) {
    for(Value::ConstMemberIterator it = languagesDocument.MemberBegin(); it != languagesDocument.MemberEnd(); ++it) {
        if(strcasecmp(it->name.GetString(), newLang) == 0) {
            LanguageClass::s_chosenLanguage = newLang;
            CCUserDefault::sharedUserDefault()->setStringForKey("language", newLang);
            return;
        }
    }
}

void LanguageClass::loadLanguages() {
    if(DEBUG)
        CCLog("Loading Languages file...");

    std::string chosenLanguage = CCUserDefault::sharedUserDefault()->getStringForKey("language", "en");
    s_chosenLanguage = chosenLanguage.c_str();

    // Parse Languages Document
    if(languagesDocument.Parse<0>(s_LanguagesJson).HasParseError()) {
        if(DEBUG) {
            CCLog(languagesDocument.GetParseError());
        }
    }
    if(DEBUG) {
        if(!languagesDocument.HasMember(chosenLanguage.c_str())) {
            CCLog("Language '%s' is not supported.", chosenLanguage.c_str());
        } else {
            CCLog("Languages Loaded.");
        }
        for(Value::ConstMemberIterator it = languagesDocument.MemberBegin(); it != languagesDocument.MemberEnd(); ++it) {
            CCLog(it->name.GetString());
            for(Value::ConstMemberIterator it2 = it->value.MemberBegin(); it2 != it->value.MemberEnd(); ++it2) {
                CCLog("%s = %s", it2->name.GetString(), it2->value.GetString());
            }
        }
    }
}

const char* LanguageClass::getStringFromKey(const char* key) {
    for(Value::ConstMemberIterator it = languagesDocument.MemberBegin(); it != languagesDocument.MemberEnd(); ++it) {
        if(strcasecmp(it->name.GetString(), CCUserDefault::sharedUserDefault()->getStringForKey("language", "en").c_str()) == 0) {
            for(Value::ConstMemberIterator it2 = it->value.MemberBegin(); it2 != it->value.MemberEnd(); ++it2) {
                if(strcasecmp(it2->name.GetString(), key) == 0) {
                    return (it2->value.GetString());
                }
            }
        }
    }
    return "NULL";
}

LanguageClass::LanguageClass() {
    loadLanguages();
}
LanguageClass* LanguageClass::sharedLanguageClass() {
    if(!s_languageClass) {
        s_languageClass = new LanguageClass();
    }
    return s_languageClass;
}

#10

This is one issue I’d really like to see fixed.

Trying to figure out what the patch is for getFileData() but I’m struggling.


#11

Thanks to all the various forum posts and helpful blogs, I was able to get this to work to copy a SQLITE file from a ‘.mp3’ file in ‘Resources’ to the writable path.

Tested on Windows, Android, Mac and iOS…

@ dbPath = CCFileUtils::sharedFileUtils()->getWriteablePath();
dbPath.append(“data.db”);
unsigned long tmpSize;
unsigned char* xmlData = CCFileUtils::sharedFileUtils()->getFileData(“data.mp3”, “rb”, &tmpSize);
FILE *fp = fopen(dbPath.c_str(), “wb”);
fwrite(xmlData, tmpSize, 1, fp);
fclose(fp);
db.open(dbPath.c_str());
@

cocos2d-2.0-x-2.0.4


#12

Thanks for that, I’ll give it a go on another project. I ended up getting round this by using an SQL script to populate the database. The problem I seemed to have was reading a binary file, and the script was the simplest way to get round this as it was a plain text file.


#13

i had the same problem trying to get the contents of a json file and finding parts missing. in the end it turned out that cclog just displays a portion of very long strings, which i was not aware of. so the file was actually perfectly loaded and also parsing went ok.