Public Member Functions | Static Public Member Functions | Public Attributes | Private Member Functions | Private Attributes | Static Private Attributes

fio_JourFile Class Reference
[Classes for large file i/o, relative file, and DataBase]

<Description:> A more elaborate class description that describes all aspects of class usage etc. More...

#include <fio_JourFile.h>

Inheritance diagram for fio_JourFile:
Inheritance graph
[legend]
Collaboration diagram for fio_JourFile:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 fio_JourFile (fio_permDBinfo_t *a_permDBinfo_p, bool a_readOnly, ivd_FileID_t a_fileID, cmn_Path a_path, string a_name="")
fio_JourHandleGetJourWriteHandle (ivd_FileBufSize_t a_transSize)
 get journal handle for write from JourFile instance
fio_PermJourInfo_tGetJourInfo ()
void DecRefCount ()
 decrement reference counter, if counter is zero then jourFile obj is removed
void IncRefCount ()
 increment reference counter
fio_JourHandleGetJourReadHandle (ivd_FilePosition_t a_position)
 get journal handle for read from JourFile instance
ivd_FilePosition_t GetFreeTransPosition ()
void ResetFreeTransPosition (ivd_FilePosition_t a_freeTransPosition)
 Used to set new file size after reaply of transactions.
void DumpDamagedTrans (ostream &os, char *data, ivd_FilePosition_t transFilePos, fio_JourHandle *jourHnd)
void Dump (ostream &os, fio_SizeOff_v &a_sizeOff_v, fio_jourDumpArguments_t &a_jda)

Static Public Member Functions

static string JourFileName (ivd_FileID_t a_jourFileNum)
 Return a name for journal file, it's fileID represent in hex number compound "j" + "xxxxxxxx" + c_fio_JourFileExt where xxxxxxxx is ascii hex presentation of a_jourFileNum.
static ivd_FileID_t JourFileNumber (const string &a_fileName)
 Convert jour file name to jour file number as fileID.

Public Attributes

 log_CLASSID_m
 Description of a class member.

Private Member Functions

void TestFreedVector (ostream &os, fio_UV_t &uv)
void TestUsedVector (ostream &os, fio_UV_t &uv)

Private Attributes

ivd_FilePosition_t m_freeTransPosition
 where next transaction could to be storen inside jour file
cmn_Mutex m_freeTransPosition_x
 exclusive lock for m_freeTransPosition
fio_PermJourInfo_t m_permJourInfo
 journal permanent information
int m_refCount
 number of active journal handle
cmn_Mutex m_refCount_x
 mutex for reference counter

Static Private Attributes

static fsc_RecordIDX_vv_t m_uvChkDeposit

Detailed Description

<Description:> A more elaborate class description that describes all aspects of class usage etc.

Author:
Dejan Volk Lupo, HERMES SoftLab
See also:
<reference>

Definition at line 175 of file fio_JourFile.h.


Constructor & Destructor Documentation

fio_JourFile::fio_JourFile ( fio_permDBinfo_t a_permDBinfo_p,
bool  a_readOnly,
ivd_FileID_t  a_fileID,
cmn_Path  a_path,
string  a_name = "" 
)

Definition at line 155 of file fio_JourFile.cpp.

References c_fio_maxJourFileHeadLen, dbg_DETAIL, fio_PermJourInfo_t::Init(), ivd_NULLCHK_m, log_DBG_m, log_FUNC_m, m_freeTransPosition, m_permJourInfo, m_uvChkDeposit, s_className, cmn_File::SeekEndF(), and fio_PermJourInfo_t::WriteInfo().

                       :
                       fio_File(a_readOnly,
                                a_fileID,
                                a_path,
                                a_name.length() == 0
                                    ? JourFileName(a_fileID)
                                    : a_name),
// fix bug 507
                       m_refCount(0)
// end
{
    log_FUNC_m(fio_JourFile);

    m_freeTransPosition = SeekEndF();  // check if file is new
    if (m_freeTransPosition < c_fio_maxJourFileHeadLen) {
        m_freeTransPosition = c_fio_maxJourFileHeadLen;

        ivd_NULLCHK_m(a_permDBinfo_p, fio_PermDBinfoMgr::s_className);
        m_permJourInfo.Init(a_permDBinfo_p);
        m_permJourInfo.WriteInfo(this); // write header on this file
    }
    else {
        // get header from this file
        m_permJourInfo.Init(this);
    }
    log_DBG_m(dbg_DETAIL, "Create journal file " << hex << a_fileID << dec );
    m_uvChkDeposit.resize(20); // per 20 relative files
}

Here is the call graph for this function:


Member Function Documentation

void fio_JourFile::DecRefCount (  ) 

decrement reference counter, if counter is zero then jourFile obj is removed

Definition at line 193 of file fio_JourFile.cpp.

References cmn_File::GetFullPathRef(), ie_FATAL_ERROR, cmn_Mutex::Lock(), log_FUNC_m, m_refCount, m_refCount_x, and cmn_Mutex::Unlock().

Referenced by fio_JourMgr::AddNewJourFile(), fio_JourHandle::~fio_JourHandle(), and fio_JourMgr::~fio_JourMgr().

                               {
    m_refCount_x.Lock();
    if (m_refCount == 0) {
        log_FUNC_m(DecRefCount);
        m_refCount_x.Unlock();
        throw ivd_InternalError(ie_FATAL_ERROR,
               "Reference counter went negative." + GetFullPathRef());
    }
    m_refCount--;
    if (m_refCount == 0) {
        m_refCount_x.Unlock();
        delete this;
        return;
    }
    m_refCount_x.Unlock();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fio_JourFile::Dump ( ostream &  os,
fio_SizeOff_v a_sizeOff_v,
fio_jourDumpArguments_t a_jda 
)

Definition at line 307 of file fio_JourFile.cpp.

References alloca(), fio_Alter_t::bufSize, c_fio_MaxAlterDataBufSize, cmn_HexDump(), fio_Alter_t::Dump(), fio_UV_t::Dump(), DumpDamagedTrans(), fio_jourDumpArguments_t::fileID, fio_Alter_t::fileID, fio_Alter_t::filePosition, fio_JourHandle::GetCurrentPosition(), ivd_BaseException::GetError(), fio_File::GetFileName(), GetFreeTransPosition(), GetJourReadHandle(), fio_jourDumpArguments_t::idx, fio_EndTransaction_t::Init(), fio_StartTransaction_t::Init(), fio_JourHandle::JourRead(), fio_JourHandle::JourSkip(), log_FUNC_m, m_freeTransPosition, recIdx, fio_JourHandle::ResetPosition(), fio_JourHandle::SetStartPositionIsCurrent(), fio_JourHandle::SetTransNumber(), fio_JourHandle::SetTransSize(), fio_jourDumpArguments_t::showData, fio_jourDumpArguments_t::showVector, TestFreedVector(), TestUsedVector(), fio_jourDumpArguments_t::testUVIdx, fio_StartTransaction_t::TransNumber, and fio_StartTransaction_t::TransSize.

Referenced by JournalDump().

                                                        {
    log_FUNC_m(Dump);
    os << "Journal file  " << GetFileName() <<
          "\n    free transaction position" << m_freeTransPosition << '\n';
    fio_JourHandle *jourHnd = GetJourReadHandle(0);
    jourHnd->SetTransNumber(0);

    fio_StartTransaction_t startTrans;
    fio_EndTransaction_t   endTrans;

    // reserved data buffer on stact of size c_fio_MaxAlterDataBufSize
    // !!! DO NOT reserve stack in loops cause allocate only per function.
    char *data = (char*)(alloca(c_fio_MaxAlterDataBufSize));
//    ivd_NULLCHK_m(data, "alloca(c_fio_MaxAlterDataBufSize)");

    ivd_FilePosition_t fileSize = GetFreeTransPosition();
    while (jourHnd->GetCurrentPosition() < fileSize) {
        ivd_FilePosition_t transFilePos = jourHnd->GetCurrentPosition();
        jourHnd->SetStartPositionIsCurrent();
        try {
            startTrans.Init(*jourHnd);
            jourHnd->SetTransNumber(startTrans.TransNumber);
        }
        catch (ivd_Error &err) {
            if (err.GetError() == ie_NOT_STARTTRANS_T) {
                // when read start trans failed then jourHeader file pos is reset.
                os << "Pos = "  << transFilePos
                   << " damaged start trans:\n" << startTrans;
                DumpDamagedTrans(os, data, transFilePos, jourHnd);
                continue;
            }
            else {
                throw(err);
            }
        }

        jourHnd->SetTransSize(startTrans.TransSize);

        os <<  "Pos = "  << transFilePos << " " << startTrans;

        fio_UV_t uv;
        unsigned int i;

        if (a_jda.showVector) {
            os << "     File ID  VectorSize       Index\n" <<
                  "------------------------------------\n";
        }

        for (i = 0; i < startTrans.FreedVector; i++) {
            jourHnd->JourRead(&uv, sizeof(uv));
            if (a_jda.testUVIdx) {
                TestFreedVector(os, uv);
            }
            if (a_jda.showVector) {
                uv.Dump(os, a_jda.idx);
            }
        }

        if (a_jda.showVector) {
            os << "     File ID  VectorSize       Index\n" <<
                  "------------------------------------\n";
        }
        for (i = 0; i < startTrans.UsedVector; i++) {
            jourHnd->JourRead(&uv, sizeof(uv));
            if (a_jda.testUVIdx) {
                TestUsedVector(os, uv);
            }
            if (a_jda.showVector) {
                uv.Dump(os, a_jda.idx);
            }
        }

        os << "Alt.numb" << "  Jour pos"
           << " RFID"<< "        IDX" << "  File pos" << "  Buf size" << '\n';
        ivd_AlterNumber_t jourAlterNum = 0; // will be checked after all alteration are insert
        while (jourAlterNum < startTrans.AlterNumber) {  // read all alteration
            try {
                fio_Alter_t alter(jourHnd);
                jourAlterNum++;

                int recSize = 0;
                if (alter.fileID <= a_sizeOff_v.size() ) {
                    recSize = a_sizeOff_v[alter.fileID].recSize;
                }
                int offset = a_sizeOff_v[alter.fileID].relFileOffset;
                ivd_RecordIDX_t recIdx =  (alter.filePosition - offset) / recSize;

                if ( (  (alter.filePosition < offset)
                     || (a_jda.idx == 0)
                     || (a_jda.idx == recIdx) )
                   &&(  (a_jda.fileID == 0)
                     || (a_jda.fileID == alter.fileID) ) ) {
                    os <<  setw(8) << jourAlterNum << setw(10) << jourHnd->GetCurrentPosition();

                    alter.Dump(os, recSize, offset);
                    if (a_jda.showData) {
                        ivd_FileBufSize_t dataSize = alter.bufSize;
                        ivd_FileBufSize_t maxFitBuff = (c_fio_MaxAlterDataBufSize / recSize) * recSize;
                        int blockCount = 0;

                        while (dataSize > 0) {
                            ivd_FileBufSize_t  dataToRead  = dataSize > maxFitBuff
                                                        ? maxFitBuff
                                                        : dataSize;

                            jourHnd->JourRead(data, dataToRead);
                            blockCount++;
                            os << cmn_HexDump(data, dataToRead, recSize, false);
                            dataSize -= dataToRead;
                        }
                    }
                    else {
                        jourHnd->JourSkip(alter.bufSize);
                    }
                }
                else {
                    // skip data
                    jourHnd->JourSkip(alter.bufSize);
                }
            }
            catch (ivd_Error &err) {
                if (err.GetError() == ie_NOT_ALTER_T) {
                    os << "    Alteration damaged " << endl;
                    jourHnd->ResetPosition();
                    DumpDamagedTrans(os, data, transFilePos, jourHnd);
                    goto nextTransaction;
                }
                else {
                    throw(err);
                }
            }
         }  // while (m_alterNumber <= jourAlterNum)  // loop until all alteration is read
        // read end thransaction header
        try {
            endTrans.Init(*jourHnd);
            if (  startTrans == endTrans ) {
                os << startTrans.TransNumber << " OK.\n" << endl;
            }
            else {
                os << "Transaction is NOT valid  start and end transaction header are different.\n\n";
            }
        }
        catch (ivd_Error &err) {
            if (err.GetError() == ie_NOT_ENDTRANS_T) {
                    os << "    EndTrans header damaged" << endl << endTrans;
                    DumpDamagedTrans(os, data, transFilePos, jourHnd);
                    goto nextTransaction;
            }
            else {
                throw(err);
            }
        }
nextTransaction: ;

        // when transactionis succesfuly finished then
        // transaciton number in jourHnd must be updated
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fio_JourFile::DumpDamagedTrans ( ostream &  os,
char *  data,
ivd_FilePosition_t  transFilePos,
fio_JourHandle jourHnd 
)

Definition at line 283 of file fio_JourFile.cpp.

References c_fio_MaxAlterDataBufSize, cmn_HexDump(), fio_JourHandle::FindNextTransHdr(), and fio_JourHandle::JourRead().

Referenced by Dump().

                                                             {
    // find next transaction header
    ivd_FilePosition_t pos = jourHnd->FindNextTransHdr(false);

    ivd_FileBufSize_t dataSize = pos - transFilePos;
    os << " Data size = " << dataSize << " nex positon = " << pos << endl;
    int blockCount = 0;
    while (dataSize > 0) {
        ivd_FileBufSize_t  dataToRead  = dataSize > c_fio_MaxAlterDataBufSize
                                       ? c_fio_MaxAlterDataBufSize
                                       : dataSize;

        jourHnd->JourRead(data, dataToRead);
        blockCount++;
        os << cmn_HexDump(data, dataToRead, 16, true);
        dataSize -= dataToRead;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

ivd_FilePosition_t fio_JourFile::GetFreeTransPosition (  )  [inline]

Definition at line 248 of file fio_JourFile.h.

Referenced by Dump(), fio_JourMgr::GetJourWriteHandle(), fio_JourMgr::Replay(), and fio_JourMgr::SetLastSyncPosition().

{ return m_freeTransPosition; };

Here is the caller graph for this function:

fio_PermJourInfo_t& fio_JourFile::GetJourInfo (  )  [inline]

Definition at line 236 of file fio_JourFile.h.

{ return m_permJourInfo; };

fio_JourHandle * fio_JourFile::GetJourReadHandle ( ivd_FilePosition_t  a_position  ) 

get journal handle for read from JourFile instance

Definition at line 237 of file fio_JourFile.cpp.

References c_fio_maxJourFileHeadLen.

Referenced by Dump(), and fio_JourMgr::GetJourReadHandle().

                                                                             {
//    log_FUNC_m(GetJourReadHandle);

    ivd_FilePosition_t pos = a_position;
    if (pos == 0) {
        pos = c_fio_maxJourFileHeadLen;
    }
    fio_JourHandle* jourHnd_p = new fio_JourHandle(this, pos, 0);

    return jourHnd_p;
}

Here is the caller graph for this function:

fio_JourHandle * fio_JourFile::GetJourWriteHandle ( ivd_FileBufSize_t  a_transSize  ) 

get journal handle for write from JourFile instance

Definition at line 221 of file fio_JourFile.cpp.

References m_freeTransPosition, and m_freeTransPosition_x.

Referenced by fio_JourMgr::GetJourWriteHandle().

                                                                              {
//    log_FUNC_m(GetJourWriteHandle);

    ivd_FilePosition_t pos;
    {
        cmn_MutexLock l(m_freeTransPosition_x);
        pos = m_freeTransPosition;          // get current possition
        m_freeTransPosition += a_transSize; // increment it
    }
    fio_JourHandle* jourHnd_p = new fio_JourHandle(this, pos, a_transSize);

    return jourHnd_p;
}

Here is the caller graph for this function:

void fio_JourFile::IncRefCount (  ) 

increment reference counter

Definition at line 212 of file fio_JourFile.cpp.

References log_FUNC_m, m_refCount, and m_refCount_x.

Referenced by fio_JourMgr::AddNewJourFile(), fio_JourHandle::fio_JourHandle(), and fio_JourMgr::fio_JourMgr().

Here is the caller graph for this function:

string fio_JourFile::JourFileName ( ivd_FileID_t  a_jourFileNum  )  [static]

Return a name for journal file, it's fileID represent in hex number compound "j" + "xxxxxxxx" + c_fio_JourFileExt where xxxxxxxx is ascii hex presentation of a_jourFileNum.

Definition at line 272 of file fio_JourFile.cpp.

References c_fio_JourFileExt, log_FUNC_m, and snprintf.

Referenced by fio_JourMgr::fio_JourMgr().

                                                            {
    log_FUNC_m(JourFileName);

    char sNum[10];
    snprintf(sNum, 10, "%08x", a_jourFileNum);
    string seqNumber(sNum);
    return "j"  + seqNumber + c_fio_JourFileExt;
}

Here is the caller graph for this function:

ivd_FileID_t fio_JourFile::JourFileNumber ( const string &  a_fileName  )  [static]

Convert jour file name to jour file number as fileID.

Definition at line 257 of file fio_JourFile.cpp.

References ie_INV_FILE_NAME, and ivd_Error.

Referenced by fio_JourMgr::fio_JourMgr(), JournalDump(), and fio_JourMgr::RemoveArhivedJourFile().

                                                                   {

    char *p;
    //    journal file name format is jxxxxxxxx.ljr
    //                                               skip the 'j'
    ivd_FileID_t fileID = strtoul(a_fileName.data() + 1, &p, 16);
    if (*p != '.') {
        throw ivd_Error(ie_INV_FILE_NAME, "Missing '.' in journal file name.", true);
    };

    return fileID;
}

Here is the caller graph for this function:

void fio_JourFile::ResetFreeTransPosition ( ivd_FilePosition_t  a_freeTransPosition  ) 

Used to set new file size after reaply of transactions.

Definition at line 250 of file fio_JourFile.cpp.

References log_FUNC_m, m_freeTransPosition, and m_freeTransPosition_x.

Referenced by fio_JourHandle::FindNextTransHdr().

                                                                                {
    log_FUNC_m(SetFreeTransPosition);
    cmn_MutexLock l(m_freeTransPosition_x);
    m_freeTransPosition = a_freeTransPosition;
}

Here is the caller graph for this function:

void fio_JourFile::TestFreedVector ( ostream &  os,
fio_UV_t uv 
) [private]

Definition at line 469 of file fio_JourFile.cpp.

References fio_UV_t::m_fileID, fio_UV_t::m_idx, m_uvChkDeposit, and fio_UV_t::m_vectorSize.

Referenced by Dump().

                                                            {
    vector<ivd_RecordIDX_t> &uvPerRelFile = m_uvChkDeposit[uv.m_fileID];

    ivd_RecordIDX_t *ip = &(*uvPerRelFile.begin());
    ivd_RecordIDX_t *ep = &(*uvPerRelFile.end());
    ivd_RecordIDX_t toIdx = uv.m_idx + uv.m_vectorSize;
    for(; ip != ep; ++ip) {
        if (*ip == uv.m_idx) {
            os << "FATAL ERROR at freed UV with idx " << uv.m_idx
               << ", with size " << uv.m_vectorSize
               << " already exist in free list."
               << " Relative file ID " << (int)uv.m_fileID
               << endl;
        }
        else { // check cross linked index
            for (ivd_RecordIDX_t idx = uv.m_idx; idx < toIdx; ++idx) {
                if (*ip == idx) {
                    os << "FATAL ERROR at freed UV with idx " << uv.m_idx
                    << ", with size " << uv.m_vectorSize
                    << " Freed vector is cross linked with vector on index " << idx
                    << " Relative file ID " << (int)uv.m_fileID
                    << endl;
                }
            }
        }
    }
    uvPerRelFile.push_back(uv.m_idx);
}

Here is the caller graph for this function:

void fio_JourFile::TestUsedVector ( ostream &  os,
fio_UV_t uv 
) [private]

Definition at line 499 of file fio_JourFile.cpp.

References fio_UV_t::m_fileID, fio_UV_t::m_idx, m_uvChkDeposit, and fio_UV_t::m_vectorSize.

Referenced by Dump().

                                                           {
    vector<ivd_RecordIDX_t> &uvPerRelFile = m_uvChkDeposit[uv.m_fileID];

    ivd_RecordIDX_t *ip = &(*uvPerRelFile.begin());
    ivd_RecordIDX_t *ep = &(*uvPerRelFile.end());
    for(; ip != ep; ++ip) {
        if (*ip == uv.m_idx) {
            if (ip != --ep) { // endel become last
                //os << "idx " << *ip << " is used "
                //   << " overwritten by idx " << *ep << endl;
                *ip = *ep;
            }
//            os << " removed idx " << uvPerRelFile.back() << endl;
            uvPerRelFile.pop_back();
            return;
        }
    }
    os << "FATAL ERROR at used UV with idx " << uv.m_idx
        << ", with size " << uv.m_vectorSize
        << " not found in free list."
        << " Relative file ID " << (int)uv.m_fileID
        << endl;
}

Here is the caller graph for this function:


Member Data Documentation

Description of a class member.

See also:
<reference>
<reference> Macro to add class name member s_className.

Reimplemented from fio_File.

Definition at line 221 of file fio_JourFile.h.

where next transaction could to be storen inside jour file

Definition at line 188 of file fio_JourFile.h.

Referenced by Dump(), fio_JourFile(), GetJourWriteHandle(), and ResetFreeTransPosition().

exclusive lock for m_freeTransPosition

Definition at line 191 of file fio_JourFile.h.

Referenced by GetJourWriteHandle(), and ResetFreeTransPosition().

journal permanent information

Definition at line 194 of file fio_JourFile.h.

Referenced by fio_JourFile().

int fio_JourFile::m_refCount [private]

number of active journal handle

Definition at line 198 of file fio_JourFile.h.

Referenced by DecRefCount(), and IncRefCount().

mutex for reference counter

Definition at line 200 of file fio_JourFile.h.

Referenced by DecRefCount(), and IncRefCount().

Definition at line 208 of file fio_JourFile.h.

Referenced by fio_JourFile(), TestFreedVector(), and TestUsedVector().


The documentation for this class was generated from the following files: