Public Member Functions | Public Attributes | Protected Member Functions | Protected Attributes | Friends

fio_RelFileTrans Class Reference
[Classes for large file i/o, relative file, and DataBase]

#include <fio_RelFileTrans.h>

Inheritance diagram for fio_RelFileTrans:
Inheritance graph
[legend]
Collaboration diagram for fio_RelFileTrans:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 fio_RelFileTrans (bool a_readOnly, const int a_fileID, cmn_Path a_path, string a_fileName, const int a_recordSize, const Int16_t a_recordStructVersion, const UInt16_t a_numOfDifVector=ivd_MAX_NUM_OF_DIF_VECTOR_IN_REL_FILES_d, const UInt16_t a_smallestVectorSize=1, const bool a_accessToIdx0=false, int a_recommendUVBufferSize=fio_RECOMENDED_UV_BUFFER_SIZE_d)
virtual ~fio_RelFileTrans ()
virtual void SyncRelFileObj ()
 Sync relative file object with data on disk after transaction reply.
void SetDataBase (fio_DataBase *a_database_p)
 set DB
void SetRecommendBufferSize (int a_size)
 Set recommended buffer size to all UV, Tuning reason.
void SetRecommendBufferSize (int a_numOfRec, int a_size)
 Set recommended buffer size to particular UV, Tuning reason.
int GetRecommendBufferSize (unsigned int a_numOfRec)
 get the recomend buffer size for vector with proper number of record
void PutVectorToBufferAndFlush (const fio_RecordIDX_t a_index, unsigned int a_numOfRec)
 Put released vector to UVbuffer.
void RemoveVectorFromBuffer (const fio_RecordIDX_t a_index, unsigned int a_numOfRec)
 This method is used only by replied transaction at start up.
void AddVectorToBuffer (const fio_RecordIDX_t a_index, unsigned int a_numOfRec)
 This method is used only by reply method in transaction.
void ReleaseVectorIdx (const fio_RecordIDX_t a_index, unsigned int a_numOfRec, fio_Transaction *a_transaction_p)
 Add released vector to free list inside transaciton.
ivd_RecordIDX_t GetVectorIDX (unsigned int a_numOfRec, fio_Transaction *a_transaction_p)
 Get one unused proper sized vectore.
void GetBunchOfVectorIDXes (ivd_RecIdxVec_t &a_recIdxVec, unsigned int a_numOfRec, fio_Transaction *a_transaction_p)
 Get a bunch of unused vectors same size and put its indeces to vector of ivd_RecordIDX_t set as element as vector size.
void Synchronize (fio_Transaction *a_transaction_p)
 All unused vectors in lists have to be saved, called by DB.

Public Attributes

 log_CLASSID_m
 Macro for class name member.

Protected Member Functions

virtual int PosWrite (ivd_FilePosition_t a_position, const void *a_buffer_p, ivd_FileBufSize_t a_size, fio_Transaction *a_transaction_p) const
 PosWrite method is virtual.

Protected Attributes

fio_DataBasem_database_p
 The relative file is not responsible to destruct databse.
fio_uvListMgr_v_t m_uvListMgr_v
 vector of list managers of unused vectors

Friends

class fio_uvListMgr

Detailed Description

Definition at line 62 of file fio_RelFileTrans.h.


Constructor & Destructor Documentation

fio_RelFileTrans::fio_RelFileTrans ( bool  a_readOnly,
const int  a_fileID,
cmn_Path  a_path,
string  a_fileName,
const int  a_recordSize,
const Int16_t  a_recordStructVersion,
const UInt16_t  a_numOfDifVector = ivd_MAX_NUM_OF_DIF_VECTOR_IN_REL_FILES_d,
const UInt16_t  a_smallestVectorSize = 1,
const bool  a_accessToIdx0 = false,
int  a_recommendUVBufferSize = fio_RECOMENDED_UV_BUFFER_SIZE_d 
)

Definition at line 53 of file fio_RelFileTrans.cpp.

References assert, c_fio_maxRelFileHeadLen, dbg_DETAIL, log_DBG_m, log_FUNC_m, fio_RelFile::m_listOfUV_v, fio_RelFile::m_loUVHeaderLen, fio_RelFile::m_numOfDifVector, fio_RelFile::m_recordSize, and m_uvListMgr_v.

        :
        fio_RelFile(a_readOnly,
                    a_fileID, 
                    a_path, 
                    a_fileName, 
                    a_recordSize, 
                    a_recordStructVersion,
                    a_numOfDifVector,
                    a_smallestVectorSize,
                    a_accessToIdx0),
        m_database_p(NULL) {

    log_FUNC_m(fio_RelFileTrans);

    log_DBG_m(dbg_DETAIL, 
                    "fileID                " << a_fileID  << endl
                 << "path                  " << a_path  << endl
                 << "fileName              " << a_fileName  << endl
                 << "recordSize            " << a_recordSize  << endl 
                 << "recordStructVersion   " << a_recordStructVersion << endl
                 << "numOfDifVector        " << a_numOfDifVector << endl
                 << "smallestVectorSize    " << a_smallestVectorSize << endl
                 << "recommendUVBufferSize " << a_recommendUVBufferSize );
    
    
    //  reserve space for UV list manager instances from one element deafault constructor
    //  const fio_uvListMgr listMgr();
    m_uvListMgr_v.resize(m_numOfDifVector); 

    //  initialize all UV list managers now
    for (unsigned int i = 0; i < m_numOfDifVector; i++) {
        m_uvListMgr_v[i].Init(this,  // which relative file use manager
                             &m_listOfUV_v[i], // addres of header that is stored in file header
                              i + a_smallestVectorSize,  // size of vector (vector index + smalest vector)
                              i * m_loUVHeaderLen + c_fio_maxRelFileHeadLen, // file offset
                              a_recommendUVBufferSize); // recomend buffer size
    }       

    assert(sizeof(fio_UnusedRec_t) > m_recordSize); 
}  

fio_RelFileTrans::~fio_RelFileTrans (  )  [virtual]

Definition at line 108 of file fio_RelFileTrans.cpp.

References log_FUNC_m.


Member Function Documentation

void fio_RelFileTrans::AddVectorToBuffer ( const fio_RecordIDX_t  a_index,
unsigned int  a_numOfRec 
)

This method is used only by reply method in transaction.

Its behaviour is to add vector to buffer directly without trig flush vectors to disk.

See also:
PutVectorToBuffer

Definition at line 168 of file fio_RelFileTrans.cpp.

References dbg_DETAIL, fio_File::GetFileName(), ie_INVALID_ARG, log_DBG_m, log_FUNC_m, fio_RelFile::m_smallestVectorSize, and m_uvListMgr_v.

{
//    log_FUNC_m(AddVectorToBuffer);

    int listIDX = a_numOfRec - m_smallestVectorSize;
    if (listIDX < 0) {
        log_FUNC_m(AddVectorToBuffer);
        log_DBG_m(dbg_DETAIL, 
                    "File    " << GetFileName() << endl <<
                    "VecIdx  " << a_index << endl <<
                    "Vecsize " << a_numOfRec << endl);     
        throw ivd_InternalError(ie_INVALID_ARG, "Too small vector. VecSize = ", true);
    }
//    log_DBG_m(dbg_DETAIL, "AddVect of size " << a_numOfRec);
    m_uvListMgr_v[listIDX].AddVectorToBuff(a_index);
}

Here is the call graph for this function:

void fio_RelFileTrans::GetBunchOfVectorIDXes ( ivd_RecIdxVec_t &  a_recIdxVec,
unsigned int  a_numOfRec,
fio_Transaction a_transaction_p 
)

Get a bunch of unused vectors same size and put its indeces to vector of ivd_RecordIDX_t set as element as vector size.

Definition at line 228 of file fio_RelFileTrans.cpp.

References ie_INVALID_ARG, log_FUNC_m, fio_RelFile::m_smallestVectorSize, and m_uvListMgr_v.

Referenced by fsc_DataLMgr::GetBunchOfFileIDs().

                                                             {

    ivd_RecIdxVec_i recIdxVectIter = a_recIdxVec.begin(); // set a iterator

    //a_transaction_p->GetBunchOfVectorIDXes(a_recIdxVec,
    //                                       recIdxVectIter,
    //                                       GetFileID(), 
    //                                       a_numOfRec);

    // try to get vector from appropriate list manager
    if (recIdxVectIter != a_recIdxVec.end()) { // not filled yet
        int listIDX = a_numOfRec - m_smallestVectorSize;
        if (listIDX < 0) {
            log_FUNC_m(GetBunchOfVectorIDXes);
            throw ivd_InternalError(ie_INVALID_ARG, "Too small vector.", true);
        }
        m_uvListMgr_v[listIDX].GetBunchOfVectorIDXes(a_recIdxVec, 
                                                     recIdxVectIter, 
                                                     a_transaction_p);
    }
}

Here is the caller graph for this function:

int fio_RelFileTrans::GetRecommendBufferSize ( unsigned int  a_numOfRec  ) 

get the recomend buffer size for vector with proper number of record

Definition at line 322 of file fio_RelFileTrans.cpp.

References ie_INVALID_ARG, log_FUNC_m, fio_RelFile::m_smallestVectorSize, and m_uvListMgr_v.

                                                                    {

    int listIDX = a_numOfRec - m_smallestVectorSize;
    if (listIDX < 0) {
        log_FUNC_m(GetRecommendBufferSize);
        throw ivd_InternalError(ie_INVALID_ARG, "Too small vector.", true);
    }
    return m_uvListMgr_v[listIDX].GetRecommendBufferSize();
}

fio_RecordIDX_t fio_RelFileTrans::GetVectorIDX ( unsigned int  a_numOfRec,
fio_Transaction a_transaction_p 
)

Get one unused proper sized vectore.

First try to find it inside transaction, then try get it form UV buffer

Definition at line 189 of file fio_RelFileTrans.cpp.

References fio_File::GetFileID(), ie_FATAL_ERROR, ie_INVALID_ARG, log_FUNC_m, fio_RelFile::m_smallestVectorSize, m_uvListMgr_v, and fio_Transaction::VectorIsUsed().

Referenced by fio_CirFileTrans::NewWriteBackBlock(), fsc_SplitMgr::Write2DB(), fsc_CopyMgr::Write2DB(), and fio_Vector::Write2DB().

{
//    log_FUNC_m(GetVectorIDX);
//    log_DBG_m(dbg_DETAIL, "Search new vector of size = " << a_numOfRec << endl);

    // try to find freed vectore inside current transaction
// to much effort for so small hits
//    fio_RecordIDX_t newIndex = a_transaction_p->GetVectorIDX(GetFileID(), a_numOfRec);

    // try to get vector from appropriate list manager
    fio_RecordIDX_t newIndex;
//    if (newIndex == 0) {
        int listIDX = a_numOfRec - m_smallestVectorSize;
        if (listIDX < 0) {
            log_FUNC_m(GetVectorIDX);
            throw ivd_InternalError(ie_INVALID_ARG, "Too small vector.", true);
        }
        newIndex = m_uvListMgr_v[listIDX].GetVectorIDX();
        if (newIndex != 0) {
            // vector is removed from buffer, so is used now
            a_transaction_p->VectorIsUsed(GetFileID(), a_numOfRec, newIndex);
        }
        else {
            log_FUNC_m(GetVectorIDX);
            throw ivd_InternalError(ie_FATAL_ERROR, "Can't find free vector", true);
        }
//        log_DBG_m(dbg_DETAIL, " Got idx from m_uvListMgr_v[" <<listIDX << "] " << newIndex << endl);
//    }
//    else {
//        log_DBG_m(dbg_DETAIL, " Got idx from transaction " << newIndex << endl);
//    }

    return newIndex;
}

Here is the call graph for this function:

Here is the caller graph for this function:

int fio_RelFileTrans::PosWrite ( ivd_FilePosition_t  a_position,
const void *  a_buffer_p,
ivd_FileBufSize_t  a_size,
fio_Transaction a_transaction_p 
) const [protected, virtual]

PosWrite method is virtual.

In this RelFileTrans class put alteration to transaction.

Reimplemented from fio_RelFile.

Definition at line 256 of file fio_RelFileTrans.cpp.

References fio_File::GetFileID(), fio_File::GetFileName(), ie_REL_FILE_NO_TRANS_OBJ, fio_Transaction::InsertAlteration(), ivd_Error, log_FUNC_m, and NULL.

Referenced by fio_CirFileTrans::fio_CirFileTrans(), fio_CirFileTrans::NewWriteBackBlock(), fio_uvListMgr::ReadUV(), fio_CirFileTrans::UpdateReadFrontBlockIdx(), fio_CirFileTrans::UpdateWriteBackBlockIdx(), and fio_uvListMgr::WriteUV().

                                                       {

    if (a_transaction == NULL) {
        log_FUNC_m(PosWrite);
        string name = GetFileName();
        throw ivd_Error(ie_REL_FILE_NO_TRANS_OBJ, 
            "Can't write to file " + name + " without transaction object.", true);
    }
    else {
        // reserve space for data on heap
//        char *data_p = new char[a_size]; 
        //log_DBG_m(dbg_DETAIL, "Write : pos = " << a_position << endl <<
        //    cmn_HexDump(a_buffer_p, a_size, 16, false) );

//        memcpy(data_p, a_buffer_p, a_size);

        //fio_Alteration *alteration_p = new fio_Alteration(const_cast<fio_RelFileTrans*>(this), 
                                //                        a_position, 
                                //                        a_size,
        //                                                  data_p);

        a_transaction->InsertAlteration(GetFileID(), a_position, a_size, (const UInt8_t*)a_buffer_p);  
    }
    return a_size;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fio_RelFileTrans::PutVectorToBufferAndFlush ( const fio_RecordIDX_t  a_index,
unsigned int  a_numOfRec 
)

Put released vector to UVbuffer.

This is done by transaction after commit When max buffer size is reached then half vectors is flushed to file.

Definition at line 113 of file fio_RelFileTrans.cpp.

References cmn_Num2Str(), fio_File::GetFileName(), ie_INVALID_ARG, log_FUNC_m, fio_RelFile::m_smallestVectorSize, and m_uvListMgr_v.

                                                          {
//    log_FUNC_m(PutVectorToBufferAndFlush);
    // put vector to appropriate list manager
    int listIDX = a_numOfRec - m_smallestVectorSize;
    if (listIDX < 0) {
        log_FUNC_m(PutVectorToBufferAndFlush);
        throw ivd_InternalError(ie_INVALID_ARG, "Too small vector. size = "
                + cmn_Num2Str(a_numOfRec)
                + " idx = " + cmn_Num2Str(a_index)
                + " file = " + GetFileName(), true);
    }
//    log_DBG_m(dbg_DETAIL, "PutVect of size " << a_numOfRec);
    m_uvListMgr_v[listIDX].PutVector(a_index);
}

Here is the call graph for this function:

void fio_RelFileTrans::ReleaseVectorIdx ( const fio_RecordIDX_t  a_index,
unsigned int  a_numOfRec,
fio_Transaction a_transaction_p 
)

Add released vector to free list inside transaciton.

Definition at line 135 of file fio_RelFileTrans.cpp.

References fio_File::GetFileID(), log_ERR_m, log_FUNC_m, and fio_Transaction::VectorIsFreed().

Referenced by fio_CirFileTrans::GetRFIdxFromNextBlock(), hdb_RemoveName(), hsm_FileHeader::RemoveFromActiveList(), fsc_nsStream::RemoveLeftVectors(), hsm_FileHeader::RemoveNameFromHSMDB(), hsm_FHrelc::Truncate(), fsc_SplitMgr::Write2DB(), fsc_GenerationMgr::Write2DB(), fsc_CopyMgr::Write2DB(), and fio_Vector::Write2DB().

{
    if (a_index == 0) {
        log_FUNC_m(ReleaseVectorIdx);
        log_ERR_m("Can't release vector. Idx = 0  RelFileID = " << (int)GetFileID()
               << "  vectorSize = " << a_numOfRec);        
    }
    else {
        // put vector to transaction free list
        a_transaction_p->VectorIsFreed(GetFileID(), a_numOfRec, a_index);
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fio_RelFileTrans::RemoveVectorFromBuffer ( const fio_RecordIDX_t  a_index,
unsigned int  a_numOfRec 
)

This method is used only by replied transaction at start up.

Transaction remove used vectore from buffer before is commited. It has the same functionality as at normal work when DB need some vector and pull it out before commit. Vector must be in buffer fore sure, othervise internal error occure.

Definition at line 153 of file fio_RelFileTrans.cpp.

References ie_INVALID_ARG, log_FUNC_m, fio_RelFile::m_smallestVectorSize, and m_uvListMgr_v.

{

    int listIDX = a_numOfRec - m_smallestVectorSize;
    if (listIDX < 0) {
        log_FUNC_m(RemoveVectorFromBuffer);
        throw ivd_InternalError(ie_INVALID_ARG, "Too small vector.", true);
    }
    m_uvListMgr_v[listIDX].RemoveVectorFromBuff(a_index);
}

void fio_RelFileTrans::SetDataBase ( fio_DataBase a_database_p  )  [inline]

set DB

Definition at line 111 of file fio_RelFileTrans.h.

Referenced by fio_RelFileTransLstMgr::SetDataBase().

                                                             { 
        m_database_p = a_database_p;
    };

Here is the caller graph for this function:

void fio_RelFileTrans::SetRecommendBufferSize ( int  a_numOfRec,
int  a_size 
)

Set recommended buffer size to particular UV, Tuning reason.

Definition at line 310 of file fio_RelFileTrans.cpp.

References ie_INVALID_ARG, log_FUNC_m, fio_RelFile::m_smallestVectorSize, m_uvListMgr_v, SetRecommendBufferSize(), and size.

                                                                        {

    int listIDX = a_numOfRec - m_smallestVectorSize;
    if (listIDX < 0) {
        log_FUNC_m(SetRecommendBufferSize(numOfRec, size));
        throw ivd_InternalError(ie_INVALID_ARG, "Too small vector.", true);
    }
    m_uvListMgr_v[listIDX].SetRecommendBufferSize(a_size);
}

Here is the call graph for this function:

void fio_RelFileTrans::SetRecommendBufferSize ( int  a_size  ) 

Set recommended buffer size to all UV, Tuning reason.

Definition at line 299 of file fio_RelFileTrans.cpp.

References m_uvListMgr_v.

Referenced by SetRecommendBufferSize().

                                                        {
//    log_FUNC_m(SetRecommendBufferSize(size));

    fio_uvListMgr_v_t::iterator p;
    for (p = m_uvListMgr_v.begin(); p != m_uvListMgr_v.end(); p++) {
         p->SetRecommendBufferSize(a_size);
    }
}

Here is the caller graph for this function:

void fio_RelFileTrans::Synchronize ( fio_Transaction a_transaction_p  ) 

All unused vectors in lists have to be saved, called by DB.

Definition at line 288 of file fio_RelFileTrans.cpp.

References log_FUNC_m, and m_uvListMgr_v.

                                                                    {
    log_FUNC_m(Synchronize);

    fio_uvListMgr_v_t::iterator p;
    for (p = m_uvListMgr_v.begin(); p != m_uvListMgr_v.end(); p++) {
         p->Synchronize(a_transaction_p);
    }
}

virtual void fio_RelFileTrans::SyncRelFileObj (  )  [inline, virtual]

Sync relative file object with data on disk after transaction reply.

Reimplemented from fio_RelFile.

Reimplemented in fio_CirFileTrans.

Definition at line 83 of file fio_RelFileTrans.h.

References fio_RelFile::SyncRelFileObj().

Here is the call graph for this function:


Friends And Related Function Documentation

friend class fio_uvListMgr [friend]

Reimplemented from fio_RelFile.

Definition at line 63 of file fio_RelFileTrans.h.


Member Data Documentation

Macro for class name member.

Reimplemented from fio_RelFile.

Reimplemented in fio_CirFileTrans, and fio_VarLenRFT.

Definition at line 107 of file fio_RelFileTrans.h.

The relative file is not responsible to destruct databse.

Definition at line 83 of file fio_RelFileTrans.h.

Referenced by fio_CirFileTrans::GetRFIdxFromNextBlock(), fio_CirFileTrans::NewWriteBackBlock(), fio_uvListMgr::PutVector(), and fio_uvListMgr::ReadUV().


The documentation for this class was generated from the following files: