Public Member Functions | Public Attributes | Private Member Functions | Private Attributes | Friends

fsc_DataCollectorStorage Class Reference
[FSC]

buffered file. More...

#include <fsc_DataCollectorStorage.h>

Inheritance diagram for fsc_DataCollectorStorage:
Inheritance graph
[legend]
Collaboration diagram for fsc_DataCollectorStorage:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 fsc_DataCollectorStorage (ivd_MediaKey_t a_mediumKey, ivd_MedVolNum_t a_medVolNum)
virtual ~fsc_DataCollectorStorage (void)
void CreateFile ()
void CheckHeader ()
void WriteToBuffer (const UInt8_t *a_buff_p)
 writes m_recSize bytes from a_buff to the end of the m_buffer
void ReadContent (ivd_FileLocationData_t_v_t &a_fileLoc_v)
bool GetNextRecord (ivd_FileLocationData_t &a_fileLocationData)

Public Attributes

 log_CLASSID_m

Private Member Functions

void InitBuffer ()
 reserve space for buffer
void Flush ()
 When buffer getting full a Flush is called.

Private Attributes

UInt8_tm_buffer
UInt8_tm_bufPos
UInt8_tm_bufEnd
UInt32_t m_recSize
UInt32_t m_bufSize
const ivd_MediaKey_t m_mediumKey
 RMDB medium key.
const ivd_MedVolNum_t m_medVolNum
 RMDB volume num.

Friends

class ut_fsc_DataCollectorStorage
class fsc_RedunCopyDataVolStorageWriter
class fsc_ReorgScanReorgVolStorage
class fsc_ReorgScanContentVolStorage

Detailed Description

buffered file.

Why buffered: When many objects of this class is used it is not possible to have all files open all the time. The buffer is used to avoid opening and closing the file for each write. When it is full then open file, flush and close operations are performed.

Definition at line 57 of file fsc_DataCollectorStorage.h.


Constructor & Destructor Documentation

fsc_DataCollectorStorage::fsc_DataCollectorStorage ( ivd_MediaKey_t  a_mediumKey,
ivd_MedVolNum_t  a_medVolNum 
)

Definition at line 63 of file fsc_DataCollectorStorage.cpp.

References dbg_DETAIL, InitBuffer(), log_DBG_m, and log_FUNC_m.

    :
    m_buffer(NULL),
    m_bufPos(NULL),
    m_bufEnd(NULL),

    m_recSize(sizeof(ivd_FileLocationData_t)),
    m_bufSize(fsc_DefBufSize_c),
    m_mediumKey(a_mediumKey),
    m_medVolNum(a_medVolNum)
{
    log_FUNC_m(fsc_DataCollectorStorage);
    log_DBG_m(dbg_DETAIL,"a_mediumKey: " << a_mediumKey <<
                         " a_medVolNum: " << a_medVolNum);

    InitBuffer();
}

Here is the call graph for this function:

fsc_DataCollectorStorage::~fsc_DataCollectorStorage ( void   )  [virtual]

Definition at line 82 of file fsc_DataCollectorStorage.cpp.

References Flush(), m_buffer, and NULL.

{
    Flush();
    if (m_buffer != NULL) {
        delete [] m_buffer;
    }
}

Here is the call graph for this function:


Member Function Documentation

void fsc_DataCollectorStorage::CheckHeader (  ) 

Definition at line 108 of file fsc_DataCollectorStorage.cpp.

References cmn_File::CloseF(), DATACOLLECTOR_FILE_HEADER_SIZE, dbg_DETAIL, fsc_ReorgFileHeader(), cmn_File::GetFullPathRef(), ie_DATA_CORRUPTION, ivd_Error, log_DBG_m, log_FUNC_m, cmn_File::OpenF(), and cmn_File::ReadF().

Referenced by fsc_RedunCopyDataVolStorageReader::fsc_RedunCopyDataVolStorageReader(), and i_ReorgJob_i::i_ReorgJob_i().

                                          {
    log_FUNC_m(CheckHeader);

    OpenF();
    vector<char> buffer;
    buffer.resize(DATACOLLECTOR_FILE_HEADER_SIZE+1);
    buffer[DATACOLLECTOR_FILE_HEADER_SIZE] = '\0';
    char *buff_p = &buffer[0];

    ivd_FileRetSize_t read = ReadF(buff_p, DATACOLLECTOR_FILE_HEADER_SIZE);
    log_DBG_m(dbg_DETAIL,"Data read: " << read << " header: " << buff_p);

    // compare fsc_FileHeader_c with fileHeader
    if  (strcmp(fsc_ReorgFileHeader, buff_p) != 0) {
        ostringstream msg;
        msg << "File Location Data File (" << GetFullPathRef() << ") is corrupted.";
        throw ivd_Error(ie_DATA_CORRUPTION, msg.str() );
    }
    CloseF();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fsc_DataCollectorStorage::CreateFile (  ) 
void fsc_DataCollectorStorage::Flush (  )  [private]

When buffer getting full a Flush is called.

Definition at line 192 of file fsc_DataCollectorStorage.cpp.

References cmn_File::CloseF(), dbg_DETAIL, fom_OPEN_ALWAYS, fom_WRITE, cmn_File::GetFullPathRef(), log_DBG_m, log_FUNC_m, m_buffer, m_bufPos, cmn_File::OpenF(), cmn_File::SeekEndF(), size, and cmn_File::WriteF().

Referenced by WriteToBuffer(), and ~fsc_DataCollectorStorage().

                                     {
    log_FUNC_m(Flush);

    UInt32_t size = (char*)m_bufPos - (char*)m_buffer;
    if (size > 0) {
        OpenF(fom_WRITE | fom_OPEN_ALWAYS); // binary | create | rdwr
        ivd_FilePosition_t pos = SeekEndF();
        log_DBG_m(dbg_DETAIL, GetFullPathRef() << " File pos " << pos
                            <<  " buff size " << size);
        WriteF(m_buffer, size);
        CloseF();
        m_bufPos  = m_buffer;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

bool fsc_DataCollectorStorage::GetNextRecord ( ivd_FileLocationData_t a_fileLocationData  ) 

Definition at line 246 of file fsc_DataCollectorStorage.cpp.

References ivd_FileLocationData_t::blockOffset, cmn_File::CloseF(), DATACOLLECTOR_FILE_HEADER_SIZE, dbg_DETAIL, ivd_FileLocationData_t::fileID, cmn_File::IsOpen(), log_DBG_m, log_FUNC_m, ivd_FileLocationData_t::migrationID, cmn_File::OpenF(), cmn_File::ReadF(), cmn_File::SeekF(), and ivd_FileLocationData_t::splitSize.

Referenced by fsc_DataLMgr::Remove().

                                                                                      {
    log_FUNC_m(GetNextRecord);

    if (!IsOpen()){
        OpenF();
        SeekF(DATACOLLECTOR_FILE_HEADER_SIZE);
    }

    ivd_FileRetSize_t ret = ReadF(&a_fileLocationData, sizeof(ivd_FileLocationData_t));
    log_DBG_m(dbg_DETAIL, "a_fileLocationData:  ret = " << ret
                       << "\n  blockOffset " << a_fileLocationData.blockOffset
                       << "\n  fileID      " << a_fileLocationData.fileID
                       << "\n  migrationID " << a_fileLocationData.migrationID
                       << "\n  splitSize   " << a_fileLocationData.splitSize);
    if (ret < (ivd_FileRetSize_t)sizeof(ivd_FileLocationData_t)) {
        CloseF();
        return true;
    } else {
        return false;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fsc_DataCollectorStorage::InitBuffer (  )  [private]

reserve space for buffer

Definition at line 151 of file fsc_DataCollectorStorage.cpp.

References assert, cmn_GetEnvVariable(), dbg_DETAIL, fsc_CollectorBufSize_c(), fsc_MaxBufSize_c, cmn_File::GetFullPathRef(), log_DBG_m, log_FUNC_m, log_WRN_m, m_bufEnd, m_buffer, m_bufPos, m_bufSize, and m_recSize.

Referenced by fsc_DataCollectorStorage().

                                          {
    log_FUNC_m(InitBuffer);

    assert(m_recSize > 0);
    assert(m_recSize <= 32768);

    string sBuffSize = cmn_GetEnvVariable(fsc_CollectorBufSize_c);
    if (!sBuffSize.empty()) {
        unsigned int buffSize = atoi(sBuffSize.c_str());
        m_bufSize = buffSize;
    }

    if (m_bufSize > fsc_MaxBufSize_c) {
        log_WRN_m("Buffes size for data collection is greater than allowed. "
                << " want size = " <<  m_bufSize
                << " max size will be used = " << fsc_MaxBufSize_c);
        m_bufSize = fsc_MaxBufSize_c;
    }
    if (m_bufSize < m_recSize) {
        log_WRN_m("Buffes size for data collection is less than recordSize. "
                << " want size = " << m_bufSize
                << " record size will be used = " << m_recSize);
        m_bufSize = m_recSize;
    }

    // bufferSize must be multiplier of record size.
    m_bufSize /= m_recSize;
    m_bufSize *= m_recSize;

    m_buffer = new  UInt8_t[m_bufSize];

    m_bufPos  = m_buffer;
    m_bufEnd  = m_buffer + m_bufSize;
    log_DBG_m(dbg_DETAIL, GetFullPathRef()
                          <<  " bufSize =" << m_bufSize
                          <<  ", recSize =" << m_recSize
                          <<  ", bufPos_p =" << hex << (void*)m_bufPos << dec
                          <<  ", bufEnd_p =" << hex << (void*)m_bufEnd << dec);
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fsc_DataCollectorStorage::ReadContent ( ivd_FileLocationData_t_v_t a_fileLoc_v  ) 

Definition at line 208 of file fsc_DataCollectorStorage.cpp.

References cmn_File::CloseF(), DATA_REC_SIZE, DATACOLLECTOR_FILE_HEADER_SIZE, dbg_DETAIL, dbg_LOW, dbg_NORM, ivd_BaseException::GetError(), ie_DATA_CORRUPTION, ivd_Error, log_DBG_m, log_FUNC_m, cmn_File::OpenF(), cmn_File::ReadF(), cmn_File::SeekF(), and cmn_File::StatF().

Referenced by fsc_FLSPerVolume::fsc_FLSPerVolume().

                                                                                 {
    log_FUNC_m(ReadContent);

    ivd_FileInfo_t info;
    ivd_FileSize_t dataFileSize;
    try {
        StatF(info);
        dataFileSize = info.size;
        OpenF();
    }
    catch (const ivd_SysError &se) {
        if (se.GetError() == ie_FILE_ERROR) {
            log_DBG_m(dbg_LOW, "File does not exist." << se);
            throw ivd_Error(ie_DATA_CORRUPTION, "File Location Data File is missing.");
        }
        throw;
    }

    log_DBG_m(dbg_NORM, "Data file size: " << dataFileSize);

    UInt32_t fileCountRead = (dataFileSize - DATACOLLECTOR_FILE_HEADER_SIZE)/DATA_REC_SIZE;
    if (fileCountRead > 0) {
        a_fileLoc_v.resize(fileCountRead);
        log_DBG_m(dbg_DETAIL,   "Will Read:" << fileCountRead <<
                                ", a_fileLoc_v.size()" << a_fileLoc_v.size());

        SeekF(DATACOLLECTOR_FILE_HEADER_SIZE);
        ivd_FileRetSize_t read = ReadF(&a_fileLoc_v[0], dataFileSize - DATACOLLECTOR_FILE_HEADER_SIZE);

        log_DBG_m(dbg_DETAIL,"Data read: " << read);
    } else {
        log_DBG_m(dbg_DETAIL,"no files listed");
    }
    CloseF();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void fsc_DataCollectorStorage::WriteToBuffer ( const UInt8_t a_buff_p  ) 

writes m_recSize bytes from a_buff to the end of the m_buffer

Definition at line 130 of file fsc_DataCollectorStorage.cpp.

References Flush(), ie_EXCEED_MAX_SIZE, log_FUNC_m, m_bufEnd, m_buffer, m_bufPos, m_bufSize, and m_recSize.

Referenced by fsc_ReorgScanDataVolStorage::Write().

                                                                    {
    log_FUNC_m(Write);
    // Note buffer size must be multipier of record size.
    UInt8_t* pos = m_bufPos + m_recSize;
    if (pos > m_bufEnd) {
        ostringstream sstr;
        sstr << "Buffer possition exceed buffer size. RecSize = " << m_recSize
            << " BufSize = " << m_bufSize
            << " BufPos = " << hex << m_bufPos
            << " BufEnd = " << m_bufEnd
            << " buffer = " << m_buffer << dec;
        throw ivd_InternalError(ie_EXCEED_MAX_SIZE, sstr.str(), true);
    }
    memcpy(m_bufPos, a_buff_p, m_recSize);
    m_bufPos = pos;
    if (pos >= m_bufEnd) {
        Flush();
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:


Friends And Related Function Documentation

friend class fsc_RedunCopyDataVolStorageWriter [friend]

Reimplemented in fsc_RedunCopyDataVolStorage.

Definition at line 59 of file fsc_DataCollectorStorage.h.

friend class fsc_ReorgScanContentVolStorage [friend]

Reimplemented in fsc_ReorgScanDataVolStorage.

Definition at line 61 of file fsc_DataCollectorStorage.h.

friend class fsc_ReorgScanReorgVolStorage [friend]

Reimplemented in fsc_ReorgScanDataVolStorage.

Definition at line 60 of file fsc_DataCollectorStorage.h.

friend class ut_fsc_DataCollectorStorage [friend]

Definition at line 58 of file fsc_DataCollectorStorage.h.


Member Data Documentation

Definition at line 92 of file fsc_DataCollectorStorage.h.

Referenced by InitBuffer(), and WriteToBuffer().

Definition at line 91 of file fsc_DataCollectorStorage.h.

Referenced by Flush(), InitBuffer(), and WriteToBuffer().

Definition at line 95 of file fsc_DataCollectorStorage.h.

Referenced by InitBuffer(), and WriteToBuffer().

Definition at line 94 of file fsc_DataCollectorStorage.h.

Referenced by InitBuffer(), and WriteToBuffer().


The documentation for this class was generated from the following files: