Public Member Functions | Private Member Functions | Private Attributes

hsm_Recall Class Reference
[G_new_group]

Object from class hsm_Recall can be created only if number of i_RecallList_t members is greter that 0. More...

#include <hsm_Recall.h>

Inheritance diagram for hsm_Recall:
Inheritance graph
[legend]
Collaboration diagram for hsm_Recall:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 hsm_Recall (df_BlockProxy *a_reader, hsm_JobElemVecMgr &a_jobElemVecMgr, UInt64_t a_pmJobId, const i_RecallList_t &a_recallList)
 ~hsm_Recall (void)
virtual void ProcRecFileHdr (const df_RecCmn_t *a_recCmn_p)
virtual void ProcRecFileEnd (const df_RecCmn_t *a_recCmn_p)
virtual void ProcRecBSStart (const df_RecCmn_t *a_recCmn_p)
virtual void ProcRecEmbData (const df_RecCmn_t *a_recCmn_p)
virtual void ProcRecRawData (const UInt8_t *a_block_p)
virtual void ProcRecBSEnd (const df_RecCmn_t *a_recCmn_p)
UInt64_t GetSizeOfRecalledData ()
UInt32_t GetNumOfRecalledFiles ()

Private Member Functions

bool IsFileFromDFforRecall (ivd_FileID_t a_fileID, ivd_MigrationID_t a_migID)
void RecallOfFileCanceled ()
void FileRecalledCheck ()

Private Attributes

 log_CLASSID_m
 Macro to add class name member s_className.
hsm_JobElemVecMgrm_jobElemVecMgr
const UInt64_t m_pmJobId
const i_RecallList_tm_recallList
int m_recalledUpToIdx
hsm_FileHeaderm_FH_p
hsm_RecallInfoGrpm_recallFileMgr_p
bool m_recallData
UInt64_t m_sizeOfRecalledData
UInt32_t m_numOfRecalledFiles
cmn_Time m_recallStartTime

Detailed Description

Object from class hsm_Recall can be created only if number of i_RecallList_t members is greter that 0.

The class does not check member count by itself.

How Group recall works? hsm_Recall object have df_BlockProxy reader that take care of data source hsm_JobElemVecMgr where FH's pointers of group recall job are stored i_RecallList_t recall list contail files that are packed in DF buffer.

Action diagram: record reader reads DF data and call ProcRec* virtual method that hsm_Recall handle it. ProcRecFileHdr() method checks if file has to be recalled. File has to be recall ed if its fileID and migID match to same fields in m_currentFileToRecall record. If file has to be recalled then m_FH_p is set. m_FH_p is set by using m_currentFileToRecall.fileIdx that point to FH in m_jobElemVecMgr Others ProcRec* methods check m_FH_p pointer to prepare and write data to destination. Whether to HSMFS or to some other place in case of recall into.

Recall into is not possible, because cannot asure that DF buffers comes in sequence. On Windows platform backup write is used.

Definition at line 71 of file hsm_Recall.h.


Constructor & Destructor Documentation

hsm_Recall::hsm_Recall ( df_BlockProxy a_reader,
hsm_JobElemVecMgr a_jobElemVecMgr,
UInt64_t  a_pmJobId,
const i_RecallList_t a_recallList 
)

Definition at line 49 of file hsm_Recall.cpp.

References log_FUNC_m.

hsm_Recall::~hsm_Recall ( void   ) 

Definition at line 70 of file hsm_Recall.cpp.

References dbg_LOW, cmn_Time::GetMilliTime(), cmn_Time::GetTime_t(), log_DBG_m, log_FUNC_m, m_numOfRecalledFiles, m_recallStartTime, and m_sizeOfRecalledData.

                            {
    log_FUNC_m(~hsm_Recall);
    cmn_Time now;
    cmn_Time recallTime(now - m_recallStartTime);
    UInt64_t msec = 
        UInt64_t(recallTime.GetTime_t()) * 1000 +
        recallTime.GetMilliTime();
    if (msec == 0) msec = 1;

    UInt64_t speed = (m_sizeOfRecalledData * 1000) / (msec * 1024) ;
    log_DBG_m (dbg_LOW, "#Chunks recalled: " << m_numOfRecalledFiles 
        << ", size: " << m_sizeOfRecalledData / 1024
        << " kB, " << speed << " kb/s");
}

Here is the call graph for this function:


Member Function Documentation

void hsm_Recall::FileRecalledCheck (  )  [private]

Definition at line 351 of file hsm_Recall.cpp.

References g_hsm_fhLock, hsm_FileHeader::GetInode(), hsm_RecallInfoGrp::IsFileRecalled(), hsm_RecallInfoGrp::IsRecallFailed(), ivd_USleep, log_FUNC_m, m_FH_p, m_recallFileMgr_p, and hsm_FileHeader::Recalled().

Referenced by ProcRecFileEnd().

                                   {
    log_FUNC_m(FileRecalledCheck);

    if (!m_recallFileMgr_p->IsFileRecalled()) {
        return;
    }
    ivd_GenInode_t inode = m_FH_p->GetInode();
    while (!g_hsm_fhLock.CanLockByID(inode)) {
        ivd_USleep(10000); // wait a 10 mili sec
    }

    m_FH_p->Recalled(!m_recallFileMgr_p->IsRecallFailed());

    g_hsm_fhLock.UnLockByID(inode);
}

Here is the call graph for this function:

Here is the caller graph for this function:

UInt32_t hsm_Recall::GetNumOfRecalledFiles (  )  [inline]

Definition at line 88 of file hsm_Recall.h.

References m_numOfRecalledFiles.

Referenced by i_HSM_i::EfficientRecall().

{ return m_numOfRecalledFiles; };

Here is the caller graph for this function:

UInt64_t hsm_Recall::GetSizeOfRecalledData (  )  [inline]

Definition at line 87 of file hsm_Recall.h.

References m_sizeOfRecalledData.

Referenced by i_HSM_i::EfficientRecall().

{ return m_sizeOfRecalledData; };

Here is the caller graph for this function:

bool hsm_Recall::IsFileFromDFforRecall ( ivd_FileID_t  a_fileID,
ivd_MigrationID_t  a_migID 
) [private]

Definition at line 250 of file hsm_Recall.cpp.

References hsm_JobElemVecMgr::At(), dbg_DETAIL, dbg_LOW, evt_ERROR, i_Recall_t::fileID, i_Recall_t::fileIdx, hsm_FileHeader::GetRecallInfoGrp(), ie_FATAL_ERROR, log_DBG_m, log_FUNC_m, log_WriteEvent(), m_FH_p, m_jobElemVecMgr, hsm_JobElemVecMgr::m_jobElemVecMgr_x, m_recalledUpToIdx, m_recallFileMgr_p, m_recallList, i_Recall_t::migrationID, and NULL.

Referenced by ProcRecFileHdr().

                                                                                       {
    log_FUNC_m(IsFileFromDFforRecall);
    
    //TODO improve searching by using block offset.
    
    log_DBG_m (dbg_DETAIL, "m_recallList.length(): " << m_recallList.length()
                        << ", a_fileID: " << a_fileID
                        << ", a_migID: " << a_migID
                        << ", sizeof i_Recall_t: " << sizeof(i_Recall_t)
                        << ", m_recalledUpToIdx: " << m_recalledUpToIdx);

    int listLen = m_recallList.length();
    const i_Recall_t *pi = &m_recallList[m_recalledUpToIdx];    
    log_DBG_m (dbg_DETAIL, "pi: " << hex << pi << dec);
    
    const i_Recall_t *pe = &m_recallList[0] + listLen;
    log_DBG_m (dbg_DETAIL, "pe: " << hex << pe << dec);
    for (;pi != pe; ++pi) {
        if (   pi->fileID      == a_fileID
            && pi->migrationID == a_migID) {

            log_DBG_m (dbg_DETAIL, "File needs to be recalled." << *pi);

            // NOTE:
            // Temporary fix to prevent crashing of HSM if
            // the fileIdx is set to -1.
            //
            if (pi->fileIdx == -1) {
                ostringstream sstr;
                sstr
                    << "File ID: " << a_fileID << ", mig ID: " << a_migID << ". "
                    << "Recall can't complete: file split already marked as processed. "
                    << "Please remount HSMFS and contact support.";
                log_WriteEvent(evt_ERROR, sstr.str(), "RECALL");
                break;
            }
            // End of temporary fix.

            {
                cmn_MutexLock l(m_jobElemVecMgr.m_jobElemVecMgr_x);
                m_FH_p = m_jobElemVecMgr.At(pi->fileIdx);

                if (m_FH_p == NULL) {
                    log_DBG_m(dbg_LOW,
                        "File header not found in job list. "
                        << "Already aborted/failed job for file ID: " << pi->fileID);
                    break;
                }

                m_recallFileMgr_p = m_FH_p->GetRecallInfoGrp();
                if (m_recallFileMgr_p == NULL) {
                    ostringstream sstr;
                    sstr << "Recall file is not set. " << *m_FH_p;
                    throw ivd_InternalError(ie_FATAL_ERROR, sstr.str(), true);        
                }
            } // lock to here

            if (listLen > 1) { // single recall, optimization not needed
                const_cast<i_Recall_t *>(pi)->fileIdx = -1; // this file is already handled
                while (   m_recalledUpToIdx < listLen
                    && m_recallList[m_recalledUpToIdx].fileIdx == -1) {
                    ++m_recalledUpToIdx;
                }
            }
            return true;
        }
    }
    return false;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_Recall::ProcRecBSEnd ( const df_RecCmn_t a_recCmn_p  )  [virtual]

Reimplemented from df_RecReader.

Definition at line 236 of file hsm_Recall.cpp.

References hsm_RecallInfoGrp::Close(), df_ST_DATA, log_FUNC_m, df_RecReader::m_curStreamType, m_FH_p, m_recallData, and m_recallFileMgr_p.

                                                           {
    log_FUNC_m(ProcRecBSEnd);

    if (  !m_FH_p
       ||  m_curStreamType != df_ST_DATA ) {
        return;
    }

//    const df_RecByteStreamEnd_t *bse_p = df_GetSpecificRec<df_RecByteStreamEnd_t>(a_recCmn_p);
    m_recallData = false;
    m_recallFileMgr_p->Close();
}

Here is the call graph for this function:

void hsm_Recall::ProcRecBSStart ( const df_RecCmn_t a_recCmn_p  )  [virtual]

Reimplemented from df_RecReader.

Definition at line 117 of file hsm_Recall.cpp.

References cmn_Num2Str(), dbg_DETAIL, dbg_NORM, df_ST_DATA, evt_WARNING, g_fs_api_p, cmn_File::GetFileType(), hsm_FileHeader::GetInode(), hsm_FileHeader::GetRecallJobIdx(), ivd_FileSystemAPI::GetRootPath(), log_DBG_m, log_FUNC_m, log_WriteEvent(), df_RecReader::m_curStreamType, m_FH_p, m_pmJobId, m_recallData, m_recallFileMgr_p, hsm_FileHeader::MakePath(), ntoh(), hsm_RecallInfoGrp::OpenF(), RecallOfFileCanceled(), hsm_RecallInfoGrp::SeekF(), and df_RecByteStreamStart_t::streamOffset.

                                                             {
    log_FUNC_m(ProcRecBSStart);
    
    if (  !m_FH_p
       ||  m_curStreamType != df_ST_DATA ) {
        return;
    }
    
    // always recall data stream...
    m_recallData = true; 
        
    const df_RecByteStreamStart_t *bss_p = df_GetSpecificRec<df_RecByteStreamStart_t>(a_recCmn_p);

    try {
        cmn_Path filePath = g_fs_api_p->GetRootPath() + m_FH_p->MakePath();
        cmn_File fileN(filePath);
        if ( fileN.GetFileType()  != ift_FILE){
            ostringstream sstr;
            sstr << ".inode/" + m_FH_p->GetInode();
            cmn_Path inodeP( sstr.str() );
            filePath = g_fs_api_p->GetRootPath() + inodeP;
        }
        log_DBG_m(dbg_NORM, "Opening file " << filePath << " for RECALL.");
        m_recallFileMgr_p->OpenF(filePath);
//        m_FH_p->OpenFile(*m_recallFileMgr_p, ivd_FS_File::e_Recall);
        // We got the data to recall
        log_DBG_m(dbg_DETAIL, "  Data of file " << filePath << " IS_FOUND. RECALL proceed.");
        // TODO Seek and set data chunk
        m_recallFileMgr_p->SeekF(m_pmJobId, ntoh(bss_p->streamOffset));
    }
    catch (ivd_SysError) {
        ostringstream sstr;
        sstr << "Cannot open a file for recall. File is probably deleted. "
             <<  m_FH_p->MakePath();
        log_WriteEvent(evt_WARNING,
            sstr.str(), "RECALL",
            m_pmJobId,
            cmn_Num2Str(m_FH_p->GetRecallJobIdx()));
        RecallOfFileCanceled();
        return;
    }
}

Here is the call graph for this function:

void hsm_Recall::ProcRecEmbData ( const df_RecCmn_t a_recCmn_p  )  [virtual]

Reimplemented from df_RecReader.

Definition at line 161 of file hsm_Recall.cpp.

References cmn_Num2Str(), df_RecEmbeddedData_t::data, df_ST_DATA, evt_WARNING, hsm_FileHeader::GetRecallJobIdx(), hsm_RecallInfoGrp::GetStreamSize(), df_RecCmn_t::GetVarDataNet(), hsm_RecallInfoGrp::IsRecallFailed(), log_FUNC_m, log_WriteEvent(), log_WRN_m, df_RecReader::m_curStreamType, m_FH_p, m_pmJobId, m_recallFileMgr_p, m_sizeOfRecalledData, hsm_FileHeader::MakePath(), ntoh(), RecallOfFileCanceled(), ivd_VarData_t::size, size, and hsm_RecallInfoGrp::WriteBufferedF().

                                                             {
    log_FUNC_m(ProcRecEmbData);

    if (  !m_FH_p
       ||  m_curStreamType != df_ST_DATA ) {
        return;
    }

    const df_RecEmbeddedData_t *embData_p = df_GetSpecificRec<df_RecEmbeddedData_t>(a_recCmn_p);
    
    Int32_t size = ntoh(embData_p->data.size);
    ivd_FileRetSize_t ret = 
        m_recallFileMgr_p->WriteBufferedF(
            a_recCmn_p->GetVarDataNet(embData_p->data), 
            size);

    if (ret != size) {
        ostringstream sstr;
        sstr << "HSMFS is full! Cannot recall a file: "
             << m_FH_p->MakePath();
        log_WriteEvent(evt_WARNING,
            sstr.str(), "RECALL",
            m_pmJobId,
            cmn_Num2Str(m_FH_p->GetRecallJobIdx()));
            
        sstr << endl << " Try to write: " << size
             << ", but " << ret << " bytes is written.";
        log_WRN_m(sstr.str());
        
        RecallOfFileCanceled();
        m_sizeOfRecalledData += size; // stream size is already increased by size.
        m_sizeOfRecalledData -= m_recallFileMgr_p->GetStreamSize();
        return;
    }
    
    if (!m_recallFileMgr_p->IsRecallFailed()) {
        m_sizeOfRecalledData += size;
    }
}

Here is the call graph for this function:

void hsm_Recall::ProcRecFileEnd ( const df_RecCmn_t a_recCmn_p  )  [virtual]

Reimplemented from df_RecReader.

Definition at line 103 of file hsm_Recall.cpp.

References hsm_RecallInfoGrp::Close(), FileRecalledCheck(), log_FUNC_m, m_FH_p, m_numOfRecalledFiles, m_recallData, and m_recallFileMgr_p.

Here is the call graph for this function:

void hsm_Recall::ProcRecFileHdr ( const df_RecCmn_t a_recCmn_p  )  [virtual]

Reimplemented from df_RecReader.

Definition at line 86 of file hsm_Recall.cpp.

References dbg_NORM, df_RecFile_t::idFile, df_RecFile_t::idMig, IsFileFromDFforRecall(), log_DBG_m, log_FUNC_m, and ntoh().

                                                            {
    log_FUNC_m(ProcRecFileHdr);
    const df_RecFile_t *file_p = df_GetSpecificRec<df_RecFile_t>(a_recCmn_p);

    ivd_FileID_t      fileID = (ivd_FileID_t)ntoh(file_p->idFile);
    ivd_MigrationID_t migID  = ntoh(file_p->idMig);

    if ( !IsFileFromDFforRecall(fileID, migID)) {
        
        log_DBG_m (dbg_NORM, "  Unpacker SKIPPED wrong fileId or MigID. " << endl <<
                                "\t\t\t\tFileID: " << fileID << " MigID: " << migID);
        return;
    }
//    cmn_Path fileName(g_rootPath + a_recCmn_p->GetVarDataNet(file_p->name));
}

Here is the call graph for this function:

void hsm_Recall::ProcRecRawData ( const UInt8_t a_block_p  )  [virtual]

Reimplemented from df_RecReader.

Definition at line 202 of file hsm_Recall.cpp.

References cmn_Num2Str(), df_ST_DATA, evt_WARNING, df_RecReader::GetBlkSize(), hsm_FileHeader::GetRecallJobIdx(), hsm_RecallInfoGrp::GetStreamSize(), hsm_RecallInfoGrp::IsRecallFailed(), log_FUNC_m, log_WriteEvent(), log_WRN_m, df_RecReader::m_curStreamType, m_FH_p, m_pmJobId, m_recallFileMgr_p, m_sizeOfRecalledData, hsm_FileHeader::MakePath(), RecallOfFileCanceled(), and hsm_RecallInfoGrp::WriteBufferedF().

                                                        {
    log_FUNC_m(ProcRecRawData);

    if (  !m_FH_p
       ||  m_curStreamType != df_ST_DATA ) {
        return;
    }

    UInt32_t blkSize = GetBlkSize();
    ivd_FileRetSize_t ret = m_recallFileMgr_p->WriteBufferedF(a_block_p, blkSize);
    if (ret != blkSize) {
        ostringstream sstr;
        sstr << "HSMFS is full! Cannot recall a file: "
             << m_FH_p->MakePath();
        log_WriteEvent(evt_WARNING,
            sstr.str(), "RECALL",
            m_pmJobId,
            cmn_Num2Str(m_FH_p->GetRecallJobIdx()));
            
        sstr << endl << " Try to write: " << blkSize
             << ", but " << ret << " bytes is written.";
        log_WRN_m(sstr.str());
        
        RecallOfFileCanceled();
        m_sizeOfRecalledData += blkSize; // stream size is already increased by size.
        m_sizeOfRecalledData -= m_recallFileMgr_p->GetStreamSize();
        return;
    }
    if (!m_recallFileMgr_p->IsRecallFailed()) {
        m_sizeOfRecalledData += blkSize;
    }
}

Here is the call graph for this function:

void hsm_Recall::RecallOfFileCanceled (  )  [private]

Definition at line 321 of file hsm_Recall.cpp.

References g_hsm_fhLock, hsm_FileHeader::GetInode(), hsm_FileHeader::IsDeleted(), ivd_USleep, log_FUNC_m, log_WRN_m, m_FH_p, m_recallFileMgr_p, hsm_FileHeader::MakePath(), hsm_RecallInfoGrp::SetRecallFailed(), and hsm_FileHeader::TruncateFile().

Referenced by ProcRecBSStart(), ProcRecEmbData(), and ProcRecRawData().

                                      {
    log_FUNC_m(RecallOfFileCanceled);
    
    ivd_GenInode_t inode = m_FH_p->GetInode();
    while (!g_hsm_fhLock.CanLockByID(inode)) {
        ivd_USleep(10000); // wait a 10 mili sec
    }

    try {
        // free already written data.
        m_FH_p->TruncateFile();
        // NOTE!!! Complete recall must be call when all splits are got.
        // So set file as recall-failed,
        // disable write to it
        // and call Recalled(false) at the end
        m_recallFileMgr_p->SetRecallFailed();
    }
    catch (ivd_SysError) {
        if (m_FH_p->IsDeleted()) { 
            ostringstream sstr;
            sstr << "Cannot cancel recall, file " 
                << m_FH_p->MakePath() << " is deleted. ";
            log_WRN_m(sstr.str());
        }       
    }

    g_hsm_fhLock.UnLockByID(inode);
}

Here is the call graph for this function:

Here is the caller graph for this function:


Member Data Documentation

Macro to add class name member s_className.

Reimplemented from df_RecReader.

Definition at line 94 of file hsm_Recall.h.

Definition at line 96 of file hsm_Recall.h.

Referenced by IsFileFromDFforRecall().

Definition at line 120 of file hsm_Recall.h.

Referenced by GetNumOfRecalledFiles(), ProcRecFileEnd(), and ~hsm_Recall().

Definition at line 99 of file hsm_Recall.h.

Referenced by ProcRecBSStart(), ProcRecEmbData(), and ProcRecRawData().

bool hsm_Recall::m_recallData [private]

Definition at line 117 of file hsm_Recall.h.

Referenced by ProcRecBSEnd(), ProcRecBSStart(), and ProcRecFileEnd().

Definition at line 106 of file hsm_Recall.h.

Referenced by IsFileFromDFforRecall().

Definition at line 101 of file hsm_Recall.h.

Referenced by IsFileFromDFforRecall().

Definition at line 121 of file hsm_Recall.h.

Referenced by ~hsm_Recall().

Definition at line 119 of file hsm_Recall.h.

Referenced by GetSizeOfRecalledData(), ProcRecEmbData(), ProcRecRawData(), and ~hsm_Recall().


The documentation for this class was generated from the following files: