fsc_RawReorgScanDataCollector Class Reference
[FSC]

#include <fsc_RawReorgScanDataCollector.h>

Inheritance diagram for fsc_RawReorgScanDataCollector:

[legend]

Collaboration diagram for fsc_RawReorgScanDataCollector:

[legend]

List of all members.

---

Detailed Description

Scan through FSC to collect file splits.

Collected data are stored into m_fileLocationData member of ivd_FileLocationData_t structure type. Method ProcRecMedPosMgr() is lief in FSC generation tree hierarchy and when called a complete information about one data location (split) is got. Data location information is stored to particular medium volume storage. Medium volume storage is searched inside m_cache member by using cache element key. If medium volume storage doesn't exist then is generated.

Definition at line 43 of file fsc_RawReorgScanDataCollector.h.

Public Member Functions
	fsc_RawReorgScanDataCollector (const string &a_partName, ivd_Time32_t a_expDate, UInt32_t a_numGener)
	~fsc_RawReorgScanDataCollector (void)
virtual bool	ProcFileID (ivd_RecordIDX_t a_fileID)
	Called when new file in process.
virtual bool	ProcVecGeneration (fsc_Generation_v_t &a_gen_v)
	Called when all generations are packed in vector.
virtual bool	ProcRecGeneration (fsc_Generation_t &a_gen, UInt32_t a_status)
	Called for each generation.
virtual bool	ProcRecSplit (fsc_Split_t &a_split)
	Called when generation or copy has splits.
virtual bool	ProcRecMedPosMgr (fsc_MediaPosition_t &a_medPos)
	Called when media position is known.
UInt32_t	GetMedPosCounter ()
Public Attributes
	log_CLASSID_m
Private Member Functions
bool	IsReorgCandidate ()
UInt64_t	KeyGen (ivd_MediaKey_t a_mediaKey, ivd_MedVolNum_t a_medVolNum)
	Key to searc element in stx_cache.
Private Attributes
const string	m_partName
const ivd_MigrationID_t	m_expGen
const UInt32_t	m_maxNumOfGen
ivd_FileLocationData_t	m_fileLocationData
	Collected data are stored to this structure.
bool	m_removeGen
	Set if file is deleted and last data mig is oldest than m_expGener.
UInt32_t	m_fileDataGenCount
	count only generations that contaion data.
ivd_MigrationID_t	m_migIdOfLastGen
ivd_MigrationID_t	m_migIdOfLastDataGen
ivd_FileSize_t	m_streamSize
	stream size stored in generation
UInt8_t	m_dataType
	each generation set this member
stx_Cache	m_reorgCache
	MedVolume file storage.
stx_Cache	m_contentCache
UInt32_t	m_medPosCounter
Friends
class	ut_fsc_RawReorgScanDataCollector

---

Constructor & Destructor Documentation

fsc_RawReorgScanDataCollector::fsc_RawReorgScanDataCollector	(	const string &	a_partName,
		ivd_Time32_t	a_expDate,
		UInt32_t	a_numGener
	)

Definition at line 40 of file fsc_RawReorgScanDataCollector.cpp.

References cmn_CleanDir(), dbg_DETAIL, fsc_GetReorgDir(), log_DBG_m, log_FUNC_m, and m_partName.

00043     :
00044     m_partName(a_partName),
00045     m_expGen((ivd_MigrationID_t)a_expDate << 8),
00046     m_maxNumOfGen(a_numGen),
00047     m_removeGen(false),
00048     m_fileDataGenCount(0),
00049     m_migIdOfLastGen(0),
00050     m_migIdOfLastDataGen(0),
00051     m_streamSize(0),
00052     m_dataType(0),
00053     m_reorgCache(1024 * 64), // cache size
00054     m_contentCache(1024 * 64), // cache size
00055     m_medPosCounter(0)
00056 {
00057     log_FUNC_m(fsc_RawReorgScanDataCollector);
00058     cmn_Path reorgDir = fsc_GetReorgDir(m_partName);
00059     log_DBG_m(dbg_DETAIL, "Create dir for reorg scan results : '" << reorgDir << "'");
00060     cmn_CleanDir(reorgDir);
00061 }
//============================================================================//

Here is the call graph for this function:

fsc_RawReorgScanDataCollector::~fsc_RawReorgScanDataCollector

(

void

)

[inline]

Definition at line 50 of file fsc_RawReorgScanDataCollector.h.

{};

---

Member Function Documentation

bool fsc_RawReorgScanDataCollector::ProcFileID

(

ivd_RecordIDX_t

a_fileID

)

[virtual]

Called when new file in process.

Reimplemented from fsc_RawCollector.

Definition at line 64 of file fsc_RawReorgScanDataCollector.cpp.

References dbg_DETAIL, ivd_FileLocationData_t::fileID, log_DBG_m, log_FUNC_m, m_fileDataGenCount, m_fileLocationData, m_migIdOfLastDataGen, m_migIdOfLastGen, and m_removeGen.

                                                                       {
    log_FUNC_m(ProcFileID);
    m_fileLocationData.fileID = a_fileID;
    log_DBG_m(dbg_DETAIL, "New locData.  fileID = " << m_fileLocationData.fileID);
    m_fileDataGenCount   = 0;
    m_migIdOfLastGen     = 0;
    m_migIdOfLastDataGen = 0;
    m_removeGen          = false;
    return true;
}

bool fsc_RawReorgScanDataCollector::ProcVecGeneration

(

fsc_Generation_v_t &

a_gen_v

)

[virtual]

Called when all generations are packed in vector.

Definition at line 76 of file fsc_RawReorgScanDataCollector.cpp.

References fsc_NextEntity_t::dataType, dbg_DETAIL, ivd_FileLocationData_t::fileID, ivd_DATA_d, log_DBG_m, log_FUNC_m, fsc_RawCollector::m_curNSNameType, m_expGen, m_fileLocationData, m_removeGen, fsc_Generation_t::migrationID, fsc_Generation_t::nextEntity, and nit_DELETED.

                                                                                 {
    log_FUNC_m(ProcVecGeneration);
    if (m_curNSNameType == nit_DELETED) {
        // check if all generations need to be removed
        // Last data generation is older than m_expGen
        bool isDir(true);
        fsc_Generation_t *ip = &(*a_gen_v.begin());
        fsc_Generation_t *ep = &(*a_gen_v.end());
        fsc_Generation_t *delGenIp = &(*a_gen_v.begin());
        for (; ip < ep; ip++) {
            if (ip->nextEntity.dataType & ivd_DATA_d) { // find first generation with data
                // Directory generations can't contain data
                isDir = false;
                if (ip->migrationID < m_expGen) {
                    m_removeGen = true;
                    delGenIp = ip;
                }
                break;
            }
        }

        // If isDir is true then delGenIp points to a_gen_v.begin()
        if ( (isDir) && (delGenIp->migrationID < m_expGen) ) {
                m_removeGen = true;
        }
        if (m_removeGen) {
            log_DBG_m(dbg_DETAIL, "File with fileID:" << m_fileLocationData.fileID
                        << " is deleted and its data was migrated before expiration date, so remove it."
                        << " migID:" << delGenIp->migrationID);
        }
    }
    return true;
}

bool fsc_RawReorgScanDataCollector::ProcRecGeneration	(	fsc_Generation_t &	a_gen,
		UInt32_t	a_status
	)			[virtual]

Called for each generation.

Reimplemented from fsc_RawCollector.

Definition at line 111 of file fsc_RawReorgScanDataCollector.cpp.

References fsc_NextEntity_t::dataType, ivd_DATA_d, m_dataType, m_fileDataGenCount, m_fileLocationData, m_migIdOfLastDataGen, m_migIdOfLastGen, m_streamSize, fsc_Generation_t::migrationID, ivd_FileLocationData_t::migrationID, fsc_Generation_t::nextEntity, fsc_Generation_t::size, and ivd_FileLocationData_t::splitSize.

                                                                                  {
//    log_FUNC_m(ProcRecGeneration);      
    m_fileLocationData.migrationID = a_gen.migrationID;
    m_streamSize = m_fileLocationData.splitSize = a_gen.size;
    m_dataType   = a_gen.nextEntity.dataType;
    if (m_migIdOfLastGen == 0) {
        m_migIdOfLastGen = a_gen.migrationID;
    }

    if (m_dataType & ivd_DATA_d) {
        m_fileDataGenCount++;
        if (m_migIdOfLastDataGen == 0) {
            m_migIdOfLastDataGen = a_gen.migrationID;
        }
    }
    return true;
}

bool fsc_RawReorgScanDataCollector::ProcRecSplit

(

fsc_Split_t &

a_split

)

[virtual]

Called when generation or copy has splits.

Reimplemented from fsc_RawCollector.

Definition at line 131 of file fsc_RawReorgScanDataCollector.cpp.

References fsc_Split_t::dataSize, m_fileLocationData, and ivd_FileLocationData_t::splitSize.

                                                                     {
//    log_FUNC_m(ProcRecSplit);      
    m_fileLocationData.splitSize   = a_split.dataSize;
    return true;
}

bool fsc_RawReorgScanDataCollector::ProcRecMedPosMgr

(

fsc_MediaPosition_t &

a_medPos

)

[virtual]

Called when media position is known.

Reimplemented from fsc_RawCollector.

Definition at line 138 of file fsc_RawReorgScanDataCollector.cpp.

References fsc_MediaPosition_t::blockOffset, ivd_FileLocationData_t::blockOffset, dbg_DETAIL, ivd_FileLocationData_t::fileID, stx_Cache::Get(), stx_Cache::Insert(), IsReorgCandidate(), KeyGen(), log_DBG_m, log_FUNC_m, m_contentCache, m_fileLocationData, m_medPosCounter, m_partName, m_reorgCache, m_streamSize, fsc_MediaPosition_t::mediaKey, fsc_MediaPosition_t::medVolNum, ivd_FileLocationData_t::migrationID, NULL, ivd_FileLocationData_t::splitSize, and fsc_ReorgScanDataVolStorage::Write().

                                                                                 {
    log_FUNC_m(ProcRecMedPosMgr);      

    m_medPosCounter++;
    
    m_fileLocationData.blockOffset = a_medPos.blockOffset;

    UInt64_t key = KeyGen(a_medPos.mediaKey, a_medPos.medVolNum);

    fsc_ReorgScanReorgVolStorage *reorgVolStor = (fsc_ReorgScanReorgVolStorage*)(m_reorgCache.Get(key));
    if (reorgVolStor == NULL) {
        reorgVolStor = new fsc_ReorgScanReorgVolStorage(key,
                                                        m_partName,
                                                        a_medPos.mediaKey, 
                                                        a_medPos.medVolNum,
                                                        true);
        m_reorgCache.Insert(reorgVolStor);
    }
    
    fsc_ReorgScanContentVolStorage *contentVolStor = (fsc_ReorgScanContentVolStorage*)(m_contentCache.Get(key));
    if (contentVolStor == NULL) {
        contentVolStor = new fsc_ReorgScanContentVolStorage(key,
                                                            m_partName,
                                                            a_medPos.mediaKey, 
                                                            a_medPos.medVolNum,
                                                            true);
        m_contentCache.Insert(contentVolStor);
    }
    //write all files to content
    contentVolStor->Write(&m_fileLocationData);
    
    if (IsReorgCandidate()){
        reorgVolStor->Write(&m_fileLocationData);
    }
    
    log_DBG_m(dbg_DETAIL, "Write locData = "
                        << "\n  blockOffset " << m_fileLocationData.blockOffset
                        << "\n  fileID      " << m_fileLocationData.fileID 
                        << "\n  migrationID " << m_fileLocationData.migrationID
                        << "\n  splitSize   " << m_fileLocationData.splitSize);

    // set to whole size, that other copies of same generation has splitSize set properlly
    m_fileLocationData.splitSize   = m_streamSize;
    return true;
}

Here is the call graph for this function:

UInt32_t fsc_RawReorgScanDataCollector::GetMedPosCounter

(

)

[inline]

Definition at line 68 of file fsc_RawReorgScanDataCollector.h.

{ return m_medPosCounter;};

bool fsc_RawReorgScanDataCollector::IsReorgCandidate

(

)

[inline, private]

Definition at line 72 of file fsc_RawReorgScanDataCollector.h.

References ivd_DATA_d, m_dataType, m_expGen, m_fileDataGenCount, m_fileLocationData, m_maxNumOfGen, m_migIdOfLastDataGen, m_migIdOfLastGen, m_removeGen, and ivd_FileLocationData_t::migrationID.

Referenced by ProcRecMedPosMgr().

                            {
        return     !m_removeGen                 // not for removed AND
                        // newest generation is always to reorg
                && (  (m_migIdOfLastGen     == m_fileLocationData.migrationID) 
                        // newest data generation too
                   || (m_migIdOfLastDataGen == m_fileLocationData.migrationID) 
                   || (  (m_dataType & ivd_DATA_d) // contain data 
                      && (m_fileLocationData.migrationID >= m_expGen) // and is fresh
                      && (  (m_maxNumOfGen == 0)   // and if max number of gen is defined 
                         || (m_maxNumOfGen >= m_fileDataGenCount) // check amount of newest data gen
                         )
                      )
                   );    
    };

Here is the caller graph for this function:

UInt64_t fsc_RawReorgScanDataCollector::KeyGen	(	ivd_MediaKey_t	a_mediaKey,
		ivd_MedVolNum_t	a_medVolNum
	)			[inline, private]

Key to searc element in stx_cache.

Definition at line 88 of file fsc_RawReorgScanDataCollector.h.

Referenced by ProcRecMedPosMgr().

                                                    {
        return UInt64_t(a_mediaKey << 8) + a_medVolNum;
    };

Here is the caller graph for this function:

---

Friends And Related Function Documentation

friend class ut_fsc_RawReorgScanDataCollector [friend]

Definition at line 44 of file fsc_RawReorgScanDataCollector.h.

---

Member Data Documentation

fsc_RawReorgScanDataCollector::log_CLASSID_m

Reimplemented from fsc_RawCollector.

Definition at line 68 of file fsc_RawReorgScanDataCollector.h.

const string fsc_RawReorgScanDataCollector::m_partName [private]

Definition at line 91 of file fsc_RawReorgScanDataCollector.h.

Referenced by fsc_RawReorgScanDataCollector(), and ProcRecMedPosMgr().

const ivd_MigrationID_t fsc_RawReorgScanDataCollector::m_expGen [private]

Definition at line 95 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate(), and ProcVecGeneration().

const UInt32_t fsc_RawReorgScanDataCollector::m_maxNumOfGen [private]

Definition at line 96 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate().

ivd_FileLocationData_t fsc_RawReorgScanDataCollector::m_fileLocationData [private]

Collected data are stored to this structure.

Definition at line 99 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate(), ProcFileID(), ProcRecGeneration(), ProcRecMedPosMgr(), ProcRecSplit(), and ProcVecGeneration().

bool fsc_RawReorgScanDataCollector::m_removeGen [private]

Set if file is deleted and last data mig is oldest than m_expGener.

Definition at line 102 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate(), ProcFileID(), and ProcVecGeneration().

UInt32_t fsc_RawReorgScanDataCollector::m_fileDataGenCount [private]

count only generations that contaion data.

Definition at line 105 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate(), ProcFileID(), and ProcRecGeneration().

ivd_MigrationID_t fsc_RawReorgScanDataCollector::m_migIdOfLastGen [private]

Definition at line 107 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate(), ProcFileID(), and ProcRecGeneration().

ivd_MigrationID_t fsc_RawReorgScanDataCollector::m_migIdOfLastDataGen [private]

Definition at line 108 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate(), ProcFileID(), and ProcRecGeneration().

ivd_FileSize_t fsc_RawReorgScanDataCollector::m_streamSize [private]

stream size stored in generation

Definition at line 111 of file fsc_RawReorgScanDataCollector.h.

Referenced by ProcRecGeneration(), and ProcRecMedPosMgr().

UInt8_t fsc_RawReorgScanDataCollector::m_dataType [private]

each generation set this member

Definition at line 114 of file fsc_RawReorgScanDataCollector.h.

Referenced by IsReorgCandidate(), and ProcRecGeneration().

stx_Cache fsc_RawReorgScanDataCollector::m_reorgCache [private]

MedVolume file storage.

Definition at line 117 of file fsc_RawReorgScanDataCollector.h.

Referenced by ProcRecMedPosMgr().

stx_Cache fsc_RawReorgScanDataCollector::m_contentCache [private]

Definition at line 118 of file fsc_RawReorgScanDataCollector.h.

Referenced by ProcRecMedPosMgr().

UInt32_t fsc_RawReorgScanDataCollector::m_medPosCounter [private]

Definition at line 120 of file fsc_RawReorgScanDataCollector.h.

Referenced by ProcRecMedPosMgr().

---

The documentation for this class was generated from the following files:

• fsc_RawReorgScanDataCollector.h
• fsc_RawReorgScanDataCollector.cpp