Public Member Functions | Static Public Member Functions | Protected Attributes | Private Attributes

df_BlockScanner Class Reference
[IVD Data format.]

#include <df.h>

List of all members.

Public Member Functions

 df_BlockScanner (UInt32_t a_blkType, UInt32_t a_blkSize)
virtual ~df_BlockScanner ()
df_BlockHeader_tSetBlock (UInt8_t *a_buf)
 Set current block.
df_RecCmn_tGetNextRecord ()
 Get next record in current block - returns NULL if no more records This method performs the following checks:

  • record magic cookie
  • record sequence number
  • record type sequence (records in correct order).

UInt8_tGetPosition () const
 Get pointer to current position in block - useful in case of error.
UInt32_t GetBlockLeft () const
 Get number of bytes left to the end of block (from current position).
void Reset ()
 Reset sequence counters and start from beginning.
UInt32_t GetNextBlkNum () const
UInt32_t GetNextRecNum () const
UInt64_t GetStreamLeft () const

Static Public Member Functions

static bool IsBlockCookie (const void *a_data)
static void GetVolumeID (UInt8_t *a_volHdrBlock_p, df_ECMAVolumeLabel_t *&a_ecmaVolHdr_p, df_RecVolHdr_t *&a_ivdVolHdr_p)
 This function parses the ECMA and IVD Volume IDs from the given input buffer.

Protected Attributes

UInt32_t m_blkType
UInt32_t m_blkSize
UInt8_tm_curBlock
UInt32_t m_curBlockPos
UInt32_t m_curBlockUsed
UInt32_t m_blkNum
 Current block number.
UInt32_t m_recNum
 Current record number & previous record type (for sequence checks).
UInt32_t m_prevRec
UInt32_t m_streamType
 Current stream info - used for handling RAW blocks.
string m_streamName
UInt64_t m_streamLeft

Private Attributes

 log_CLASSID_m

Detailed Description

Definition at line 746 of file df.h.


Constructor & Destructor Documentation

df_BlockScanner::df_BlockScanner ( UInt32_t  a_blkType,
UInt32_t  a_blkSize 
)

Definition at line 64 of file df_scanner.cpp.

References log_FUNC_m.

df_BlockScanner::~df_BlockScanner (  )  [virtual]

Definition at line 80 of file df_scanner.cpp.

References log_FUNC_m.


Member Function Documentation

UInt32_t df_BlockScanner::GetBlockLeft (  )  const

Get number of bytes left to the end of block (from current position).

Definition at line 479 of file df_scanner.cpp.

References m_blkSize, and m_curBlockPos.

Referenced by ParseBlock().

                                    {
    return (m_blkSize - m_curBlockPos);
}

Here is the caller graph for this function:

UInt32_t df_BlockScanner::GetNextBlkNum (  )  const [inline]

Definition at line 777 of file df.h.

Referenced by df_FRI::MediumVolumeStart().

{ return m_blkNum; }

Here is the caller graph for this function:

UInt32_t df_BlockScanner::GetNextRecNum (  )  const [inline]

Definition at line 778 of file df.h.

Referenced by df_FRI::MediumVolumeStart().

{ return m_recNum; }

Here is the caller graph for this function:

df_RecCmn_t * df_BlockScanner::GetNextRecord (  ) 

Get next record in current block - returns NULL if no more records This method performs the following checks:

  • record magic cookie
  • record sequence number
  • record type sequence (records in correct order).

For BSSt, we need to save stream size in order to count how many bytes to stream end (decreased by EDta record and RAW block)

Definition at line 283 of file df_scanner.cpp.

References blk_FRI_c, cmn_Num2Str(), df_RecEmbeddedData_t::data, dbg_NORM, df_IsValidPredecessor(), df_RF_CONTINUED, df_RF_SPLITTED, df_ST_ALTDATA, df_RecCmn_t::flags, df_RecFile_t::idFile, df_RecFile_t::idMig, ie_DF_INV_BLK, ie_DF_INVRECCOOKIE, ie_DF_INVRECSEQNUM, ie_DF_INVSEQ, ie_DF_INVSTREAMFMT, IsRecordCookie(), ivd_Error, log_DBG_m, log_ERR_m, log_FUNC_m, m_blkType, m_curBlock, m_curBlockPos, m_curBlockUsed, m_prevRec, m_recNum, m_streamLeft, m_streamName, m_streamType, df_RecCmn_t::magicCookie, ntoh(), rec_BSEnd_c, rec_BSStart_c, rec_EmbData_c, rec_FileHdr_c, rec_NoRec_c, df_RecCmn_t::seqNum, df_RecCmn_t::size, ivd_VarData_t::size, df_RecByteStreamStart_t::streamOffset, df_RecByteStreamStart_t::streamSize, df_RecByteStreamStart_t::streamSplitSize, df_RecByteStreamEnd_t::streamType, df_RecEmbeddedData_t::streamType, df_RecByteStreamStart_t::streamType, and df_RecCmn_t::type.

Referenced by df_FRI::BlockScan(), df_Unpacker::GetNextRecord(), and ParseBlock().

                                            {

    log_FUNC_m(GetNextRecord);

    // check if there are more records in the block
    if (m_curBlockPos == m_curBlockUsed) {
        return NULL;
    }

    df_RecCmn_t *recCmn_p = 
        reinterpret_cast<df_RecCmn_t*>(m_curBlock + m_curBlockPos);

    if (!IsRecordCookie(recCmn_p->magicCookie)) {
        throw ivd_Error(
            ie_DF_INVRECCOOKIE, 
            "Got=" + string((char*)recCmn_p->magicCookie,
                            sizeof(recCmn_p->magicCookie)));
    }


    // if this is first record (or continuation)- set recNum appropriately
    UInt32_t recFlags = ntoh(recCmn_p->flags);
    UInt32_t recNum   = ntoh(recCmn_p->seqNum);
    if (m_recNum == 0 || (recFlags & df_RF_CONTINUED)) {
        m_recNum = recNum;
        m_prevRec = rec_NoRec_c;
    }
    else if (recNum == 1) {
        // Beginning of new migration/FRI section
        m_recNum = recNum;
        m_prevRec = rec_NoRec_c;
    }

    if (recNum != m_recNum) {
        log_MARKLINE_m;
        throw ivd_DFError(
            ie_DF_INVRECSEQNUM, 0,
            "Got=" + cmn_Num2Str(recNum) + 
            ", expected=" + cmn_Num2Str(m_recNum));
    }

    if (m_prevRec != rec_NoRec_c
        && !df_IsValidPredecessor(m_blkType, m_prevRec, recCmn_p->type)) {
        string prevStr = string((char*)(&m_prevRec), sizeof(m_prevRec));

        string curStr =
            (recCmn_p->type == rec_NoRec_c)
            ? string("NULL REC")
            : string((char*)(&recCmn_p->type), sizeof(recCmn_p->type));

        throw ivd_DFError(ie_DF_INVSEQ, 0,
            "Record " + prevStr + " can't be followed by " + curStr);
    }

    if (recCmn_p->type == rec_BSStart_c) {
        if (m_streamType != 0) {
            throw ivd_DFError(ie_DF_INVSTREAMFMT, 0,
                              "BSStart, previous stream not complete:" 
                              " strType=" + cmn_Num2Str(m_streamType) + 
                              ", strName=" + m_streamName +
                              ", strLeft=" + cmn_Num2Str(m_streamLeft));
        }

        df_RecByteStreamStart_t *bss_p =
            reinterpret_cast<df_RecByteStreamStart_t*>(m_curBlock + 
                                                       m_curBlockPos +
                                                       sizeof(df_RecCmn_t));

        m_streamType = ntoh(bss_p->streamType);
        
        UInt64_t streamSize = ntoh(bss_p->streamSize);
        UInt64_t streamOffset = ntoh(bss_p->streamOffset);
        UInt64_t streamSplitSize = ntoh(bss_p->streamSplitSize);

        log_DBG_m(dbg_NORM, "StreamSize      : " << streamSize);
        log_DBG_m(dbg_NORM, "StreamOffset    : " << streamOffset);
        log_DBG_m(dbg_NORM, "StreamSplitSize : " << streamSplitSize);
        
        m_streamLeft = streamSize - streamOffset;
        
        if (  streamSplitSize > 0
           || ntoh(recCmn_p->flags) & df_RF_SPLITTED) {
            if (m_streamLeft < streamSplitSize) {
                log_ERR_m(
                    "StreamSplitSize > (StreamSize - StreamOffset): " <<
                    "Possible internal error!!");
            }
            
            m_streamLeft = streamSplitSize;
        }
        if (    m_blkType == blk_FRI_c
            && m_streamType == df_ST_ALTDATA) {

            log_DBG_m(dbg_NORM, "ADS in FRI. Only name is stored in FRI.");
            m_streamLeft = 0;
        }

        log_DBG_m(dbg_NORM, "StreamLeft : " << m_streamLeft);

        if (m_streamType == 0) {
            throw ivd_DFError(ie_DF_INVSTREAMFMT, 0,
                              "BSStart, but invalid streamType:" 
                              " strType=" + cmn_Num2Str(m_streamType) + 
                              ", strName=" + m_streamName +
                              ", strLeft=" + cmn_Num2Str(m_streamLeft));
        }

    }
    else if (recCmn_p->type == rec_EmbData_c) {
        df_RecEmbeddedData_t *eData_p =
            reinterpret_cast<df_RecEmbeddedData_t*>(m_curBlock +
                                                    m_curBlockPos +
                                                    sizeof(df_RecCmn_t));

        if (m_streamType != 0) {
            if (ntoh(eData_p->streamType) != m_streamType) {
                throw ivd_DFError(ie_DF_INVSTREAMFMT, 0,
                                  "EDta, but not from current stream:" 
                                  " strType=" + cmn_Num2Str(m_streamType) + 
                                  ", strName=" + m_streamName +
                                  ", strLeft=" + cmn_Num2Str(m_streamLeft));
            }
            UInt32_t embSize = ntoh(eData_p->data.size);
            if (embSize == 0 || embSize > m_streamLeft) {
                throw ivd_DFError(ie_DF_INVSTREAMFMT, 0,
                                  "EDta - less/more embedded than expected:" 
                                  " embSize=" + cmn_Num2Str(embSize) + 
                                  ", strLeft=" + cmn_Num2Str(m_streamLeft));
            }

            m_streamLeft -= embSize;
        }

    }
    else if (recCmn_p->type == rec_BSEnd_c) {
        df_RecByteStreamEnd_t *bse_p =
            reinterpret_cast<df_RecByteStreamEnd_t*>(m_curBlock +
                                                     m_curBlockPos +
                                                     sizeof(df_RecCmn_t));

        if (m_streamType != 0) {
            if (ntoh(bse_p->streamType) != m_streamType) {
                throw ivd_DFError(ie_DF_INVSTREAMFMT, 0,
                                  "BSEnd, but not from current stream:" 
                                  " strType=" + cmn_Num2Str(m_streamType) + 
                                  ", strName=" + m_streamName +
                                  ", strLeft=" + cmn_Num2Str(m_streamLeft));
            }
            if (m_streamLeft > 0) {
                throw ivd_DFError(ie_DF_INVSTREAMFMT, 0,
                                  "BSEnd, but we expect more data:" 
                                  " strLeft=" + cmn_Num2Str(m_streamLeft));
            }

            m_streamType = 0;
            m_streamName = string("");
        }
    }
    else if (recCmn_p->type == rec_FileHdr_c) {
        df_RecFile_t *fh_p =
            reinterpret_cast<df_RecFile_t*>(
                m_curBlock + m_curBlockPos + sizeof(df_RecCmn_t));

        ivd_MigrationID migID(ntoh(fh_p->idMig));
        log_DBG_m(dbg_NORM,
            "Start of file ID: " << ntoh(fh_p->idFile) <<
            ", migID" << migID);
    }


    UInt32_t recSize= ntoh(recCmn_p->size);
    // if next record position is above blocke used size -> very strange! 
    if (m_curBlockPos + recSize > m_curBlockUsed) {
        throw ivd_DFError(ie_DF_INV_BLK, 0,
                          "Block Used=" + cmn_Num2Str(m_curBlockUsed) +
                          ", Record End=" + cmn_Num2Str(m_curBlockPos
                                                        + recSize));
    }

    m_recNum++;
    m_prevRec = recCmn_p->type;
    m_curBlockPos += recSize; // TODO: align to record boundary 

    return recCmn_p;
}

Here is the call graph for this function:

Here is the caller graph for this function:

UInt8_t * df_BlockScanner::GetPosition (  )  const

Get pointer to current position in block - useful in case of error.

Definition at line 474 of file df_scanner.cpp.

References m_curBlock, and m_curBlockPos.

Referenced by ParseBlock().

                                   {
    return (UInt8_t*)(m_curBlock + m_curBlockPos);
}

Here is the caller graph for this function:

UInt64_t df_BlockScanner::GetStreamLeft (  )  const [inline]

Definition at line 779 of file df.h.

Referenced by df_FRI::MediumVolumeStart().

{ return m_streamLeft; }

Here is the caller graph for this function:

void df_BlockScanner::GetVolumeID ( UInt8_t a_volHdrBlock_p,
df_ECMAVolumeLabel_t *&  a_ecmaVolHdr_p,
df_RecVolHdr_t *&  a_ivdVolHdr_p 
) [static]

This function parses the ECMA and IVD Volume IDs from the given input buffer.

Volume ID record has a predefined size df_VOLHDR_REC_SIZE_d.

Parameters:
a_volHdrBlock (in) input volume header block from a tape
a_ecmaVolHdr (out) ECMA volume header (NULL if not detected)
a_ivdVolHdr (out) IVD volume header (NULL if not detected)

Definition at line 93 of file df_scanner.cpp.

References cmn_Num2Str(), dbg_LOW, dbg_NORM, df_recCookie_c, ecma_ImplID_c, ecma_LblStdVer_c, ecma_VolID_c, df_ECMAVolumeLabel_t::ecmaHdr, df_ECMAVolumeLabel_t::first, ie_DF_HDRCORRUPT, df_ECMAFirstVolumeLabel_t::implID, IsRecordCookie(), df_ECMALblHdr_t::lblID, df_ECMAFirstVolumeLabel_t::lblStdVer, log_DBG_m, log_FUNC_m, df_RecCmn_t::magicCookie, df_ECMALblHdr_t::number, rec_VolHdr_c, and df_RecCmn_t::type.

Referenced by CheckIVDHeader(), and ParseECMAHeader().

                                             {

    log_FUNC_m(GetVolumeID);

    a_ecmaVolHdr_p = NULL;
    a_ivdVolHdr_p = NULL;

    // Try to detect ECMA volume header
    log_DBG_m(dbg_NORM, "Detecting ECMA volume header.");

    df_ECMAVolumeLabel_t *ecmaHdr_p =
        reinterpret_cast<df_ECMAVolumeLabel_t*>(a_volHdrBlock_p);

    if (memcmp(ecmaHdr_p->ecmaHdr.lblID, ecma_VolID_c,
            sizeof(ecmaHdr_p->ecmaHdr.lblID)) != 0) {

        log_DBG_m(dbg_NORM, "ECMA volume header not found.");
        return;
    }

    if (ecmaHdr_p->ecmaHdr.number == 0) {
        log_DBG_m(dbg_NORM, "ECMA volume number can't be 0.");
        return;
    }
    a_ecmaVolHdr_p = ecmaHdr_p;

    // Try to detect IVD volume header
    log_DBG_m(dbg_NORM, "Detecting HSM volume header.");
    if (ecmaHdr_p->ecmaHdr.number == 1) {
        if (ecmaHdr_p->first.lblStdVer != ecma_LblStdVer_c) {
            log_DBG_m(dbg_NORM, "ECMA version not correct: " +
                cmn_Num2Str(ecmaHdr_p->first.lblStdVer));
            return;
        }

        string implIdShort(df_recCookie_c);
        string implOnHdrShort((char*)(ecmaHdr_p->first.implID), implIdShort.length());
        string implOnHdr((char*)(ecmaHdr_p->first.implID), ecma_ImplID_c.length());
        
        log_DBG_m(dbg_LOW, "ECMA implementation ID: " << implOnHdr);
        
        // Detect the data format type by checking the first part of the
        // Implementation ID
        if (implOnHdrShort != implIdShort) {
            log_DBG_m(dbg_NORM, "Invalid ECMA ImplID: " + implOnHdr);
            return;
        }
        else if (implOnHdr != ecma_ImplID_c) {
            // Compare the version part as well.
            log_DBG_m(dbg_LOW,
                "Detected a volume with different data format version than " <<
                ecma_ImplID_c);
        }
        
        // TODO: Use ecmaHdr_p->first.volID??
    }

    log_DBG_m(dbg_NORM, "Detecting HSM df_RecCmn_t.");

    df_RecCmn_t *recCmn_p =
        reinterpret_cast<df_RecCmn_t*>(a_volHdrBlock_p + sizeof(*ecmaHdr_p));

    if (!IsRecordCookie(recCmn_p->magicCookie)) {
        log_DBG_m(dbg_NORM, "Record magic cookie not found.");
        return;
    }

    if (recCmn_p->type != rec_VolHdr_c) {
        throw ivd_DFError(
            ie_DF_HDRCORRUPT, 0,
            "Detected HSM common record, but type is not rec_VolHdr_c.");
    }

    log_DBG_m(dbg_NORM, "Detected HSM df_RecVolHdr_t.");

    df_RecVolHdr_t *ivdHdr_p = 
        reinterpret_cast<df_RecVolHdr_t*>(
          a_volHdrBlock_p + sizeof(*ecmaHdr_p) + sizeof(*recCmn_p));

    a_ivdVolHdr_p = ivdHdr_p;
}

Here is the call graph for this function:

Here is the caller graph for this function:

bool df_BlockScanner::IsBlockCookie ( const void *  a_data  )  [static]

Definition at line 55 of file df_scanner.cpp.

References df_blkCookie_c, and df_BLKCOOKIE_SIZE_d.

Referenced by ReadOneBackupFromMedium(), and SetBlock().

                                                      {
    return (
        memcmp(
            a_data, df_blkCookie_c,
            df_BLKCOOKIE_SIZE_d) == 0);
}

Here is the caller graph for this function:

void df_BlockScanner::Reset (  ) 

Reset sequence counters and start from beginning.

Definition at line 484 of file df_scanner.cpp.

References m_blkNum, m_curBlock, m_curBlockPos, m_curBlockUsed, m_prevRec, m_recNum, m_streamLeft, m_streamName, m_streamType, and rec_NoRec_c.

Referenced by df_FRI::MediumVolumeStart(), and ParseBlock().

                       {

    // reset block - caller need to call SetBlock again
    m_curBlock      = NULL;
    m_curBlockPos   = 0;
    m_curBlockUsed  = 0;

    // reset sequence numbers
    m_blkNum        = 0;
    m_recNum        = 0;

    // reset info about previous record and stream
    m_prevRec       = rec_NoRec_c;
    m_streamType    = 0;
    m_streamName    = "";
    m_streamLeft    = 0; 
}

Here is the caller graph for this function:

df_BlockHeader_t * df_BlockScanner::SetBlock ( UInt8_t a_buf  ) 

Set current block.

Returns NULL if raw block. This method performs the following checks:

  • block magic cookie (unless the block is raw)
  • block sequence number

Definition at line 184 of file df_scanner.cpp.

References df_BlockHeader_t::blkCookie, cmn_Num2Str(), dbg_DETAIL, df_BF_CONTINUED, df_BlockHeader_t::flags, ie_DF_INV_BLK, ie_DF_INVBLKCOOKIE, ie_DF_INVBLKSEQNUM, IsBlockCookie(), log_DBG_m, log_FUNC_m, m_blkNum, m_blkSize, m_blkType, m_curBlock, m_curBlockPos, m_curBlockUsed, m_prevRec, m_recNum, m_streamLeft, ntoh(), rec_Data_c, df_BlockHeader_t::seqNum, df_BlockHeader_t::type, and df_BlockHeader_t::used.

Referenced by df_FRI::BlockScan(), df_Unpacker::GetNextBlock(), and ParseBlock().

                                                          {

    log_FUNC_m(SetBlock);

    m_curBlock      = a_buf;
    m_curBlockPos   = 0;

    // if RAW block
    if (m_streamLeft >= m_blkSize) {
        m_curBlockUsed  = 0;

        log_DBG_m(dbg_DETAIL, "RAW block: blkNum=" << m_blkNum);
        m_blkNum++; // RAW block does not have seqNum, but it is counted
        m_prevRec = rec_Data_c;     // mark that we read data

        m_streamLeft   -= m_blkSize; // RAW block is always full of stream data

        return NULL;
    }

    df_BlockHeader_t *blkHdr_p = 
        reinterpret_cast<df_BlockHeader_t*>(m_curBlock);
 
    if (!IsBlockCookie(blkHdr_p->blkCookie)) {
        throw ivd_DFError(
            ie_DF_INVBLKCOOKIE, 0,
            "Got=" + string((char*)blkHdr_p->blkCookie, 
                             sizeof(blkHdr_p->blkCookie)));
    }

    if (blkHdr_p->type != m_blkType) {
        throw ivd_DFError(
            ie_DF_INV_BLK, 0,
            "Invalid block type \"" + 
            string((char*)&blkHdr_p->type, sizeof(blkHdr_p->type)) +
            "\" - expected \"" + 
            string((char*)&m_blkType, sizeof(m_blkType)) + "\"");
    }

    // Check that current block number matches
    UInt32_t curBlkNum = ntoh(blkHdr_p->seqNum); 


    // if this is first block we got (or continuation)- set blkNum appropriately
    UInt32_t blkFlags = ntoh(blkHdr_p->flags);
    if (!m_blkNum || (blkFlags & df_BF_CONTINUED)) {
        m_blkNum = curBlkNum;
    } 

    // if this is first block from new stream - restart counting
    if (curBlkNum == 1) {
        m_blkNum = 1;
        m_recNum = 1;
    }

    if (curBlkNum != m_blkNum) {
        throw ivd_DFError(
            ie_DF_INVBLKSEQNUM, 0,
            "Got=" + cmn_Num2Str(curBlkNum) + 
            ", expected=" + cmn_Num2Str(m_blkNum));
    }

    m_curBlockUsed = ntoh(blkHdr_p->used);

    // if first record position is above block used size -> very strange! 
    if (m_curBlockUsed > m_blkSize) {
        throw ivd_DFError(
            ie_DF_INV_BLK, 0,
            "Block Used=" + cmn_Num2Str(m_curBlockUsed) +
            " larger than specified Block Size=" + 
            cmn_Num2Str(m_blkSize));
    }

    // if first record position is above block used size -> very strange! 
    if (sizeof(df_BlockHeader_t) > m_curBlockUsed) {
        throw ivd_DFError(
            ie_DF_INV_BLK, 0,
            "Block Used=" + cmn_Num2Str(m_curBlockUsed) +
            ", First Rec=" + 
            cmn_Num2Str(sizeof(df_BlockHeader_t)));
    }

    log_DBG_m(dbg_DETAIL, "HSM block: blkNum=" << m_blkNum << " flags=" << 
              cmn_Num2Str(blkFlags, true));

    m_blkNum++;
    m_curBlockPos = sizeof(df_BlockHeader_t); // TODO: align to record boundary

    return blkHdr_p;
}

Here is the call graph for this function:

Here is the caller graph for this function:


Member Data Documentation

Definition at line 810 of file df.h.

Current block number.

Warning:
Raw data blocks are not marked but they are counted.

Definition at line 794 of file df.h.

Referenced by Reset(), and SetBlock().

Definition at line 785 of file df.h.

Referenced by GetBlockLeft(), and SetBlock().

Definition at line 784 of file df.h.

Referenced by GetNextRecord(), and SetBlock().

Definition at line 786 of file df.h.

Referenced by GetNextRecord(), GetPosition(), Reset(), and SetBlock().

Definition at line 787 of file df.h.

Referenced by GetBlockLeft(), GetNextRecord(), GetPosition(), Reset(), and SetBlock().

Definition at line 788 of file df.h.

Referenced by GetNextRecord(), Reset(), and SetBlock().

Definition at line 800 of file df.h.

Referenced by GetNextRecord(), Reset(), and SetBlock().

Current record number & previous record type (for sequence checks).

Definition at line 799 of file df.h.

Referenced by GetNextRecord(), Reset(), and SetBlock().

Definition at line 807 of file df.h.

Referenced by GetNextRecord(), Reset(), and SetBlock().

string df_BlockScanner::m_streamName [protected]

Definition at line 806 of file df.h.

Referenced by GetNextRecord(), and Reset().

Current stream info - used for handling RAW blocks.

Definition at line 805 of file df.h.

Referenced by GetNextRecord(), and Reset().


The documentation for this class was generated from the following files: