Partition Manager
[Server]

Collaboration diagram for Partition Manager:

Detailed Description

The Main objective of the Partition Manager is to Create new Migration or Recall Jobs.

Each new Job is a new CORBA Object. Within this job a executor thread is started which starts the Job activity.

Migration Job:

  1. DiskBuffer space check and allocation
  2. DiskBuffer file is created by Client
  3. Resources request from Resource Manger
  4. Job waits for Resources When Job get resources:
  5. Library Agent for Load/Unload is called
  6. Back-End Agent (BEA) is started (SVC)
  7. Use Resources is invoked on Back-End Agent to start actually migrating data
  8. When BEA finishes CompleteJob is called
  9. Resources are released
  10. Complete Migration on Client is called
  11. DiskBuffer File is deleted

Recall Job:

  1. Positions of Copies are requested from FSC (File System Catalog)
  2. Best Copy is choosen by RM
  3. Resources request from Resource Manger
  4. Job waits for Resources When Job get resources:
  5. Library Agent for Load/Unload is called
  6. Back-End Agent (BEA) is started (SVC)
  7. Use Resources is invoked on Back-End Agent to start actually migrating data
  8. When BEA finishes CompleteJob is called
  9. Resources are released
  10. Recall on Client is called
  11. DiskBuffer File is deleted


Classes

class  job_finishedStatus
class  job_Manager
class  i_DelExpiredFilesJob_i
class  i_FSCCheckJob_i
struct  friVolStatus_t
struct  MediumInfo_t
class  i_MaintFriJob_i
class  i_MediumCheckJob_i
class  pm_AssignResThread
class  i_MigrationJob_i
struct  cfg_Tree
class  cfg_PM
class  i_PartitionManager_i
 CORBA servant for i_PartitionManager. More...
class  pm_RecallClientThread
class  i_RecallJob_i
class  pm_EfficientRecallClientThread
class  i_EfficientRecallJob_i
class  i_RecoveryJob_i
class  i_RecreatedIVDFSJob_i
class  i_RedundantCopyJob_i
class  i_ReorgJob_i
class  i_ReorgScanJob_i
class  i_ThreadJob_i
struct  pm_File_t
class  pm_FileList

Defines

#define job_COUNT_NO_LIMITS   (0)
 <Description:> A more elaborate class description that describes all aspects of class usage etc.
#define job_RECALL_FACTOR   (2)
#define MAX_WAIT_FOR_BEA_TO_START   (10*cfg_MINUTE)
#define evt_MAINTJOB   "MAINTENANCE"
 <Description:> A more elaborate class description that describes all aspects of class usage etc.

Typedefs

typedef vector< i_Job_i * > job_JobList_v_t
typedef job_JobList_v_t::iterator job_Iterator_t
typedef vector< pm_File_tpm_FileList_v

Enumerations

enum  mj_operationType_e {
  mj_CHECK_FSC_VS_IVDFS, mj_CHECK_FSC_VS_MEDIA,
  mj_SENTINEL
}

Functions

 i_MediumCheckJob_i::~i_MediumCheckJob_i ()
virtual void i_MediumCheckJob_i::CompleteJob (i_CompletionStatus_e a_status)
void i_MediumCheckJob_i::Execute ()
void i_MediumCheckJob_i::ProcessFRI (pm_FRIInfo a_FRIInfo)
virtual i_ThreadJob_i::~i_ThreadJob_i ()
virtual void i_ThreadJob_i::MediumOperationComplete (i_Index_t a_beaNum, i_CompletionStatus_e a_status)
virtual void i_ThreadJob_i::CompleteJob (i_CompletionStatus_e a_status)
virtual void i_ThreadJob_i::Remove ()
i_FSC_ptr i_ThreadJob_i::GetFSC ()
virtual void i_ThreadJob_i::GetNewResources (i_Index_t a_resNum)
virtual i_JobRequestList_ti_ThreadJob_i::GetFiles ()
virtual void i_ThreadJob_i::AssignResources (const i_ResourceList_t &a_resources)
virtual void i_ThreadJob_i::Execute ()=0

Variables

i_PartitionManager_ii_MediumCheckJob_i::m_iPM
bool i_MediumCheckJob_i::m_autoCorrect
bool i_MediumCheckJob_i::m_removeMissingOnMedia
ivd_MediaChkStat_t i_MediumCheckJob_i::m_totalChkStat
 i_MediumCheckJob_i::log_CLASSID_m
 i_ThreadJob_i::log_CLASSID_m
i_PartitionManager_ii_ThreadJob_i::m_iPM
ui_MsgWriter i_ThreadJob_i::m_uiMsgWriter


Define Documentation

#define evt_MAINTJOB   "MAINTENANCE"

#define job_COUNT_NO_LIMITS   (0)

<Description:> A more elaborate class description that describes all aspects of class usage etc.

Author:
Toni Pljakoski, HERMES SoftLab
See also:
<reference>

Definition at line 45 of file jobs.h.

Referenced by job_Manager::AddJobToList(), and job_Manager::RemoveFromList().

#define job_RECALL_FACTOR   (2)

Definition at line 46 of file jobs.h.

Referenced by job_Manager::SetMaxRunningJobs().

#define MAX_WAIT_FOR_BEA_TO_START   (10*cfg_MINUTE)

Definition at line 47 of file jobs.h.

Referenced by i_Job_i::WaitBea().


Typedef Documentation

typedef job_JobList_v_t::iterator job_Iterator_t

Definition at line 58 of file jobs.h.

Definition at line 57 of file jobs.h.

Definition at line 55 of file pm.h.


Enumeration Type Documentation

Enumerator:
mj_CHECK_FSC_VS_IVDFS 
mj_CHECK_FSC_VS_MEDIA 
mj_SENTINEL 

Definition at line 41 of file pm.h.


Function Documentation

void i_ThreadJob_i::AssignResources ( const i_ResourceList_t a_resources  )  [virtual, inherited]

Reimplemented from i_Job_i.

Definition at line 153 of file i_threadjob_impl.cpp.

References ie_NYI.

00153                                                                       { 
00154     throw ivd_InternalError(ie_NYI, "");
00155 }

void i_ThreadJob_i::CompleteJob ( i_CompletionStatus_e  a_status  )  [virtual, inherited]

Implements i_Job_i.

Definition at line 92 of file i_threadjob_impl.cpp.

References evt_ERROR, evt_WARNING, i_Job_i::GetJobTypeText(), i_CLIENT_ERROR, i_SUCCEDED, ipc_EXEC_m, i_JobParams::jobID, log_ERR_m, log_FUNC_m, log_WriteEvent(), i_Job_i::m_aborted, i_Job_i::m_iJobParams, i_Job_i::m_status, and i_ThreadJob_i::Remove().

00092                                                               {
00093     log_FUNC_m(CompleteJob);
00094 
00095     m_status = a_status;
00096 
00097     try {
00098         if (a_status == i_SUCCEDED) {
00099             log_WriteEvent("Finished.", 
00100                            GetJobTypeText(), m_iJobParams.jobID);
00101         }
00102         else if (m_aborted) {
00103             log_WriteEvent(evt_WARNING, "Aborted.", 
00104                            GetJobTypeText(), m_iJobParams.jobID);
00105         } 
00106         else if (a_status == i_CLIENT_ERROR) {
00107             log_WriteEvent(evt_ERROR, "Failed (client error).",
00108                            GetJobTypeText(), m_iJobParams.jobID);
00109         }
00110         else {
00111             ostringstream os;
00112             os << "Failed (" << a_status << ").";
00113             log_WriteEvent(evt_ERROR, os.str(),
00114                            GetJobTypeText(), m_iJobParams.jobID);
00115         }
00116         //DEL        m_iJobParams.status = job_FINISHED;
00117 
00118         ipc_EXEC_m( Remove(); );
00119 
00120     }
00121     catch (ivd_Exception& e) {
00122         log_ERR_m(e);
00123     }
00124     catch (ivd_InternalError& ie) {
00125         log_ERR_m(ie);
00126     }
00127     catch (...) {
00128         log_ERR_m("Caught unknown" );
00129     }
00130 }

Here is the call graph for this function:

void i_MediumCheckJob_i::CompleteJob ( i_CompletionStatus_e  a_status  )  [virtual, inherited]

Reimplemented from i_MaintFriJob_i.

Definition at line 256 of file i_mediumcheckjob_impl.cpp.

References i_MaintFriJob_i::CompleteJob(), dbg_DETAIL, ui_MsgWriter::DisplayMessage(), i_Job_i::GetJobTypeText(), ui_MsgWriter::GetStreamPath(), i_SUCCEDED, i_Job_i::IsAborted(), i_JobParams::jobID, log_DBG_m, log_FUNC_m, log_WriteEvent(), i_Job_i::m_iJobParams, i_MediumCheckJob_i::m_totalChkStat, and i_MaintFriJob_i::m_uiMsgWriter.

00256                                                                   {
00257     log_FUNC_m(CompleteJob);
00258     
00259     if (a_status == i_SUCCEDED
00260         && !IsAborted()) {
00261     
00262         log_DBG_m(dbg_DETAIL, "Check for not checked volumes.");
00263     }
00264     ostringstream sstr;
00265     sstr
00266         << "Finished FSC check "
00267         << "(total):"
00268         << m_totalChkStat;
00269         
00270     if (  m_totalChkStat.missingInFSC 
00271        || m_totalChkStat.mismatch) {
00272        sstr << endl << "    Look into log file for details. '" 
00273             << m_uiMsgWriter.GetStreamPath() << "'";
00274     }
00275     
00276     m_uiMsgWriter.DisplayMessage(sstr.str());
00277     log_WriteEvent(sstr.str(), GetJobTypeText(), m_iJobParams.jobID);
00278 
00279     i_MaintFriJob_i::CompleteJob(a_status);
00280 }

Here is the call graph for this function:

virtual void i_ThreadJob_i::Execute (  )  [protected, pure virtual, inherited]

void i_MediumCheckJob_i::Execute (  )  [private, virtual, inherited]

Implements i_MaintFriJob_i.

Definition at line 219 of file i_mediumcheckjob_impl.cpp.

References i_FSC_i::CollectByVolume(), dbg_DETAIL, ui_MsgWriter::DisplayError(), ui_MsgWriter::DisplayMessage(), i_MaintFriJob_i::Execute(), ivd_BaseException::GetFriendly(), i_ABORTED, i_FAILED, ipc_EXEC_m, i_Job_i::IsAborted(), log_DBG_m, log_FUNC_m, i_PartitionManager_i::m_iFSCi, i_MediumCheckJob_i::m_iPM, i_Job_i::m_status, and i_MaintFriJob_i::m_uiMsgWriter.

00219                                  {
00220     log_FUNC_m(Execute);
00221 
00222      try {
00223         ipc_EXEC_m(
00224             ostringstream msg;
00225             msg << "Preparing FSC for check.";
00226             m_uiMsgWriter.DisplayMessage(msg.str());
00227 
00228             try {
00229                 m_iPM.m_iFSCi->CollectByVolume();
00230                 msg.str("");
00231                 msg << "Preparing of FSC finished.";
00232                 m_uiMsgWriter.DisplayMessage(msg.str());
00233 
00234                 if (IsAborted()) {
00235                     m_status = i_ABORTED;
00236                     return;
00237                 }
00238             }
00239             catch (...) {
00240             }
00241             log_DBG_m(dbg_DETAIL, msg.str());
00242 
00243         );
00244     } 
00245     catch (ivd_Exception &e) {
00246         ostringstream err;
00247         err << e.GetFriendly() << endl;
00248         m_uiMsgWriter.DisplayError(err.str());
00249         m_status = i_FAILED;
00250         throw;
00251     }
00252     i_MaintFriJob_i::Execute();
00253    
00254 }

Here is the call graph for this function:

i_JobRequestList_t * i_ThreadJob_i::GetFiles (  )  [virtual, inherited]

Implements i_Job_i.

Definition at line 148 of file i_threadjob_impl.cpp.

References ie_NYI.

00148                                            { 
00149     throw ivd_InternalError(ie_NYI, "");
00150 }

i_FSC_ptr i_ThreadJob_i::GetFSC (  )  [virtual, inherited]

Implements i_Job_i.

Definition at line 138 of file i_threadjob_impl.cpp.

References ie_NYI.

00138                                { 
00139     throw ivd_InternalError(ie_NYI, "");
00140 }

void i_ThreadJob_i::GetNewResources ( i_Index_t  a_resNum  )  [virtual, inherited]

Implements i_Job_i.

Definition at line 143 of file i_threadjob_impl.cpp.

References ie_NYI.

00143                                                       { 
00144     throw ivd_InternalError(ie_NYI, "");
00145 }

void i_ThreadJob_i::MediumOperationComplete ( i_Index_t  a_beaNum,
i_CompletionStatus_e  a_status 
) [virtual, inherited]

Implements i_Job_i.

Definition at line 78 of file i_threadjob_impl.cpp.

References dbg_NORM, i_Job_i::GetJobTypeText(), i_JobParams::jobID, log_DBG_m, log_FUNC_m, ipc_Log::LogStatus(), and i_Job_i::m_iJobParams.

00081 {
00082     log_FUNC_m(MediumOperationComplete);
00083     log_DBG_m(dbg_NORM,
00084         GetJobTypeText() <<
00085         " Completed " << a_beaNum <<
00086         "  status: [" << a_status << "]" <<
00087         ipc_Log::LogStatus(a_status) << endl <<
00088         m_iJobParams.jobID);
00089 }

Here is the call graph for this function:

void i_MediumCheckJob_i::ProcessFRI ( pm_FRIInfo  a_FRIInfo  )  [private, virtual, inherited]

Implements i_MaintFriJob_i.

Definition at line 101 of file i_mediumcheckjob_impl.cpp.

References i_Medium_t::barcode, i_VolInfo_t::blockSize, i_FSC_i::CheckWithMedVolume(), dbg_DETAIL, dbg_LOW, cmn_File::DeleteF(), cmn_Global::dirs, ui_MsgWriter::DisplayMessage(), evt_ERROR, evt_WARNING, g_cmn, i_Job_i::GetJobTypeText(), i_Job_i::GetResources(), ie_FATAL_ERROR, ipc_EXEC_m, ivd_Error, i_JobParams::jobID, log_DBG_m, log_ERR_m, log_FUNC_A_m, log_WriteEvent(), ipc_Log::LogMedium(), ipc_Log::LogVolInfo(), i_MediumCheckJob_i::m_autoCorrect, pm_FRIInfo::m_beaNum, MediumInfo_t::m_friVolStatus, i_PartitionManager_i::m_iFSCi, i_Job_i::m_iJobParams, i_MediumCheckJob_i::m_iPM, pm_FRIInfo::m_lastDataPos, MediumInfo_t::m_medium, i_MaintFriJob_i::m_mediumInfoVec, ivd_Product::m_nameShortLC, i_MediumCheckJob_i::m_removeMissingOnMedia, i_MediumCheckJob_i::m_totalChkStat, i_MaintFriJob_i::m_uiMsgWriter, pm_FRIInfo::m_volNum, i_VolInfo_t::medVolID, ivd_MediaChkStat_t::mismatch, ivd_MediaChkStat_t::missingInFSC, i_JobParams::partName, cmn_Global::prod, ivd_Directories::tmp, and i_VolInfo_t::volNum.

00101                                                        {
00102     log_FUNC_A_m(ProcessFRI,    "m_beaNum:" << a_FRIInfo.m_beaNum <<
00103                                 ", m_volNum:" << a_FRIInfo.m_volNum <<
00104                                 ", m_lastDataPos:" << a_FRIInfo.m_lastDataPos);
00105 
00106     log_DBG_m(dbg_DETAIL,"Calling FSC for Medium Check");
00107 
00108     ivd_MediaChkStat_t   mediumChkStat;
00109 
00110     UInt32_t mediumKey    = GetResources()[a_FRIInfo.m_beaNum].mediumKey;
00111     MediumInfo_t &medInfo = m_mediumInfoVec[a_FRIInfo.m_beaNum];
00112     log_DBG_m(dbg_DETAIL, "Processing medium: " << endl <<
00113                         ipc_Log::LogMedium(medInfo.m_medium));
00114 
00115     UInt32_t currentVol(0);
00116     bool found(false);
00117     for (UInt32_t i(0); i < medInfo.m_friVolStatus.size(); i++){
00118         if (medInfo.m_friVolStatus[i].m_volInfo.volNum == a_FRIInfo.m_volNum  ){
00119             found = true;
00120             log_DBG_m(dbg_DETAIL,"i=" << i << endl <<
00121                 ipc_Log::LogVolInfo(medInfo.m_friVolStatus[i].m_volInfo));
00122             currentVol = i;
00123             break;
00124         }
00125     }
00126     if (!found){
00127         throw ivd_Error(ie_FATAL_ERROR, "Medium Volume does not exist.");
00128     }
00129     
00130     i_VolInfo_t &volInfo = medInfo.m_friVolStatus[currentVol].m_volInfo;
00131        
00132     try {
00133         try {
00134             ipc_EXEC_m(
00135                 m_iPM.m_iFSCi->CheckWithMedVolume(
00136                     string(volInfo.medVolID), 
00137                     volInfo.volNum,
00138                     string(medInfo.m_medium.barcode),
00139                     mediumKey,
00140                     volInfo.blockSize,
00141                     string(m_iJobParams.partName),
00142                     m_autoCorrect,
00143                     m_removeMissingOnMedia,
00144                     m_uiMsgWriter,
00145                     mediumChkStat,
00146                     a_FRIInfo.m_lastDataPos);
00147             );
00148 
00149             // set FRI processed
00150             medInfo.m_friVolStatus[currentVol].m_processed = true;
00151 
00152             try {
00153                 cmn_Path friPath =  g_cmn.dirs.tmp + 
00154                                     string("fri") + 
00155                                     string(volInfo.medVolID);
00156 
00157                 cmn_File delFri(friPath);
00158                 delFri.DeleteF();
00159             } catch (ivd_Exception& e){
00160                 log_DBG_m(dbg_LOW,
00161                     "Non-fatal Error while deleting tmp fri File: " << endl << e
00162                     << endl << "Continuing check");
00163                 //ignore this error                                        
00164             }
00165 
00166         }
00167         catch (ivd_DFError &dfe) {
00168             log_ERR_m("Error while Checking Volume: " << dfe);
00169             if (medInfo.m_friVolStatus[currentVol].m_recreated) {
00170                 ostringstream sstr;
00171                 sstr
00172                     << "Got bad data even on data volume"
00173                     << volInfo.medVolID
00174                     << ". Set volume as unused and skip it.";
00175 
00176                 log_WriteEvent(evt_ERROR, sstr.str(), 
00177                             GetJobTypeText(), 
00178                             m_iJobParams.jobID );
00179                 m_uiMsgWriter.DisplayMessage(sstr.str());
00180             }
00181             else {
00182                 ostringstream sstr;
00183                 sstr << "Corrupted FRI file for volume "
00184                         << volInfo.medVolID
00185                         << ". Use '" << g_cmn.prod.m_nameShortLC
00186                         << "medium --recreate-fri' and run "
00187                         << g_cmn.prod.m_nameShortLC << "check again.";
00188 
00189                 log_WriteEvent(
00190                     evt_WARNING, sstr.str(),
00191                     GetJobTypeText(), m_iJobParams.jobID );
00192                 
00193                 m_uiMsgWriter.DisplayMessage(sstr.str());
00194         #if TGT_OS_linux
00195             #warning "TODO: restart fri recreation for this volume"
00196         #elif TGT_OS_windows
00197             #pragma message ("TODO: restart fri recreation for this volume")
00198         #endif
00199             }
00200         }
00201     } catch (ivd_Exception& e) {
00202         log_ERR_m("Check of volume " << volInfo.medVolID << " failed" << endl << e)
00203     }
00204     
00205     ostringstream sstr;
00206     sstr << "Finished FSC check (medium: " << medInfo.m_medium.barcode << "):"
00207          << mediumChkStat;
00208          
00209     if (  mediumChkStat.missingInFSC 
00210        || mediumChkStat.mismatch) {
00211         log_WriteEvent(sstr.str(), GetJobTypeText(), m_iJobParams.jobID);
00212     }
00213 
00214     m_uiMsgWriter.DisplayMessage(sstr.str());
00215     m_totalChkStat += mediumChkStat;
00216 }

Here is the call graph for this function:

void i_ThreadJob_i::Remove (  )  [virtual, inherited]

Reimplemented from i_Job_i.

Definition at line 133 of file i_threadjob_impl.cpp.

References i_Job_i::Remove().

Referenced by i_ThreadJob_i::CompleteJob().

00133                            { 
00134     i_Job_i::Remove();
00135 }

Here is the call graph for this function:

Here is the caller graph for this function:

i_MediumCheckJob_i::~i_MediumCheckJob_i (  )  [inherited]

Definition at line 93 of file i_mediumcheckjob_impl.cpp.

References i_PartitionManager_i::ClearCheckAutoCorrectRunning(), i_PartitionManager_i::ClearCheckRunning(), i_MediumCheckJob_i::m_autoCorrect, and i_MediumCheckJob_i::m_iPM.

00093                                        {
00094     if (m_autoCorrect){
00095         m_iPM.ClearCheckAutoCorrectRunning();
00096     } else {
00097         m_iPM.ClearCheckRunning();
00098     }
00099 }

Here is the call graph for this function:

i_ThreadJob_i::~i_ThreadJob_i (  )  [virtual, inherited]

Definition at line 70 of file i_threadjob_impl.cpp.

References ui_MsgWriter::DisplayMessage(), i_JobParams::jobID, log_FUNC_m, i_Job_i::m_iJobParams, and i_ThreadJob_i::m_uiMsgWriter.

00070                              {
00071     log_FUNC_m(~i_ThreadJob_i);
00072     ostringstream msg;
00073     msg << "Finished (JobID: " << m_iJobParams.jobID << ").";
00074     m_uiMsgWriter.DisplayMessage(msg.str());
00075 }

Here is the call graph for this function:


Variable Documentation

Reimplemented from i_Job_i.

Reimplemented in i_DelExpiredFilesJob_i, i_FSCCheckJob_i, i_RecreatedIVDFSJob_i, and i_ReorgScanJob_i.

Definition at line 47 of file i_threadjob_impl.h.

Reimplemented from i_MaintFriJob_i.

Definition at line 50 of file i_mediumcheckjob_impl.h.

bool i_MediumCheckJob_i::m_autoCorrect [private, inherited]

i_PartitionManager_i& i_ThreadJob_i::m_iPM [protected, inherited]

bool i_MediumCheckJob_i::m_removeMissingOnMedia [private, inherited]

Definition at line 46 of file i_mediumcheckjob_impl.h.

Referenced by i_MediumCheckJob_i::ProcessFRI().


Generated on Mon Feb 27 18:56:23 2012 for OPENARCHIVE by  doxygen 1.5.6