hsm_FHmigc Class Reference
[Classes for managing object list]

#include <hsm_FHmigc.h>

Inheritance diagram for hsm_FHmigc:

Inheritance graph
[legend]
Collaboration diagram for hsm_FHmigc:

Collaboration graph
[legend]

List of all members.

Detailed Description

hsm_FHmigc class manages files which are "Dirty".

Definition at line 102 of file hsm_FHmigc.h.


Public Member Functions

 hsm_FHmigc (bool &a_running, i_PartitionManager_ptr a_iPM, ivd_Time32_t a_minWaitTime, UInt32_t a_minNumFiles, ivd_FileSize_t a_minMigSize, ivd_Time32_t a_maxWaitTime, UInt32_t a_maxNumFiles, ivd_FileSize_t a_maxMigSize, hsm_MigByAPI &a_migJobs)
virtual ~hsm_FHmigc ()
virtual hsm_FH_p_li Append (hsm_FileHeader *a_fh_p)
 list methods
void Remove (hsm_FileHeader *a_fh_p)
ivd_FileSize_t GetFilesListSize ()
void Reconfigure (ivd_Time32_t a_CfgMinWaitTime, UInt32_t a_CfgMinNumFiles, ivd_FileSize_t a_CfgMinMigSize, ivd_Time32_t a_CfgMaxWaitTime, UInt32_t a_CfgMaxNumFiles, ivd_FileSize_t a_CfgMaxMigSize)
ivd_Time32_t GetWakeUpTime ()
UInt32_t TrigMigration (void)
virtual void Run (void *arg)
void Shutdown ()
void WakeUp (void)
void Migrate (UInt32_t jobSize)
void SendToPM (UInt32_t sizeTL, UInt32_t filesNoFID, UInt32_t preJobIdx)
bool GetFileIDs (UInt32_t filesNoFID, i_FileIDs_t_var &tFIDs)
void DumpStatus (ostringstream &sstr)
void DumpList (ostringstream &sstr)

Public Attributes

 log_CLASSID_m

Private Attributes

i_PartitionManager_var m_iPM
bool & m_running
 outside variable that set when thread is down
bool m_goDown
cmn_Mutex m_thread_x
cmn_Condition m_thread_c
ivd_Time32_t m_minWaitTime
 If nothing change in minWaitTime perion and see m_minNumMigFile, m_minMigSize.
UInt32_t m_minNumFiles
 at least this number of files to be mig at minWaitTime
ivd_FileSize_t m_minMigSize
 Amount of data that has to be achive to migrate in minWaitTime.
ivd_Time32_t m_maxWaitTime
 If maxWaitTime passed from last migration then mig now.
UInt32_t m_maxNumFiles
 If m_maxNumFiles lay on list then migrate immediately.
ivd_FileSize_t m_maxMigSize
 If m_maxMigSize data is add to mig list then migrate it.
hsm_MigByAPI & m_migJobs
ivd_Time32_t m_wakeUpTime
 thread wake up time
ivd_FileSize_t m_migDataSize
ivd_Time32_t m_firstAppendTime
ivd_Time32_t m_lastAppendTime
bool m_forcedMigration

Constructor & Destructor Documentation

hsm_FHmigc::hsm_FHmigc ( bool &  a_running,
i_PartitionManager_ptr  a_iPM,
ivd_Time32_t  a_minWaitTime,
UInt32_t  a_minNumFiles,
ivd_FileSize_t  a_minMigSize,
ivd_Time32_t  a_maxWaitTime,
UInt32_t  a_maxNumFiles,
ivd_FileSize_t  a_maxMigSize,
hsm_MigByAPI &  a_migJobs 
)

Definition at line 49 of file hsm_FHmigc.cpp.

References dbg_LOW, log_DBG_m, log_FUNC_m, m_iPM, m_maxMigSize, m_maxNumFiles, m_maxWaitTime, m_minMigSize, m_minNumFiles, and m_minWaitTime.

00059         :
00060         m_running(a_running),
00061         m_goDown(false),
00062         m_thread_c(&m_thread_x),
00063         m_minWaitTime(a_minWaitTime),
00064         m_minNumFiles(a_minNumFiles),
00065         m_minMigSize(a_minMigSize),
00066         m_maxWaitTime(a_maxWaitTime),
00067         m_maxNumFiles(a_maxNumFiles),
00068         m_maxMigSize(a_maxMigSize),
00069         m_migJobs(a_migJobs),
00070         m_wakeUpTime(0),
00071         m_migDataSize(0),
00072         m_firstAppendTime(0),
00073         m_lastAppendTime(0),
00074         m_forcedMigration(false)
00075 {
00076     log_FUNC_m(hsm_FHmigc);
00077 
00078     log_DBG_m(dbg_LOW, "Create migration candidate list." << endl <<
00079         "-----------------------------------------------------------------------" << endl <<
00080         "MinWaitTime "   << setw(8)  << m_minWaitTime <<
00081         "\tMinNumFiles " << setw(8)  << m_minNumFiles <<
00082         "\tMinMigSize "  << setw(10) << m_minMigSize  << endl <<
00083         "MaxWaitTime "   << setw(8)  << m_maxWaitTime <<
00084         "\tMaxNumFiles " << setw(8)  << m_maxNumFiles <<
00085         "\tMaxMigSize "  << setw(10) << m_maxMigSize  << endl <<
00086         "-----------------------------------------------------------------------");
00087     m_iPM = i_PartitionManager::_duplicate(a_iPM);
00088 }
//============================================================================//

virtual hsm_FHmigc::~hsm_FHmigc (  )  [inline, virtual]

Definition at line 117 of file hsm_FHmigc.h.

00117 {};

Member Function Documentation

hsm_FH_p_li hsm_FHmigc::Append ( hsm_FileHeader *  a_fh_p  )  [virtual]

list methods

Reimplemented from hsm_FHlist.

Definition at line 92 of file hsm_FHmigc.cpp.

References dbg_NORM, hsm_FileHeader::GetFileSizeForMig(), hsm_FileHeader::IncrRef(), log_DBG_m, log_FUNC_m, hsm_FHlist::m_FHlist_x, m_firstAppendTime, m_lastAppendTime, m_maxMigSize, m_maxNumFiles, m_migDataSize, NULL, size, and WakeUp().

Referenced by hsm_FileHeader::ToMigCand().

00092                                                      {
00093     log_FUNC_m(Append);
00094     a_fh_p->IncrRef();
00095     cmn_MutexLock l(m_FHlist_x);
00096 
00097     // check if file is deletted is implement inside GetFileSizeForMig() method
00098     ivd_FileSize_t prevDataSize = m_migDataSize;
00099     m_migDataSize += a_fh_p->GetFileSizeForMig();
00100     if (  m_migDataSize >= m_maxMigSize
00101         || size()       >= m_maxNumFiles) {
00102         WakeUp();
00103 // fix bug 463
00104         if (prevDataSize <  m_maxMigSize
00105             || size()    == m_maxNumFiles ) { // show only first time
00106             log_DBG_m(dbg_NORM, "Migration is trigged." <<
00107                                 "  DataSize "  <<  m_migDataSize  <<
00108                                 "  Number of entryes " << size());
00109         }
00110     }
00111 
00112     m_lastAppendTime = time(NULL);
00113     if (m_firstAppendTime == 0) {
00114         m_firstAppendTime = m_lastAppendTime;
00115     }
00116     return insert(end(), a_fh_p);
00117 }

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_FHmigc::Remove ( hsm_FileHeader *  a_fh_p  ) 

Definition at line 121 of file hsm_FHmigc.cpp.

References hsm_FileHeader::DecrRef(), hsm_FileHeader::GetFileSizeForMig(), hsm_ListPos::GetMigCanPos(), log_FUNC_m, hsm_FHlist::m_FHlist_x, and m_migDataSize.

Referenced by hsm_ListPos::RemoveFromList().

00121                                               { // hsm_FH_p_li &a_pos
00122     log_FUNC_m(Remove);
00123     cmn_MutexLock l(m_FHlist_x);
00124 
00125     m_migDataSize -= a_fh_p->GetFileSizeForMig();
00126     a_fh_p->DecrRef();
00127     erase(a_fh_p->GetMigCanPos());
00128 }

Here is the call graph for this function:

Here is the caller graph for this function:

ivd_FileSize_t hsm_FHmigc::GetFilesListSize (  )  [inline]

Definition at line 167 of file hsm_FHmigc.h.

References m_migDataSize.

Referenced by DumpStatus(), and i_HSM_i::GetInfo().

00167 {return m_migDataSize;};

Here is the caller graph for this function:

void hsm_FHmigc::Reconfigure ( ivd_Time32_t  a_CfgMinWaitTime,
UInt32_t  a_CfgMinNumFiles,
ivd_FileSize_t  a_CfgMinMigSize,
ivd_Time32_t  a_CfgMaxWaitTime,
UInt32_t  a_CfgMaxNumFiles,
ivd_FileSize_t  a_CfgMaxMigSize 
)

Definition at line 675 of file hsm_FHmigc.cpp.

References dbg_LOW, log_DBG_m, log_FUNC_m, m_maxMigSize, m_maxNumFiles, m_maxWaitTime, m_minMigSize, m_minNumFiles, and m_minWaitTime.

Referenced by i_HSM_i::Reconfigure().

00680                                                                   {
00681     log_FUNC_m(Reconfigure);
00682 
00683     m_minWaitTime   = a_CfgMinWaitTime;
00684     m_minNumFiles   = a_CfgMinNumFiles;
00685     m_minMigSize    = a_CfgMinMigSize;
00686     m_maxWaitTime   = a_CfgMaxWaitTime;
00687     m_maxNumFiles   = a_CfgMaxNumFiles;
00688     m_maxMigSize    = a_CfgMaxMigSize;
00689 
00690     log_DBG_m(dbg_LOW, "Reconfigure migration candidate list." << endl <<
00691         "-----------------------------------------------------------------------" << endl <<
00692         "MinWaitTime "   << setw(8)  << m_minWaitTime <<
00693         "\tMinNumFiles " << setw(8)  << m_minNumFiles <<
00694         "\tMinMigSize "  << setw(10) << m_minMigSize  << endl <<
00695         "MaxWaitTime "   << setw(8)  << m_maxWaitTime <<
00696         "\tMaxNumFiles " << setw(8)  << m_maxNumFiles <<
00697         "\tMaxMigSize "  << setw(10) << m_maxMigSize  << endl <<
00698         "-----------------------------------------------------------------------");
00699 }

Here is the caller graph for this function:

ivd_Time32_t hsm_FHmigc::GetWakeUpTime (  )  [inline]

Definition at line 175 of file hsm_FHmigc.h.

References m_wakeUpTime.

Referenced by DumpStatus(), and i_HSM_i::GetInfo().

00175 {return m_wakeUpTime;};

Here is the caller graph for this function:

UInt32_t hsm_FHmigc::TrigMigration ( void   ) 

Definition at line 132 of file hsm_FHmigc.cpp.

References cmn_Condition::Broadcast(), ClientConf_t::CfgRecallOnly, g_clientConf_p, g_hsm_preJobList_p, cmn_Mutex::Lock(), log_FUNC_m, hsm_FHlist::m_FHlist_x, m_forcedMigration, m_thread_c, m_thread_x, hsm_JobListMgr::MigrateFailedJob(), size, and cmn_Mutex::Unlock().

Referenced by i_HSM_i::MigrateByAPI(), and i_HSM_i::TrigMigration().

00132                                        {
00133     log_FUNC_m(TrigMigration);
00134 // maybe will be used in future    g_hsm_activeList_p->OrphanedToMigList();
00135     UInt32_t count(0);
00136     if (!g_clientConf_p->CfgRecallOnly) {
00137         count += g_hsm_preJobList_p->MigrateFailedJob();
00138     }
00139 
00140     cmn_MutexLock l(m_thread_x);
00141     m_forcedMigration = true;
00142     m_thread_c.Broadcast();
00143     m_FHlist_x.Lock();
00144     count += size();
00145     m_FHlist_x.Unlock();
00146     return count;
00147 }

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_FHmigc::Run ( void *  arg  )  [virtual]

Wait MinWaitTime or wait to broadcast

Reimplemented from cmn_Thread.

Definition at line 151 of file hsm_FHmigc.cpp.

References cmn_Condition::Broadcast(), cfg_DAY, cmn_Global::dbg, dbg_DETAIL, dbg_NORM, errno, g_cmn, g_jobList_c, g_jobList_x, g_mig, hsm_MigByAPI::GetAPIMode(), log_Debugger::GetLevel(), cmn_Mutex::Lock(), log_DBG_m, log_FUNC_m, hsm_FHlist::m_FHlist_x, m_firstAppendTime, m_forcedMigration, m_goDown, m_lastAppendTime, m_maxMigSize, m_maxNumFiles, m_maxWaitTime, m_migDataSize, m_minMigSize, m_minNumFiles, m_minWaitTime, m_running, m_thread_c, m_thread_x, m_wakeUpTime, Migrate(), NULL, size, cmn_Time::Time2hms(), cmn_Condition::TimedWait(), TIMEXPIRE_d, and cmn_Mutex::Unlock().

00151                              {
00152     log_FUNC_m(Run);
00153     m_running = true;
00154 
00155     log_DBG_m(dbg_DETAIL, "Migration cand. thread is starting up");
00156 
00157     while (!m_goDown) {
00158 
00159         if (g_mig.GetAPIMode()){
00160             //recall only mode
00161             //let migration candidate list migrate once a day
00162             log_DBG_m(dbg_NORM, "minWaitTime set to 24h");
00163             m_minWaitTime = cfg_DAY;
00164         }
00165 
00166         m_wakeUpTime = time(NULL) + m_minWaitTime;
00167 
00168         {
00169             cmn_Time cmntimewakeup(m_wakeUpTime, 0);
00170             log_DBG_m(dbg_DETAIL, "MigCndLThread will wake up at latest "
00171                                 << cmntimewakeup.Time2hms());
00172         }
00173 
00174         try {
00175             m_thread_x.Lock();
00176 
00178             int retv = m_thread_c.TimedWait(m_wakeUpTime);
00179             m_thread_x.Unlock();
00180 
00181             if (m_goDown) {
00182                 break;
00183             }
00184 
00185             m_FHlist_x.Lock();
00186             UInt32_t numFile = size();
00187 
00188             if (numFile == 0) {
00189                 m_FHlist_x.Unlock();
00190                 log_DBG_m(dbg_DETAIL, "NO files to migrate");
00191 
00192                 //Broadcast to thread waiting for triggered migrations
00193                 log_DBG_m(dbg_DETAIL, "before Lock");
00194                 cmn_MutexLock l(g_jobList_x);
00195                 g_jobList_c.Broadcast();
00196 
00197                 continue;
00198             }
00199 
00200             if (g_cmn.dbg.GetLevel() == dbg_DETAIL) {
00201                 cmn_Time firstAppTime(m_firstAppendTime);
00202 
00203                 cmn_Time lastAppTime(m_lastAppendTime);
00204 
00205                 log_DBG_m(dbg_DETAIL,
00206                       "MigCndLThread properties of List: " << endl <<
00207                       "\tlast append time " << lastAppTime.Time2hms() <<
00208                       "\tminWaitTime "      << m_minWaitTime <<
00209                       " \tNumFiles "        << numFile << endl <<
00210                       "\tfirst append time " << firstAppTime.Time2hms() <<
00211                       "\tmaxWaitTime "      << m_maxWaitTime <<
00212                       " \tMigSize  "        << m_migDataSize <<
00213                       ((retv == 0) ? " TIMEOUT" : " BROADCAST"));
00214             }
00215             log_DBG_m(dbg_DETAIL, "APIMode()" << boolalpha << g_mig.GetAPIMode());
00216 
00217             if (  !g_mig.GetAPIMode() &&
00218                 ((  retv == TIMEXPIRE_d
00219                   && m_wakeUpTime    >= m_lastAppendTime + m_minWaitTime
00220                   && numFile       >= m_minNumFiles
00221                   && m_migDataSize >= m_minMigSize )
00222                   || m_wakeUpTime    >= m_firstAppendTime + m_maxWaitTime
00223                   || numFile       >= m_maxNumFiles
00224                   || m_migDataSize >= m_maxMigSize
00225                   || m_forcedMigration)) {
00226 
00227                 m_forcedMigration = false;
00228                 // reset list kumulative values
00229                 m_firstAppendTime = 0;
00230                 m_FHlist_x.Unlock();
00231                 log_DBG_m(dbg_DETAIL, "MigCndLThread - Migration NEED to be started!");
00232                 Migrate(numFile);
00233             } else if (g_mig.GetAPIMode()){
00234                 m_forcedMigration = false;
00235                 m_firstAppendTime = 0;
00236                 m_FHlist_x.Unlock();
00237                 log_DBG_m(dbg_DETAIL, "MigCndLThread - started by API");
00238                 Migrate(numFile);
00239 
00240                 //Broadcast to thread waiting for triggered migrations
00241                 log_DBG_m(dbg_DETAIL, "before Lock");
00242                 cmn_MutexLock l(g_jobList_x);
00243                 g_jobList_c.Broadcast();
00244             }
00245             else {
00246                 m_FHlist_x.Unlock();
00247                 log_DBG_m(dbg_DETAIL, "MigCndLThread - Migration DO_NOT_NEED to be started yet!");
00248             }
00249         }
00250         catch (ivd_Error &ie) {
00251             log_DBG_m(dbg_DETAIL, "MigCndLThread return ERROR " << ie);
00252             // TODO manage lost old MigCnd List
00253         }
00254         catch (ivd_SysError &ise) {
00255             log_DBG_m(dbg_DETAIL, "MigCndLThread WILL GO_DOWN on ERROR " << ise);
00256             break;
00257         }
00258         catch (...) {
00259             log_DBG_m(dbg_DETAIL, "MigCndListMgrThrd WILL GO_DOWN after thrown ERROR. errno = " << errno);
00260             break;
00261         }
00262     } // while (!m_goDown)
00263     log_DBG_m(dbg_DETAIL, "Migration cand. thread IS_DOWN.");
00264     m_running = false;
00265 }

Here is the call graph for this function:

void hsm_FHmigc::Shutdown (  ) 

Definition at line 269 of file hsm_FHmigc.cpp.

References log_FUNC_m, m_goDown, and WakeUp().

Referenced by hsm_Containers::Stop().

00269                           {
00270     log_FUNC_m(Shutdown);
00271     m_goDown = true;
00272     WakeUp();
00273 }

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_FHmigc::WakeUp ( void   ) 

Definition at line 277 of file hsm_FHmigc.cpp.

References cmn_Condition::Broadcast(), log_FUNC_m, m_thread_c, and m_thread_x.

Referenced by Append(), and Shutdown().

00277                             {
00278     log_FUNC_m(WakeUp);
00279 
00280     cmn_MutexLock l(m_thread_x);
00281     m_thread_c.Broadcast();
00282 }

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_FHmigc::Migrate ( UInt32_t  jobSize  ) 

Definition at line 286 of file hsm_FHmigc.cpp.

References dbg_DETAIL, dbg_LOW, dbg_NORM, g_hsm_fhLock, g_hsm_preJobList_p, g_mig, hsm_MigByAPI::GetAPIMode(), hsm_FileHeader::GetFileID(), hsm_FileHeader::GetFileSizeForMig(), hsm_FileHeader::GetFileType(), hsm_FileHeader::GetInode(), hsm_JobListMgr::GetJobIdx(), hjs_PREPARATION, ie_NULLPTR, ift_FILE, hsm_FileHeader::IsDeleted(), IVD_PRINT_ID_FS, cmn_Mutex::Lock(), log_DBG_m, log_FUNC_m, log_NOTE_m, hsm_FHlist::m_FHlist_x, m_goDown, m_maxMigSize, m_maxNumFiles, m_migDataSize, hsm_FileHeader::MigCanToPreJob(), NULL, hsm_JobListMgr::ReleaseJobIdx(), SendToPM(), hsm_JobListMgr::SetJobStatus(), size, cmn_Mutex::Unlock(), and hsm_MigByAPI::WasFileRequested().

Referenced by Run().

00286                                          {
00287     log_FUNC_m(Migrate);
00288 
00289     bool continueMig;
00290     do {
00291         UInt32_t preJobIdx  = g_hsm_preJobList_p->GetJobIdx(jobSize);
00292         log_DBG_m(dbg_LOW, "preJobIdx: " << preJobIdx << ", jobSize: " << jobSize);
00293         g_hsm_preJobList_p->SetJobStatus(preJobIdx, hjs_PREPARATION);
00294         UInt32_t count      = 0;
00295         UInt32_t filesNoFID = 0;
00296 
00297         ivd_FileSize_t migDataSize = 0;
00298 
00299         m_FHlist_x.Lock();
00300         hsm_FH_p_li iter = begin();
00301         while (  iter  != end()
00302             && count <= jobSize) {
00303             if (m_goDown) {
00304                 m_FHlist_x.Unlock();
00305                 g_hsm_preJobList_p->ReleaseJobIdx(preJobIdx);
00306                 return;
00307             }
00308 
00309             hsm_FileHeader &fh = (**iter);
00310             if (&fh == NULL) {
00311                 log_FUNC_m(MngDirty);
00312                 throw ivd_InternalError(ie_NULLPTR, "Mig candidate list contain NULL pointer.", true);
00313             }
00314 
00315             //  try to lock FH by inode
00316             ivd_GenInode_t inode = fh.GetInode();
00317             if (!g_hsm_fhLock.CanLockByID(inode)) {
00318                 iter++;
00319                 continue;  // can't use this FH already locked, so skip it
00320             }
00321             // now FH is locked, so list can be unlock
00322             m_FHlist_x.Unlock();
00323 
00324             if ( fh.GetFileID() == 0) {  // candidate for fileID
00325                 filesNoFID++;
00326             }
00327             if (   g_mig.GetAPIMode()
00328                 && (fh.GetFileType() == ift_FILE)
00329                 && (!fh.IsDeleted())
00330                 && (!g_mig.WasFileRequested(inode))) {
00331 
00332                 log_NOTE_m("File " << IVD_PRINT_ID_FS(inode) << " migration was not requested by API.");
00333             }
00334             fh.MigCanToPreJob(preJobIdx);
00335             count++;
00336             migDataSize += fh.GetFileSizeForMig();
00337 
00338             g_hsm_fhLock.UnLockByID(inode);
00339 
00340             if (   migDataSize >= m_maxMigSize
00341                 || count       >= m_maxNumFiles) {
00342                 log_DBG_m(dbg_NORM, "Job is large enough.  DataSize "  <<  m_migDataSize
00343                                 << " number of files " << count);
00344                 goto sendit;
00345             }
00346 
00347             m_FHlist_x.Lock();
00348             iter = begin();
00349         }
00350 
00351         m_FHlist_x.Unlock();
00352 
00353     sendit:
00354         if (count > 0) {
00355             SendToPM(count, filesNoFID, preJobIdx);
00356         }
00357         else {
00358             g_hsm_preJobList_p->ReleaseJobIdx(preJobIdx);
00359             log_DBG_m(dbg_DETAIL, "No more files to do preJob.");
00360         }
00361 
00362         m_FHlist_x.Lock();
00363         hsm_FH_p_l::size_type listsize = size();
00364         continueMig  =   m_migDataSize >= m_maxMigSize
00365                       || listsize      >= m_maxNumFiles;
00366         m_FHlist_x.Unlock();
00367         log_DBG_m(dbg_DETAIL, "m_migDataSize=" << m_migDataSize
00368                         << ", mig list size=" << listsize
00369                         << ", continueMig=" << boolalpha << continueMig);
00370     } while (continueMig);
00371 
00372 }

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_FHmigc::SendToPM ( UInt32_t  sizeTL,
UInt32_t  filesNoFID,
UInt32_t  preJobIdx 
)

Definition at line 376 of file hsm_FHmigc.cpp.

References hsm_MigByAPI::AddJob(), hsm_JobListMgr::At(), ivd_FS_File::Close(), cmn_Num2Str(), dbg_DETAIL, dbg_NORM, ivd_FS_File::e_Cache, fs_api::eDelFile, hsm_FileHeader::Event(), evt_ERROR, file, g_cmn, g_fs_api_p, g_hsm_fhLock, g_hsm_preJobList_p, g_hsmDB_p, g_jobList_x, g_mig, hsm_MigByAPI::GetAPIMode(), hsm_FileHeader::GetFileID(), GetFileIDs(), hsm_FileHeader::GetFName(), ivd_BaseException::GetFriendly(), hsm_FileHeader::GetInode(), ivd_FileSystemAPI::GetRootPath(), fio_DataBase::GetTransObj(), hjs_MIGFAILED, hjs_WAITFORDOMIG, hsm_MigByAPI::InsertJobInode(), ipc_EXEC_m, ivd_USleep, cmn_Mutex::Lock(), log_DBG_m, log_ERR_m, log_FUNC_m, log_WriteEvent(), hsm_MigByAPI::m_flags, m_goDown, m_iPM, hsm_JobElemVecMgr::m_jobElemVecMgr_x, hsm_MigByAPI::m_majColId, hsm_MigByAPI::m_minColId, ivd_Product::m_nameShort, hsm_FileHeader::MakePath(), hsm_FileHeader::MigPreJobToPM(), NULL, ivd_FS_File::Open(), cmn_Global::prod, hsm_JobListMgr::ReleaseJobIdx(), fio_DataBase::ReleaseTransObj(), hsm_FileHeader::SetFileID(), hsm_JobListMgr::SetJobStatus(), hsm_JobListMgr::SetMigJobID(), true, and cmn_Mutex::Unlock().

Referenced by Migrate().

00378                                                 {
00379 
00380     log_FUNC_m(SendToPM);
00381     int FileIDidx  = 0;
00382     UInt32_t    numfiles = 0;
00383 
00384     i_FileIDs_t_var tFIDs;
00385     if (!GetFileIDs(filesNoFID, tFIDs)) {
00386         log_ERR_m("Can't get fileIDs");
00388         return;
00389     }
00390 
00391     log_DBG_m(dbg_DETAIL, "  Sent To PM");
00392 
00393     log_DBG_m(dbg_DETAIL, "  To resolve " << sizeTL << " files for Migration");
00394 
00395     hsm_JobElemVecMgr &preJobVect = g_hsm_preJobList_p->At(a_preJobIdx);
00396     if (&preJobVect == NULL) {
00397         log_ERR_m("Can't get hsm_JobElemVecMgr");
00399         return;
00400     }
00401 
00402     list<hsm_FileHeader**> skipped;
00403 
00405     i_JobRequestList_t seqTFileList(sizeTL);
00406     seqTFileList.length(sizeTL);
00407 
00408     preJobVect.m_jobElemVecMgr_x.Lock();
00409     for (hsm_JobElem_v_i iter = preJobVect.begin(); iter != preJobVect.end(); iter++) {
00410         if (m_goDown) {
00411             preJobVect.m_jobElemVecMgr_x.Unlock();
00412             log_DBG_m (dbg_NORM, "Shut down. Abort SendToPM.");
00413             return;
00414         }
00415 
00416         hsm_FileHeader &fh = (**iter);
00417         if (&fh == NULL) {
00418             continue; // probably moved to dirty list
00419         }
00420         //  try to lock FH by inode
00421         ivd_GenInode_t inode = fh.GetInode();
00422         if (!g_hsm_fhLock.CanLockByID(inode)) {
00423             skipped.push_back(&*iter); // proceed latter
00424             continue;  // can't use this FH already locked, so skip it
00425         }
00426         preJobVect.m_jobElemVecMgr_x.Unlock();
00427 
00428         ivd_FileID_t fileID = fh.GetFileID();
00429         ivd_FS_File file(*g_fs_api_p, (g_fs_api_p->GetRootPath() + fh.MakePath()));
00430 
00431         if ( fileID == 0) {
00432             log_DBG_m (dbg_DETAIL, "SetFileID. tFIDs[" << FileIDidx << "]");
00433             try {
00434                 //HPUX - The file needs to be opened (to be in the kernel cache)
00435                 file.Open(ivd_FS_File::e_Cache);
00436                 fh.SetFileID((tFIDs)[FileIDidx++]);
00437             }
00438             catch (ivd_SysError &ise) {
00439                 // file is probably deleted in meantime
00440                 log_DBG_m(dbg_NORM, "Try 1. File is probably deleted. Simulate delete event.");
00441                 fio_Transaction &trans = *g_hsmDB_p->GetTransObj();
00442                 cmn_Path oldName(fh.GetFName());
00443                 fh.Event(fs_api::eDelFile, trans, &oldName);
00444                 g_hsmDB_p->ReleaseTransObj(trans);
00445 
00446                 g_hsm_fhLock.UnLockByID(inode);
00447                 preJobVect.m_jobElemVecMgr_x.Lock();
00448                 continue;
00449             }
00450         }
00451         fh.MigPreJobToPM(seqTFileList[numfiles++]);
00452 
00453         if ( g_mig.GetAPIMode() == true ){
00454             g_mig.InsertJobInode(fh.GetInode(), a_preJobIdx);
00455         }
00456 
00457         //HPUX Close file
00458         file.Close();
00459         g_hsm_fhLock.UnLockByID(inode);
00460         preJobVect.m_jobElemVecMgr_x.Lock();
00461     }
00462     preJobVect.m_jobElemVecMgr_x.Unlock();
00463 
00464     //  Proceed previous locked FH inode until are handled
00465     if (skipped.size() > 0) {
00466         log_DBG_m(dbg_DETAIL, "Some FH were locked. Try again.");
00467         ivd_USleep(10);
00468     // fix lock bug
00469         preJobVect.m_jobElemVecMgr_x.Lock();
00470         for (list<hsm_FileHeader**>::iterator iskip = skipped.begin();
00471              iskip  != skipped.end();) {
00472             if (m_goDown) {
00473                 preJobVect.m_jobElemVecMgr_x.Unlock();
00474                 log_DBG_m (dbg_NORM, "Shut down. Abort SendToPM.");
00475                 return;
00476             }
00477 
00478             hsm_FileHeader &fh = ***iskip; // these *** are not HOTEL stars
00479             if (&fh == NULL) {
00480                 skipped.erase(iskip++);
00481                 continue; // probably moved to dirty list
00482             }
00483             //  try to lock FH by inode
00484             ivd_GenInode_t inode = fh.GetInode();
00485             if (!g_hsm_fhLock.CanLockByID(inode)) {
00486                 // swap this element with last one
00487                 hsm_FileHeader** temp = *iskip;
00488                 *iskip = skipped.back();
00489                 skipped.back() = temp;
00490                 preJobVect.m_jobElemVecMgr_x.Unlock(); // to allow changes
00491                 log_DBG_m(dbg_DETAIL, "Unlock job list and sleep a while. ");
00492                 ivd_USleep(10);
00493                 preJobVect.m_jobElemVecMgr_x.Lock();
00494                 continue;  // can't use this FH already locked, so skip it
00495             }
00496             preJobVect.m_jobElemVecMgr_x.Unlock();
00497 
00498             ivd_FileID_t fileID = fh.GetFileID();
00499 
00500             if ( fileID == 0) {
00501                 try { // TODO update fileID in HSMDB
00502                     fh.SetFileID((tFIDs)[FileIDidx++]);
00503                 }
00504                 catch (ivd_SysError &ise) {
00505                     // file is probably deletted in meantime
00506                     log_DBG_m(dbg_NORM, "Try 2. File is probably deleted. Simulate delete event.");
00507                     fio_Transaction &trans = *g_hsmDB_p->GetTransObj();
00508                     cmn_Path oldName(fh.GetFName());
00509                     fh.Event(fs_api::eDelFile, trans, &oldName);
00510                     g_hsmDB_p->ReleaseTransObj(trans);
00511 
00512                     skipped.erase(iskip++);
00513                     g_hsm_fhLock.UnLockByID(inode);
00514                     preJobVect.m_jobElemVecMgr_x.Lock();
00515                     continue;
00516                 }
00517             }
00518             fh.MigPreJobToPM(seqTFileList[numfiles++]);
00519 
00520             if ( g_mig.GetAPIMode() == true ){
00521                 g_mig.InsertJobInode(fh.GetInode(), a_preJobIdx);
00522             }
00523 
00524 
00525             skipped.erase(iskip++);
00526             g_hsm_fhLock.UnLockByID(inode);
00527             preJobVect.m_jobElemVecMgr_x.Lock();
00528         }
00529         preJobVect.m_jobElemVecMgr_x.Unlock();
00530     }
00531 
00532     if (numfiles == 0) {
00533         log_DBG_m(dbg_NORM, "Sequence is EMPTY, all files are changed in meantime. JobList "
00534                              + cmn_Num2Str(a_preJobIdx));
00535         g_hsm_preJobList_p->ReleaseJobIdx(a_preJobIdx);
00536         return;
00537     }
00538 
00539 //    delete tFIDs;
00540 
00541     seqTFileList.length(numfiles);
00542 
00543     log_DBG_m(dbg_DETAIL, "  Sequence is prepared to call Migrate on PM of "
00544                         << seqTFileList.length() << " files");
00545 
00546     try {
00547         ipc_EXEC_m(
00548             i_JobID_t jobID;
00549             if ( g_mig.GetAPIMode() == true ) {
00550                 jobID = m_iPM->Migrate(seqTFileList, 0, g_mig.m_majColId, g_mig.m_minColId, g_mig.m_flags);
00551                 cmn_MutexLock l(g_jobList_x);
00552                 g_mig.AddJob(jobID);
00553             } else {
00554                 jobID = m_iPM->Migrate(seqTFileList, 0, 0, 0, 0);
00555             }
00556 
00557             log_DBG_m(dbg_NORM, "Job: " << jobID)
00558 
00559             g_hsm_preJobList_p->SetJobStatus(a_preJobIdx, hjs_WAITFORDOMIG);
00560             g_hsm_preJobList_p->SetMigJobID(a_preJobIdx, jobID, 0);
00561 
00562             log_DBG_m(dbg_DETAIL, "Called PM Migrate returned. Waiting for PM to call DoMigration.");
00563         );
00564     }
00565     catch (ivd_Exception &ie) {
00566         log_ERR_m("PM->Migrate() failed: " + ie.GetFriendly());
00567         ostringstream sstr;
00568         sstr << "Failed to start migration on " << g_cmn.prod.m_nameShort
00569              << "-PM (" << ie.GetFriendly() << ").";
00570         log_WriteEvent(evt_ERROR, sstr.str());
00571         g_hsm_preJobList_p->SetJobStatus(a_preJobIdx, hjs_MIGFAILED);
00572     }
00573     catch (...){
00574         log_ERR_m("PM->Migrate() failed: unknown exception");
00575         log_WriteEvent(evt_ERROR, "Failed to start migration on " + g_cmn.prod.m_nameShort + "-PM.");
00576         g_hsm_preJobList_p->SetJobStatus(a_preJobIdx, hjs_MIGFAILED);
00577     }
00578     return;
00579 }

Here is the call graph for this function:

Here is the caller graph for this function:

bool hsm_FHmigc::GetFileIDs ( UInt32_t  filesNoFID,
i_FileIDs_t_var &  tFIDs 
)

Definition at line 583 of file hsm_FHmigc.cpp.

References dbg_DETAIL, fio_Transaction::EndTransaction(), evt_ERROR, fio_RelFile::FirstNewIDX(), g_File2hdbID_p, g_hsmDB_p, ivd_BaseException::GetDescription(), ivd_BaseException::GetFriendly(), fio_DataBase::GetTransObj(), ipc_EXEC_m, log_DBG_m, log_ERR_m, log_FUNC_m, log_WriteEvent(), m_iPM, fio_DataBase::ReleaseTransObj(), fio_Transaction::StartTransaction(), and fio_RelFile::WriteRec().

Referenced by SendToPM().

00583                                                                        {
00584     log_FUNC_m(GetFileIDs);
00585 
00586     if (filesNoFID == 0) {
00587         return true;
00588     }
00589 
00590     i_FSC_var tFSC;
00591 
00592     log_DBG_m(dbg_DETAIL, "  MigList ask PM to get FSC");
00593 
00594     try {
00595         ipc_EXEC_m(
00596             tFSC = m_iPM->GetFSC();
00597         );
00598     }
00599     catch (ivd_Exception &ie) {
00600         log_ERR_m("Caught Corba or HSM exception: " + ie.GetDescription());
00601         return false;
00602     }
00603     catch (...){
00604         log_ERR_m("Caught other exception.");
00605         return false;
00606     }
00607 
00608     if( CORBA::is_nil(tFSC) ) {
00609         log_ERR_m("i_FSC is_nil!");
00610         return false;
00611     }
00612 //        log_DBG_m(dbg_DETAIL, "  MigList called FSC to get " << filesNumFID << " FileIDs");
00613     try {
00614         ipc_EXEC_m(
00615             (tFIDs) = tFSC->GetFileIDs(filesNoFID);
00616         );
00617     }
00618     catch (ivd_Exception &ie) {
00619         log_ERR_m("FSC->GetFileIDs() failed: " + ie.GetFriendly());
00620         ostringstream sstr;
00621         sstr << "Failed to allocate File IDs from FSC ("
00622                 << ie.GetFriendly() << ").";
00623         log_WriteEvent(evt_ERROR, sstr.str());
00624         return false;
00625     }
00626     catch (...){
00627         log_ERR_m("FSC->GetFileIDs() failed: unknown exception");
00628         log_WriteEvent(evt_ERROR, "Failed to allocate File IDs from FSC.");
00629         return false;
00630     }
00631 
00632     log_DBG_m(dbg_DETAIL, "  FSC return " << (tFIDs)->length() <<
00633                             " elements");
00634 
00635     if (filesNoFID != (tFIDs)->length()){
00636         return false;
00637 //            UInt32_t numids = tFIDs->length();
00638 //            throw ivd_InternalError(ie_EMPTYLIST ,"Number of returned FileID's " + cmn_Num2Str(numids) + " are not as requested " + cmn_Num2Str(filesNumFID));
00639     }
00640     //     Save max FileID;
00641     ivd_RecordIDX_t maxFileID = 0;
00642     i_FileIDs_t &fidseq = (tFIDs);
00643 
00644     int i = fidseq.length();
00645     while (i > 0) {
00646         --i;
00647         if (maxFileID < fidseq[i]) {
00648             maxFileID = fidseq[i];
00649         }
00650     }
00651     //
00652     ostringstream ostr;
00653     ostr << "Last fileID: " << maxFileID;
00654     log_WriteEvent(ostr.str(), "FSC");
00655 
00656     ivd_RecordIDX_t hdbMaxFileID = g_File2hdbID_p->FirstNewIDX();
00657     if (hdbMaxFileID <= maxFileID) {
00658         log_DBG_m(dbg_DETAIL, "Extend file2hdbID file up to idx " << maxFileID
00659                             << " prevoius max fileID = " << hdbMaxFileID);
00660         fio_Transaction *trans = g_hsmDB_p->GetTransObj();
00661         trans->StartTransaction();
00662         hdb_file2hdbID_t file2hdbID; // an empty record
00663         g_File2hdbID_p->WriteRec(maxFileID, &file2hdbID, 1, trans);
00664         trans->EndTransaction();
00665         g_hsmDB_p->ReleaseTransObj(&trans);
00666         log_DBG_m(dbg_DETAIL, "Max fileID after extenson = " << g_File2hdbID_p->FirstNewIDX());
00667     }
00668 
00669     return true;
00670 }

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_FHmigc::DumpStatus ( ostringstream &  sstr  ) 

Definition at line 702 of file hsm_FHmigc.cpp.

References cmn_Num2PowOfK(), GetFilesListSize(), GetWakeUpTime(), m_firstAppendTime, m_lastAppendTime, m_maxMigSize, m_maxNumFiles, m_maxWaitTime, m_minMigSize, m_minNumFiles, m_minWaitTime, size, cmn_Time::Time2hms(), and cmn_Time::Time2YMDhms().

Referenced by i_HSM_i::ShowStatus().

00702                                                {
00703     cmn_Time upMtime(GetWakeUpTime() ,0);
00704     cmn_Time firstAppendTime(m_firstAppendTime);
00705     cmn_Time lastAppendTime(m_lastAppendTime);
00706     cmn_Time minWaitTime(m_minWaitTime, 0, true);
00707     cmn_Time maxWaitTime(m_maxWaitTime, 0, true);
00708 
00709     sstr << "Number of files waiting for migration:  " << size() << endl;
00710     sstr << "Size of files waiting for migration:    " << cmn_Num2PowOfK(GetFilesListSize()) << endl;
00711     sstr << "Cfg: Max mig size                       " << m_maxMigSize << endl;
00712     sstr << "Cfg: Max number of files                " << m_maxNumFiles << endl;
00713     sstr << "Cfg: Min mig size                       " << m_minMigSize << endl;
00714     sstr << "Cfg: Min number of files                " << m_minNumFiles << endl;
00715 
00716     sstr << "Next Migration will wake up at:         " << upMtime.Time2YMDhms() << endl;
00717     sstr << "First append time:                      " << firstAppendTime.Time2YMDhms() << endl;
00718     sstr << "Last  append time:                      " << lastAppendTime.Time2YMDhms() << endl;
00719     sstr << "Cfg: min wait time:                     " << minWaitTime.Time2hms() << endl;
00720     sstr << "Cfg: max wait time:                     " << maxWaitTime.Time2hms() << endl;
00721 
00722                //   && m_wakeUpTime    >= m_lastAppendTime + m_minWaitTime
00723                //   && numFile       >= m_minNumFiles
00724                //   && m_migDataSize >= m_minMigSize )
00725                //|| m_wakeUpTime    >= m_firstAppendTime + m_maxWaitTime
00726                //|| numFile       >= m_maxNumFiles
00727                //|| m_migDataSize >= m_maxMigSize
00728                //|| m_forcedMigration) {
00729 }

Here is the call graph for this function:

Here is the caller graph for this function:

void hsm_FHmigc::DumpList ( ostringstream &  sstr  ) 

Definition at line 732 of file hsm_FHmigc.cpp.

References hsm_FileHeader::DumpListEl(), FSC_MAX_CORBA_BUFFERSIZE, hsm_FHlist::m_FHlist_x, m_goDown, NULL, and size.

Referenced by i_HSM_i::ShowStatus().

00732                                              {
00733     cmn_MutexLock l(m_FHlist_x);
00734 
00735     sstr << endl << "MIG Candidate List Contents" << endl;
00736     sstr << "ChgFlg     FileID               Inode   Name... Old name" << endl;
00737     sstr << "------------------------------------------------------------" << endl;
00738     hsm_FH_p_li iter = begin();
00739     for (;iter != end(); ++iter) {
00740         if (m_goDown) {
00741             sstr << "Shut down. Abort Dirty List." << endl;
00742             break;
00743         }
00744         hsm_FileHeader &fh = (**iter);
00745         if (&fh == NULL) {
00746             sstr << "NULL pointer found." << endl;
00747             break;
00748         }
00749         fh.DumpListEl(sstr);
00750         if (sstr.str().size() > FSC_MAX_CORBA_BUFFERSIZE) {
00751             sstr << "Stop dumping. Too much elements " << size()
00752                  << ". Buff size " << sstr.str().size() << endl;
00753             break;
00754         }
00755     }
00756     sstr << "Total files " << size() << endl;
00757 }

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

i_PartitionManager_var hsm_FHmigc::m_iPM [private]

Definition at line 117 of file hsm_FHmigc.h.

Referenced by GetFileIDs(), hsm_FHmigc(), and SendToPM().

bool& hsm_FHmigc::m_running [private]

outside variable that set when thread is down

Definition at line 122 of file hsm_FHmigc.h.

Referenced by Run().

bool hsm_FHmigc::m_goDown [private]

Definition at line 124 of file hsm_FHmigc.h.

Referenced by DumpList(), Migrate(), Run(), SendToPM(), and Shutdown().

cmn_Mutex hsm_FHmigc::m_thread_x [private]

Definition at line 127 of file hsm_FHmigc.h.

Referenced by Run(), TrigMigration(), and WakeUp().

cmn_Condition hsm_FHmigc::m_thread_c [private]

Definition at line 128 of file hsm_FHmigc.h.

Referenced by Run(), TrigMigration(), and WakeUp().

ivd_Time32_t hsm_FHmigc::m_minWaitTime [private]

If nothing change in minWaitTime perion and see m_minNumMigFile, m_minMigSize.

Definition at line 132 of file hsm_FHmigc.h.

Referenced by DumpStatus(), hsm_FHmigc(), Reconfigure(), and Run().

UInt32_t hsm_FHmigc::m_minNumFiles [private]

at least this number of files to be mig at minWaitTime

Definition at line 134 of file hsm_FHmigc.h.

Referenced by DumpStatus(), hsm_FHmigc(), Reconfigure(), and Run().

ivd_FileSize_t hsm_FHmigc::m_minMigSize [private]

Amount of data that has to be achive to migrate in minWaitTime.

Definition at line 136 of file hsm_FHmigc.h.

Referenced by DumpStatus(), hsm_FHmigc(), Reconfigure(), and Run().

ivd_Time32_t hsm_FHmigc::m_maxWaitTime [private]

If maxWaitTime passed from last migration then mig now.

Definition at line 139 of file hsm_FHmigc.h.

Referenced by DumpStatus(), hsm_FHmigc(), Reconfigure(), and Run().

UInt32_t hsm_FHmigc::m_maxNumFiles [private]

If m_maxNumFiles lay on list then migrate immediately.

Definition at line 141 of file hsm_FHmigc.h.

Referenced by Append(), DumpStatus(), hsm_FHmigc(), Migrate(), Reconfigure(), and Run().

ivd_FileSize_t hsm_FHmigc::m_maxMigSize [private]

If m_maxMigSize data is add to mig list then migrate it.

Definition at line 143 of file hsm_FHmigc.h.

Referenced by Append(), DumpStatus(), hsm_FHmigc(), Migrate(), Reconfigure(), and Run().

hsm_MigByAPI& hsm_FHmigc::m_migJobs [private]

Definition at line 145 of file hsm_FHmigc.h.

ivd_Time32_t hsm_FHmigc::m_wakeUpTime [private]

thread wake up time

Definition at line 149 of file hsm_FHmigc.h.

Referenced by GetWakeUpTime(), and Run().

ivd_FileSize_t hsm_FHmigc::m_migDataSize [private]

Definition at line 151 of file hsm_FHmigc.h.

Referenced by Append(), GetFilesListSize(), Migrate(), Remove(), and Run().

ivd_Time32_t hsm_FHmigc::m_firstAppendTime [private]

Definition at line 153 of file hsm_FHmigc.h.

Referenced by Append(), DumpStatus(), and Run().

ivd_Time32_t hsm_FHmigc::m_lastAppendTime [private]

Definition at line 155 of file hsm_FHmigc.h.

Referenced by Append(), DumpStatus(), and Run().

bool hsm_FHmigc::m_forcedMigration [private]

Definition at line 157 of file hsm_FHmigc.h.

Referenced by Run(), and TrigMigration().

hsm_FHmigc::log_CLASSID_m

Reimplemented from hsm_FHlist.

Definition at line 160 of file hsm_FHmigc.h.

The documentation for this class was generated from the following files:
Generated on Mon Feb 27 19:26:18 2012 for OPENARCHIVE by  doxygen 1.5.6