Sequential Inspection And Maintenance Optimization For Partially Observable Multi-State Systems

Maintenance is an important way to enhance reliability of a running system in its operation phases.In many engineering scenarios,systems are desired to complete a sequence of missions with finite breaks between two consecutive missions.Due to the limited resources,not all the desired maintenance activities could be performed in a break.One,therefore,has to choose an optimal subset of the maintenance activities to implement.Such a maintenance optimization strategy is known as selective maintenance.Traditional selective maintenance studies in the literature assumed that the components’ states can be known in advance without additional efforts.However,this basic premise may not always hold in real-world situations.The states of the components in a system need to be revealed by inspection activities,which also consume the limited resources shared with maintenance activities.On the one hand,the states of components can be better revealed by implementing inspection activities and further be used to derive a more effective maintenance strategy.On the other hand,an increasing number of inspections might reduce the maintenance activities that could be implemented.Furthermore,due to the complex operating environment and limited accuracy of sensors or measurement instruments,the inspections usually cannot fully reveal the true component states.Thereby,by taking account of the resource sharing between inspection and maintenance,and the uncertainty of inspection data,a sequential inspection and maintenance optimization strategy is needed to ensure the mission’s success.By taking account of the resource shared by inspection and maintenance activities,the multi-state nature of engineered systems,the interaction between inspections and maintenance activities,and the hierarchical system structure,this dissertation devotes to a sequential inspection and maintenance strategy for partially observable multi-state systems.The primary contributions to the existing body of knowledge are trifold:（1）Developing a sequential inspection and maintenance optimization method for multi-state systems with component-level imperfect inspections.The observation probability matrix and the state transition probability matrix of maintenance are introduced to characterize the uncertainties associated with inspection and maintenance actions,respectively.As the components’ states are only partially observable whereas the remaining time resource is fully observable,a finite-horizon mixed observability Markov decision process is formulated to model the sequential inspection and maintenance optimization problem.The dynamic programming algorithm is implemented to obtain the optimal inspection and maintenance policy.By the illustrative examples,the inspection and maintenance actions can be dynamically selected to achieve the maximum probability of the system successfully completing the next mission.（2）Developing a deep reinforcement learning method for sequential inspection and maintenance optimization with component-level imperfect inspections.Due to the continuous belief state,uncountable state space,and complex decision variables,a deep reinforcement learning algorithm,namely deep value network（DVN）algorithm,is customized to resolve the optimal inspection and maintenance policy.In the customized DVN algorithm,an artificial neural network is utilized to approximate the value function of the mixed observability Markov decision process.Moreover,the target network and experience replay techniques are implemented to break the correlations between the training data and ensure stability during the training process.As demonstrated by two numerical examples,the customized DVN algorithm can approximate the value function accurately and computational effectively.（3）Developing the sequential inspection and maintenance optimization for multistate systems with imperfect multi-level inspections.For the hierarchical structure system,the inspection data can be collected for multiple physical levels,such as the system level and component level.Firstly,the probability distribution of the component state combinations is utilized to characterize the system state.Secondly,the imperfect inspection,imperfect maintenance,and mission success model are developed under the imperfect multi-level inspection.Thirdly,a new mixed observability Markov decision model is formulated to model the resulted optimization problem.Finally,both the dynamic programming and DVN algorithms are extended to resolve the optimal inspection and maintenance policy in this scenario.Two illustrative examples reveal that taking account of multi-level inspection can enhance the probability of the system successfully completing the next mission.