Research On Resource Scheduling Methods For Cross-media Knowledge Fusio

With the rapid development of social networks and the explosive growth of multi-modal media data,users’ retrieval needs for different types of data of the same object continue to increase.Therefore,the cross-media knowledge fusion technology used to meet this demand has attracted widespread attention.However,the increasing number of users and the increasing demand for efficient cross-media retrieval services make it difficult for the existing centralized cross-media knowledge fusion scheme to effectively respond to this large-scale user request scenario.This kind of cross-media retrieval application that contains multiple tasks is often constructed as a workflow model.Therefore,how to deploy distributed cross-media knowledge fusion services to handle such large-scale workflow applications has become an important issue for service providers to consider.In addition,affected by the continuous changes in user needs and dynamic changes in load,the system will have a large number of idle resources or excessive resource utilization.Therefore,how to implement reasonable resource scheduling in response to load changes is also an important issue.Aiming at the service deployment problem of distributed cross-media knowledge integration,this article proposes a resource scheduling algorithm that minimizes the cost of container leasing that starts from the perspective of the service provider.This algorithm meets the constraints of the task deadline specified by the user,and considers the complexity of the scheduling architecture,the heterogeneity of resources and the dynamics of the workload.The main work is as follows:(1)A unified modeling method is proposed for the diversity of cross-media knowledge integration applications.A resource scheduling framework in Cloud-Edge environment is proposed to solve the deployment and resource scheduling problems of distributed cross-media knowledge integration services.In addition,in order to simulate a large-scale cluster environment and evaluate the performance and stability of the algorithm,a cross-media knowledge fusion simulation platform oriented to the cloud side has been developed to simulate workflow scheduling in the cloud side environment.(2)In order to schedule tasks to the container with the lowest execution cost while trying to meet the deadline of the workflow,a scheduling scheme that combines online workflow scheduling and resource scaling strategies is proposed.The scheme sorts tasks based on the length of part of the critical path and assigns sub-deadlines to tasks in the workflow,and uses simulated deployment to perform automatic scaling.The simulation experiment results show that the algorithm has lower cost and higher success rate than the latest algorithm.(3)An automatic scaling strategy based on the average execution deviation and feedback coefficient of recently completed tasks is proposed.The strategy uses LSTM neural network to predict the arrival rate of the request,in order to obtain the trend of the request rate and guide the implementation of the scaling strategy.At the same time,the strategy uses parameter adjustments to obtain reasonable feedback coefficients to determine the amount of resource reduction,so as to reduce the frequency of oscillation problems.The simulation experiment results show that the algorithm has a lower cost and a higher and more stable success rate than the latest algorithm in most cases.