DOUBLE LAYER PROGRAMMING MODEL TO THE SCHEDULING OF REMOTE SENSING DATA PROCESSING TASKS

. Remotely sensed data are widely used in disaster and environment monitoring. To complete the tasks associated with processing these data, it is a practical and pressing problem to match the resources for these data with data processing centers in real or near-real time and complete as many tasks on time as possible. However, scheduling remotely sensed data processing tasks has two phases, namely, task assignment and task scheduling. This paper presents a model using bilevel optimization, which considers task assignment and task scheduling as a single problem. Using this architecture, a mathematical model for both levels of the problem is presented. To solve the mathematical model, this paper presents a cooperative coevolution algorithm that combines the ad- vantages of a very fast simulated annealing algorithm with a learnable ant colony optimization algorithm. Finally, the eﬀectiveness and feasibility of the proposed approach compared with the conventional method is demonstrated through empirical results.


1.
Introduction. There has been a rapid increase in the number of onboard satellite sensors as a result of the emergence of satellite technologies. Remotely sensed data originate from multiple sources and are complex in structure. Hence, processing these data using scheduling algorithms of the Earth observing satellites is becoming more important [39,20,4,5]. Recently, remotely sensed data have been widely used in disaster and environmental monitoring [27,6,9]. However, as the resolution of remotely sensed images becomes much higher, the automatic scheduling of remotely sensed data processing is becoming a challenge and a severe problem [21].
The main objective of remotely sensed data processing task comprehensive scheduling (RSDPTCS) is to assign remotely sensed data processing resources in a way that ensures the optimal use of limited resources to satisfy multifarious remotely sensed data processing requests from customers. RSDPTCS consists of two phases, namely, task assignment and task scheduling. Task assignment is responsible for rationally distributing the remotely sensed data tasks from the stations to the processing centers. Task scheduling is responsible for rationally distributing the remotely sensed data tasks to processing resources given the constraints of each processing center. The objective of the problem is to complete as many remotely sensed data processing tasks on time as possible. The research of this problem has important theoretical and practical value.
So far, the study of task scheduling and data processing has led to fairly mature approaches. Most of the research focuses on topics such as problem decomposition and recombination [34,42,15,7,25], algorithm optimization [31,37,16,35,28,40,18,26], and model building [13,23,24,10]. Over the past few decades, RSDPTCS has attracted increasing interest in the mission planning and scheduling area. Aloisio presented an architecture that shares the resources of different agencies and attaches importance to the access policies defined by the owners of the resources in a grid environment [1,2,3]. Karamchand established a cost model based on a centralized scheduling mode to evaluate continuous remotely sensed data processing [17]. Tehranian presented a robust distributed processing framework for processing remotely sensed data obtained from the NOAA satellite [30]. Xiang discussed the task management and scheduling techniques of the general high performance preprocessing system(GHIPS) and presented a task-level and algorithm-level parallel scheduling strategy to resolve the problem of remotely sensed data processing [36].Zhu developed a framework for a distributed computing system for remotely sensed images to support distributed and high-performance computing for geospatial images. Zhu further designed a new algorithm that supports parallel computing with a dynamic workload balance for large images [41]. Chen and Dong designed a hierarchical cluster to create a global task scheduler for the workflow process, improved the remote sensing quantitative retrieval system performance [8,11,12]. Ma proposed task-tree based mosaicking for remote sensed imagery at a large scale. The approach uses dynamic DAG scheduling to obtain an optimized schedule on a multi-core cluster with minimal completion time [22]. Recent years, the method of machine learning has been gradually applied to solve the scheduling problem [29].
The research mentioned above only considers the problem of scheduling the remotely sensed data processing task and ignores the importance of task assignment. The goal of this study is to address both parts of the problem by building a mathematical model and using a bilevel optimization approach to propose an algorithm for solving it.
2. Problem formulations. The RSDPTCS problem has a hierarchical nature: the upper level is the task-assignment level and the lower level is the task-scheduling level. As Fig. 1 shows, these levels are interoperable and collaborative decisionmaking processes. Upper-level optimization. The task assignment process that occurs at the upper level can be described as follows: processing centers form a cooperative plan to allocate the remotely sensed data processing task set on the basis of their own capabilities and the task constraints. By predicting the task transmission and completion times, a schedule is created that has the minimum expected penalty for tasks that are not completed on time.
Suppose M P C = {M P C1 , M P C2 , · · · , M P Cm } denotes the set of processing centers, and the number of processing centers is m. Let T = {T 1 , T 2 , · · · , T N T } denote the set of tasks, and the number of tasks is N T . Here, T i is a multi-component system: where T askID, ω i , and T ype i denote the identifier, priority, and data type of the task, respectively, T idf and T idl denote the predefined time and deadline of the task, and L imin and L imax denote the initial level and final level of task production. Let i is the distance between the station and processing center, and t idelay is the network delay, which is acquired by the adaptive model in [32]. Let t icomp denote the estimated completion time of T i . The mathematical model of task assignment is as follows: Equation 1 expresses the objective of task assignment, i.e.,to minimize the latecompletion penalty, Equation 2 indicates that every task only can be transferred to one processing center. Here, t icomp can be acquired by low-level processing.

2.2.
Low-level optimization. At the lower level, the task scheduling of remotely sensed data can be treated as the processing of data in a remotely sensed data processing grid consisting of resource processing nodes. The processing of each task involves the execution of dissimilar operations on the resource processing nodes in a sequence. Different resource nodes possess different processing capabilities (processing speed, and processing type). They are able to process only one operation at a time. Each operation utilizes only one resource node to finish the processing and is non-preemptive in nature. The schedule of remotely sensed data processing tasks is formed that minimizes the expected late-completion penalty, maximum computation time, and balances the resource node load.
Let M P R = {M P R1 , M P R2 , · · · , M P R m } and m denote the set and number of process resources respectively. T m = {T m 1 , T m 2 , · · · , T m n } ⊂ T denotes the tasks received by processing center M P Cm . Every task T m i consists of a number of procedures, i.e., P roc = {P rocID, T askID, ω i , t impl , t start , t end , F arP rocID} where P rocID denotes p(i, l, j) and ω i is the priority of the procedure, which is initially assigned to be the priority of the task, but can change at a later time. In addition, t impl is the minimum processing time of p(i, l, j) and is denoted as t p(i,l,j) , t start is the start processing time of p(i, l, j), and t end is the end processing time of p(i, l, j). F arP rocID is the procedure at the next level up. The meta-task is denoted by (p(i, l, j), M P Rm ), t B(p(i,l,j)) denotes the waiting time of p(i, l, j), and S p(i,l,j),m = {0, 1} denotes the coefficient of p(i, l, j) processed on m P Rm .
The mathematical model of the task scheduling is expressed as: t B(p(i,0,j)) = 0 (13) where c i denotes the completion time of T m i . Equations 3,4,5 express the objectives of the task scheduling: to minimize the late-completion penalty, minimize the maximum computation time, and balance the resource node load. Equation 6 indicates that each resource can only handle at most one procedure at a time. Equation 7 specifies that each procedure must be executed uninterrupted on a given resource. Equations 8,9,10 are the processing time constraints of the procedure. Equations 11,12,13 are the sequence constraints of the procedure.
3. Cooperative evolution algorithm. Figure 2 presents the cooperative coevolution algorithm proposed in this study for solving the mathematical model using bi-level programming. A cooperative coevolution algorithm is an algorithm combining the advantages of very fast simulated annealing and learnable ant colony optimization. The main part of the algorithm is the very fast simulated annealing algorithm. Learnable ant colony optimization is used to computation the completion task time and the scheduling plan.
The cooperative coevolution algorithm adopts parallel processing in the following two ways: 1) during task assignment, the data stations send the initial distribution plan (the remotely sensed data processing tasks) in parallel to the processing centers and 2) after initial distribution plan has been received, the processing centers computation the scheduling plans and lag costs in parallel and send feedback simultaneously. Based on the feedback, the algorithm adjusts the distribution plan until it generates the optimized plan. 1) Occupied resources prediction: the processing resources are used to process the remotely sensed data task, so this processing time is occupied time.
2) Fault prediction: when processing resources are not operational, they cannot be used and the tasks cannot be processed. Faults can be predicted using a Markov model. By analyzing the fault logs of resources, dynamic status space without faults and transition matrix are built into the Markov model to predict the availability of the resources and center.
The available time windows of the processing resources and centers are the times other than the occupied and fault times.
3.2. Plan evolution. According to following rules for plan evolution, the cooperative coevolution algorithm modifies distribution plan as follows: 1) Choose the best solution S best ; 2) Choose the worst tasks (i.e., a late-completion time greater than 0) in S best . If the quantity of tasks in S best is S, then the total quantity of the worst tasks is 0.1S; 3) Delete these worst tasks in S best and redistribute them (to any processing centers except the current one) to generate the new distribution plan S new ; 4) Compare with results of S best and S new and choose the better solution.
3.3. Learnable ant colony optimization algorithm. Based on the methods in [33,38], a learnable ant colony optimization algorithm is suggested to solve the low-level model using a coarsely parallel computation: 1) Attractive factor: let α s p(i,l,j),m be the attractive factor of an s-type pheromone for the selection of processing resource m for p(i, l, j) in available time window set p(i,l,j) . Hence, where τ s p(i,l,j),m is the s-type pheromone. 2) Rejection factor: let β ( p(i, l, j), m) s be the rejection factor of an s-type pheromone for the selection of processing resource m for p(i, l, j) in available time windows set p(i,l,j) . Hence, The learnable ant colony optimization algorithm is performed in two steps: 1)State transition rule The operations are randomly allocated by the ants to the machines proportionally as per the state transition rule. The proposed method adopts a pseudorandom proportional state transition rule to enhance the efficiency of the optimization algorithm [14].
where P r p(i,l,j),k (t) is the probability that procedure p(i, l, j) selects resource to process at time t, τ p(i,l,j),k (t) is the pheromone, P i is the priority of p(i, l, j), Q i is the data capacity of p(i, l, j), a is the pheromone value, b is the priority value of p(i, l, j), c is the data size value of the corresponding task for p(i, l, j). Parameters a, b, and c are set heuristically.
2) Pheromone update rule In the proposed method, we adopted a policy associated with a global update as well as local update. The pheromone is updated as follows: where ρ local is local the volatility of the pheromones, ρ global is the global volatility of the pheromones, Q is total pheromones, f is the objective function.

Results and discussions.
To evaluate the performance of the algorithm, we performed the following simulation experiments. Table 1 shows the distances between stations and processing centers used in the evaluation. station1 station2 PC1 PC2 PC3 PC1 PC2 PC3 3420 3568 2954 2486 1471 1770 Table 1. Distances between stations and processing Both stations have the same number of tasks (m = 50, 100, 200) and the processing centers have the same number of processing resources (n = 5, 10, 25).
In the present study, we set the processing start time of a task as a random number in the interval [9 : 00 : 00, 9 : 00 : 10] and the deadline of a task as a random number in the interval [10 : 30 : 00, 11 : 00 : 00] to generate tasks at different scales from 9:00:00 h to 11:00:00 h in a day. We assigned the processing resource nodes and capabilities randomly. We formulated nine problem instances and ran each of them 10 times. We compare our results to those obtained using the limited concentration model [19].
The empirical results obtained from the execution of the proposed approach are depicted in Figs. 3C6. The results are compared with respect to maximum computation time, average resource node load, late completion time, and processing time.
The results indicate that the proposed approach is better than the representative model in the field (the limited concentration model).    Conclusions. In the RSDPTCS problem, it is quite challenging to coordinate objects and constraints. This paper presented a model using bilevel optimization, which considers task assignment and task scheduling as a whole. At the upper level, processing centers make a cooperative plan to allocate the remotely sensed data processing tasks based on their own capabilities and constraints. At the lower level, the processing of each task involves the execution of dissimilar operations on the resource processing nodes in a sequence. We mathematically modeled the two levels of the problem, and proposed a cooperative coevolution algorithm that combines the advantages of very fast simulated annealing and learnable ant colony optimization to solve the proposed model. Experimental results comparing the results of the proposed method with those of the limited concentration model show the effectiveness and feasibility of our approach.