1 Introduction

Ever-increasing large data processing requires a high-performance computing platform with cloudification in order to provide quality of service (QoS) to the end users with optimized infrastructure resource allocation. Despite the advantage of cloud computing systems such as a web-based structure, which gives the cloud computing system a great potential to be deployed by large data processing workloads from all over the world, the high operating and maintenance cost of the clouds makes them economically unfeasible to apply large batch/stream processing. Cloud computing as the next generation of IT platforms has to facilitate the use and availability of its resources for the large data processing workloads. Thus, one of the main challenges addressed here is the efficiency of the cloud computing systems for data processing.

The elasticity characteristic of cloud computing systems brings about a great consolidation gain in terms of energy saving and adaptability to workload dynamicity. However, as discussed in [1,2,3], the total resource utilization in cloud computing systems is around 20%, while the same metric in large data processing jobs varies between 60 and 80%, according to [4,5,6]. This motivated us to propose a preemptive approach for resource allocation of virtual machines (VMs) similar to that in [7] to create a more economically feasible solution for large data stream processing in the cloud computing environment.

Thus, the problem of improving the utilization of cloud computing systems via efficient allocation of active cloud resources to data stream processing workload is addressed in this paper.

However, an efficient resource allocation of processing workloads requires a precise modeling of the tasks of the data stream. Modeling the large data streams is relatively complex because of the dynamicity and heterogeneity of its tasks [8, 9]. Each large data stream can be modeled with parallel and sequential tasks. The performance of cloud computing services for task computing is analyzed in [6] by focusing on the prime characteristics of data streams, such as structures. Characteristics of a large data stream processing workload resemble those described in parallel production infrastructures (PPIs) [5] and the bag of tasks (BoTs) [10] where arriving jobs to a system comprise parallel and sequential tasks. Because of the high level of parallel processing in large data streams, scheduling of stream execution is more complicated than in a typical cloud job [10]. Owing to the existence of a substantial number of distributed system files coping with data streams and the complexity of the BoTs and PPIs in the large data stream processing workloads, current platforms cannot usually support the large data processing efficiently. As a result, large data processing workloads need a more complicated resource management and scheduling framework.

Existing queuing system models lack the abstractions to address both the large scale of processing job and heterogeneity of resources. For instance, G / G / c queue, as the most general type of the queueing systems assume similarity among the servers. Despite that, there is no closed-form exact analysis for resource allocation of these types of queues and different algorithms [11] are developed to approximate the performance metrics of these systems which are not accurate in large scale. Thus, heterogeneity and hugeness of resources and processing workloads compelled us to develop a new Markovian model to be able to analyze the resource allocation of large-scale processing jobs.

We first model the large data streams by considering sequential super-tasks. We assume a super-task is similar to BoTs so that each super-task consists of several tasks executed parallel to one another. According to the MapReduce, a workload consists of multiple flows of super-tasks where different super-tasks may include a different number of tasks. The super-tasks may arrive to the system according to a Poisson process [12,13,14]. Service time of a super-task begins with its arrival to the system and ends when there are no more tasks belonging to that super-task left in the system. Afterward, we present a model for cloud computing systems in which cloud users request resources in the form of VMs.

For further description of the model, it is required to define few phrases. CCRU: In general, VMs are hosted in physical servers occupying several types of resources, namely CPU, memory and storage. Within the current research, similar to [15, 16], a set or fragment of processors, servers or networks is considered as a base scale unit. In the rest of this paper, therefore, we use the phrase “Cloud Computing Resource Unit” (CCRU) to indicate the base cloud computing resource unit, including CPU, memory and disk. Heterogeneity of servers in a cloud center is defined by possessing different number of CCRUs, while heterogeneity of applications/services is defined by demanding various number of CCRUS for various amount of time. Without loss of generality, we presume that each task occupies a CCRU during its execution. Regular Job (primary job): typical cloud computing jobs run on Infrastructure as a Service (IaaS) cloud, which have a long life cycle in the cloud center. The average resource utilization of these types of jobs is not constantly high but dynamically varies over the time. The QoS metrics of these types of jobs cannot be affected by other jobs in the system. Jobs such as relational database service (RDS) and WebServers are categorized in this domain which stay in the cloud during their long life cycle. The type of a VM is determined by the type and size of regular job users’ demands. A regular job can reserve a fixed number of resources, while only a portion of the reserved CCRUs is active. In addition, regular jobs can dynamically generate new tasks during their lease and use their reserved CCRUs at any time. A semi-birth–death queuing system has also been utilized to model the task generation events for a regular job. Data stream processing job (secondary jobs): some sort of the jobs submitted to cloud is highly process-intensive. They need large cluster of instances on IaaS for a short period. Tasks associated with these type of jobs are not QoS-sensitive and can be suspended. The resource utilization of these types of jobs are high. Batch MapReduce-based processing jobs are considered in this category.

In this paper, we study the cloud computing system serving both large data stream processing jobs and regular jobs. It is important to understand the main differences between large data stream processing workloads and regular jobs in cloud computing systems [10]. The large data stream processing workloads require a vast number of resources [10], while typical cloud computing tenants require a relatively small to medium number of CCRUs to operate. Moreover, as was mentioned earlier, the resource utilization in high-demand stream tasks is much higher than in the typical workloads in clouds. We model the data processing workloads in the cloud computing systems in order to analyze preemption-based scheduling for the data streams. Next, a preemptive base scheduling technique similar to that in [7] is proposed to increase the resource utilization of the cloud computing system. We propose that the idle computing resource reserved by the regular jobs can be utilized temporarily by the secondary streams for improving resource utilization. However, secondary streams have to plan for migration to new computing resources, i.e., CCRUs, whenever the priority of regular jobs requires back their reserved resources.

Fig. 1
figure 1

Generic diagram representing the large data stream using idle resources of regular cloud users

Full size image

Figure 1 shows the topology of the proposed system. As depicted, a data stream comprises the flows of super-tasks. Each super-task includes a number of tasks. A task can occupy the idle computing resources reserved by the regular cloud users. The orange CCRUs in Fig. 1 indicate the CCRUs occupied by data streams that are reserved but not utilized by the cloud’s regular jobs, while the blue ones show the active utilized CCRUs occupied by the regular cloud users. Thus, Fig. 1 sheds light on the relation between data stream, super-tasks, tasks and the sequential/parallel execution of tasks within a super-task and across super-tasks.

The objective of this research is to make cloud computing datacenters better suited to delivering service for large data processing by optimizing the infrastructure resource utilization in the cloud computing system. The contribution of this paper is twofold. The main contributions of this paper are as follows:

  1. 1.

    A processing job comprising sequential and parallel tasks is modeled. Then, according to the proposed model, including the possibility of task failure, the resource demands for the data streams are investigated. Dynamic resource demand in the cloud computing platform is also analyzed, and the possible idle capacity of the cloud computing datacenter is calculated.

  2. 2.

    A preemptive VM handling algorithm is proposed to enhance the utilization of cloud computing systems. Accordingly, without loss of generality, an architecture is also proposed in order to make the algorithm applicable to common cloud computing platforms.

The remainder of this paper is ordered as follows: In Sect. 2, related works are discussed. In Sect. 3, the model for the data stream processing job is proposed. The resource requirements of the workload are analyzed in Sect. 4. In Sect. 5, the performance of the cloud computing system is investigated, followed by an analysis of the dynamicity of the cloud computing system. In Sect. 6, a preemptive scheduling algorithm is proposed. Next, simulation and experimental results are presented in Sect. 7. Section 8 discusses the challenges, and finally, in Sect. 9, the conclusion and findings of the paper are presented.

2 Related work

There are different proposed solutions to offer resources for cloud stream computing, some of which model the large data stream to achieve a better schedule for the resources. Many studies have investigated cloud computing systems as a platform for applications with streams. For instance, the Pegasus, proposed in [17], maps the tasks of a data stream into computing resources available at runtime in order to serve the jobs in grid systems efficiently. Ramakrishnan et al. [18] study the programmatic methods for scheduling, resource management and distributed launch and fault tolerance and provide a resource allocation platform in cloud resources appropriate for the large data streams. Furthermore, the Cumulus project [19] by combining the existing Grid infrastructures with cloud technologies intends to build a scalable cloud architecture adaptive to various VM hypervisors for a datacenter. Nephele, presented in [20], is a data processing framework to schedule and manage task executions over different types of VMs that are automatically created and terminated during the job execution.

The cost of stream processing on the Amazon cloud infrastructure is investigated in [21] via simulation. The results in [21] indicated that a cloud platform is rationally expensive for scientists to run their applications. Ludascher et al. [22] has a generic web service actor that works for large datasets. Handling web services, it provides an access to remote resources and services of large data workloads. Given the URL of a web service, the generic WebService actor of Kepler can be instantiated to serve operations. Scheduling algorithms for stream processing proposed in [19, 20] are performance effective; but according to the analysis in [22] for the data flow models discussed in [23], and experimental examples, they are too expensive to be applied for big data stream processing. This fact motivates us to make cloud systems more cost-efficient for stream processing.

On the other hand, [12,13,14] modeled and analyzed the performance of the cloud computing systems by applying Markov chains and queuing theory. In [14], a cloud computing center is modeled as an \(M/G/m/m+r\) queue, where r is the buffer size. The steady-state distribution of the queue length is determined via the transition probability matrix of the embedded Markov chain at arrival points, thus solving the equilibrium equations. Khojasteh et al. [24] also applied multi-dimensional Markov system based on a birth–death queueing system and G/m/c queues to model virtual machine provisioning systems.

The large data streams submit parallel tasks in the form of BoTs or PPIs, respectively. Lublin and Feitelson [5, 10] modeled the large data stream as a collection of BoTs and PPIs. According to the models proposed in [5, 10] and considering many-task computing jobs, the performance of commercial cloud computing services to serve the data streams are evaluated in [6]. Their results again indicate that the current clouds are not capable of serving the data streams efficiently and that they should be more adaptive to large workloads. In this paper, the stochastic model proposed for the cloud computing system is similar to the models in [12, 14], and the proposed data stream is similar to the streams in [5, 10]. The differences arise from the structure of the jobs. We allow a more complicated structure to find more accurate consolidation gains of preemptive resource allocation.

The preemptive base scheduling in distributed computing systems and cloud computing systems was first introduced in [7]. In [7], two types of users are considered: external and local. Local users have higher priority compared to external users. External users should submit their requests to the InterGrid Gateway to have access to the resources. They can only use the resources in the absence of local users. Contention between external and local users is resolved by preempting external leases in favor of local requests. VM-based leases of external users are preempted when the local users require them.

An algorithmic resource provisioning and scheduling approach based on particle swarm optimization (PSO) for the large data streams on IaaS that minimizes execution cost while taking into account deadlines is proposed in [25]. They modeled a workflow application as a directed acyclic graph (DAG) considering sets of tasks, dependency and inheritance between the tasks, which is similar to the assumption and modeling in this paper. Besides, for workflow, a set of different types of VMs may be leased. For each VM type, resource amounts, estimated lease start time and lease end time are associated. PSO minimizes the overall execution cost while ensuring constraints on user-defined deadlines.

The problem of energy-aware scheduling for independent, aperiodic real-time tasks in clouds is addressed in [25, 26]. Sossa and Buyya [25] first proposed scheduling architecture for real-time tasks in clouds. Secondly, the task-oriented energy consumption model is given and analyzed. Sossa and Buyya [25] also proposed the energy-aware scheduling algorithm called EARH that employs a rolling-horizon optimization method. Tasks following the last version of the Google cloud tracelogs were conducted to validate the superiority of EARH by comparing it with other baselines. The experimental results show it is suitable for real-time task scheduling in virtualized clouds. They consider independent tasks attributed by arrival time, task length, deadline and VM requirements.

In [27], a Hadoop scheduler, called LsPS, is developed to lessen the job response time of Hadoop systems by leveraging the job size patterns. First, the statistical information of the finished jobs is gathered; then, Welford’s one-pass algorithm is applied to update job size distribution statistics. Afterward, a self-tuning scheduling policy is designed to schedule Hadoop jobs to diminish the job response time. In this paper, the job response time is considered as a constraint, whereas in [27] improving job response time is the objective. In [28], a resource rental planning model is developed to make optimal job allocation and resource rental decisions so as to minimize cloud expenditures. As in this paper, the main contribution in [28] is the formulation of a planning model that approximates resource rental decisions. However, the approach in this paper is different from that in [28]. Zhao et al. [28] used a planning model as a guide to make cost-efficient resource allocation decisions in real time: but in this paper, our proposed model is used to allocate processing tasks preemptively so that they could result in cost efficiency.

In this paper, initially, a data stream consisting of parallel and sequential tasks similar to those in [25, 26] is modeled. Cloud computing regular jobs are also modeled according to [12,13,14, 24]. Then, according to these models, data stream requirements are investigated. Finally, the idea of a preemption-based scheduler is simulated in line with the work done in [7].

Fig. 2
figure 2

Data stream structure

Full size image
Table 1 Parameter definition
Full size table

3 Data stream model

In this section, we model the data streams, assuming that they consist of parallel flows of sequential super-tasks where each of these super-tasks consist of different numbers of tasks, as presented in Fig. 2 [9]. Note that, similar to most of processing jobs, e.g., batch jobs, the processing is initiated by a super-task named Super_ Task(Init) and is ended up by another super-task named Super_ Task(End). For more details, please refer to [9].

We also assume that class r super-task has r parallel tasks and occupies \(b_{r}\) CCRUs during its execution, and task execution times are exponentially distributed with parameter \(\mu \). Hence, heterogeneity of data streams is defined by demanding various number of CCRUS for various amount of time. Service time of a super-task begins with its arrival to the system and is completed when there are no more tasks belonging to that super-task in the system. The notation is given in Table 1.

It is worth pointing out that “method of stages” [29] permits one to study Erlang distribution by decomposing the service time distribution into a collection of structured sequential exponential distributions. Although the service time of the tasks is exponentially distributed, r sequential single tasks hold Erlang r distribution [29]. Thus, for jobs having tasks with Erlang distributed service times, analysis of this paper stays credible by decomposing a super-task with Erlang r distributed service times to r sequential super-tasks encompass a single task.

We derived the average service time of class r super-tasks, equal to the maximum of service time of the tasks of the super-task in [12],

$$\begin{aligned} \bar{T}_{r}= \frac{1}{\mu }\mathop {\sum }\limits _{i=1}^r \frac{\left( \begin{array}{c} r\\ i\\ \end{array} \right) }{i} {(-1)}^{i+1}\quad \quad \quad \quad \forall r\in \{1,\ldots ,R\} \end{aligned}$$
(1)

For details, please refer to [12]. Over time, there are some dependencies among the sequential super-tasks so that the complete execution of super-tasks may be required for the execution of previous trial super-tasks. We presume that a data stream has I parallel workflows of sequential super-tasks and that ith parallel sequential workflow has \(N_{r}^{i}\) class r super-tasks, as well. According to the mentioned dependency, the average total (cumulative) delay of a data stream, \(E\left[ d_{T} \right] \), is equal to

$$\begin{aligned} {\bar{d}}_{T}={\max }_{i\in \{1,\ldots ,I\}}\left( \mathop {\sum }\limits _{r=1}^R {N_{r}^{i}\bar{T}_{r}} \right) \end{aligned}$$
(2)

Equation (2) denotes that a data stream is not processed until all the workflows of the data stream are executed. Thus, the total average delay of the stream processing is strictly tied up to the workflow with maximum processing delay.

We assume also the following expression to approximate the upper and lower bounds of the total (cumulative) delay, \(\bar{d}_{T},\)

$$\begin{aligned} {\max }_{i}\left( \mathop {\sum }\limits _{r=1}^R {N_{r}^{i}\bar{T}_{r}} \mp 4\sigma _{i} \right) \end{aligned}$$
(3)

\(\mathop {\sum }\nolimits _{r=1}^R {N_{r}^{i}\bar{T}_{r}} \) represents the mean delay of the ith workflow and \(\sigma _{i}\) represents the standard deviation of ith parallel sequential workflow of the data stream. The mean and standard deviation were calculated by the analysis explained in “Appendix A.”

In addition, there are some factors that may lead to task execution failure. We identify two main reasons for stream task failure:

  1. 1.

    Jobs with higher priority (regular job) utilize the CCRU.

  2. 2.

    Server failure is due to hardware or software errors.

As mentioned earlier, regular jobs have higher priority than the super-tasks of the data stream. When the tasks of a regular jobs arrive to the CCRU, the CCRU utilization becomes high, and the secondary task of the stream leaves it. So, it is convenient that arrival of a task of a job with higher priority should preempt the task with less priority.

On the other hand, because of software and hardware problems, it is possible for task to fail; thus, when a primary job (regular job) comes to the CCRU, or there is hardware failure, the thread will be removed or halted to be executed. Then, however, through VM migration, the thread will be transferred into another appropriate CCRU to be executed. Under these circumstances, we need to find the service time characteristics of a stream task and the searching time for the secondary stream tasks to find the appropriate free CCRU in the datacenter.

We have presumed that CCRU failure inter-arrival time \((f_{C})\) and inter-arrival time of primary tasks (associated with regular jobs) \((f_{A})\) are exponentially distributed with parameters \(f_{C}\) and \(f_{A}\), respectively. x represents the CCRU task failure rate. Afterward, because of their independency and according to the [29] failure of inter-arrival time, a CCRU will have an exponential distribution with a mean equal to,

$$\begin{aligned} \bar{x}=\frac{1}{f_{C}}+\frac{1}{f_{A}} \end{aligned}$$
(4)

We know that successful service for a stream task happens when there is no arrival of a primary task during its execution and no CCRU failure.

Lemma 1

The average probability of successful service of a task is equal to

$$\begin{aligned} {\varvec{p}}_{{\varvec{sucess}}}=\frac{\varvec{\mu }}{\varvec{\mu } +\bar{{\varvec{x}}}} \end{aligned}$$

Proof

Please refer to “Appendix B.” \(\square \)

Consequently, the average number of preemptions, represented by e, experienced by a stream task follows a geometric distribution,

$$\begin{aligned} p_{F_{e}}=\left( \frac{\bar{x}}{\mu +\bar{x}} \right) ^{e}\left( \frac{\mu }{\mu +\bar{x}} \right) \end{aligned}$$
(5)

As mentioned earlier, after the task failure, the task should be migrated into another CCRU. The delay in finding an existing idle CCRU has a Coxian distribution [14]. The scheduler checks the server and if all CCRUs are busy, it will check the next server and so on. Hence, the average search delay for each server is:

$$\begin{aligned} \bar{\tau }_{n}=\mathop {\sum }\limits _{m=0}^M {(m+1)\tau _{a}\left( p_{b} \right) ^{m}} \end{aligned}$$
(6)

where \(\tau _{a}\) is the constant indicating the search delay for each server and M represents the total number of servers. \(p_{b}\) represents the probability of having all CCRUS of a server busy, which will be discussed in the next sections. Assuming that a load balancer assigns the load equally to the servers, the blocking probability, \(p_\mathrm{block}\), for each server will be the same [30]. Accordingly, the expectation of the search delay per preemption for a substantial number of servers is equal to

$$\begin{aligned} \bar{\tau }_{n}= \frac{\tau _{a}}{\left( {1-p}_{b} \right) ^{2}} \end{aligned}$$
(7)

On the other hand, the wasted time of the task execution for each preemption has to be calculated.

Proposition 1

Average wasted time of the stream task execution for each preemption will be equal to

$$\begin{aligned} \bar{\tau }_{w}=\frac{\bar{x}\mu }{\left( \bar{x}+\mu \right) ^{2}} \end{aligned}$$
(8)

Proof

Please refer to “Appendix B.” \(\square \)

Consequently, the average delay for an existing idle CCRU when considering the number of preemptions that have occurred will be as follows:

$$\begin{aligned} \frac{1}{\bar{\mu }}=\mathop {\sum }\limits _{e=1}^\infty {(\bar{\tau }_{n}+\bar{\tau }_{w})ep_{F_{e}}} \end{aligned}$$
(9)

where e, \(\tau _{n}\) and \(\tau _{w}\) are all mutually independent of each other. Accordingly, the average service time of a stream task is given by

$$\begin{aligned} \frac{1}{\bar{\mu }}=\frac{\bar{x}}{\mu }\left( \frac{\tau _{a}}{\left( {1-p}_{b} \right) ^{2}}+\frac{\mu \bar{x}}{\left( \mu +\bar{x} \right) ^{2}} \right) +\frac{1}{\mu } \end{aligned}$$
(10)

To obtain the second moment of the service time, the gross processing time characteristics discussed in preemptive repeat queuing in [31] can provide more details.

Obviously, the service time of a stream task is impacted by the primary tasks pertaining to the cloud computing customers (regular jobs). When the average probability of successful service for a stream task, \({\varvec{p}}_{{\varvec{sucess}}}\), is not high enough, it is better to execute multiple versions of secondary tasks in parallel, as is done in [32, 33]. Let us assume there are L multiple copies of a stream task executing simultaneously in parallel. Once one of the L replicated stream task executions is finished, the stream task could be considered done and all the other running tasks should be terminated accordingly. Under these circumstances, we will have preemption only if all the running tasks fail. Hence, success probability and fail probability in geometric distribution will change into \(1-\left( \frac{\bar{x}}{\mu +\bar{x}} \right) ^{L}\) and \(\left( \frac{\bar{x}}{\mu +\bar{x}} \right) ^{L}\), respectively. In such a condition, the average service time of a task is equal to

$$\begin{aligned} \frac{1}{\bar{\mu }}=\frac{\left( \frac{\bar{x}}{\mu +\bar{x}} \right) ^{L}}{1-\left( \frac{\bar{x}}{\mu +\bar{x}} \right) ^{L}}\left( \frac{\tau _{a}}{\left( {1-p}_{b} \right) ^{2}}+\frac{\mu \bar{x}}{\left( \mu +\bar{x} \right) ^{2}} \right) +\frac{1}{L\mu } \end{aligned}$$
(11)

Consequently, replication of stream task execution helps to decrease the service time of the task. Note that the value of Ldoes depend on the failure rate in the system.

4 Resource requirements of data streams

Super-tasks arrive at the system according to a Poisson process. \(\lambda =\left[ \lambda _{1}\ldots \lambda _{r}\ldots \lambda _{R} \right] \) denotes the arrival rates of different super-tasks consisting of \(b_{r}=[b_{1},\ldots b_{R}]\) tasks.

Fig. 3
figure 3

State transition rate diagram of the data stream

Full size image

In this section, the semi-birth–death queuing system illustrated in Fig. 3 is proposed to model the discussed cloud computing data streams. Figure 3 shows the state transition diagram in which birth coefficients are set equal to \(\lambda \) elements and death coefficients are dependent on the service time of the tasks found in Eqs. (10) and (11). It can be seen that in state k, there are k CCRUs running to serve their jobs, so the death coefficient is equal to \(k\mu \). The objective of this part is to determine the distribution and average of the total CCRUs required to serve the stream.

We assume that class r super-task occupies \(b_{r}\) CCRU during its execution and task execution time is exponentially distributed with parameter \(\mu \). In the simplest case, this can be narrowed down to \(b_{r}=r\), where each task occupies only one CCRU during its execution time.

We first analyze the system of a datacenter without considering the constraint on the number of CCRUs serving different classes of super-tasks \((S=\infty ,r=1,\ldots R)\).

Theorem 1

The probability generating function (PGF) of the number of occupied CCRUs by the data stream, \(P\left( z \right) \), is given by

$$\begin{aligned} P\left( z \right) =e^{\frac{\left( -\mathop {\sum }\nolimits _{r=1}^R \lambda _{r} \mathop {\sum }\nolimits _{i=1}^r \frac{1}{i} \right) +\left( \mathop {\sum }\nolimits _{r=1}^R \lambda _{r} \mathop {\sum }\nolimits _{i=1}^r \frac{z^{i}}{i} \right) }{\mu }} \end{aligned}$$
(12)

Proof

Please refer to “Appendix C.” \(\square \)

Hence, the average and variance of the number of CCRUs that should be allocated to the stream are equal to

$$\begin{aligned} \bar{k}_{T}= & {} \frac{dP\left( z \right) }{dz}\vert _{z=1}=\frac{\mathop {\sum }\nolimits _{r=1}^R {r\lambda }_{r} }{\mu } \end{aligned}$$
(13)
$$\begin{aligned} \textit{VAR}_{k_{T}}= & {} {\left( \frac{d^{2}P\left( z \right) }{dz}+\frac{dP\left( z \right) }{dz}-\frac{dP\left( z \right) }{dz}^{2}\right) \vert }_{z=1}=\frac{\mathop {\sum }\nolimits _{r=1}^R {r(\frac{r+3}{2})\lambda }_{r} }{\mu } \end{aligned}$$
(14)

Equations (13) and (14) can be used to estimate the number ofresources required to serve the stream. Now let us assume finite resources with S CCRUs and model the system with semi-birth–death processes. Let \(p_{k}\) denote the probability that there will k tasks in the system; then, the global balance equation (GBE) of the system may be written as

$$\begin{aligned} \left\{ {\begin{array}{ll} \left( \mathop {\sum }\nolimits _{r=1}^R \lambda _{r} +k\mu \right) p_{k}=\left( k+1 \right) \mu p_{k+1}+ \mathop {\sum }\nolimits _{r=1}^R {p_{k-b_{r}}\lambda }_{r}, &{} 0<k<S \\ \mathop {\sum }\nolimits _{r=1}^R \lambda _{r} p_{0}={\mu p}_{1},&{} k=0 \\ S\mu p_{S}=\mathop {\sum }\nolimits _{r=1}^R {{p_{S-b_{r}}\lambda }_{r}},&{}k=S\\ \end{array}} \right. \end{aligned}$$

The above equations cannot be solved through transform analysis, but the distribution of the number of busy CCRUs may be determined from the above recursive equations together with the normalization condition. Later, the average of the total number of the busy CCRUs is given by

$$\begin{aligned} \bar{k}_{T}=\mathop {\sum }\limits _{k=0}^S {kp_{k}} \end{aligned}$$
(15)

Let \(P_{B_{r}}\) denote the blocking probability of class r super-tasks given by

$$\begin{aligned} p_{B_{r}}=\mathop {\sum }\limits _{k\vert (k+b_{r}>S)} {p_{k}=} \mathop {\sum }\limits _{k=S-r+1}^S p_{k} \end{aligned}$$
(16)

4.1 Distributed stream computing

In this section, we extend our analysis to the realistic distributed scenario in which regular jobs and stream tasks are distributed among multiple servers. Communication among the output of sequential tasks running on different servers adversely affects the stream processing performance. In particular, data access latencies become a challenge in stream processing [34]. Thus, we continue our analysis to find the probability distribution of stream super-tasks over the servers.

Let us consider that we have M separate servers in a datacenter where each of them includes \(m= S/M\) CCRUs. Resource sharing policies in cloud computing aim to minimize the number of active servers of the datacenter while supporting users’ QoS requirements. Therefore, the position of the mapped server for the tasks of a super-task could influence the total performance of the system. Each CCRU of the datacenter has an ID. Let us simply observe that CCRU ID of the first server ranges from 1 to m, for the second server the IDs are in the range of \(m+\)1 to 2m and so on. For this reason, for the last server the CCRUs are in the range of \(mM{+}1-m\), mM. Note that according to the heterogeneity of servers analysis can be easily modified. For instance, if the capacity of the second server is two times bigger than the first one, for the second server the CCRU IDs will be in the range of \(m+\)1 to 3m. We consider a resource sharing policy \((\pi )\) that minimizes the number of active servers in the datacenter required to serve the data stream. Because of this, it always starts from the first free computing spots. When a super-task arrives at the system, the system assigns the arriving tasks to the first idle CCRU of the first idle server. Hence, the position of the occupied computing unit is important in the sharing policy. Let us define \(k_{D}\) as the maximum ID number of the occupied CCRU in the datacenter. In the resource sharing policy, \(\pi \) minimizes (controls) \(k_{D}\) so that

$$\begin{aligned} k_{\pi }=\min k_{D} \end{aligned}$$
(17)

Let us also define n as follows,

$$\begin{aligned} n=\frac{k_{\pi }}{m}+1 \end{aligned}$$
(18)

Thus, the probability distribution of n is equal to:

$$\begin{aligned} pr(n)=mp_{k_{\pi }}(mn-m) \end{aligned}$$
(19)

To obtain the probability distribution of the number of active servers, we have to use the following equation (the probability massive function of the number of active servers):

$$\begin{aligned} q_{\vartheta }=\mathop {\int }\nolimits _{\vartheta -1}^\vartheta {pr\left( n \right) dn} \end{aligned}$$

Then,

$$\begin{aligned} \bar{\vartheta }=\mathop {\sum }\limits _{m=1}^\infty {\vartheta q_{\vartheta }} \end{aligned}$$
(20)
Fig. 4
figure 4

State transition diagram of maximum ID of in-service CCRUs

Full size image

It is important to note that \(k_{\pi }\) (the maximum ID number of occupied computing units) is completely different from k (the number of the busy CCRU in the system). k and \(k_{\pi }\) together give us visibility on the number and position of the busy CCRUs and servers required to serve the super-tasks of the stream. Using the proposed policy \(\pi (k,k_{\pi })\), we analyze the cloud computing system. First, we need to obtain the probability distribution of \(k_{\pi },\) which may be modeled as a semi-Markov process (Fig. 4).

Note that in Figs. 3 and 4, it is possible to get to any state from any state, which makes state space a single communicating class, and consequently, the associated Markov Chains in these figures are irreducible. Moreover, states are contained in cycles in which the greatest common divisor of the cycle length is greater than one that makes them aperiodic. It is convenient that a finite state irreducible Markov chain with aperiodic states has ergodic property [35].

Figure 4 also shows the transition diagram of \(k_{\pi }\) conditioning on variable k. As a result of obtaining \(k_{\pi }^{'}\) distribution, a birth coefficient from state \(k_{\pi }\) to \(k_{\pi }+j\) is equal to:

$$\begin{aligned} \hat{\lambda }_{j}^{k_{\pi }}=\mathop {\sum }\limits _{i=j}^R {\lambda _{j}\phi \left( {(i-j)}_{k_{\pi }} \right) } \end{aligned}$$
(21)

where \(\phi \left( l_{k_{\pi }} \right) \) denotes the probability of having \(l_{\acute{k}_{\pi }}\) task arrive into the system without any departure, with the following binomial distribution,

$$\begin{aligned} \phi \left( l_{k_{\pi }} \right) =\left( \frac{\mu }{\mu +\mathop {\sum }\nolimits _{i=1}^R \lambda _{i} } \right) ^{l_{k_{\pi }}}\left( \frac{\mathop {\sum }\nolimits _{i=1}^R \lambda _{i} }{\mu +\mathop {\sum }\nolimits _{i=1}^R \lambda _{i} } \right) ^{k_{\pi }-l_{k_{\pi }}} \end{aligned}$$
(22)

Death coefficients are given by

$$\begin{aligned} \mu _{i}^{k_{\pi }}=\mu p_{k_{\pi }-i}^{b} \end{aligned}$$
(23)

where \(p_{k_{\pi }-i}^{b}\) denotes probability of having \((i-1)\) idle CCRUs in the range of \((k_{\pi }-1,\ldots ,k_{\pi }-i+1)\) and busy status for the \(k_{\pi }-i\) CCRU step.

$$\begin{aligned} p_{k_{\pi }-i}^{b}=\frac{\left( {\begin{array}{l} k_{\pi }-i-1 \\ k-2 \\ \end{array}} \right) }{\left( {\begin{array}{l} k_{\pi }-1 \\ k-1 \\ \end{array}} \right) } \end{aligned}$$
(24)

Now we can model the system as a two-dimensional semi-birth–death process. We analyze the two-dimensional Markov chain \((k_{\pi },k)\) via matrix-analytic methods, which are numerical methods tailored to such chains (see [36, 37]) to obtain, \(p(k_{\pi },k)\) the joint probability distribution of \(k_{\pi },k\).

4.2 Numerical results for stream processing requirements

Next, we present the numerical results for modeling a data stream. We consider two real different data streams: fluid mechanic scientific processing stream, finite element and cellular automata co-simulation (with low task arrival rate), and commercial video processing stream, AVI to MPEG compression (large data stream with large task arrival rate).

Figures 5 and 6 show the \(p(k_{\pi },k)\) distribution for these two streams with different parameters, such as total super-task arrival rate and capacity. As indicated, when the scale of the stream increases, the average difference between \(k_{\pi }\) and k will be perpetual. Hence, our analysis sheds light on the requirements of data streams. For instance, Fig. 5 shows that there is no need for more than 15 CCRUs to process the scientific stream task. For more details, please refer to [8].

4.3 Simulation results for generalizing the service time distribution

In our analysis, we assume that a data stream is a collection of sequential super-tasks. A super-task consists of several tasks executed independently in parallel to one another. According to [12, 30], the average service time of super-tasks is calculated in Eq. (1). However, in real-life scenarios, super-tasks could be distributed according to other types of distributions.

In this subsection, we maintain that the analysis in Eqs. (17)–(24) can provide a good approximation for other types of processing streams as well.

Fig. 5
figure 5

Joint probability distribution of \({k,\acute{k}}_{\pi }\) with \(\lambda _{T}=5 , \mu =1 S=20\) (fluid mechanic scientific stream)

Full size image
Fig. 6
figure 6

Joint probability distribution of \({k,\acute{k}}_{\pi }\) with \(\lambda _{T}=30, \mu =1\) and \(S = 100\), \(R=5\) (commercial video processing stream)

Full size image
Fig. 7
figure 7

Numerical and simulation results for the probability distribution of active servers with \(\lambda _{T}= 30\), \(\mu =1\)

Full size image

We simulated a video processing data stream in a way similar to that in the previous subsection with \(R=5\), in which tasks are, instead of being exponentially distributed, normally distributed, with \(\mu =1, \sigma =0.2\). Next, we present numerical results based on Eqs. (17)–(24) and the simulation results of the data stream with normally distributed tasks. The presumed CCRU is the Amazon EC2 elastic compute unit (ECU) since Amazon ECU delivers approximately 1.0 GHz CPU. Each server contains 20 ECUs. Figure 7 shows the numerical results for the probability distribution of the number of active servers required to serve the video processing data stream for \(\lambda _{T}=30\). Figure 8 shows the average number of active servers required to serve the large data stream as a function of the total super-task arrival rate.

Fig. 8
figure 8

Numerical and simulation results for the average number of active servers as a function of total stream super-task arrival rate, \(\mu =1\)

Full size image

The findings reveal that the required resources of the numerical results obtained for exponential distribution and the simulation results of data stream with normally distributed tasks are not only different but also in good agreement in both figures, which verifies the modeling and analysis done in Sects. 3 and 4.

5 Cloud computing system model

After modeling the large data stream and its required resources, it is time to model the cloud computing center as the stream processing platform. A cloud computing center has to support and serve the regular jobs as well as the data stream. Sections 5 and 6 shed light on the interaction of the regular jobs and the data streams in order to assess the stream processing capacity of the cloud center occupied with regular heterogeneous types of jobs. Here, we try to model a regular job pattern in the cloud computing system.

Assuming multiple classes of regular jobs, the arrival rates of which follow a different Poisson process, each class has its own service rate and job size. The number of CCRUs a regular job requires determines its size, which remains constant during its service time. Each task of a regular job requires a CCRU for its execution. We will assume that there are \(R^{'}\) distinct VM configurations, where each VM configuration is specified in terms of job requirements. The service time of all classes is arbitrarily distributed with different service rates (general service time distribution). The service time of a job begins with its arrival to the system; at the end of that service time, all of its occupied CCRUs are released simultaneously. In other words, the processing units related to an arriving regular job are provisioned and released together.

We assume finite resources; thus, a job will be blocked if there are not enough idle CCRUs to serve it. The objective of the following analysis is to determine a joint distribution of the number of jobs from each class, the job blocking probabilities and the distribution of the utilization of resources.

Let us define a state of the system as a number of regular jobs from each class in the system, \({\vec {n}=(n}_{1}{,n}_{2}\ldots ,n_{r},\ldots n_{R^{'}})\), and \(p\left( \vec {n} \right) \) as the distribution of \(\vec {n}\).

First, we consider a system with finite resources of S CCRUs. In this case, an arriving job will be lost if there are not enough free CCRUs to serve it. This model is the same as blocking in a shared resources environment studied in [38]. From there, the joint probability distribution of the number of jobs in the system is given by

$$\begin{aligned} \varphi \left( \vec {n} \right) =\frac{1}{G}\mathop {\prod }\nolimits _{r=1}^{R^{'}} \frac{\rho _{r}^{n_{r}}}{n_{r}!} \end{aligned}$$
(25)

where G is the normalization constant and \(\rho _{r}=\frac{\lambda _{r}^{'}}{\mu _{r}^{'}}\). Here \(\lambda _{r}^{'}\) denotes the arrival rate of the class r jobs of the cloud computing center and \(\mu _{r}^{'}\) denotes the associated service time. Let j denote the number of reserved CCRUs at the computing center; then, \(j=\vec {n}.\vec {B}^{T}\) where \(\vec {B}=[b_{1}^{'},\ldots b_{r}^{'},\ldots b_{R^{'}}^{'}]\). Defining probability distribution of the number of busy CCRUs in the cloud computing center as

$$\begin{aligned} q_{j}=Pr\left( j=\vec {n}.\vec {B}^{T} \right) \end{aligned}$$
(26)

we apply the algorithm proposed in [12, 38] to calculate \(q_{j}\) recursively. For more details, please refer to [12]. The average number of busy CCRUs in the system,\( \bar{k}_{T}^{'},\) is given by

$$\begin{aligned} \bar{k}_{T}^{'}=\mathop {\sum }\limits _{j=1}^S {jq_{j}}. \end{aligned}$$
(27)

5.1 Dynamicity of busy CCRUs by regular jobs

This is about how users with different demands rent CCRUs; however, these users do not use all of their rented resources. Now let us assume that using parallel computing, execution time of the data stream task is smaller than the lifetime of a regular job in the system. As a result, we can presume a system in which there are always N regular jobs (VMs) in service. The number of regular jobs for stream tasks in the system can be considered fixed. Also, the new job initially requires service for a single task and generates new tasks according to a Poisson process with parameter \(\alpha \). Nevertheless, because of the limited capacity of some users, the following analysis is a reflection of a plausible upper bound for the dynamicity of the system.

The service time of a regular job remains as it was in the previous case. The objective of analysis is to determine the total number of idle CCRUs in the system. Since each job has at least a single task in the system, the minimum number of active tasks in the system will be N. Defining \(k_{b}^{'}\) to be the total number of inuse and busy CCRUs occupied by regular jobs, it can be modeled as a semi-Markov birth–death process with the following birth and death coefficients at state k,

$$\begin{aligned} \alpha _{k}=N\alpha ,y_{k_{b}^{'}}=k_{b}^{'}yk_{b}^{'}\ge N \end{aligned}$$
(28)

where y represents the execution rate of the cloud computing regular jobs tasks. \(\alpha \) and y differ from \(\lambda _{r}^{'}\)s and \(\mu _{r}^{'}\)s in the sense that \(\alpha \) and ystrictly depend on the utilization dynamicity of regular jobs rather than their arrival and departure. More details about their definitions are given in the next section. Regarding the inter-arrival time of the regular jobs tasks defined in Eq. (28), note that \(\alpha =1/f_{A}\).

Thus, the distribution of the total number of busy CCRUs in the system is given by the product form solution,

$$\begin{aligned} p_{k_{b}^{'}}\left( k\vert N \right) =p_{N}\mathop {\prod }\nolimits _{i=N}^{k-1} \frac{\alpha _{i}}{y_{i+1}}\quad k \ge N \end{aligned}$$
(29)

Substituting Eq. (28) in the above,

$$\begin{aligned} p_{k_{b}^{'}}\left( k\vert N \right) =p_{N}\frac{N!}{k!}\left( \frac{N\alpha }{y} \right) ^{k-N}\quad k\ge N \end{aligned}$$
(30)

where \(p_{N}\) is determined from the normalization condition, \(\mathop {\sum }\nolimits _{k=N}^\infty {p_{k_{b}^{'}}\left( k \right) =1}\),

$$\begin{aligned} p_{N}=\frac{1}{\mathop {\sum }\nolimits _{k=N}^\infty {\frac{N!}{k!}\left( \frac{N\alpha }{y} \right) ^{k-N}} }=\frac{\left( \frac{N\alpha }{y} \right) ^{N}}{N!\left( e^{\frac{N\alpha }{\mu }}-\mathop {\sum }\nolimits _{i=0}^{N-1} \frac{\left( \frac{N\alpha }{y} \right) ^{i}}{i!} \right) } \end{aligned}$$
(31)

Then substituting (3) in Eq. (31) will lead to:

$$\begin{aligned} p_{k_{b}^{'}} \left( k\vert N \right) =\left( e^{\frac{N\alpha }{\mu }}-\mathop {\sum }\limits _{i=0}^{N-1} \frac{\left( \frac{N\alpha }{y} \right) ^{i}}{i!} \right) ^{-1}\frac{\left( \frac{N\alpha }{y} \right) ^{k}}{k!}\quad k\ge N \end{aligned}$$
(32)

From the above, the average number of the cloud computing busy CCRUs in the system is obtained by

$$\begin{aligned} E\left[ k_{b}^{'}\vert N \right]= & {} \mathop {\sum }\limits _{k=N}^\infty {{k\,p}_{k_{b}^{'}}\left( k \right) } \nonumber \\= & {} \left( \frac{N\alpha }{y} \right) \left( 1+\left( e^{\frac{N\alpha }{y}} -\mathop {\sum }\limits _{i=0}^{N-1} \frac{\left( \frac{N\alpha }{y} \right) ^{i}}{i!} \right) ^{-1}\frac{\left( \frac{N\alpha }{y} \right) ^{N-1}}{(N-1)!} \right) \end{aligned}$$
(33)

The distribution of total cloud computing regular jobs in the system, \(\varphi (n)\), can also be obtained from Eq. (25). Therefore, the average number of busy CCRUs in the system is equal to:

$$\begin{aligned} k_{b}^{'}=\mathop {\sum }\limits _{n=1}^\infty {\varphi (n)E\left[ k_{b}^{'}\vert n \right] } \end{aligned}$$
(34)

\(k_{b}^{'}\) indicates that the busy CCRUs occupied by regular jobs that are not accessible for the secondary tasks.

6 Priority-based scheduling of data stream analysis

One of the approaches to increase the resource utilization in the cloud datacenter is the assignment of idle reserved resources to the secondary QoS-insensitive jobs with shorter service time. When the service time of the secondary jobs is truly smaller than that of the primary (regular) jobs, it is possible to serve many secondary tasks in the system without impacting the performance of the primary jobs, thereby increasing the throughput of the system.

We have explained the number of reserved CCRUs in the system and at the same time the average number of busy CCRUs in the system. By subtracting the total reserved CCRUs from the busy ones, the number of idle reserved CCRUs can be obtained. Therefore, the average number of idle reserved CCRUs that are not used by the dynamic regular jobs in the cloud computing system is calculated as

$$\begin{aligned} \bar{k}_{\gamma }^{'}=\bar{k}_{T}^{'}-\bar{k}_{b}^{'} \end{aligned}$$
(35)
figure a

We consider the situation in which the computing resources reserved by a user which are not currently busy may be used to serve the secondary data streams. The idle reserved resources can be allocated to the secondary data streams so that cloud resources are utilized effectively without affecting the performance of the primary user. Therefore, secondary tasks must release the resources immediately and search for other CCRUs, giving priority to the primary task that has arrived at the system. Consequently, the cloud computing data streams as secondary applications should be QoS-insensitive. From now on, we consider the regular cloud computing task as the primary task. As a result, the stream task (as a secondary task) service time should be much shorter than the lifetime of the cloud computing jobs. When a primary task comes to the system, the secondary VM instance is immediately removed even though pre-built or customized disk images that are used to create the tasks are still available. This scenario corresponds to the preemptive repeat priority queuing system in which each CCRU has two customer classes, namely primary and secondary.

When the primary job comes to the CCRU, the stream thread will be removed; but then through internal migration, the thread will be transferred. Under these circumstances, we need to find the service time characteristics of a stream task as well as the searching time for the secondary tasks to find the appropriate free CCRU in the server.

Fig. 9
figure 9

Proposed architecture diagram for preemptive resource allocation of stream processing (an OpenStack-based cloud with KVM as the hypervisor \(+\) batch/Stream Hadoop/Storm big data Processing)

Full size image

Now let us assume that the allowance of the large data streams using the idle CCRUs of a job may cause a raise in the lease of the cloud computing system tenants and that they have to pay just \(\theta \) percent of the regular lease. The large data streams have to pay just \(\delta \) percent of the leases of resources that they use. Definitely. \(\theta +\delta \) percent of the workloads should compensate the extra power consumption of the cloud computing systems as a result of the high utilization of the large data streams. For example, in the case of twenty percent extra cost due to the high power consumption: \(\theta +\delta =120\%\). Hence, the cloud service providers, users and scientists will gain more benefits. The details concerning the cost alignment are beyond the scope of this paper.

Next, we propose an architecture to implement the proposed preemptive scheduling algorithm. We consider two layers of virtualization [39, 40]. In the first layer of virtualization, hypervisors manage the VMs allocated to the regular jobs. VMs are created, migrated or removed from physical machines to serve the cloud jobs. On top of hypervisors, another multi-node cloud management platform should be installed to manage the secondary tasks of the stream. The VMs resulting from the first virtualization layer are considered resources for the distributed cluster computing platform. Figure 9 represents the proposed architecture for preemptive resource allocation of stream processing. As can be seen, hypervisors such as KVM/qemu are considered the first virtualization layer and multi-node OpenStack/Nova is considered a candidate for distributed cluster computing on top of the virtual environment.

The CPU/memory utilization of each VM should be measured. If a VM is underutilized, then it could be eligible for the stream to execute secondary tasks. During the secondary task execution on the CCRU, resource utilization should be measured. As soon as the resource utilization passes the threshold, the task should be migrated to another instance. Nova, the computing service of OpenStack, eases the procedure of adding/removing instances dedicated to the stream. Moreover, during the execution of secondary tasks, the utilization has to be continually inspected to avoid undesirable impacts on the performance of primary jobs. Algorithm 1 describes the details of the proposed preemptive scheduling for job processing.

7 Performance evaluation

In this section, we study the performance of the proposed preemptive resource allocation platform dedicated to data stream processing. First, the approximation of service time obtained in Eqs. (1) and (2) is validated through simulation. Then, the performance of the proposed preemptive resource allocation model is evaluated in a real virtualized test bed. Next, we present the numerical and simulation results for a computing center using the proposed preemptive base scheduling.

7.1 Simulation results

The details of simulation are provided as follows:

Resources We assume a datacenter with 100 high-performance computing servers. In order to make it easy to compare CPU capacity between different servers, CCRU is defined as 1.0 GHz processing power of a CPU (similar to Amazon EC2 Compute Unit (ECU)). It provides the relative measure of the required integer processing power. Let us assume that the average number of CCRUs in each server is equal to 20. Then, the total number of CCRUs in the Datacenter is equal to \(S=100\times 20=2000\).

Workloads We assume twenty different classes of regular jobs \((R^{'}=20)\) with different sizes in which the size of a regular job is determined by the number of CCRUs that it requires. It is deemed that the class r jobs arrive into the system according to a Poisson process with parameter \(\lambda _{r}^{'}=60\) jobs/second for all different classes of regular jobs \((r=1,\ldots R^{'})\). It is also assumed that there is a secondary data stream of super-tasks. Size of the super-tasks varies from 1 to 20 \((R=20)\). Super-tasks of the data stream also arrive into the system according to a Poisson process with parameter \(\lambda _{T}=10\) jobs/second.

The proposed preemption-based scheduling is applied to serve the large data stream using the idle resources of primary jobs.

Fig. 10
figure 10

Numerical and simulation results for the average service time of a stream job as a function of task generation rate of the cloud jobs with \(\Lambda =5\) (1/min)

Full size image
Fig. 11
figure 11

Numerical and simulation results for average service time of a stream job as a function of cloud computing jobs arrival rate, \(\Lambda \) with \(\alpha =1\) (1/min)

Full size image

The performance of the cloud system is impacted by the dynamicity of the busy CCRUs occupied by regular cloud computing jobs. As depicted in Fig. 10, the low task generation rate of the primary cloud users benefits the stream tasks’ execution. An increase in the task generation rate, \(\alpha \), for the cloud computing jobs makes the system more dynamic; therefore, the preemption probability of the data streams will increase, which will eventually increase the service time of a stream task. Please note that the gap for higher \(\alpha \) (3%) is due to the distribution fitting error.

In Fig. 11, we present the average service time of a stream job as a function of the arrival rate of regular cloud computing jobs. This demonstrates how the low arrival rate of cloud users causes the service time of the stream to be relatively higher. The trend reveals the increase in the arrival rate of the regular cloud jobs through providing idler resources, which improves the stream tasks’ service time. Hence, the larger the scale and the lower the utilization of primary jobs (lower \(\alpha \)), better for Secondary jobs service itme. Furthermore, duplication of the tasks with \(L=2\) (for cloud computing users’ lower arrival rates) causes a negative effect; but for a higher arrival rate of cloud users, it decreases the service time. In both figures, there are close agreements between simulation and numerical results, which validate the approximation done in Eqs. (1) and (2).

7.2 Experimental results

The cloud platform used to evaluate the proposed solution comprises 5 physical super-servers interconnected by a Gigabit Ethernet switch. Each server is a Cisco UCS B200 M3 Blade Server System with two Intel Xeon Processor E5-2660 version 2 CPUs and 8 \(\times \) 16 GB DDR3 (M393B2G70DB0-CMA) RAM. Each of the Xeon CPUs has 10 Cores and 25 MB of L2 Cache and is working at 2.2 GHz frequency. Also, each server has 500 GB SCSCI 1500 rpm HDD for storage. All the servers run Kernel-based Virtual Machine (KVM) QEMU 2.0.0, a virtualization infrastructure as the hypervisor. Here, the CCRU includes 1 VCPU core(1GHZ), 512 MB and 10 GB. Thus, the resource bottleneck is the CPU. Primary users occupy different VM types demanding different ranges of CCRUs from 1 to 10 CCRUs. It is assumed that all the primary users run Ubuntu 14.04 with Linux kernel 3.16.

On top of the KVM hypervisor is OpenStack Liberty with Nova, cinder, ceilometer and heat enabled in a node called the controller node. Nova module is responsible for management of the pools of resources dedicated to secondary jobs. The heat is the orchestration module which get the operational commands through REST API from the external resource utilization management module (represented in Fig. 9). As mentioned in the previous section, if the primary user needs its reserved resources, instances devoted to the data stream should migrate to the new resources, which degrades their performance. Migration downtime of the large Nova computing instances is observably long (more than 10 s). Thus, we consider just Tiny and Micro OpenStack flavors that require 1 and 2 VCPUs, and 256 and 512 MB memory, respectively, running CentOS 5.05.

The word count scenario in Apache Storm [41], an open-source distributed real-time computation system written in Java and Clojure, is considered as the data stream process. There are two main components in Storm: spouts, bolts. A spout is the data source in Storm, which retrieves data from external sources and emits them into the bolt. A bolt processes streams and produces the output. So, most of the logic of a computation is done in bolts. Network of spouts and bolts forms the Storm topology. A topology is an arbitrarily stream computation architecture that runs indiscriminately as soon as it deployed and allows distributed processing of streaming data. In the topology, each component, spout or bolt, handles the data via the worker processes. Worker process spawns one or more Java threads called executors running as an instance within its JVM. For more details, please refer to [41]. We set one spout component with four bolt instances processed in four executors. All the spout instances emit random sentences as tuples to the bolts using shuffle grouping methods. The emitted tuples are passed to the bolt, which contains four instances running in four different executors. In order to run the WordCountTopology, for one case, we write our own sentences like Twitter posts and let them be emitted downstream in a random sequence. Using Psutil library in Python language, a batch script is applied to measure the CPU utilization of the VMs leased by the primary tenants. For more details, please refer to [9].

As soon as the CPU utilization of the primary users becomes higher than the threshold, an event will be created and will be sent through the heat module to the Nova Controller node to migrate the running instances of secondary jobs.

Fig. 12
figure 12

Primary user’s CPU utilizations as a function of time (indicating idle periods)

Full size image
Fig. 13
figure 13

Total delay of a word count process as a function of the primary task arrival rate both for numerical and experimental results

Full size image

Figure 12 shows the CPU utilization of the primary jobs over time. The primary VM is considered an Apache tomcat web server which, according to the load arrival rate, should communicate with a SQL-based database. Thus, according to the request, CPU utilization varies over time. Idle times are detected in this figure. As it is shown, primary web server is idle for noticeable amount of time for periods which adversely affects the utilization of the cloud center. Hence, the secondary tasks could be executed in these periods to alleviate this underutilization problem. A threshold of 5% for CPU utilization percentage, shown by the red line in the figure, is selected to avoid the performance degradation of primary jobs.

Figure 13 shows the delay of the Storm word count process as a function of total task arrival rate of the primary jobs. As can be seen, by increasing the primary task arrival rate, \(\alpha \), the number of preemptions increases, and consequently, the total delay of the stream process increases. The blue line shows the numerical result of the upper bound of stream processing delay resulting from substituting Eqs. (10) and (1) in Eq. (3). The proposed upper bound can be useful in determining the average delay and checking the delay constraints. From the results of Figs. 1 and 11, it should be noticed that the most effective parameter in preemptive resource allocation of tasks is the task generation rate of the primary users and the attributed dynamicity of busy occupied CCRUs, which means that the more static the system, the lower the delay of the secondary tasks. Moreover, in Fig. 14, the total number of the counted words is presented for the limited amount of time equal to 150 s. Again, it should be noticed that the dynamicity of the primary tasks degrades the system performance of the secondary primary processes. However, it can be concluded that if the secondary processes are not QoS-sensitive or their required service rate is not high, then knowing the lower price preemptive resource allocation could be an excellent option for low process computing or QoS processes that are non-sensitive.

Fig. 14
figure 14

Number of counted words as a function of the primary task arrival rate in 150 s

Full size image

7.3 Comparison with the literature

In [12,13,14], the cloud center is stochastically modeled as a queuing system with general distribution of response time. The authors computed many performance metrics such as waiting, service and execution periods for batch processing jobs. In their model, it is assumed that a cloud computing job is a group of parallel tasks which enters, is processed and then leaves the system. Thus, their work is accurate for cloud batch processing jobs and is not suitable for data streams. A batch processing system contains MapReduce and is generally more concerned with size and complexity of jobs than latency of computation. However, exponential growth of real-time applications in a large scale, where computations can be divided into independent tasks, urges a new computing paradigm called stream processing platform. For more details, please refer to [9].

In this work, two types of jobs are modeled, namely regular jobs (primary jobs) and data streams (secondary jobs). The modeling of the proposed regular cloud jobs (primary jobs) in this work (Sect. 5) is the same as [12,13,14] except that we also investigate the utilization dynamicity of occupied resources in Sect. 5.1. It gives us information about the state of the occupied CCRUs that they are either busy or idle.

Fig. 15
figure 15

Performance comparison of the proposed algorithm with the literature

Full size image

For distributed stream processing, closed-form formulas (Eqs. 13 and 15) are calculated for the average and variance of cloud computing resources occupied by data stream of workload along with the other performance metrics. It will also calculate the joint probability of the number and position of the occupied resources required for the distributed jobs. Please note that since execution dependency among the tasks and of the cloud computing job is not considered in [12,13,14], these models are not

For clarification, the performance of the proposed algorithm is compared with the typical cloud computing system proposed in [12, 14] with parameters explained in Sect. 7.1. It can be seen that if there is no data stream in the system, \(\lambda _{T}=0\), the number of occupied resources, are almost the same for our proposed method and the models in [12, 14].

If a data stream arrives into the cloud datacenter, models in [12, 14] consider it as a regular job and allocate new resources into it, while the proposed method detects the idle resources assigned to regular jobs and reassign them to the super-tasks of the data stream. Thus, the proposed algorithm saves the resources and increases the utilization of the cloud computing system. As shown in Fig. 15, for a fixed value of total job arrival rate of primary and secondary tasks (\(\lambda _{T}+\Lambda )\), if the percentage of the data stream tasks increases (from 0.2 to 0.4), as long as there are enough idle resources in regular jobs, the performance of the cloud system can be drastically increase using the proposed algorithm. In this case (Fig. 15), the number of active servers is approximately 10% less than the models used in [12, 14]. It should be also noted that in the worst case scenario where the dynamicity of the CCRUs of regular jobs is high, tasks of secondary jobs cannot use the CCRUs assigned to primary jobs and performance of the propose algorithm tends to models performance in [12, 14].

Fig. 16
figure 16

Job assignment method pipelines of [13] (red arrow) and proposed algorithm (green line) on Google Cloud Platform (color figure online)

Full size image

To detail the proposed algorithm, an example on Google Cloud Platform is also provided [42]. Please note that all the security relevant issues are out of the scope of this paper. In this example, regular jobs require instances with different flavors on Google Compute Engine. DATAFLOW service is also provided to process the incoming data streams. Performance metrics of instances at Google Compute Engine is monitored and is input into the proposed algorithm implemented in a container. If the resource utilization of the instances on Google Compute Engine instances is less than a threshold (Let say 2%), proposed algorithm sends a YAML configuration to the Google Kubernetes Engine to create containers for DATAFLOW service on Google Compute Engine unutilized instances. Thus, instead of demanding new resources from Google Cloud Platform, the proposed method saves the budget and increases the resource utilization of the system via reusing the unutilized instances of the regular jobs.

Figure 16 compares the architecture of the proposed model and the literature [13] on Google Cloud Platform. The red arrow shows the performance of the model in [13] in which the data stream is treated as same as the regular jobs, while the green arrows show the pipelines of the proposed algorithm. As shown, the proposed method instead of creating new instances, create processing containers on idle Google Compute Engine instances assigned to regular jobs according to their utilization status.

8 Discussions

Though a high-level and reasonable scenario is proposed for cloud systems, cloud monitoring service has a very complex job to bill the real consumer of resources. Another critical issue with the proposed idea is the security and privacy of primary users data. One of the main features that IaaS clouds strive to provide for their users is a strong isolated environment which will be violated with the approach presented in this paper. However, two assumptions may mitigate this issue:

  1. (a)

    considering the primary and secondary jobs as opposed to users. In other words, there is one user that has two types of jobs including long time and high-priority jobs for which reservation of cloud resources is required; then secondary, temporary and short-term jobs with low or no QoS requirements that can leverage the reserved resources without conflicting with primary jobs. Since both jobs are belong to the same user, the security and privacy issue are naturally mitigated;

  2. (b)

    another technique would be using constrained and limited containers to run secondary jobs; in this approach, there is no need that jobs originate from the same user as the privacy and security concerns can be addressed by firing controlled and isolated containers on top of VMs.

If secondary jobs are going to be run in the same VM, then upon arrival of regular jobs, the resources cannot be realized immediately and become available. Even immediate killing of secondary jobs may not be translated to immediate release of resources (i.e., depending on the type of job, memory, CPU, network or disk may remain engaged after the killing command), let alone migration of jobs to another VM. De-provisioning a VM also takes time which might not be negligible in the eyes of the primary user. The provisioning of new VMs should also be taken into account; it is not something that can be done too frequently as opposed to containers. Thus, for clarification of the placement strategy of secondary jobs in the primary user premises, it should be noted that the size of the VMs allocated to the secondary jobs should be tiny enough or containers have to be applied.

Data movement and initialization of big data jobs are indeed expensive; these kinds of jobs are not suitable as secondary jobs as they are neither short nor migratory unless they got stored in very distributed manner. In Fig. 12, the maximum amount of time for idle CPU utilization is 78 s which is not enough for doing most (if not all) realistic data analytic jobs. For example, most of the Hadoop jobs takes couple of seconds to be initialized. They need minutes to complete; thus, according to the MapReduce the processing should be vastly fragmented to small pieces. Also, in big data analytics the assumption is that we don’t move the data. So migration task in case of big data scenarios should be done accurately. QoS-sensitive data stream jobs are also not a good candidate for secondary jobs that are usually time-sensitive. CPU-intensive jobs (eg, data crunching) might be also good candidate as secondary jobs for the scenario proposed in this paper.

9 Conclusion

In this paper, we modeled the data stream as a two-dimensional semi-birth–death queuing process. Then, we applied the model to calculate the required resources and active servers for serving the data stream. The two-dimensional semi-birth–death processes helped us in obtaining the performance measures for the incoming jobs, such as blocking probability of an incoming stream task in a datacenter. In addition, we also obtained the number of active resources in the cloud computing center, which can help in finding the idle reserved resources that can be occupied by secondary jobs. Finally, we propose a preemptive base scheduling technique for data stream processing to efficiently utilize the idle reserved resources of the cloud computing system, thereby improving the utilization efficiency of the computing platform. The service time and other performance metrics for the streams were investigated under the preemptive circumstances. An architecture was designed and implemented using the proposed preemptive resource allocation algorithm. The numerical simulation and experimental results indicate the impressive performance of our proposed preemptive resource allocation technique. They also show that the preemptive resource allocation is too sensitive to the dynamicity of the system and that the static primary jobs provide a better test bed for the secondary processing jobs.