Main.SpeQuloS History

Hide minor edits - Show changes to output

March 14, 2012, at 06:09 AM by 129.175.15.11 -
Added lines 5-6:

The SpeQuloS Software Home Page is here : [[http://graal.ens-lyon.fr/~sdelamar/spequlos/]]
October 28, 2011, at 06:55 AM by 129.175.15.11 -
Changed line 4 from:
'''Participants:''' Simon Delamare, Gilles Fedak, Derrick Kondo
to:
'''Participants:''' Simon Delamare, Gilles Fedak, Derrick Kondo, Oleg Lodygensky
March 16, 2011, at 07:43 PM by 129.175.15.11 -
Changed line 1 from:
!! SpeQuloS: Quality of Service for Hybrid Distributed Computing Infrastructures
to:
!! SpeQuloS: Quality of Service for Best-Effort Distributed Computing Infrastructures
November 30, 2010, at 02:28 PM by 129.175.15.11 -
Changed line 47 from:
!! Financial support
to:
!!! Financial support
November 30, 2010, at 02:28 PM by 129.175.15.11 -
Changed line 10 from:
%rframe text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/fp7.png"FP7 Support"
to:
%rframe text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/edges.png"FP7 Support"
November 30, 2010, at 02:16 PM by 129.175.15.11 -
Changed lines 36-38 from:
%frame text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/fspequlos.png"The SpeQuloS architecture"

to:
%frame text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/spequlos.png"The SpeQuloS architecture"

Deleted line 51:
%rframe text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/fp7.png"FP7 Support"
November 30, 2010, at 02:15 PM by 129.175.15.11 -
Added lines 10-11:
%rframe text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/fp7.png"FP7 Support"
Added lines 36-38:
%frame text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/fspequlos.png"The SpeQuloS architecture"

Changed line 54 from:
http://www.lri.fr/~fedak/fp7.png
to:
http://www.lri.fr/~fedak/fp7.png"FP7 Support"
November 30, 2010, at 01:45 PM by 129.175.15.11 -
Changed lines 47-49 from:
[[http://www.lri.fr/~fedak/fp7.png]]
to:
%rframe text-align=center margin-top=5px margin-right=15px margin bottom=5px margin-left=25px% http://www.lri.fr/~fedak/fp7.png"FP7 Support"

http://www.lri.fr/~fedak/fp7.png
November 30, 2010, at 01:40 PM by 129.175.15.11 -
Changed line 47 from:
[[http://www.lri.fr/~fedak/fp7.png  | FP7 EDGI]]
to:
[[http://www.lri.fr/~fedak/fp7.png]]
November 30, 2010, at 01:40 PM by 129.175.15.11 -
Changed line 47 from:
[[http://degisco.eu/image/image_gallery?uuid=cb4e2997-4c81-49ad-b720-683824793d00&groupId=10508&t=1279438340221 | FP7 EDGI]]
to:
[[http://www.lri.fr/~fedak/fp7.png | FP7 EDGI]]
November 30, 2010, at 01:34 PM by 129.175.15.11 -
Added lines 41-47:

!! Financial support


EDGI is supported by the FP7 Capacities Programme under grant agreement nr RI-261556.

[[http://degisco.eu/image/image_gallery?uuid=cb4e2997-4c81-49ad-b720-683824793d00&groupId=10508&t=1279438340221 | FP7 EDGI]]
November 30, 2010, at 01:27 PM by 129.175.15.11 -
Changed line 1 from:
! SpeQuloS: Quality of Service for Hybrid Distributed Computing Infrastructures
to:
!! SpeQuloS: Quality of Service for Hybrid Distributed Computing Infrastructures
November 30, 2010, at 01:26 PM by 129.175.15.11 -
Changed lines 1-3 from:
! SpeQuloS

!!
Quality of Service for Hybrid Distributed Computing Infrastructures
to:
! SpeQuloS: Quality of Service for Hybrid Distributed Computing Infrastructures
November 30, 2010, at 01:26 PM by 129.175.15.11 -
Changed lines 40-41 from:
* The goal of the QoS scheduler is to provide guaranties that series or batch of jobs submitted by SG users to a particular DG will be finished by a certain deadline with a given probability. To do so, the QoS scheduler  extends DG systems with additional Cloud resources. In EDGI we will deal with two different Cloud systems implementations (likely candidates are Eucalyptus and OpenNebula) in order to develop a generic solution not tightly connected to one particular Cloud interface. Once jobs are submitted to DG, the QoS scheduler monitor jobsí execution. According to user QoS ability, the QoS Scheduler adds additional computing resources if it detects that the DG will not be able to complete the jobs on time.
Technically, this will be achieved by using the libcloud library.
to:
* The goal of the QoS scheduler is to provide guaranties that series or batch of jobs submitted by SG users to a particular DG will be finished by a certain deadline with a given probability. To do so, the QoS scheduler  extends DG systems with additional Cloud resources. In EDGI we will deal with two different Cloud systems implementations (likely candidates are Eucalyptus and OpenNebula) in order to develop a generic solution not tightly connected to one particular Cloud interface. Once jobs are submitted to DG, the QoS scheduler monitor jobsí execution. According to user QoS ability, the QoS Scheduler adds additional computing resources if it detects that the DG will not be able to complete the jobs on time. Technically, this will be achieved by using the libcloud library.
November 30, 2010, at 01:25 PM by 129.175.15.11 -
Changed lines 40-41 from:
* The goal of the QoS scheduler is to provide guaranties that series or batch of jobs submitted by SG users to a particular DG will be finished by a certain deadline with a given probability. To do so, the QoS scheduler
will extend DG systems with additional Cloud resources. In EDGI we will deal with two different Cloud systems implementations (likely candidates are Eucalyptus and OpenNebula) in order to develop a generic solution not tightly connected to one particular Cloud interface. Once jobs are submitted to DG, the QoS scheduler monitor jobsí execution. According to user QoS ability, the QoS Scheduler adds additional computing resources if it detects that the DG will not be able to complete the jobs on time.
to:
* The goal of the QoS scheduler is to provide guaranties that series or batch of jobs submitted by SG users to a particular DG will be finished by a certain deadline with a given probability. To do so, the QoS scheduler  extends DG systems with additional Cloud resources. In EDGI we will deal with two different Cloud systems implementations (likely candidates are Eucalyptus and OpenNebula) in order to develop a generic solution not tightly connected to one particular Cloud interface. Once jobs are submitted to DG, the QoS scheduler monitor jobsí execution. According to user QoS ability, the QoS Scheduler adds additional computing resources if it detects that the DG will not be able to complete the jobs on time.
November 30, 2010, at 01:25 PM by 129.175.15.11 -
Changed lines 38-40 from:
*  The QoS Oracle has two responsibilities. First, the scheduling oracle  determines feasible deadlines on each DG. A deadline is feasible if the DG can meet it with 95% confidence. To do this, the scheduling oracle takes into account the job's quality of service requirements, and the past performance of each Desktop Grid. Second, for jobs needing fast turnaround time, the oracle should determine when tasks of the job submitted to the Desktop Grid must be run on the dedicated Cloud, and give this recommendation to the QoS
scheduler.
to:
*  The QoS Oracle has two responsibilities. First, the scheduling oracle  determines feasible deadlines on each DG. A deadline is feasible if the DG can meet it with 95% confidence. To do this, the scheduling oracle takes into account the job's quality of service requirements, and the past performance of each Desktop Grid. Second, for jobs needing fast turnaround time, the oracle should determine when tasks of the job submitted to the Desktop Grid must be run on the dedicated Cloud, and give this recommendation to the QoS scheduler.
Changed lines 44-45 from:
* There is one more important aspect of QoS support which is fairness. Obviously every user will want to submit applications with QoS request. If they can do it in an unlimited way we can not provide QoS to anyone. So we need a mechanism by which users can claim QoS resources according to their contributions to the resources of the EDGI infrastructure. The QoS credit system allows to measure the institutional contributions and  to allocate
QoS resources in proportion with the collected credits.
to:
* There is one more important aspect of QoS support which is fairness. Obviously every user will want to submit applications with QoS request. If they can do it in an unlimited way we can not provide QoS to anyone. So we need a mechanism by which users can claim QoS resources according to their contributions to the resources of the EDGI infrastructure. The QoS credit system allows to measure the institutional contributions and  to allocate QoS resources in proportion with the collected credits.
November 30, 2010, at 01:23 PM by 129.175.15.11 -
Changed lines 38-42 from:
*  The QoS Oracle has two responsibilities. First, the scheduling oracle  determines feasible deadlines on
each DG. A deadline is feasible if the DG can meet it with 95% confidence. To do this, the scheduling oracle
takes into account the job's quality of service requirements, and the past performance of each Desktop Grid.
Second, for jobs needing fast turnaround time, the oracle should determine when tasks of the job submitted
to the Desktop Grid must be run on the dedicated Cloud, and give this recommendation to the QoS
to:
*  The QoS Oracle has two responsibilities. First, the scheduling oracle  determines feasible deadlines on each DG. A deadline is feasible if the DG can meet it with 95% confidence. To do this, the scheduling oracle takes into account the job's quality of service requirements, and the past performance of each Desktop Grid. Second, for jobs needing fast turnaround time, the oracle should determine when tasks of the job submitted to the Desktop Grid must be run on the dedicated Cloud, and give this recommendation to the QoS
Changed lines 41-46 from:
* The goal of the QoS scheduler is to provide guaranties that series or batch of jobs submitted by SG users to a
particular DG will be finished by a certain deadline with a given probability. To do so, the QoS scheduler
will extend DG systems with additional Cloud resources. In EDGI we will deal with two different Cloud
systems implementations (likely candidates are Eucalyptus and OpenNebula) in order to develop a generic
solution not tightly connected to one particular Cloud interface. Once jobs are submitted to DG, the QoS scheduler monitor jobsí execution. According to user QoS ability, the QoS Scheduler
adds additional computing resources if it detects that the DG will not be able to complete the jobs on time.
to:
* The goal of the QoS scheduler is to provide guaranties that series or batch of jobs submitted by SG users to a particular DG will be finished by a certain deadline with a given probability. To do so, the QoS scheduler
will extend DG systems with additional Cloud resources. In EDGI we will deal with two different Cloud systems implementations (likely candidates are Eucalyptus and OpenNebula) in order to develop a generic solution not tightly connected to one particular Cloud interface. Once jobs are submitted to DG, the QoS scheduler monitor jobsí execution. According to user QoS ability, the QoS Scheduler adds additional computing resources if it detects that the DG will not be able to complete the jobs on time.
Changed lines 45-48 from:
* There is one more important aspect of QoS support which is fairness. Obviously every user will want to
submit applications with QoS request. If they can do it in an unlimited way we can not provide QoS to
anyone. So we need a mechanism by which users can claim QoS resources according to their contributions to
the resources of the EDGI infrastructure. The QoS credit system allows to measure the institutional contributions and  to allocate
to:
* There is one more important aspect of QoS support which is fairness. Obviously every user will want to submit applications with QoS request. If they can do it in an unlimited way we can not provide QoS to anyone. So we need a mechanism by which users can claim QoS resources according to their contributions to the resources of the EDGI infrastructure. The QoS credit system allows to measure the institutional contributions and  to allocate
November 30, 2010, at 01:22 PM by 129.175.15.11 -
Changed lines 33-34 from:
 It  takes
the right decision about assigning the necessary number of trusted clients and Cloud clients for the QoS
to:
It  takes the right decision about assigning the necessary number of trusted clients and Cloud clients for the QoS
November 30, 2010, at 01:18 PM by 129.175.15.11 -
Changed lines 1-5 from:
!! SpeQuloS

!!! Quality of Service for Hybrid Distributed Computing Infrastructures

to:
! SpeQuloS

!! Quality of Service for Hybrid Distributed Computing Infrastructures

Added lines 21-22:
!!! The SpeQuloS middleware
Changed lines 28-31 from:
The first approach will classify the DG clients according to their historical behavior and allocates applications with QoS needs to the more trustable and faster clients.  However, even in this case it can happen that some of the work-units are not completed in time. For such critical work-units the DG system will be able to dynamically deploy fast and trustable clients from some
to:
The first approach  classifies the DG clients according to their historical behavior and allocates applications with QoS needs to the more trustable and faster clients.  However, even in this case it can happen that some of the work-units are not completed in time.

The second approach is based on the
extension of DG systems with Cloud resources. For such critical work-units the SpeQuloS system is
able to dynamically deploy fast and trustable clients from some
Changed lines 33-36 from:


So the second approach will be based on the
extension of DG systems with Cloud resources. EDGI will elaborate the technical solution to be able to take
to:
 It  takes
Changed lines 35-37 from:
applications by developing 4 components: a Qos Info system, QoS credit system, QoS Oracle and a QoS Scheduler

*  The
QoS Oracle has two responsibilities. First, the scheduling oracle should determine feasible deadlines on
to:
applications.

The Spequlos middleware is composed of  4 components: a Qos Info
system, QoS credit system, QoS Oracle and a QoS Scheduler

*  The QoS Oracle has two responsibilities. First, the scheduling oracle  determines
feasible deadlines on
Changed lines 52-54 from:
Technically, this will be achieved by the Cloud execution plugin that will be implemented as an extension to
the 3G bridge (see Section 1.2.2.1)
.
to:
Technically, this will be achieved by using the libcloud library.
Changed lines 57-63 from:
the resources of the EDGI infrastructure. The main idea of the solution is based on the credit mechanism
provided by BOINC. Although it is not perfect it is a good starting point to measure contributions of
institutes to the resources of the DG systems of the EDGI infrastructure.
In the project we will develop a policy how to measure the institutional contributions and how to allocate
QoS resources in proportion with the collected credits. This requires new kind of monitoring and scheduling
in DG systems. This information should also be presented to the users by the EDGI
monitoring/accounting/credit portal.
to:
the resources of the EDGI infrastructure. The QoS credit system allows to measure the institutional contributions and  to allocate
QoS resources in proportion with the collected credits.
November 30, 2010, at 11:11 AM by 129.175.15.11 -
Changed lines 8-9 from:
The aim of the project is to provide an middleware to implement Quality of Service for unreliable DCI (e.g. Desktop Grid) by provisionning additional stable and on-demand resources (e.g. Cloud).
to:
The aim of the project is to provide an middleware to implement Quality of Service for unreliable DCI (e.g. Desktop Grid) by provisionning additional stable and on-demand resources (e.g. Cloud) in the scope of the European FP7 EDGI project.

!!! The EDGI project

EDGI  is an FP7 European project, following the successful FP7 EDGeS project, whose goal is build a Grid infrastructure composed of "Desktop Grids", such as BOINC or XtremWeb, where computing resources are provided by Internet volunteers, and "Service Grids", where computing resources are provided by institutional Grid such as EGEE, gLite, Unicore and "Clouds systems" such as OpenNebula and Eucalyptus, where resources are provided on-demand.  The goal of the EDGI project is to provide an infrastructure where Service Grids are extended with public and institutional Desktop Grids and Clouds.  Our partners include SZTAKI insitute (Hungary), CIEMAT (Spain), Univ. Coimbra (Portugal), Univ Cardiff (UK), Univ Westminster (UK), AlmereGrid (NL), IN2P3 (FR) and more.


The main problem with the currentinfrastructure is that it cannot give any QoS support
for running their application in the Desktop Grid part of the infrastructure. For example,
a public DG system enables clients to return work-unit results in the range of weeks. Although there are
EGEE applications (e.g. the fusion communityís applications) that can tolerate such a long latency most of
the user communities want much smaller latency.

The INRIA leads the activities of the Work Package JRA2 : QoS support for Desktop Grids to solve this
critical problem.

We define QoS concretely as a probabilistic guarantee of job makespan or throughput. Providing QoS features even in Service Grids is hard and not solved yet satisfactorily. It is even more difficult in an environment where there are no guaranteed resources. In DG systems resources can leave the system at any time for a long time or forever even after taking several work-units with the promise of computing them. Two main approaches will be investigated within EDGI and deployed in the EDGI production infrastructure.

The first approach will classify the DG clients according to their historical behavior and allocates applications with QoS needs to the more trustable and faster clients.  However, even in this case it can happen that some of the work-units are not completed in time. For such critical work-units the DG system will be able to dynamically deploy fast and trustable clients from some
Clouds that are available to support the EDGI DG systems.


So the second approach will be based on the
extension of DG systems with Cloud resources. EDGI will elaborate the technical solution to be able to take
the right decision about assigning the necessary number of trusted clients and Cloud clients for the QoS
applications by developing 4 components: a Qos Info system, QoS credit system, QoS Oracle and a QoS Scheduler

*  The QoS Oracle has two responsibilities. First, the scheduling oracle should determine feasible deadlines on
each DG. A deadline is feasible if the DG can meet it with 95% confidence. To do this, the scheduling oracle
takes into account the job's quality of service requirements, and the past performance of each Desktop Grid.
Second, for jobs needing fast turnaround time, the oracle should determine when tasks of the job submitted
to the Desktop Grid must be run on the dedicated Cloud, and give this recommendation to the QoS
scheduler.

* The goal of the QoS scheduler is to provide guaranties that series or batch of jobs submitted by SG users to a
particular DG will be finished by a certain deadline with a given probability. To do so, the QoS scheduler
will extend DG systems with additional Cloud resources. In EDGI we will deal with two different Cloud
systems implementations (likely candidates are Eucalyptus and OpenNebula) in order to develop a generic
solution not tightly connected to one particular Cloud interface. Once jobs are submitted to DG, the QoS scheduler monitor jobsí execution. According to user QoS ability, the QoS Scheduler
adds additional computing resources if it detects that the DG will not be able to complete the jobs on time.
Technically, this will be achieved by the Cloud execution plugin that will be implemented as an extension to
the 3G bridge (see Section 1.2.2.1).

* There is one more important aspect of QoS support which is fairness. Obviously every user will want to
submit applications with QoS request. If they can do it in an unlimited way we can not provide QoS to
anyone. So we need a mechanism by which users can claim QoS resources according to their contributions to
the resources of the EDGI infrastructure. The main idea of the solution is based on the credit mechanism
provided by BOINC. Although it is not perfect it is a good starting point to measure contributions of
institutes to the resources of the DG systems of the EDGI infrastructure.
In the project we will develop a policy how to measure the institutional contributions and how to allocate
QoS resources in proportion with the collected credits. This requires new kind of monitoring and scheduling
in DG systems. This information should also be presented to the users by the EDGI
monitoring/accounting/credit portal.
November 30, 2010, at 10:40 AM by 129.175.15.11 -
Changed lines 8-9 from:
The aim of the project is to provide an middleware to implement Quality of Service for unreliable DCI (e.g. Desktop Grid) by providing additional stable and on-demand resources (e.g. Cloud).
to:
The aim of the project is to provide an middleware to implement Quality of Service for unreliable DCI (e.g. Desktop Grid) by provisionning additional stable and on-demand resources (e.g. Cloud).
October 25, 2010, at 11:51 AM by 129.175.15.11 -
Added line 4:
October 25, 2010, at 11:49 AM by 129.175.15.11 -
Added lines 1-8:
!! SpeQuloS

!!! Quality of Service for Hybrid Distributed Computing Infrastructures

'''Participants:''' Simon Delamare, Gilles Fedak, Derrick Kondo

The aim of the project is to provide an middleware to implement Quality of Service for unreliable DCI (e.g. Desktop Grid) by providing additional stable and on-demand resources (e.g. Cloud).
 
GlossyBlue theme adapted by David Gilbert
Powered by PmWiki