DIET Cloud Features

DIET: A middleware designed for Cloud Federation

We propose a Cloud middleware designed for High performance Computing that takes advantage of federated clouds and other virtual resources and is also able to use classic HPC resources. Using DIET Cloud, we are able to deploy a large scale distributed HPC platform that spans on a large pool of resources aggregated from different providers. Moreover, using cutting edge advances in research, we are able to ease the usage of our platform and made the platform itself automatically adapt depending on user needs. Users will be able to submit large computational jobs as well as big data jobs without losing control of their data and by taking advantage of all resources they can access and doing this at a small cost.

 

As a proof of concept many PaaS interface exist:

• DIET with Amazon EC2 and Ecalyptus
• DIET with Nimbus
• DIET with Proxmox
• DIET with Delta-cloud
• DIET with OpenStack
• etc.

The DIET Cloud Architecture

This section is based on the ongoing work in the DIET team and we introduce here a prototype (one of our goals). We want to open discussions and contribution around this topic, join us.

DIET already implements many prerequisites, such as service calls, scalable scheduling and data management. This allowed us to implement a Cloud middleware with minimal effort. Now we will describe the architecture that upgrades DIET to a Cloud middleware designed to manage multi-cloud infrastructures. This architecture is shown here:

 

 In this Figure where the DIET hierarchical agent connection is represented by the DIET logo. The aim of the SeD Cloud is to provide a component that deals with a large number of Cloud IaaS. Thus it hides the complexity and heterogeneity of the Cloud layer (using an API as Delta Cloud). the Delta Cloud API is a good solution. Nevertheless the DIET SeD Cloud could be interfaced with different APIs if it is required but we try to minimize the number of interface through the good choice of API. DIET can benefit from the IaaS capabilities and manage Virtual Machines. Nevertheless Virtual Machine management decisions will be taken according to the scheduler and the required SLA (Service Level Agreement). DIET has successfully interacted with Amazon EC2, thus the feasibly of this approach was validated (see DIET with Amazon EC2 web page for more information).  Delta Cloud increases the amount of supported Cloud platforms.

DIET Cloud for every one

The DIET Cloud is designed to be used easily by every one who wants to :

• use a service regardless whether the service is performed in a cloud or a federation of clouds and be a client

• deploy a service on a cloud or a federation of clouds and be a service administrator.

If the user acts as a client : he invokes the service like a standard service deployed in the grid, i.e with a classical diet_call(diet_profile).

If the user wants to act as a service administrator, DIET Cloud offers several convenient features to make deployments and access to services easy and fast :

Automatic behavior:

• when a service X is requested virtual machines are instanciated in the cloud and destroyed when the service X is terminated
• when a service X is requested virtual machines are instanciated in the cloud and they are destroyed only on demand

By hand (expert mode):

• the service administrator choose to instantiate the cloud with virtual machines and provision them with services, he decides on his own the instant to destroy the virtual machines

To provide these user-friendly features, a customizable SeD Cloud  »Server Daemon for Cloud » was designed :

• the SeD Cloud provides to the client the services cited above.
• the deployment of virtual machines is done with delta-cloud, so DIETCloud can address many kind of clouds (OpenNebula, OpenStack, an many others)
• The SeD Cloud offers interfaces to deploy services in virtual machines with tools like Chef or Puppet ; by default the SeD Cloud uses ssh/rsync/scp.

Besides, we Introduce the notion of « CloudServiceBinary » which is a executable defined by the designer of the SeD Cloud and is intended to run in one virtual machine of the Cloud.

The bellow illustration shows that a SeD Cloud offers a matrix multiplication as a service and the correponding CloudServiceBinary’s name is matrixMult. As we can see the CloudServiceBinary is firstly copied in one virtual machine.

 

 

The following scheme shows the execution of the CloudServiceBinary « matrixMult » ; when a client wants to perform a matrix multiplication, he make a classical diet_call with two files : matrices A and B. These two files a then copied to a specific folder (corresponding to the DIET request ID) in the virtual machine and the CloudServiceBinary performs the operation and sends the result back to the SeD Cloud.

---