satis egitimisatis egitimitengda.pro

Open@Blog

Discussion on the state of cloud computing and open source software that helps build, manage, and deliver everything-as-a-service.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Login
Subscribe to this list via RSS Blog posts tagged in amazon

Tb2ap3_thumbnail_xenproject.pnghis week the Linux Foundation announced that Xen was becoming a Linux Collaboration project, Xen Project. In the announcement Amazon Web Services, AMD, Bromium, Calxeda, CA Technologies, Cisco, Citrix, Google, Intel, Oracle, Samsung and Verizon all pledged their support both monetarily and through continued contribution to the development of Xen Project.

Why is this Good For Xen?

Xen Community manager, Lars Kurth, lists a number of reasons why this move is a good for the Xen community on the Xen blog. 

  • An increase in Diversity 
  • Bringing Users and Developers Together
  • More Collaboration

That's interesting with over 10 million users worldwide and over 60% of the code base coming from outside the walls of Citrix (the former sponsor of the Xen.org project) there was already a fair amount of diversity, users, developers and collaboration so things can only get better. 

What the Industry is Saying

The industry is abuzz talking about this move and it's been overwhelmingly positive. First off I think you would be hard-pressed to find a more elite group of users and software developers collectively behind a single virtualization solution. Secondly, the technology has over ten years of software history one of the most mature technologies in it's field. Finally, the software has the support and committment of some of the world's biggest virtualization "power users". 

For example, Verizon also added a very publics statement about their use of Xen and Apache CloudStack on their blog, Chris Drumgoole, Senior Vice President, Global Operations for Verizon Terramark wrote:

...
Hits: 10529
Rate this blog entry:
Continue reading Comments

To REST or not to REST

Posted by on in CloudStack Tips

I wish I could write like Shakespeare but since I don't you are left with this blog about Representational State Transfer (REST) and specifically a discussion on whether the CloudStack API is a REST API or not. The short answer is that the CloudStack API is RESTlike but not RESTfull since it is only based on the GET method. Being an http based API that can return JSON does not make it a RESTfull API. This should not be seen as negative criticism but just a clarification.

A few words first about the CloudStack API and a few pointers. It is very extensive, from user creation, vm management to firewall configuration and more advanced networking features. Github has lots of clients written by the community and I am sure you can find your favorite language in there. The Developer Guide explains in details how to make requests. With CloudMonkey interacting with the API has become even easier. It is a terrific way to learn the API and check the required parameters of each call. Coupled with devcloud you can have a fully functional local CloudStack testbed.

To explore the API, I often also look at the GUI and see how the API calls are made. To do this, I access the UI in Firefox and launch Firebug console. I can then see the calls that I am interested in and specifically check the parameters/headers etc or the http call being made.

REST is known as an architectural style and to put it in french-english it is basically a way to design your API to create a web service that only uses the HTTP methods to manipulate the state of web resources. In my opinion it was really a response to the complexity of the SOAP based web services and the many standards that came out. REST really took off with Web 2.0 and was seen as a way to create easy to use web services. I recently found a few articles by Luis Rei and they clearly explain how to design a REST web service as well as implement on with Flask a lightweight python based web framework.

...
Hits: 19398
Rate this blog entry:
Continue reading Comments

I was asked the other day what was the connection between High Performance Computing (HPC) and Clouds, so I thought I would try to post an answer here. Let's first talk a little bit about HPC.

High Performance Computing is about finding every single flops and every single iops on the largest machine you can get your hands on, in order to run your code as fast as possible. It is about batch processing on as many cores as you can get, so you can solve the largest problem you are facing. For a while, supercomputers, were large shared memory machines but in the late nineties distributed memory systems appeared, they were cheaper and you could assemble lots of nodes to get hundreds of cpus. Today the Top500 supercomputers are ranked every 6 months, this ranking is the theater of great technological battle between countries, vendors, research labs and programmers. In the latest ranking, Sequoia the supercomputer from Lawrence Livermore National lab topped the ranking at 16.32 PetaFlop/s and 1,572,864 cores. Weather modeling, atomic weapons simulation, molecular dynamics, genomics and high energy physics are among those that benefit from HPC.

There is big difference however within HPC itself. It is the difference between applications that rely heavily on inter-process communication and need a low latency network for message passing, and applications where each process runs an independent task, the so-called embarrassingly parallel applications. (e.g Map-reduce is an example of how to express an embarrassingly parallel problem ). High Throughput Computing (HTC) defines the type of application where access to a large number of cores over a specific amount of time is needed. Protein folding popularized by the Folding@home project running on PS3 as well as desktops is a good example. Financial simulation such as stock price forecasting and portfolio analysis also tend to fall in that category due to their statistical nature. Graphics rendering for animated movies also falls under HTC. HTC cares less about performance -as measured by FLOPS- and more about productivity -as in processing lots of jobs-.

The HPC quest for performance seems totally antagonist with the IaaS layer of clouds, at least when one thinks of true HPC workload that consumes every flop. Virtualization, the key enabler of IaaS, introduces overhead, both in cpus and network latency, and thus has been deemed "evil" for true HPC. Despite directed I/O, pass thrus, VM pinning and other tuning possibilities to reduce the overhead of virtualization, you might think that this would be it, no connection between HPC and Clouds. However according to a recent academic study of hypervisor performance from a group at Indiana University, this may not be entirely true and it would also be forgetting the users and their specific workloads.

In november 2010 a new player in the Top 500 arrived: Amazon EC2. Amazon submitted a benchmark result which placed an EC2 cluster 233rd on the top 500 list. By June 2011, this cluster was down to rank 451. Yet it proved a point: that a Cloud based cluster could do High Performance computing, raking up 82.5 TFlops peak using VM instances and 10GigE network. In november 2011, Amazon followed with a new EC2 cluster ranked 42nd with 17,023 cores and 354 TFlops peak. This cluster is made of "Cluster Compute Eight Extra Large" instances with 16 cores, 60 GB of RAM and 10 GigE interconnect and now ranked 72nd. For $1000 per hour this allows users to get an on-demand HPC cluster that itself ranks in the top500. This is done on-demand and provides users with their personal cluster.

...
Hits: 27939
Rate this blog entry:
Continue reading Comments

CloudBridge is a free, open-source, downloadable add-on to CloudStack that translates Amazon EC2 API calls to native CloudStack API calls so that clients can continue using existing EC2-compatible tools with CloudStack.

CloudBridge 1.0.2 contains a significant refactoring of much of the internal code as well as improvements to the overal architecture. As a result of these changes, every supported command in CloudBridge has been refactored and in many cases actually work now.

Beyond just the code, the documentation has been improved and made more accessible.

You can find the installation package here:
http://sourceforge.net/projects/cloudstack/files/CloudBridge/

You can find the latest revision of the documentation here:
http://docs.cloud.com/CloudBridge_Documentation

Please let us know if you have any questions or problems using this release candidate for 1.0.2 by communicating on the cloudstack-devel mailing list, or filing bugs at http://bugs.cloud.com

Hits: 4064
Rate this blog entry:
Continue reading Comments

Open@Citrix

Citrix supports the open source community via developer support and evangeslism. We have a number of developers and evangelists that participate actively in the open source community in Apache Cloudstack, OpenDaylight, Xen Project and XenServer. We also conduct educational activities via the Build A Cloud events held all over the world. 

Connect