Table of Contents for

Programming for PaaS

Consider a common scenario for large businesses and financial or government institutions, where you’re building an application that has data-constrained portions that can only run in house. This data-restricted part of your application usually ends up being a small fraction of the app, perhaps 5% to 10%.

If you build your application monolithically, the entire application will have to be run in house due to the data constraints. However, if you build it to run as a distributed system, with many APIs feeding into a lightweight JavaScript frontend, or even a simple dynamic frontend that consumes other service APIs, you can still host 90% of your application safely in the public cloud. All the mundane pieces that do not interact with sensitive data can be run on commodity hardware that is cheaper to run in the cloud.

We’ve seen many successful examples of this modern service-oriented, API-driven architecture. We already mentioned Twitter, but Gmail also took a very API-driven, client-side approach, putting together APIs that interact with the mail and having the frontend JavaScript client consume it all. Apple did something similar with MobileMe: the backends are simple APIs, and the frontend is a thick, client-side application that handles much of the logic in real time. This enables more flexibility, more scalability, and more foresight, leading to better, more maintainable applications.

This approach of building many smaller services works especially well with Platform-as-a-Service. In fact, it is a best practice to do it this way with PaaS. Making small independent services that all tie together on the frontend is not only a modern approach to developing both web and mobile applications, but also ties in well when you are trying to decide how to incorporate a Platform-as-a-Service strategy.

JSON (short for JavaScript Object Notation) has emerged as a standard format for transferring data within API services, serving as an alternative to XML. JSON is used in a wide range of applications, including Gmail, MobileMe, iCloud, Twitter, LinkedIn, and almost every major website you’re likely to interact with. One of the reasons that it has been so predominant is because it is easy and natural for JavaScript on the web frontends to consume the JSON data. JSON has strong support on native mobile clients as well. The great libraries for parsing JSON within mobile and web APIs have made it a natural standard for generating and presenting the data for modern application design.

The JSON scheme is not hard to read, both for computers and for humans. Here is an example:

{
      "id": 1,
      "name": "Programming for PaaS",
      "price": 123,
      "tags": ["PaaS", "Programming"]
}

One of the nice aspects of the JSON format is how easy it is for web browsers to parse. This makes it a great format for Ajax programming techniques. Here is an example of how easy it is for a native JavaScript client like the one in your browser or in any Node.js application to parse:

var object = eval(text);

This is not the most secure technique (validating the text before executing it is always best practice). However, it shows the power of the JSON format.

REST (short for Representational State Transfer) is an architecture pattern (or style) for distributed systems like the Web. REST has become a predominant service design model, reusing HTTP’s basic vocabulary like GET, POST, PUT, and DELETE. Because REST uses simple HTTP commands, it uses much less bandwidth than services like SOAP that demand lengthy headers and heavy XML parsing.

The basic principle behind REST is to provide a shared network vocabulary for transferring resources in a standard and predictable format. What does that mean? Let’s look at an example of what a RESTful JSON service for serving and managing user data might look like:

With a RESTful approach, getting and managing services becomes predictable and straightforward. Creating and scaling any particular RESTful service is not usually very challenging.

For example, if your service is unlikely to get more than 100,000 rows in the next few years, then there is little need to worry about scaling beyond a traditional MySQL or PostgreSQL database table. These traditional SQL tables can easily scale to millions or even tens of millions of rows on a single server.

However, if you anticipate the service growing to 100,000,000 rows, you might want to consider a database better suited to highly distributed growth, like Riak or CouchDB.

A metaservice is a RESTful application for serving collections of data.

In slightly more detail, it is a lightweight application that consumes a core service (e.g., MySQL, MongoDB, Redis, memcached, etc.) and produces a REST API, which generally serves JSON or XML serialized data.

A metaservice is like a core service wrapped in REST with limited application logic specific to whatever the metaservice specializes in. Examples include a user registration metaservice, a commenting metaservice, a message metaservice, and a blog metaservice. Sometimes the same database serves various metaservices, but in other cases the metaservices will have their own databases and core services.

Traditionally, instead of building metaservices for data sources, you would build a lot of code that essentially did the functionality of REST, but mixed a bunch of services together, cross-function the logic, and, instead of generating simple JSON serialized objects, resulted in a complex mess of dynamically generated HTML, CSS, and JavaScript. When you wanted to swap out components, you found it difficult, if not impossible. When you wanted to scale components, they would be intertwined. The same table that would only need 100 records accessed frequently was on the same database as the one that needed to grow to 100,000,000 records, which would be infrequently accessed but needed to support fast indexed searching. This is the nightmare of scaling web applications in a traditional way—and it is a path that has been trodden by thousands of developers before you. Those developers usually ended up decomposing these applications into metaservices, so why not go with metaservices from the start?

A metaservice is built on top of the idea of a core service. It’s powered by these services—by the MySQLs, MongoDBs, and PostgreSQLs of the world. However, they are wrapped within a lightweight API that can still talk directly to the drivers and services. Instead of presenting HTML, or a finished frontend to be delivered directly to the user, these are typically much simpler. They simply generate JSON: serialized data structures built on the existing services. So, a metaservice delivers serialized data to be consumed by the client frontend, typically using a serialization format like JSON. These metaservices have become the building blocks of modern, scalable web architecture.

There are frameworks well suited to quickly generating RESTful applications in every language you can think of. Chapter 10 has a full list of frameworks specifically built for a wide variety of languages. No matter what language you use, you should be able to find an appropriate REST framework.

The one thing all of these frameworks have in common is that they make it easy to develop RESTfully routed URLs and provide structure to the code that responds to each HTTP verb (GET, PUT, POST, DELETE, etc.).

There are many libraries in every programming language built to consume metaservices. You give the library a RESTful URL, and it will model the result for you in your code. This lets you stitch together many metaservices in a lightweight, dynamic backend that can serve the compiled data to the end user. Here is a code example using Ruby’s ActiveResource library that shows an example of how easy these libraries can be to interact with:

class Person < ActiveResource::Base
    self.site = ENV["PEOPLE_REST_URL"] // Best Practice
end

ryan = Person.new(
    :first => 'Ryan',
    :last => 'Daigle'
)

ryan.save                # => true
ryan.id                  # => 2
Person.exists?(ryan.id)  # => true
ryan.exists?             # => true

ryan = Person.find(1)
# Resource holding our newly created Person object

ryan.first = 'Rizzle'
ryan.save                # => true

ryan.destroy             # => true

You can see how powerful REST clients can be. They can be consumed very similarly to core services like MySQL and PostgreSQL, and serve many more purposes as well.

This will be covered in Chapter 6.

Metaservices can be consumed directly in the browser. In fact, you do not need any frameworks at all to process a standard RESTful URL. Any browser-based Ajax call with an exec() function can do the trick:

var url = "http://example.com/users";
var req = new XMLHttpRequest();

req.onreadystatechange = function() {
  if (req.status == 200) {
    var json = req.responseText;

    // this is not secure, only
    // showing this for example
    var object = exec(json);

    // do something with
    // the object...
  }
}

req.open("GET", url, true);
req.send();

This code can be a little tricky and cumbersome, but some really great modern JavaScript frameworks have been developed that you can add to any website that will not only automate the consumption of these metaservices, but also display the data in a structured way. They include:

For a deeper discussion on how to select the proper JavaScript framework for your needs, see Steven Anderson’s blog post, “Rich JavaScript Applications—the Seven Frameworks”.

Impressive technologies like SproutCore and Cappuccino are able to provide building blocks to bind your data against. This is a well thought out and proven method for developing applications in modern desktop environments, incorporating data sources and data bindings.

As our JavaScript frontend client layers get thicker and thicker, we’re starting to see more and more composition of data sources in the frontend. Frontends are being presented through very dynamic, well presented, structured web interfaces.

Tools like SproutCore and Cappuccino provide building blocks, the foundational pieces that are tied together in a very standard way to build modern applications that feel like desktop applications on your web or mobile clients. In fact, SproutCore and Cappuccino are similar to using Apple’s Xcode or Microsoft’s Visual Studio. To a traditional web developer who has never created a desktop or native mobile application before, these tools can feel strange at first. The visual components are more strictly defined for you, and once you bind data sources to them, they seem to magically just work.

One of the benefits of using these technologies is that they are so prescribed, they have such a strong experience tied into them, that some of these clients can actually be compiled into native mobile applications. If you write your application in Cappuccino, for example, there are services like NativeHost that will compile your application without having to change anything to run in a desktop environment as a native app, which leads to a cross-media way of thinking about application development.

These technologies are so rich and becoming so well supported that we are seeing even large companies starting to use them. Apple has used SproutCore to build out its entire iCloud suite of web applications, supporting Mail, Contacts, Calendar, Notes, Reminders, Find My Phone, and iWork. NPR has built a beautiful Cappuccino-based player to stream its radio content. There are many new examples of adoption of these thick client interfaces every day.

This new form of development yields significant benefits in four important areas: agility, time to market, scalability, and interoperability.

One of the most vexing problems for enterprises and governments is how to realize the benefits of utilizing public cloud systems while keeping sensitive data restricted. Using the methods we’ve discussed in this chapter, businesses and governments can get the best of both worlds. They can enjoy the best agility and fastest time to market. They can utilize business-critical IT applications as services that are consumed in public cloud−hosted applications and mobile apps, which are delivered to their employees and customers in a safe and secure manner. This means they can start to experience the benefits that have already been realized by many smaller technology companies.

In addition, with metaservices, there can be independent teams working together in parallel on many different aspects of an application. Thus, tackling the development of large applications becomes a simpler problem of small teams focused on doing one thing very well.