Day 3: Databases and REST APIs

March 5, 2014

This article is the third in a 5-day series about teaching beginners how to program. Read the introduction.

Yesterday, we outlined our first full unit of study, front-end development. This unit gives us the basic skills needed to create websites and web application interfaces using HTML and Bootstrap. It also introduces DOM manipulation and AJAX requests through jQuery, so we can retrieve data from external sources and display it in our own projects.

Today, we'll learn how to create, modify and save data to our own database. This will allow us to begin building full-featured, interactive web applications. Along the way, we'll learn some really important concepts about web development, such as HTTP and REST, JSON, and the difference between a server and a client.

After careful consideration and a lot of thought (remember how many databases I listed on Day 1?), I decided to go with Apache CouchDB for this unit of study. I've outlined my reasoning below.

Why CouchDB?

There are many reasons why I am convinced Apache CouchDB is the ideal database for this curriculum:

It's a top-level Apache open source project

This is important for many reasons. CouchDB doesn't seek to make a profit. There's no vendor lock-in. It's maintained by a vibrant, growing community of volunteers, and is being improved at a staggering pace. In 2013, CouchDB saw 5 major stable version releases (1.0.x to 1.5.x) that added all kinds of new features, and the roadmap for 2014 is no less promising.

The community is second-to-none.

The personalities and voices in the CouchDB community are, hands down, the most helpful, generous, welcoming, insightful, ethical, principled, future-minded group of developers I've ever encountered in one place. As far as online communities go, you can't find a safer, more encouraging group of advocates. These people are role models that will help beginners to understand what "open-source" can really be about: social justice, diversity, inclusion, and collaboration.

It's free and easy to install.

Apache CouchDB doesn't require complex configuration, and is very approachable out-of-the-box. There are installers for Windows and Mac OS X, and as soon as you launch the application, you can start creating databases and adding documents. It even has a built-in, friendly web-based administration console (called Futon) to make this easier for you. And, in situations where you can't install it on a machine, there are hosted solutions you can sign up with for free.

You can interact with it using a browser.

CouchDB has an HTTP interface. This means that every database, collection of documents, and individual document can be referenced by a specific URL, right in your browser. This makes it extremely easy to retrieve data using jQuery's AJAX features, without any additional configuration necessary. You also create, change and delete documents via URLs, so you can use familiar AJAX calls to manage the data in your database.

It's skating to where the puck will be.

The people behind CouchDB have a vision for the future of the web, and they're actively involved in building towards this vision today. You can see some examples of this vision here, here, here, and here.

It promotes a decentralized, federated web.

Over the past decade, we've seen a rise in walled-garden social networking sites. Your personal data is siloed in Facebook, Twitter, Tumblr, Google+, etc. This isn't a good norm for people to grow up with. And CouchDB can help us reclaim the notion that our personal data should belong to us.

At the heart of CouchDB is its unique replication capability, which allows us to copy or sync a database with other computer(s) and mobile devices on an ongoing basis. The CouchDB community is also working on offline-capable tools, so that periodic loss of connectivity doesn't affect our access to data. With CouchDB alone, we can easily build systems that put us in control of our own data. The sense of empowerment this can give someone is priceless.

It uses Javascript and delivers JSON.

Data in CouchDB is stored and retrieved primarily in JSON format, which means Javascript can work with it directly. Additionally, all your database views (collections of structured data) are generated using instructions in Javascript. There's a very gentle learning curve here, so learners will see results quickly (remember our mantra).

It's relevant to big data.

Data science has become a growing field in our region over the past few years. Several businesses in the Maritimes (Radian6, LeadSift, Affin.io & Tactics Cloud, IntroHive, UserEvents, etc.) were launched with data analytics at their core. There's such a demand that our universities are working with the private sector to accommodate it.

How does CouchDB fit in? It's massively concurrent (built with Erlang, which has concurrency at its heart). It encourages append-only storage, and can store and collect massive amounts of data (just ask CERN). It can easily be clustered. It's queried using the familiar map / reduce pattern that analytics engines like Hadoop use to run massive calculations. Learning how to use CouchDB will expose learners to concepts and architectures that they'll see again if they ever decide to pursue data science.

It's a great fit pedagogically.

CouchDB has many unique features that distinguish it among databases. Many of these features also make it an ideal fit for our course of study. For example: CouchDB acts as both a database and a web server, so it gives us our first exposure to a client-server setup. Normally, we'd need to set up a web server independently of our database, and use a language on the server to interact with (and serve) our data.

CouchDB also uses an HTTP interface, and using it requires you to gain a deep understanding of the protocol. So, in addition to learning about databases, we get to pick up concepts like HTTP headers, content-types, verbs like GET, POST, PUT and DELETE, and status codes (you know, like 200, 302, 404, etc). While HTTP and REST are essential concepts to web development, many programming environments try to hide them from developers. Lucky for us, CouchDB doesn't.

Finally, CouchDB has a flexible schema, which means you aren't required to think in abstract terms like columns, fields, data-types, primary keys and relationships. You can add fields to your documents as you go without affecting performance, and you don't need to specify what information you're going to store ahead-of-time. For beginners, this flexible, forgiving quality is a huge asset.

Unit overview

This unit picks up very nicely from where the last unit left off. Armed with a basic knowledge of front-end development, and the ability to pull data from an external source using AJAX, we're ready to build something that talks to a database.

The beginner's exercise is really simple: once students have downloaded and installed CouchDB (or signed up with a free hosting provider), get them to create a database and add some documents to it. This could be a list of record albums, recent meals, friends' contact information, etc. Once they have some documents in a database, challenge them to display a list of their documents on a website, using an AJAX request. On a basic configuration, this should be as simple as:

$.getJSON("http://127.0.0.1:5984/database_name/_all_docs?include_docs=true",
          function(data){…});

This should be enough to get students to see the benefits of a database. Using Futon, they could manage all their data directly in CouchDB if they wanted to, and those changes would appear in their documents when they reload.

Next, you might want to begin learning how to send data to CouchDB, so that you can create new documents from your own website or application. You can do this using HTTP POST requests. A simple example:

$.post("http://127.0.0.1:5984/database_name/",
       {"name": "Ari", "surname": "Najarian"},
       function(data){…});

A good way to teach this would be to create a simple form to collect data, which is submitted to CouchDB via AJAX. Students can, for example, create questionnaires and share them with their peers (which in my experience is a great way to engage them).

From here, there are all kinds of directions you can take. I recommend spending some time learning how to use CouchApp (Windows, Mac), so you can begin creating design documents without having to use Futon. Design documents will allow you to define special URLs for fetching organized collections of documents (views), formatting and delivering individual documents (show functions), formatting groups of documents (list functions), and updating documents on-the-fly (update handlers). You define these URLs on your CouchDB database using pure Javascript, so it's also a great chance to practice programming.

Since you can communicate with CouchDB over HTTP, working with it introduces learners to the concept of a server (as opposed to a client), and familiarizes them with the notion of a REST API (unique URLs for fetching and modifying documents). The beauty of this is that we never even had to choose or set up a web server (remember all those choices from Day 1?) - CouchDB acts as a database and a web server with a REST API built-in! So again, we've made an investment in a technology that give us quick results without introducing unnecessary complexity. It helps us focus on what we're trying to learn (Javascript and web technologies), and offers very few distractions from this goal.

CouchDB's many unique features make it possible to stay on this unit for a long time. With just CouchDB and your front-end development skills from the previous unit, you actually have everything you need to develop a fully-featured web application. CouchDB can serve websites as well as JSON (you can attach HTML, CSS and Javascript files to documents directly), so you can bundle your front-end into a design document and access it via a URL. You can even set up users & passwords, so people can log into your application and see their data. The possibilities are endless.

However, depending on time constraints, you'll likely want to move on to subsequent units so you can expose students to as many different topics as possible. The bare minimum that I recommend students learn before moving onto Day 4 is how to define views and update handlers in a design document, and how to interact with CouchDB via the HTTP REST interface. This will prepare them for the next unit, which introduces more advanced programming patterns that can help them build ambitious, database-backed web applications.

But that's a topic for tomorrow.

Up next:

Ari Najarian Follow @stickbyatlas