Containers Everywhere

This post was originally published on the Predix Developer Network Blog in 2016.

It feels like everybody is talking about containers. What is a container? When should you use one? How do you use them? What does this have to do with Predix?

Surprise! It’s containers all the way down — in the cloud, on your laptop, and on the edge.

Containers in the Cloud

While Docker, Mesos, and Kubernetes (k8s) may dominate what’s #trending — Cloud Foundry has been providing us with “boring, cloud-scale container technology” all along. It’s been working reliably in production behind the scenes such that we forget to even bring it up in conversation.

Machines in the Cloud

The value of software containers is that you can pack your code and any dependencies into a container image format that can be run nearly anywhere. If something goes wrong, you can tear down and fire up a new one (or thousand) quickly. Taking advantage of Linux isolation features for process separation, this ability to load and unload an application without shipping around the full operating system in a virtual machine hypervisor is incredible and makes the cloud even more elastic. Containers make it easier to orchestrate deployments knowing your software will run the same and has become a key enabler for building micro-services.

The Open Container Initiative (OCI) helps negotiate some standards around container image formats as Docker is not the only option, runC and rkt are alternatives as well. If you are highly opinionated on the image format of your container, Cloud Foundry’s latest release of Diego does support the Docker image format. I won’t go into too much detail because Predix hasn’t made the switch to Diego and may instead prefer developers stick with runC for security among other reasons.

If you are interested in the mechanics of how containers work — there are plenty of other well written sources — but don’t click spiral away just yet, I’m just getting to the interesting part.

Containers on your Laptop

Wi-Fi: Looking for Networks…

An Application Development Environment (ADE) is a fancy way of saying anything “used to build enterprise-wide applications that need to be executed and tested on an array of different computing devices and back-end or integrated platforms.” Basically, your IDE and a few other tools.

I’ve worked with many developers who are just getting started building apps and services for Cloud Foundry but run into environment problems. These have nothing to do with the app they are trying to build but rather silly things like having a busted ruby on osx, or a fight with node.js on ubuntu, or can’t pipe commands together on windows. Writing tutorials and guides for such a fragmented set of computer systems presents a rather unique challenge.

Further, once you get past that there are development tools you need to use and install. So, you go through installing dependencies one by one only to get into the trials and tribulations of version management. Oh, you have that version of node, you can’t use the system ruby with that gem, did you try sudo? Then after you get all your updates only to find out that you broke something else on your machine that depended on that version and BOOM, everything is terrible and on fire. There are tools that can help with this up to a point: for JavaScript there is Node Version Manager (nvm), Ruby has Ruby Version Manager (rvm), and Virtual Environment (virtualenv) for the Pythonistas.

“But wait, can I use containers to solve this?” Yes, you’re right, I haven’t said container in the last 37 words. Predix DevBox is a fine option if you want the whole Window Manager and Desktop experience, but you can use containers for this too for a native in a shell terminal experience.

Just Install Docker Already

A friend formatted his laptop and declared that he was not going to install ANYTHING on his mac other than Docker. Using that dockerize-your-shell project as inspiration there are a number of tools you can use for software development without installing them natively.

For example, when I need to do java development I haven’t even bothered installing Maven. I setup an alias to run it in a container and can’t tell the difference.

alias mvn='docker run -it --rm --name maven \
  -v "$PWD":/usr/src/maven \
  -w /usr/src/maven maven:3.2-jdk-7 mvn'

Installing Docker can be a breeze now with native versions that don’t even require VirtualBox any longer. I’ve had great success with Docker for Mac since it was released in Beta.

CF-ADE

To work with Cloud Foundry, you do need some basic tools. A Linux environment can be helpful since many developers are on osx or Linux, but windows users may feel left out. Docker can help level that playing field.

Run the following command and you’ll land yourself in an environment with the cf and uaac tools installed along with many common Linux utilities.

docker run --rm -it j12y/cf-ade

The first time you run it, the command will contact an online repository of Docker images called Docker Hub. Your individual firewall mileage may vary so check your proxy settings.

Once launched, you can login to Predix and go to work pushing apps.

cf login -a $PREDIX

You just exit when you are done. Since you’ve already pulled down all the layers, the next time you need it startup will be quick. To understand what it is doing you can check out the Dockerfile in the j12y/cf-ade github repository.

FROM phusion/passenger-full

This first line is the base image. I’ve used phusion which is a fair bit larger than other options like alpine or coreos but includes a lot more of the tools commonly found on Ubuntu Linux systems. Since this is intended for application development rather than scaling on the cloud, I felt this was an acceptable trade-off on size vs features.

ENV PREDIX https://api.system.aws-usw02-pr.ice.predix.io

You can define environment variables for the container, such as the Predix API endpoint.

# Install some Ubuntu dependencies
RUN apt-get update && apt-get install -y \
    wget \
    unzip

We’re not savages, so we may want to use wget and unzip which are surprisingly absent from phusion and ubuntu for such commonly used tools.

# Install Cloud Foundry CLI
RUN wget -O cf-cli.deb "https://cli.run.pivotal.io/stable?release=debian64&source=github" && dpkg -i cf-cli.deb
# Install Cloud Foundry UAAC
RUN gem install cf-uaac

This is the special sauce. The cloud foundry cf cli and uaac tool are fundamental to a lot of the Predix development workflows and install cleanly into this environment.

Pushing it further

Now, let’s say you are a polyglot developer and today you are working on a python project and tomorrow a javascript project. You cd into your project and fire up your environment:

docker run --rm --volume=$(pwd):/home/app -it j12y/cf-ade-py

A few differences to note in the command here. First, by using --volume we’re allowing the container access to our local file system for our project (probably a GitHub repo). From inside the container you can’t access any other files otherwise. Remember that isolation bit we talked about earlier. Second, the j12y/cf-ade-py image uses j12y/cf-ade as the base layer, so the cloud foundry tools we had before but now we pull down a new small layer to add other tools like pip, ipython, virtualenv, and other python libraries common for Predix big data and analytics application development like pandas and scikit.

Similarly, the j12y/cf-ade-js image includes tools like gulp, npm, and bower for when you work on a javascript project. The best part is that you can trust your co-workers will have the same development experience should they want to run your app.

I’ve provided j12y/cf-ade as an illustrative example. You can freely use it, but if your needs are different, fork and modify the Dockerfile to your liking and you can docker build your own local image.

Containers on the Edge

And behind door number 3, Predix Machine is the edge solution of edge-to-cloud, running software at the physical location of the device. Using the Machine SDK we have an opportunity to do computations closer to the source of sensor data. This gives us reduced latency in network transfer rates but also improved reliability for connectivity issues when those edge devices may be very far from the nearest ethernet plug or starbucks.

If you hadn’t already guessed, yes — you can container there too. Predix Machine supports Docker containers for remote deployment and execution. Setting up Predix Machine on a microprocessor and deploying containers in the fog can be a bit involved so we’ll save those details for another day.