Tag Archives: bluemix

Choose IBM’s Docker-based Container Service on Bluemix for your I/O intensive code

Few weeks ago IBM announced general availability of a new Docker-based container service[1] as part of Bluemix PaaS[2]. The service was in beta since the beginning of 2015 and looks very promising, especially for I/O heavy workloads like databases and analytics. This post will help you create your own container instance running on Bluemix and provide some pointers on how you can evaluate whether the I/O performance of the instances matches your application’s needs. It will also describe the nuances of using Docker Machine[5] if you are running Mac OSX or Windows.

Even if you are not familiar with Docker, chances are you know about virtual machines. When you order a server hosted in a cloud, in most cases you get a virtual machine instance (a guest) running on a physical server (a host) in your cloud provider’s data center. There are many advantages in getting a virtual machine (as opposed to a physical server) from a cloud provider and arguably the top one is quicker delivery. Getting access to a physical server hosted in a data center usually takes hours while you can get a virtual machine in a matter of minutes. However, many workloads like databases and analytics engines are still running on physical servers because virtual machine hypervisors introduce a non-trivial penalty on I/O operations in guest instances.

Enter Linux Containers(LXC) and Docker. Instead of virtualizing the hardware (as the case with traditional virtual machines) containers virtualize the operating system. Start up time for containers is as good or better (think seconds not minutes) than for virtual machines and the I/O overhead all but disappears. In addition, Docker makes it easier to manage both containers and their contents. Containers are not a panacea and there are situations where virtual machines make more sense but that’s a topic for another post.

In this post, you can follow along with the examples to learn whether I/O performance of Docker containers on IBM Bluemix matches your application’s needs. Before starting, make sure that you have access to Bluemix[3] and to the Container Service[4].

Getting Started

The following steps describe how to use Docker Machine[5] for access to a Docker installation. You can skip the Docker Machine instructions if you are already running Docker 1.6 or higher on your Linux instance. Otherwise, you’ll want a Docker Machine which deploys a VirtualBox guest with Tiny Core Linux to OS X or Windows and you can ssh into the guest to access docker CLI.

Start of Docker Machine specific instructions

Install Docker Machine as described here[5]. Use the following command to connect to your Docker Machine Tiny Core Linux guest

docker-machine ssh dev

Install python and the ice CLI tool in your guest so you can interface with the IBM Container Service environment. The approach used in these steps to install python is specific to TinyCore Linux and shouldn’t be used on other distros.

tce-load -wi python
curl https://bootstrap.pypa.io/get-pip.py -o - | sudo python
curl https://bootstrap.pypa.io/ez_setup.py -o - | sudo python
curl -o cf.tgz -L -O https://cli.run.pivotal.io/stable?release=linux64-binary
sudo tar -zxvf cf.tgz -C /usr/bin/
curl -O https://static-ice.ng.bluemix.net/icecli-3.0.zip
sudo pip install icecli-3.0.zip
rm -f cf.tgz icecli-3.0.zip setuptools-18.0.1.zip

End of Docker Machine specific instructions

If you are not using Docker Machine, you should follow the standard ice CLI installation instructions[6]

Before proceeding, you will create a public/private key pair which you’ll use to connect to your container. The following steps save your private key file to id_rsa and your public key file to id_rsa.pub with both in your local directory. The second step ensures that the private key file is ignored by Docker and doesn’t get included in your image.

sudo ssh-keygen -f id_rsa -t rsa
echo id_rsa > .dockerignore

Make sure that you have access to the IBM Container Service from the Bluemix dashboard as described here[8] and proceed to login using the ice CLI with the command below. Note that when prompted you’ll need to enter your Bluemix credentials and to specify the logical organization as well as the space where your container will reside.

ice login

The login command should complete with a Login Succeeded message.

Next, you will pull one of the base IBM Docker images and customize it with your own Docker file:

docker pull registry.ng.bluemix.net/ibmnode:latest

Once the image completed downloading, you will create a Dockerfile that will customize the image with your newly created credentials (so you can ssh into it) and with sysbench scripts for performance testing.

Create a Dockerfile using your favorite editor and the following contents:

FROM registry.ng.bluemix.net/ibmnode:latest
MAINTAINER Carl Osipov

ADD *.sh /bench/

EXPOSE 22
COPY id_rsa.pub /root/.ssh/
RUN cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys

RUN apt-get update && apt-get install -y sysbench

Next, create io.sh with the following contents

#!/bin/sh
SIZE="$1"
sysbench --test=fileio --file-total-size=$SIZE prepare
sysbench --test=fileio --file-total-size=$SIZE --file-test-mode=rndrw --init-rng=on --max-time=30 --max-requests=0 run
sysbench --test=fileio --file-total-size=$SIZE cleanup

And cpu.sh containing:

#!/bin/sh
PRIME="$1"
sysbench --test=cpu --cpu-max-prime=$PRIME run

add execute permissions to both scripts

chmod +x *.sh

At this point your custom Docker image is ready to be built. Run

docker build -t sysbench .

which should finish with a Successfully built message followed by an ID.

Push your custom Docker image to Bluemix

When you first accessed the Container Service via the Bluemix dashboard[4], you should have specified a custom namespace to be used when provisioning your containers. Note that in the steps below, `ice namespace get` is evaluated to retrieve the custom namespace which is unique to your account.

docker tag sysbench registry.ng.bluemix.net/`ice namespace get`/sysbench
docker push registry.ng.bluemix.net/`ice namespace get`/sysbench
ice run -p 22 -n sysbench `ice namespace get`/sysbench

After you’ve executed the commands above, your container should be in a BUILD stage which changes to the Running stage after a minute or so. You can verify that by executing

ice ps

Request a public IP address from the Container Service and note its value.

ice ip request

Bind the provided public IP address to your container instance with

ice ip bind <public_ip_address> sysbench

Now you can go ahead and ssh into the container using

sudo ssh -i id_rsa root@<public_ip_address>

Once there, notice it is running Ubuntu 14.04

lsb_release -a

on a 48 core server with Intel(R) Xeon(R) CPU E5-2690 v2 @ 2.60GHz CPUs

cat /proc/cpuinfo
...
processor : 47
vendor_id : GenuineIntel
cpu family : 6
model : 63
model name : Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz

Now you can also test out the I/O performance using

. /bench/io.sh 100M

Just for a comparison, I ordered a Softlayer virtual machine (running on a Xen hypervisor) and ran the same Docker container and the benchmark there. In my experience, the I/O benchmark results were roughly twice better on the Container Service than on a Softlayer VM. You can also get a sense of relative CPU performance using

. /bench/cpu.sh 5000

Conclusions

Benchmarks are an artificial way of measuring performance and better benchmark results don’t always mean that your application will necessarily run better or faster. However, benchmarks can help you understand if there exists potential for better performance and help you design or redesign your code accordingly.

In case of the Containers Service on IBM Bluemix, I/O benchmark performance results are significantly superior to those from a Softlayer virtual machine. This shouldn’t be surprising since Containers runs on bare metal Softlayer servers. However, unlike the hardware servers, Containers can be delivered to you in seconds compared to hours for bare metal. This level of responsiveness and workload flexibility enable Bluemix application designers to create exciting web applications built on novel and dynamic architectures.

References

[1] https://developer.ibm.com/bluemix/2015/06/22/ibm-containers-on-bluemix/
[2] https://console.ng.bluemix.net
[3] https://apps.admin.ibmcloud.com/manage/trial/bluemix.html
[4] https://console.ng.bluemix.net/catalog/?org=5b4b9dfb-8537-48e9-91c9-d95027e2ed77&space=c9e6fd6b-088d-4b2f-b47f-9ee229545c09&containerImages=true
[5] https://docs.docker.com/machine/#installation
[6] https://www.ng.bluemix.net/docs/starters/container_cli_ov_ice.html#container_install
[7] https://www.ibm.com/developerworks/community/blogs/1ba56fe3-efad-432f-a1ab-58ba3910b073/entry/ibm_containers_on_bluemix_quickstart?lang=en&utm_content=buffer894a7&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
[8] https://www.ng.bluemix.net/docs/containers/container_single_ov.html#container_single_ui

Secure your IBM BlueMix web app with OAuth 2.0 using the IBM ID Single Sign-on Service

Authentication and authorization often form the starting set of security capabilities one adds to a web application. Due to the widespread need for these capabilities across web applications of various types, numerous standards are available, including OpenID, OAuth, and others. For example, in case of authorization, OAuth 2.0 has been broadly adopted by vendors including IBM, Google, LinkedIn and others. This post describes how a Node.js application deployed to IBM BlueMix can be augmented to use IBM ID, which is an OAuth 2.0 compliant, policy-based authentication service that provides an easy to use single sign-on (SSO) capability for web and mobile applications.

IBM ID provides access to IBM applications, services, communities, support, on-line purchasing, and more. IBM ID credentials are easy to create and your profile information is centralized so you can maintain it in a convenient and secure location.[1]

This post extends an earlier entry from this blog on how to deploy a minimal, hello world style application to Node.js on IBM BlueMix. The authentication and authorization approach presented here can be easily layered onto any existing Node.js app.

Getting Started

To try the code described in this post, you have to have your own IBM ID account. It is easy to register for one using the following form: https://www.ibm.com/account/profile/us?page=reg

Throughout this post ibm-sso-nodejs-sample will be used as the BlueMix application name. The hostname for the application is going to be generated by replacing a substring ‘sample’ from the name with a random value (e.g. ibm-sso-nodejs-42) to minimize the chances of name collisions across multiple readers of this post.

Start by cloning the ibm-sso-nodejs-sample from the git repository available here: https://github.com/osipov/ibm-sso-nodejs-sample

The next section is going to describe the implementation of the code, so if you’d like you can skip forward to the Prepare and push code to BlueMix section to try out the code.

Implementation Overview

package.json file (shown below) declares key dependencies used in this sample.

Node.js Express framework[2] simplifies the setup of the routes and management of the application session state. Passport[3] is another Node.js framework for integration of various authentication providers (including OAuth2 based ones) into an Express based application. The library dependency for the IBM ID OAuth2 provider[4] is declared on line 11. If you are planning on debugging the application on your local machine (i.e. outside of BlueMix), then you should execute the npm install command from the ibm-sso-nodejs-sample directory to download and install the dependencies from package.json into your environment. Otherwise, the dependencies will be downloaded automatically inside the BlueMix runtime container during the execution of the cf push command. The application code for this sample resides in the main.js file (snippet below). The initial section of main.js[5] contains commonly used setup code to import the two frameworks mentioned earlier: Express and Passport. Note that this implementation uses the Express memory based session store[6]. While this is a reasonable approach for sample code, if you decide to take this code into production, you should migrate to using an alternative session cache. Examples of alternatives include the IBM Session Cache[7] and Redis[8], both available from the BlueMix service catalog.

The sample provides templates for callback functions that are invoked by the Passport framework when serializing/deserializing a user[9]. Although this sample does not specify any code in these functions, a production class application would insert code here to manage application specific extensions to a user object, e.g. an application specific user ID.

The application is initialized using BlueMix specific code to extract environment variables from the BlueMix runtime container and to set the variables related to the hostname, port number, URL as well as the IBM ID OAuth 2.0 client ID and client secret[10]. Note that the variables for the ID and secret are set to blank in the source code repository but will be modified according to the instructions later in this post.

To use the IBM ID OAuth2 provider, one must include the code from lines 34-52 into a Node.js application[11]. clientID, clientSecret and callbackURL are the minimal required parameters to create a new instance of the IbmIdStrategy OAuth2 strategy. The callback function[12] processes the access and the refresh tokens retrieved in case of a successful OAuth 2.0 authorization. The profile variable carries the IBM ID profile information supplied by an IBM ID user. The code used to save the profile information in the session[13] is arbitrary, an alternative implementation can choose to disregard the profile information entirely or extract just the necessary attributes of the profile to populate the session.

OAuth2.0 specification recommends the use of a state variable to help prevent cross-site request forgery[14]. Since the code in this post is not intended for production, the ‘/ibmid/auth’ route does not pass a state attribute along with the scope to the passport.authenticate function. When adopting this code for production, you should implement a random string generator to set the state attribute and then verify that the random value is correctly returned back as a req.query.state variable in the handler function of the ‘ibmid/auth/callback’ route[15].

The rest of this section describes the routes added just to demonstrate the OAuth 2.0 based authentication and authorization process, there is a “public” route that can be accessed without any authentication whatsoever[16], a “private” route that requires a client to authorize with IBM ID[17], a “logout” route that logs a client out of an IBM ID authentication[18], and an “error” route that is used to display a message to a client in case of any OAuth 2.0 errors[19].

Prepare and push code to BlueMix

Before proceeding with the rest of the instructions in this post, push the code you cloned from the github repository to BlueMix using the following command

cf push ibm-sso-nodejs-sample -n ibm-sso-nodejs-$RANDOM -c 'node main.js'

The result of the push command should provide a session similar to the one shown in the snippet. Note the value of the URL, e.g. from the session which should look like line 50 from the snippet. It will be referred to as <URI> in the instructions below.

Any IBM ID OAuth 2.0 client must be registered with IBM. To do so, Open https://idaas.ng.bluemix.net/idaas/developer/manageclients.jsp using your browser, login using your IBM ID and click on “Create a new client”. Leave client identifier and client secret as then type in ibm-sso-node-js-sample for the Display Name and specify <URI> for the Redirect URI. Check off the Enabled box, click Submit and take note of the client ID and secret values that were randomly generated. The rest of this post will refer to their values as <ID> and <SECRET> .

To propagate the OAuth 2.0 credentials that you just obtained from IBM, execute the following commands to create Cloud Foundry user defined variables:
cf set-env ibm-sso-nodejs-sample SSO_CLIENT_ID <ID> cf set-env ibm-sso-nodejs-sample SSO_CLIENT_SECRET <SECRET>

Do one last push to persist the user defined variables.

cf push ibm-sso-nodejs-sample -c 'node main.js'

Test out the app

Start by navigating to the URL assigned to you by BlueMix. You should see a ‘Hello World’ message along with a login link.

Clicking on the link should force you to login using the IBM ID you created earlier.

Once you are logged in, you should see a Hello message with the first and last name you used when registering for your IBM ID.

References