December 21, 2017 · kubernetes gcp opensource

Failing to Install Apache Pulsar in a Few Minutes with One Command

I was surfing around on Twitter today when I ran across this Tweet by @streamlio on Apache Pulsar, a pub/sub system built using Zookeeper and a few other bits.


Streamlio's claim was that you could run a single command to get Apache Pulsar "up and running" in "minutes" using "one command". In all fairness they said "on DC/OS", but then again they also said "one command", which is not true at least according to the video. The video clearly shows one command to download the code and another command to launch it on DC/OS, which itself had to be installed and configured prior to launch.

"Installing" Kubernetes on Google Cloud doesn't require any commands to be typed in. It's a simple click on a button, name the cluster and a waiting period of about how long it takes to make a cup of coffee to be off and running.

If you'd like to skip the drama here, and just get it installed, check out Apache Pulsar's GCP guide here.

Given this misinformation about an infrastructure project, I thought it was a good enough excuse to try to get Apache Pulsar running on my Kubernetes cluster on Google Cloud...from Kubernetes configs alone. If you have 15 minutes to burn for the stream I did, while I grumbling about how people don't do the work required to make their claims true, knock yourself out:

Just Skip to the Commands

To follow along with me in the video, you'll need to have a Google Cloud account, have a Kubernetes cluster spinning, and have the Google Cloud Shell open. Assuming these things are true, you should be able to run four commands to get the same results as the other video guide.

First, check out the Apache Pulsar repo and change into the container engine directory:

$ git clone
Cloning into 'incubator-pulsar'...
remote: Counting objects: 24637, done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 24637 (delta 6), reused 22 (delta 5), pack-reused 24596
Receiving objects: 100% (24637/24637), 16.77 MiB | 23.70 MiB/s, done.
Resolving deltas: 100% (11360/11360), done.

Next, change into the working directory:

$ cd incubator-pulsar/kubernetes/google-container-engine
$ ls -la
total 36
drwxr-xr-x 2 kordless kordless 4096 Dec 21 09:33 .
drwxr-xr-x 4 kordless kordless 4096 Dec 21 09:33 ..
-rw-r--r-- 1 kordless kordless 5354 Dec 21 09:33 bookie.yaml
-rw-r--r-- 1 kordless kordless 3317 Dec 21 09:33 broker.yaml
-rw-r--r-- 1 kordless kordless 5514 Dec 21 09:33 monitoring.yaml
-rw-r--r-- 1 kordless kordless 4865 Dec 21 09:33 zookeeper.yaml

Now, start the services:

$ kubectl create -f zookeeper.yaml
$ kubectl create -f monitoring.yaml
$ kubectl create -f broker.yaml
$ kubectl create -f bookie.yaml

You should get something like this for your pod list:

$ kubectl get po
bookie-2nh76                           0/1       Pending             0          1s
bookie-9h7q9                           0/1       Pending             0          1s
bookie-autorecovery-5c967b6bc9-5kwq4   0/1       Pending             0          2m
bookie-autorecovery-5c967b6bc9-ls9vs   0/1       Pending             0          2m
bookie-dmpkg                           0/1       Pending             0          1s
bookie-fwmlx                           0/1       Pending             0          0s
broker-9db7b45c7-9xxdv                 0/1       ContainerCreating   0          2m
broker-9db7b45c7-gs5qp                 0/1       ContainerCreating   0          2m
broker-9db7b45c7-pf7cb                 0/1       CrashLoopBackOff    2          2m
grafana-5dc58f45d-zxldr                1/1       Running             0          2m
prometheus-748697d8d7-whpxm            1/1       Running             0          2m
pulsar-admin                           1/1       Running             0          2m
pulsar-dashboard-67b8788b69-v4qnd      1/1       Running             0          2m
zk-0                                   1/1       Running             0          3m
zk-1                                   0/1       Pending             0          1m

Note that I failed to start these services due to the fact I did not have enough required resources in my Kubernetes cluster and probably missed a bunch of steps after that because of this fact.