Cluster User Docs
We run Kubernetes, a cluster system on our production machines. This allows you to schedule software to run without having to worry about traditional deployment, or where your particular piece of code is actually running. This document will not teach you how to use Kubernetes, but will give you a short hands-on example on how to access it, and then point you in the right direction for general documentation to follow.
Kubernetes is accessed fully via an API, for which there exists a standard command line tool:
kubectl. If you’ve check out hscloud and followed the instructions in //README.md, you should have that tool built and available for you to use.
Before you can use
kubectl, however, you will need to authenticate yourself. To do that, run
prodaccess. This will issue you short-term (~hours) credentials that
kubectl can then pass on to Kubernetes to authenticate itself.
$ prodaccess Enter SSO/LDAP password for firstname.lastname@example.org: Good evening professor. I see you have driven here in your Ferrari.
You might need to ping q3k on #hswaw-infra if you get a PermissionDenied error.
prodaccess is not on your $PATH:
$ bazel run //tools:install $ . env.sh
prodaccess will use your local user name to authenticate as
<user>@hackerspce.pl. If your Hackerspace SSO name is different, specify it using the
-u flag to prodaccess, eg.
prodaccess -u informatic.
You can now check that you indeed have access to Kubernetes:
$ kubectl version # show version of Kubernetes $ kubectl top nodes # show node (machine/server) statistics
You are now fully set up to schedule your own jobs on
k0.hswaw.net, our currently only Kubernetes cluster.
We have a fairly extensive role-based access control system set up to provide a level of multi-tenancy of our Kubernetes cluster. What this means is that you will not be able to modify other people’s stuff. Indeed, by default, you barely have any access. So as you can experiment with Kubernetes, we automatically provision you a personal namespace (
personal-$USER) in Kubernetes. This acts as your own playground, where you can run anything you want, as long as it doesn’t eat into our resources too much.
For example, to run an Alpine Linux Docker image in your own namespace:
kubectl -n personal-$USER run --image=alpine:latest -it foo
This will create a Kubernetes pod named foo, running the
alpine:latest Docker image, and drop you in an interactive shell in it. Naturally, replace
$USER with your SSO username if it’s different from your system username.
Once you’re done, delete the pod:
kubectl -n personal-$USER delete pod foo
Apart from the RBAC (role based access control) that prevents you from poling at things that you shouldn’t over the API, we have one more security measure in place. Throught a Kubernetes mechanism called ‘PodSecurityPolicy’ we limit what pods (ie. containers) can do. Notably, pods will by default not be able to access any host data, run in privileged mode, or even setuid to a different uid. The most notable side effect of this is that some basic system tools within pods will not work: ie., apt on Ubuntu.
We highly recommend following the Kubernetes Basics tutorial as a first step in using Kubernetes for real world applications.
For defining production jobs, we use a language called
Jsonnet via a tool called
kubecfg. This is to replace some more popular tools that other Kubernetes systems use, eg. Helm. For more information about that, ping q3k so that he writes a codelab about it :). Before he does, see jsonnet documentation, and basic (possibly outdated)