M3s, the Kubernetes Framework for Apache Mesos



With M3s, I create the ability to run Kubernetes Cluster on Apache Mesos very quickly and easily. The clusters are separated from each other and can thus be operated on a per-client basis.

In just a few steps, Kubernetes can be launched on an existing Apache Mesos stack.

1) In this example we use the pre-compiled binary version of M3s. The latest developer version can be found in our Github Repository as Artifact under the respective “Binary Build”. It is worthwhile to use these for tests.

2) M3s needs some parameters to connect to Mesos. The following serve only as an example.

export MESOS_SSL=falseexport DOCKER_CNI=mini
❯ export LOGLEVEL=DEBUG
❯ export AUTH_USERNAME=user
❯ export AUTH_PASSWORD=password
❯ export VOLUME_K3S_SERVER=local_k3sserver
❯ export K3S_TOKEN=df54383b5659b9280aa1e73e60ef78fc
❯ export DOMAIN=.mini
❯ export BOOTSTRAP_URL=https://raw.githubusercontent.com/AVENTER-UG/mesos-m3s/master/bootstrap/bootstrap.sh
❯ export K3S_AGENT_LABELS=[{"key":"traefik.enable","value":"true"},{"key":"traefik.http.routers.m3s.entrypoints","value":"web"},{"key":"traefik.http.routers.m3s.service","value":"m3s-http"},{"key":"traefik.http.routers.m3s.rule","value":"HostRegexp(`example.com`, `{subdomain:[a-z]+}.example.com`)"}]

The variable K3S_AGENT_LABELS gives the possibility to create labels for Traefik or other load balancers connected to mesos. In the example given here are labels for our Traefik Provider.

3) Before we launch M3s, we create in Docker in dedicated network.

docker network create --subnet 10.40.0.0/24 mini

4) Now M3s can be started:

❯ ./mesos-m3s

After a short time, the corresponding M3s tasks appear in the Mesos GUI:

5) With the M3s Mesos CLI plugin we can export the required Kubernetes configuration:

❯ mesos m3s list

ID                                         Active  WebUI                    Name
2f0fc78c-bf81-4fe0-8720-e27ba217adae-0004  True    http://andreas-pc:10000  m3s

❯ mesos m3s kubeconfig m3s > ~/.kube/config

How to install the plugin can be read here hier.

If the mesos cli is not available to us, we use the API call:

❯ curl -k -X GET -u user:password http://127.0.0.1:10000/api/m3s/v0/server/config  > ~/.kube/config

Afterwards all kubectl commands are available

❯ kubectl get nodes

NAME                             STATUS   ROLES                  AGE    VERSION
k3sagent0.weave.local-170d0b36   Ready    <none>                 4m5s   v1.21.1+k3s1
k3sserver.weave.local            Ready    control-plane,master   5m7s   v1.21.1+k3s1

If a working mesos cli is not available, we can also export the kubeconfig using curl:

❯ curl -X GET http://127.0.0.1:10000/v0/server/config

The IP address specified is that of the framework. If the variable AUTH_USERNAME/PASSWORD has been set, these must also be assigned to curl.

6) Access to the Kubernetes Dashboard:

❯ kubectl -n kubernetes-dashboard describe secret admin-user-token | grep '^token'
❯ kubectl proxy