Add k8s audit instructions for kops

examples/k8s_audit_config/README.md

@@ -1,38 +1,54 @@
 # Introduction
 
-The files in this directory can be used to configure k8s audit logging. The relevant files are:
+This page describes how to get K8s Audit Logging working with Falco. For now, we'll describe how to enable audit logging in k8s 1.11, where the audit configuration needs to be directly provided to the api server. In 1.13 there is a different mechanism that allows audit confguration to be managed like other k8s objects, but these instructions are for 1.11.
+
+The main steps are:
+
+1. Deploy Falco to your K8s cluster
+1. Define your audit policy and webhook configuration
+1. Restart the API Server to enable Audit Logging
+1. Observe K8s audit events at falco
 
-* [audit-policy.yaml](./audit-policy.yaml): The k8s audit log configuration we used to create the rules in [k8s_audit_rules.yaml](../../rules/k8s_audit_rules.yaml). You may find it useful as a reference when creating your own K8s Audit Log configuration.
-* [webhook-config.yaml.in](./webhook-config.yaml.in): A (templated) webhook configuration that sends audit events to an ip associated with the falco service, port 8765. It is templated in that the *actual* ip is defined in an environment variable `FALCO_SERVICE_CLUSTERIP`, which can be plugged in using a program like `envsubst`. You may find it useful as a starting point when deciding how to route audit events to the embedded webserver within falco.
+## Deploy Falco to your K8s cluster
 
-These files are only needed when using Minikube, which doesn't currently
-have the ability to provide an audit config/webhook config directly
-from the minikube commandline. See [this issue](https://github.com/kubernetes/minikube/issues/2741) for more details.
+Follow the [K8s Using Daemonset](../../integrations/k8s-using-daemonset/README.md) instructions to create a falco service account, service, configmap, and daemonset.
 
-* [apiserver-config.patch.sh](./apiserver-config.patch.sh): A script that changes the configuration file `/etc/kubernetes/manifests/kube-apiserver.yaml` to add necessary config options and mounts for the kube-apiserver container that runs within the minikube vm.
+## Define your audit policy and webhook configuration
 
-A way to use these files with minikube to run falco and enable audit logging would be the following:
+The files in this directory can be used to configure k8s audit logging. The relevant files are:
 
-#### Start Minikube with Audit Logging Enabled
+* [audit-policy.yaml](./audit-policy.yaml): The k8s audit log configuration we used to create the rules in [k8s_audit_rules.yaml](../../rules/k8s_audit_rules.yaml).
+* [webhook-config.yaml.in](./webhook-config.yaml.in): A (templated) webhook configuration that sends audit events to an ip associated with the falco service, port 8765. It is templated in that the *actual* ip is defined in an environment variable `FALCO_SERVICE_CLUSTERIP`, which can be plugged in using a program like `envsubst`.
 
-Run the following to start minikube with Audit Logging Enabled:
+Run the following to fill in the template file with the ClusterIP ip address you created with the `falco-service` service above. Although services like `falco-service.default.svc.cluster.local` can not be resolved from the kube-apiserver container within the minikube vm (they're run as pods but not *really* a part of the cluster), the ClusterIPs associated with those services are routable.
 
 ```
-minikube start --kubernetes-version v1.11.0 --mount --mount-string $PWD:/tmp/k8s_audit_config --feature-gates AdvancedAuditing=true
+FALCO_SERVICE_CLUSTERIP=$(kubectl get service falco-service -o=jsonpath={.spec.clusterIP}) envsubst < webhook-config.yaml.in > webhook-config.yaml
 ```
 
-#### Create a Falco DaemonSet and Supporting Accounts/Services
+## Restart the API Server to enable Audit Logging
 
-Follow the [K8s Using Daemonset](../../integrations/k8s-using-daemonset/README.md) instructions to create a falco service account, service, configmap, and daemonset.
+A script [enable-k8s-audit.sh](./enable-k8s-audit.sh) performs the necessary steps of enabling audit log support for the apiserver, including copying the audit policy/webhook files to the apiserver machine, modifying the apiserver command line to add `--audit-log-path`, `--audit-policy-file`, etc. arguments, etc. (For minikube, ideally you'd be able to pass all these options directly on the `minikube start` command line, but manual patching is necessary. See [this issue](https://github.com/kubernetes/minikube/issues/2741) for more details.)
+
+It is run as `bash ./enable-k8s-audit.sh <variant>`. `<variant>` can be one of the following:
+
+* "minikube"
+* "kops"
 
-#### Configure Audit Logging with a Policy and Webhook
+When running with variant="kops", you must either modify the script to specify the kops apiserver hostname or set it via the environment: `APISERVER_HOST=api.my-kops-cluster.com bash ./enable-k8s-audit.sh kops`
 
-Run the following commands to fill in the template file with the ClusterIP ip address you created with the `falco-service` service above, and configure audit logging to use a policy and webhook that directs the right events to the falco daemonset. Although services like `falco-service.default.svc.cluster.local` can not be resolved from the kube-apiserver container within the minikube vm (they're run as pods but not *really* a part of the cluster), the ClusterIPs associated with those services are routable.
+Its output looks like this:
 
 ```
-FALCO_SERVICE_CLUSTERIP=$(kubectl get service falco-service -o=jsonpath={.spec.clusterIP}) envsubst < webhook-config.yaml.in > webhook-config.yaml
-minikube ssh sudo bash /tmp/k8s_audit_config/apiserver-config.patch.sh
+$ bash enable-k8s-audit.sh minikube
+***Copying audit policy/webhook files to apiserver...
+audit-policy.yaml                                                                           100% 2519     1.2MB/s   00:00
+webhook-config.yaml                                                                         100%  248   362.0KB/s   00:00
+apiserver-config.patch.sh                                                                   100% 1190     1.2MB/s   00:00
+***Modifying k8s apiserver config (will result in apiserver restarting)...
+***Done!
+$
 ```
+## Observe K8s audit events at falco
 
 K8s audit events will then be routed to the falco daemonset within the cluster, which you can observe via `kubectl logs -f $(kubectl get pods -l app=falco-example -o jsonpath={.items[0].metadata.name})`.
-

examples/k8s_audit_config/apiserver-config.patch.sh

@@ -2,7 +2,8 @@
 
 IFS=''
 
-FILENAME="/etc/kubernetes/manifests/kube-apiserver.yaml"
+FILENAME=${1:-/etc/kubernetes/manifests/kube-apiserver.yaml}
+VARIANT=${2:-minikube}
 
 if grep audit-webhook-config-file $FILENAME ; then
     echo audit-webhook patch already applied
@@ -12,23 +13,31 @@ fi
 TMPFILE="/tmp/kube-apiserver.yaml.patched"
 rm -f "$TMPFILE"
 
+APISERVER_PREFIX="    -"
+APISERVER_LINE="- kube-apiserver"
+
+if [ $VARIANT == "kops" ]; then
+    APISERVER_PREFIX="     "
+    APISERVER_LINE="/usr/local/bin/kube-apiserver"
+fi
+
 while read LINE
 do
     echo "$LINE" >> "$TMPFILE"
     case "$LINE" in
-        *"- kube-apiserver"*)
-            echo "    - --audit-log-path=/tmp/k8s_audit_config/audit.log" >> "$TMPFILE"
-            echo "    - --audit-policy-file=/tmp/k8s_audit_config/audit-policy.yaml" >> "$TMPFILE"
-            echo "    - --audit-webhook-config-file=/tmp/k8s_audit_config/webhook-config.yaml" >> "$TMPFILE"
-            echo "    - --audit-webhook-batch-max-wait=5s" >> "$TMPFILE"
+        *$APISERVER_LINE*)
+            echo "$APISERVER_PREFIX --audit-log-path=/var/lib/k8s_audit/audit.log" >> "$TMPFILE"
+            echo "$APISERVER_PREFIX --audit-policy-file=/var/lib/k8s_audit/audit-policy.yaml" >> "$TMPFILE"
+            echo "$APISERVER_PREFIX --audit-webhook-config-file=/var/lib/k8s_audit/webhook-config.yaml" >> "$TMPFILE"
+            echo "$APISERVER_PREFIX --audit-webhook-batch-max-wait=5s" >> "$TMPFILE"
             ;;
         *"volumeMounts:"*)
-            echo "    - mountPath: /tmp/k8s_audit_config/" >> "$TMPFILE"
+            echo "    - mountPath: /var/lib/k8s_audit/" >> "$TMPFILE"
             echo "      name: data" >> "$TMPFILE"
             ;;
         *"volumes:"*)
             echo "  - hostPath:" >> "$TMPFILE"
-            echo "      path: /tmp/k8s_audit_config" >> "$TMPFILE"
+            echo "      path: /var/lib/k8s_audit" >> "$TMPFILE"
             echo "    name: data" >> "$TMPFILE"
             ;;
 

examples/k8s_audit_config/enable-k8s-audit.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+
+set -euo pipefail
+
+VARIANT=${1:-minikube}
+
+if [ $VARIANT == "minikube" ]; then
+    APISERVER_HOST=$(minikube ip)
+    SSH_KEY=$(minikube ssh-key)
+    SSH_USER=docker
+    MANIFEST="/etc/kubernetes/manifests/kube-apiserver.yaml"
+fi
+
+if [ $VARIANT == "kops" ]; then
+#    APISERVER_HOST=api.your-kops-cluster-name.com
+    SSH_KEY=~/.ssh/id_rsa
+    SSH_USER=admin
+    MANIFEST=/etc/kubernetes/manifests/kube-apiserver.manifest
+
+    if [ -z "${APISERVER_HOST+xxx}" ]; then
+	echo "***You must specify APISERVER_HOST with the name of your kops api server"
+	exit 1
+    fi
+fi
+
+echo "***Copying audit policy/webhook files to apiserver..."
+ssh -i $SSH_KEY $SSH_USER@$APISERVER_HOST "sudo mkdir -p /var/lib/k8s_audit && sudo chown $SSH_USER /var/lib/k8s_audit"
+scp -i $SSH_KEY audit-policy.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit
+scp -i $SSH_KEY webhook-config.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit
+scp -i $SSH_KEY apiserver-config.patch.sh $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit
+
+echo "***Modifying k8s apiserver config (will result in apiserver restarting)..."
+
+ssh -i $SSH_KEY $SSH_USER@$APISERVER_HOST "sudo bash /var/lib/k8s_audit/apiserver-config.patch.sh $MANIFEST $VARIANT"
+
+echo "***Done!"