Jour 3 Méthodologies DevOps

```

These slides have been built from commit: 86828ca

[shared/title.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/title.md)]
---

Jour 3 Méthodologies DevOps

**Slides[:](https://www.youtube.com/watch?v=h16zyxiwDLY) http://2020-02-enix.container.training/**
]

.debug[[shared/title.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/title.md)]
---
## Intros

- Hello! We are:

- .emoji[🐳] Jérôme Petazzoni ([@jpetazzo](https://twitter.com/jpetazzo), Enix SAS)

- .emoji[☸️] Julien Girardin ([Zempashi](https://github.com/zempashi), Enix SAS)

- The training will run from 9am to 5:30pm (with lunch and coffee breaks)

- For lunch, we'll invite you at [Chameleon, 70 Rue René Boulanger](https://goo.gl/maps/h2XjmJN5weDSUios8)

(please let us know if you'll eat on your own)

- Feel free to interrupt for questions at any time

- *Especially when you see full screen container pictures!*

.debug[[logistics.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/logistics.md)]
---
## A brief introduction

- This was initially written by [Jérôme Petazzoni](https://twitter.com/jpetazzo) to support in-person,
  instructor-led workshops and tutorials
  
- Credit is also due to [multiple contributors](https://github.com/jpetazzo/container.training/graphs/contributors) — thank you!

- You can also follow along on your own, at your own pace

- We included as much information as possible in these slides

- We recommend having a mentor to help you ...

- ... Or be comfortable spending some time reading the Kubernetes [documentation](https://kubernetes.io/docs/) ...

- ... And looking for answers on [StackOverflow](http://stackoverflow.com/questions/tagged/kubernetes) and other outlets

.debug[[k8s/intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/intro.md)]
---

## Hands on, you shall practice

- Nobody ever became a Jedi by spending their lives reading Wookiepedia

- Likewise, it will take more than merely *reading* these slides
  to make you an expert

- These slides include *tons* of exercises and examples

- They assume that you have access to a Kubernetes cluster

- If you are attending a workshop or tutorial:
 you will be given specific instructions to access your cluster

- If you are doing this on your own:
 the first chapter will give you various options to get your own cluster

.debug[[k8s/intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/intro.md)]
---
## Accessing these slides now

- We recommend that you open these slides in your browser:

http://2020-02-enix.container.training/

- Use arrows to move to next/previous slide

(up, down, left, right, page up, page down)

- Type a slide number + ENTER to go to that slide

- The slide number is also visible in the URL bar

(e.g. .../#123 for slide 123)

.debug[[shared/about-slides.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/about-slides.md)]
---

## Accessing these slides later

- Slides will remain online so you can review them later if needed

(let's say we'll keep them online at least 1 year, how about that?)

- You can download the slides using that URL:

http://2020-02-enix.container.training/slides.zip

(then open the file `3.yml.html`)

- You will to find new versions of these slides on:

https://container.training/

.debug[[shared/about-slides.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/about-slides.md)]
---

## These slides are open source

- You are welcome to use, re-use, share these slides

- These slides are written in markdown

- The sources of these slides are available in a public GitHub repository:

https://github.com/jpetazzo/container.training

- Typos? Mistakes? Questions? Feel free to hover over the bottom of the slide ...

.footnote[.emoji[👇] Try it! The source file will be shown and you can view it on GitHub and fork and edit it.]

.debug[[shared/about-slides.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/about-slides.md)]
---

## Extra details

- This slide has a little magnifying glass in the top left corner

- This magnifying glass indicates slides that provide extra details

- Feel free to skip them if:

- you are in a hurry

- you are new to this and want to avoid cognitive overload

- you want only the most essential information

- You can review these slides another time if you want, they'll be waiting for you ☺

.debug[[shared/about-slides.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/about-slides.md)]
---

## Chat room

- We've set up a chat room that we will monitor during the workshop

- Don't hesitate to use it to ask questions, or get help, or share feedback

- The chat room will also be available after the workshop

- Join the chat room: [Gitter](https://gitter.im/enix/formation-highfive-202002)

- Say hi in the chat room!

.debug[[shared/about-slides.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/about-slides.md)]
---

## Chapter 1

- [Pre-requirements](#toc-pre-requirements)

- [Our sample application](#toc-our-sample-application)

- [Development Workflow](#toc-development-workflow)

- [Volumes](#toc-volumes)

- [Namespaces](#toc-namespaces)

- [Controlling a Kubernetes cluster remotely](#toc-controlling-a-kubernetes-cluster-remotely)

- [Accessing internal services](#toc-accessing-internal-services)

- [Testing](#toc-testing)

## Chapter 2

- [Managing configuration](#toc-managing-configuration)

- [sealed-secrets](#toc-sealed-secrets)

- [Kustomize](#toc-kustomize)

- [Managing stacks with Helm](#toc-managing-stacks-with-helm)

- [Helm chart format](#toc-helm-chart-format)

- [Helm secrets](#toc-helm-secrets)

## Chapter 3

- [Shipping images with a registry](#toc-shipping-images-with-a-registry)

- [Registries](#toc-registries)

- [Automation && CI/CD](#toc-automation--cicd)

- [Rolling updates](#toc-rolling-updates)

- [Advanced Rollout](#toc-advanced-rollout)

## Chapter 4

- [Prometheus](#toc-prometheus)

- [OpenTelemetry](#toc-opentelemetry)

- [Security and kubernetes](#toc-security-and-kubernetes)

.debug[[shared/toc.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/toc.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/Container-Ship-Freighter-Navigation-Elbe-Romance-1782991.jpg)]

---

Pre-requirements

.nav[
[Previous section](#toc-)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-our-sample-application)
]

---
# Pre-requirements

- Be comfortable with the UNIX command line

- navigating directories

- editing files

- a little bit of bash-fu (environment variables, loops)

- Some Docker knowledge

- `docker run`, `docker ps`, `docker build`

- ideally, you know how to write a Dockerfile and build it
 
 (even if it's a `FROM` line and a couple of `RUN` commands)

- It's totally OK if you are not a Docker expert!

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

*Tell me and I forget.*
 
*Teach me and I remember.*
 
*Involve me and I learn.*

Misattributed to Benjamin Franklin

[(Probably inspired by Chinese Confucian philosopher Xunzi)](https://www.barrypopik.com/index.php/new_york_city/entry/tell_me_and_i_forget_teach_me_and_i_may_remember_involve_me_and_i_will_lear/)

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

## Hands-on sections

- The whole workshop is hands-on

- We are going to build, ship, and run containers!

- You are invited to reproduce all the demos

- All hands-on sections are clearly identified, like the gray rectangle below

- This is the stuff you're supposed to do!

- Go to http://2020-02-enix.container.training/ to view these slides

- Join the chat room: [Gitter](https://gitter.im/enix/formation-highfive-202002)

]

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

## Where are we going to run our containers?

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

![You get a cluster](images/you-get-a-cluster.jpg)

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

## You get a cluster of cloud VMs

- Each person gets a private cluster of cloud VMs (not shared with anybody else)

- They'll remain up for the duration of the workshop

- You should have a little card with login+password+IP addresses

- You can automatically SSH from one VM to another

- The nodes have aliases: `node1`, `node2`, etc.

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

## Why don't we run containers locally?

- Installing this stuff can be hard on some machines

(32 bits CPU or OS... Laptops without administrator access... etc.)

- *"The whole team downloaded all these container images from the WiFi!
 ... and it went great!"* (Literally no-one ever)

- All you need is a computer (or even a phone or tablet!), with:

- an internet connection

- a web browser

- an SSH client

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

## SSH clients

- On Linux, OS X, FreeBSD... you are probably all set

- On Windows, get one of these:

- [putty](http://www.putty.org/)
  - Microsoft [Win32 OpenSSH](https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH)
  - [Git BASH](https://git-for-windows.github.io/)
  - [MobaXterm](http://mobaxterm.mobatek.net/)

- On Android, [JuiceSSH](https://juicessh.com/)
  ([Play Store](https://play.google.com/store/apps/details?id=com.sonelli.juicessh))
  works pretty well

- Nice-to-have: [Mosh](https://mosh.org/) instead of SSH, if your internet connection tends to lose packets

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

## What is this Mosh thing?

*You don't have to use Mosh or even know about it to follow along.
 
We're just telling you about it because some of us think it's cool!*

- Mosh is "the mobile shell"

- It is essentially SSH over UDP, with roaming features

- It retransmits packets quickly, so it works great even on lossy connections

(Like hotel or conference WiFi)

- It has intelligent local echo, so it works great even in high-latency connections

(Like hotel or conference WiFi)

- It supports transparent roaming when your client IP address changes

(Like when you hop from hotel to conference WiFi)

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---

## Using Mosh

- To install it: `(apt|yum|brew) install mosh`

- It has been pre-installed on the VMs that we are using

- To connect to a remote machine: `mosh user@host`

(It is going to establish an SSH connection, then hand off to UDP)

- It requires UDP ports to be open

(By default, it uses a UDP port between 60000 and 61000)

.debug[[shared/prereqs.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/prereqs.md)]
---
class: in-person

## Connecting to our lab environment

- Log into the first VM (`node1`) with your SSH client:
  ```bash
  ssh `user`@`A.B.C.D`
  ```

(Replace `user` and `A.B.C.D` with the user and IP address provided to you)

<!--
```bash
for N in $(awk '/\Wnode/{print $2}' /etc/hosts); do
 ssh -o StrictHostKeyChecking=no $N true
done
```

```bash
### FIXME find a way to reset the cluster, maybe?
```
-->

]

You should see a prompt looking like this:
```
[A.B.C.D] (...) user@node1 ~
$
```
If anything goes wrong — ask for help!

.debug[[shared/connecting.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/connecting.md)]
---

## Doing or re-doing the workshop on your own?

- Use something like
  [Play-With-Docker](http://play-with-docker.com/) or
  [Play-With-Kubernetes](https://training.play-with-kubernetes.com/)

Zero setup effort; but environment are short-lived and
  might have limited resources

- Create your own cluster (local or cloud VMs)

Small setup effort; small cost; flexible environments

- Create a bunch of clusters for you and your friends
    ([instructions](https://github.com/jpetazzo/container.training/tree/master/prepare-vms))

Bigger setup effort; ideal for group training

.debug[[shared/connecting.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/connecting.md)]
---

## For a consistent Kubernetes experience ...

- If you are using your own Kubernetes cluster, you can use [shpod](https://github.com/jpetazzo/shpod)

- `shpod` provides a shell running in a pod on your own cluster

- It comes with many tools pre-installed (helm, stern...)

- These tools are used in many exercises in these slides

- `shpod` also gives you completion and a fancy prompt

.debug[[shared/connecting.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/connecting.md)]
---

## Get your own Docker nodes

- If you already have some Docker nodes: great!

- If not: let's get some thanks to Play-With-Docker

- Go to http://www.play-with-docker.com/

- Log in

- Create your first node

]

You will need a Docker ID to use Play-With-Docker.

(Creating a Docker ID is free.)

.debug[[shared/connecting.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/connecting.md)]
---

## We will (mostly) interact with node1 only

*These remarks apply only when using multiple nodes, of course.*

- Unless instructed, **all commands must be run from the first VM, `node1`**

- We will only check out/copy the code on `node1`

- During normal operations, we do not need access to the other nodes

- If we had to troubleshoot issues, we would use a combination of:

- SSH (to access system logs, daemon status...)

- Docker API (to check running containers and container engine status)

.debug[[shared/connecting.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/connecting.md)]
---

## Terminals

Once in a while, the instructions will say:
 "Open a new terminal."

There are multiple ways to do this:

- create a new window or tab on your machine, and SSH into the VM;

- use screen or tmux on the VM and open a new window from there.

You are welcome to use the method that you feel the most comfortable with.

.debug[[shared/connecting.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/connecting.md)]
---

## Tmux cheatsheet

[Tmux](https://en.wikipedia.org/wiki/Tmux) is a terminal multiplexer like `screen`.

*You don't have to use it or even know about it to follow along.
 
But some of us like to use it to switch between terminals.
 
It has been preinstalled on your workshop nodes.*

- Ctrl-b c → creates a new window
- Ctrl-b n → go to next window
- Ctrl-b p → go to previous window
- Ctrl-b " → split window top/bottom
- Ctrl-b % → split window left/right
- Ctrl-b Alt-1 → rearrange windows in columns
- Ctrl-b Alt-2 → rearrange windows in rows
- Ctrl-b arrows → navigate to other windows
- Ctrl-b d → detach session
- tmux attach → reattach to session

.debug[[shared/connecting.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/connecting.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/ShippingContainerSFBay.jpg)]

---

Our sample application

.nav[
[Previous section](#toc-pre-requirements)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-development-workflow)
]

---
# Our sample application

- We will clone the GitHub repository onto our `node1`

- The repository also contains scripts and tools that we will use through the workshop

- Clone the repository on `node1`:
  ```bash
  git clone https://github.com/jpetazzo/container.training
  ```

]

(You can also fork the repository on GitHub and clone your fork if you prefer that.)

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Downloading and running the application

Let's start this before we look around, as downloading will take a little time...

- Go to the `dockercoins` directory, in the cloned repo:
  ```bash
  cd ~/container.training/dockercoins
  ```

- Use Compose to build and run all containers:
  ```bash
  docker-compose up
  ```

]

Compose tells Docker to build all container images (pulling
the corresponding base images), then starts all containers,
and displays aggregated logs.

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## What's this application?

- It is a DockerCoin miner! .emoji[💰🐳📦🚢]

- No, you can't buy coffee with DockerCoins

- How DockerCoins works:

- generate a few random bytes

- hash these bytes

- increment a counter (to keep track of speed)

- repeat forever!

- DockerCoins is *not* a cryptocurrency

(the only common points are "randomness," "hashing," and "coins" in the name)

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## DockerCoins in the microservices era

- DockerCoins is made of 5 services:

- `rng` = web service generating random bytes

- `hasher` = web service computing hash of POSTed data

- `worker` = background process calling `rng` and `hasher`

- `webui` = web interface to watch progress

- `redis` = data store (holds a counter updated by `worker`)

- These 5 services are visible in the application's Compose file,
  [docker-compose.yml](
  https://github.com/jpetazzo/container.training/blob/master/dockercoins/docker-compose.yml)

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## How DockerCoins works

- `worker` invokes web service `rng` to generate random bytes

- `worker` invokes web service `hasher` to hash these bytes

- `worker` does this in an infinite loop

- every second, `worker` updates `redis` to indicate how many loops were done

- `webui` queries `redis`, and computes and exposes "hashing speed" in our browser

*(See diagram on next slide!)*

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

![Diagram showing the 5 containers of the applications](images/dockercoins-diagram.svg)

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Service discovery in container-land

How does each service find out the address of the other ones?

- We do not hard-code IP addresses in the code

- We do not hard-code FQDNs in the code, either

- We just connect to a service name, and container-magic does the rest

(And by container-magic, we mean "a crafty, dynamic, embedded DNS server")

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Example in `worker/worker.py`

```python
redis = Redis("`redis`")

def get_random_bytes():
    r = requests.get("http://`rng`/32")
    return r.content

def hash_bytes(data):
    r = requests.post("http://`hasher`/",
                      data=data,
                      headers={"Content-Type": "application/octet-stream"})
```

(Full source code available [here](
https://github.com/jpetazzo/container.training/blob/8279a3bce9398f7c1a53bdd95187c53eda4e6435/dockercoins/worker/worker.py#L17
))

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Links, naming, and service discovery

- Containers can have network aliases (resolvable through DNS)

- Compose file version 2+ makes each container reachable through its service name

- Compose file version 1 required "links" sections to accomplish this

- Network aliases are automatically namespaced

- you can have multiple apps declaring and using a service named `database`

- containers in the blue app will resolve `database` to the IP of the blue database

- containers in the green app will resolve `database` to the IP of the green database

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Show me the code!

- You can check the GitHub repository with all the materials of this workshop:
 https://github.com/jpetazzo/container.training

- The application is in the [dockercoins](
  https://github.com/jpetazzo/container.training/tree/master/dockercoins)
  subdirectory

- The Compose file ([docker-compose.yml](
  https://github.com/jpetazzo/container.training/blob/master/dockercoins/docker-compose.yml))
  lists all 5 services

- `redis` is using an official image from the Docker Hub

- `hasher`, `rng`, `worker`, `webui` are each built from a Dockerfile

- Each service's Dockerfile and source code is in its own directory

(`hasher` is in the [hasher](https://github.com/jpetazzo/container.training/blob/master/dockercoins/hasher/) directory,
  `rng` is in the [rng](https://github.com/jpetazzo/container.training/blob/master/dockercoins/rng/)
  directory, etc.)

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Compose file format version

*This is relevant only if you have used Compose before 2016...*

- Compose 1.6 introduced support for a new Compose file format (aka "v2")

- Services are no longer at the top level, but under a `services` section

- There has to be a `version` key at the top level, with value `"2"` (as a string, not an integer)

- Containers are placed on a dedicated network, making links unnecessary

- There are other minor differences, but upgrade is easy and straightforward

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Our application at work

- On the left-hand side, the "rainbow strip" shows the container names

- On the right-hand side, we see the output of our containers

- We can see the `worker` service making requests to `rng` and `hasher`

- For `rng` and `hasher`, we see HTTP access logs

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Connecting to the web UI

- "Logs are exciting and fun!" (No-one, ever)

- The `webui` container exposes a web dashboard; let's view it

- With a web browser, connect to `node1` on port 8000

- Remember: the `nodeX` aliases are valid only on the nodes themselves

- In your browser, you need to enter the IP address of your node

]

A drawing area should show up, and after a few seconds, a blue
graph will appear.

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## If the graph doesn't load

If you just see a `Page not found` error, it might be because your
Docker Engine is running on a different machine. This can be the case if:

- you are using the Docker Toolbox

- you are using a VM (local or remote) created with Docker Machine

- you are controlling a remote Docker Engine

When you run DockerCoins in development mode, the web UI static files
are mapped to the container using a volume. Alas, volumes can only
work on a local environment, or when using Docker Desktop for Mac or Windows.

How to fix this?

Stop the app with `^C`, edit `dockercoins.yml`, comment out the `volumes` section, and try again.

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Why does the speed seem irregular?

- It *looks like* the speed is approximately 4 hashes/second

- Or more precisely: 4 hashes/second, with regular dips down to zero

- Why?

- The app actually has a constant, steady speed: 3.33 hashes/second
 
 (which corresponds to 1 hash every 0.3 seconds, for *reasons*)

- Yes, and?

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## The reason why this graph is *not awesome*

- The worker doesn't update the counter after every loop, but up to once per second

- The speed is computed by the browser, checking the counter about once per second

- Between two consecutive updates, the counter will increase either by 4, or by 0

- The perceived speed will therefore be 4 - 4 - 4 - 0 - 4 - 4 - 0 etc.

- What can we conclude from this?

- "I'm clearly incapable of writing good frontend code!" 😀 — Jérôme

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---

## Stopping the application

- If we interrupt Compose (with `^C`), it will politely ask the Docker Engine to stop the app

- The Docker Engine will send a `TERM` signal to the containers

- If the containers do not exit in a timely manner, the Engine sends a `KILL` signal

- Stop the application by hitting `^C`

]

Some containers exit immediately, others take longer.

The containers that do not handle `SIGTERM` end up being killed after a 10s timeout. If we are very impatient, we can hit `^C` a second time!

.debug[[shared/sampleapp.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/sampleapp.md)]
---
## Software development

From years, decades, (centuries !), software development has followed the same principles:

- Development

- Testing

- Packaging

- Shipping

- Deployment

We will see how this map to Kubernetes world.

.debug[[k8s/software-dev-banalities.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/software-dev-banalities.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/aerial-view-of-containers.jpg)]

---

Development Workflow

.nav[
[Previous section](#toc-our-sample-application)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-volumes)
]

---
# Development Workflow

In this section we will see how to set up a local development workflow.

We will list multiple options.

Keep in mind that we don't have to use *all* these tools!

It's up to the developer to find what best suits them.

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## What does it mean to develop on Kubernetes ?

In theory, the generic workflow is:

1. Make changes to our code or edit a Dockerfile

2. Build a new Docker image with a new tag

3. Push that Docker image to a registry

4. Update the YAML or templates referencing that Docker image
 (e.g. of the corresponding Deployment, StatefulSet, Job ...)

5. Apply the YAML or templates

6. Are we satisfied with the result?
 No → go back to step 1 (or step 4 if the image is OK)
 Yes → commit and push our changes to source control

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## A few quirks

In practice, there are some details that make this workflow more complex.

- We need a Docker container registry to store our images
 
 (for Open Source projects, a free Docker Hub account works fine)

- We need to set image tags properly, hopefully automatically

- If we decide to use a fixed tag (like `:latest`) instead:

- we need to specify `imagePullPolicy=Always` to force image pull

- we need to trigger a rollout when we want to deploy a new image
 (with `kubectl rollout restart` or by killing the running pods)

- We need a fast internet connection to push the images

- We need to regularly clean up the registry to avoid accumulating old images

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## When developing locally

- If we work with a local cluster, pushes and pulls are much faster

- Even better, with a one-node cluster, most of these problems disappear

- If we build and run the images on the same node, ...

- we don't need to push images

- we don't need a fast internet connection

- we don't need a registry

- we can use bind mounts to edit code locally and make changes available immediately in running containers

- This means that it is much simpler to deploy to local development environment (like Minikube, Docker Desktop ...) than to a "real" cluster

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## Minikube

- Start a VM with the hypervisor of your choice: VirtualBox, kvm, Hyper-V ...

- Well supported by the Kubernetes community

- Lot of addons

- Easy cleanup: delete the VM with `minikube delete`

- Bind mounts depend on the underlying hypervisor

(they may require additionnal setup)

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## Docker Desktop

- Available for Mac and Windows

- Start a VM with the appropriate hypervisor (even better!)

- Bind mounts work out of the box

```yaml
volumes:
- name: repo_dir
  hostPath:
    path: /C/Users/Enix/my_code_repository
```

- Ingress and other addons need to be installed manually

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## Kind

- Kubernetes-in-Docker

- Uses Docker-in-Docker to run Kubernetes
 
 (technically, it's more like Containerd-in-Docker)

- We don't get a real Docker Engine (and cannot build Dockerfiles)

- Single-node by default, but multi-node clusters are possible

- Very convenient to test Kubernetes deployments when only Docker is available
 
 (e.g. on public CI services like Travis, Circle, GitHub Actions ...)

- Bind mounts require extra configuration

- Extra configuration for a couple of addons, totally custom for other

- Doesn't work with BTRFS (sorry BTRFS users😢)

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## microk8s

- Distribution of Kubernetes using Snap

(Snap is a container-like method to install software)

- Available on Ubuntu and derivatives

- Bind mounts work natively (but require extra setup if we run in a VM)

- Big list of addons; easy to install

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## Proper tooling

The simple workflow seems to be:

- set up a one-node cluster with one of the methods mentioned previously,

- find the remote Docker endpoint,

- configure the `DOCKER_HOST` variable to use that endpoint,

- follow the previous 7-step workflow.

Can we do better?

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

## Helpers

- Skaffold (https://skaffold.dev/):
    - build with docker, kaniko, google builder
    - install with pure yaml manifests, kustomize, helm

- Tilt (https://tilt.dev/)
    - Tiltfile is programmatic format (python ?)
    - Primitive for building with docker
    - Primitive for deploying with pure yaml manifests, kustomize, helm

- Garden (https://garden.io/)

- Forge (https://forge.sh/)

.debug[[k8s/on-desktop.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/on-desktop.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/blue-containers.jpg)]

---

Volumes

.nav[
[Previous section](#toc-development-workflow)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-namespaces)
]

---
# Volumes

- Volumes are special directories that are mounted in containers

- Volumes can have many different purposes:

- share files and directories between containers running on the same machine

- share files and directories between containers and their host

- centralize configuration information in Kubernetes and expose it to containers

- manage credentials and secrets and expose them securely to containers

- store persistent data for stateful services

- access storage systems (like Ceph, EBS, NFS, Portworx, and many others)

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Kubernetes volumes vs. Docker volumes

- Kubernetes and Docker volumes are very similar

(the [Kubernetes documentation](https://kubernetes.io/docs/concepts/storage/volumes/) says otherwise ...
 
 but it refers to Docker 1.7, which was released in 2015!)

- Docker volumes allow us to share data between containers running on the same host

- Kubernetes volumes allow us to share data between containers in the same pod

- Both Docker and Kubernetes volumes enable access to storage systems

- Kubernetes volumes are also used to expose configuration and secrets

- Docker has specific concepts for configuration and secrets
 
 (but under the hood, the technical implementation is similar)

- If you're not familiar with Docker volumes, you can safely ignore this slide!

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Volumes ≠ Persistent Volumes

- Volumes and Persistent Volumes are related, but very different!

- *Volumes*:

- appear in Pod specifications (we'll see that in a few slides)

- do not exist as API resources (**cannot** do `kubectl get volumes`)

- *Persistent Volumes*:

- are API resources (**can** do `kubectl get persistentvolumes`)

- correspond to concrete volumes (e.g. on a SAN, EBS, etc.)

- cannot be associated with a Pod directly; but through a Persistent Volume Claim

- won't be discussed further in this section

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Adding a volume to a Pod

- We will start with the simplest Pod manifest we can find

- We will add a volume to that Pod manifest

- We will mount that volume in a container in the Pod

- By default, this volume will be an `emptyDir`

(an empty directory)

- It will "shadow" the directory where it's mounted

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Our basic Pod

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-without-volume
spec:
  containers:
  - name: nginx
    image: nginx
```

This is a MVP! (Minimum Viable Pod😉)

It runs a single NGINX container.

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Trying the basic pod

- Create the Pod:
  ```bash
  kubectl create -f ~/container.training/k8s/nginx-1-without-volume.yaml
  ```

- Get its IP address:
  ```bash
  IPADDR=$(kubectl get pod nginx-without-volume -o jsonpath={.status.podIP})
  ```

- Send a request with curl:
  ```bash
  curl $IPADDR
  ```

]

(We should see the "Welcome to NGINX" page.)

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Adding a volume

- We need to add the volume in two places:

- at the Pod level (to declare the volume)

- at the container level (to mount the volume)

- We will declare a volume named `www`

- No type is specified, so it will default to `emptyDir`

(as the name implies, it will be initialized as an empty directory at pod creation)

- In that pod, there is also a container named `nginx`

- That container mounts the volume `www` to path `/usr/share/nginx/html/`

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## The Pod with a volume

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-with-volume
spec:
  volumes:
  - name: www
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: www
      mountPath: /usr/share/nginx/html/
```

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Trying the Pod with a volume

- Create the Pod:
  ```bash
  kubectl create -f ~/container.training/k8s/nginx-2-with-volume.yaml
  ```

- Get its IP address:
  ```bash
  IPADDR=$(kubectl get pod nginx-with-volume -o jsonpath={.status.podIP})
  ```

- Send a request with curl:
  ```bash
  curl $IPADDR
  ```

]

(We should now see a "403 Forbidden" error page.)

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Populating the volume with another container

- Let's add another container to the Pod

- Let's mount the volume in *both* containers

- That container will populate the volume with static files

- NGINX will then serve these static files

- To populate the volume, we will clone the Spoon-Knife repository

- this repository is https://github.com/octocat/Spoon-Knife

- it's very popular (more than 100K stars!)

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Sharing a volume between two containers

.small[
```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-with-git
spec:
  volumes:
  - name: www
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: www
      mountPath: /usr/share/nginx/html/
  - name: git
    image: alpine
    command: [ "sh", "-c", "apk add git && git clone https://github.com/octocat/Spoon-Knife /www" ]
    volumeMounts:
    - name: www
      mountPath: /www/
  restartPolicy: OnFailure
```
]

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Sharing a volume, explained

- We added another container to the pod

- That container mounts the `www` volume on a different path (`/www`)

- It uses the `alpine` image

- When started, it installs `git` and clones the `octocat/Spoon-Knife` repository

(that repository contains a tiny HTML website)

- As a result, NGINX now serves this website

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Trying the shared volume

- This one will be time-sensitive!

- We need to catch the Pod IP address *as soon as it's created*

- Then send a request to it *as fast as possible*

- Watch the pods (so that we can catch the Pod IP address)
  ```bash
  kubectl get pods -o wide --watch
  ```

]

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Shared volume in action

- Create the pod:
  ```bash
  kubectl create -f ~/container.training/k8s/nginx-3-with-git.yaml
  ```

- As soon as we see its IP address, access it:
  ```bash
  curl `$IP`
  ```

- A few seconds later, the state of the pod will change; access it again:
  ```bash
  curl `$IP`
  ```

]

The first time, we should see "403 Forbidden".

The second time, we should see the HTML file from the Spoon-Knife repository.

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Explanations

- Both containers are started at the same time

- NGINX starts very quickly

(it can serve requests immediately)

- But at this point, the volume is empty

(NGINX serves "403 Forbidden")

- The other containers installs git and clones the repository

(this takes a bit longer)

- When the other container is done, the volume holds the repository

(NGINX serves the HTML file)

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## The devil is in the details

- The default `restartPolicy` is `Always`

- This would cause our `git` container to run again ... and again ... and again

(with an exponential back-off delay, as explained [in the documentation](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy))

- That's why we specified `restartPolicy: OnFailure`

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Inconsistencies

- There is a short period of time during which the website is not available

(because the `git` container hasn't done its job yet)

- With a bigger website, we could get inconsistent results

(where only a part of the content is ready)

- In real applications, this could cause incorrect results

- How can we avoid that?

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Init Containers

- We can define containers that should execute *before* the main ones

- They will be executed in order

(instead of in parallel)

- They must all succeed before the main containers are started

- This is *exactly* what we need here!

- Let's see one in action

.footnote[See [Init Containers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) documentation for all the details.]

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Defining Init Containers

.small[
```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-with-init
spec:
  volumes:
  - name: www
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: www
      mountPath: /usr/share/nginx/html/
  initContainers:
  - name: git
    image: alpine
    command: [ "sh", "-c", "apk add --no-cache git && git clone https://github.com/octocat/Spoon-Knife /www" ]
    volumeMounts:
    - name: www
      mountPath: /www/
```
]

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Trying the init container

- Repeat the same operation as earlier

(try to send HTTP requests as soon as the pod comes up)

]

- This time, instead of "403 Forbidden" we get a "connection refused"

- NGINX doesn't start until the git container has done its job

- We never get inconsistent results

(a "half-ready" container)

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Other uses of init containers

- Load content

- Generate configuration (or certificates)

- Database migrations

- Waiting for other services to be up

(to avoid flurry of connection errors in main container)

- etc.

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

## Volume lifecycle

- The lifecycle of a volume is linked to the pod's lifecycle

- This means that a volume is created when the pod is created

- This is mostly relevant for `emptyDir` volumes

(other volumes, like remote storage, are not "created" but rather "attached" )

- A volume survives across container restarts

- A volume is destroyed (or, for remote storage, detached) when the pod is destroyed

.debug[[k8s/volumes.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/volumes.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/chinook-helicopter-container.jpg)]

---

Namespaces

.nav[
[Previous section](#toc-volumes)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-controlling-a-kubernetes-cluster-remotely)
]

---
# Namespaces

- We would like to deploy another copy of DockerCoins on our cluster

- We could rename all our deployments and services:

hasher → hasher2, redis → redis2, rng → rng2, etc.

- That would require updating the code

- There has to be a better way!

- As hinted by the title of this section, we will use *namespaces*

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Identifying a resource

- We cannot have two resources with the same name

(or can we...?)

- We cannot have two resources *of the same kind* with the same name

(but it's OK to have an `rng` service, an `rng` deployment, and an `rng` daemon set)

- We cannot have two resources of the same kind with the same name *in the same namespace*

(but it's OK to have e.g. two `rng` services in different namespaces)

- Except for resources that exist at the *cluster scope*

(these do not belong to a namespace)

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Uniquely identifying a resource

- For *namespaced* resources:

the tuple *(kind, name, namespace)* needs to be unique

- For resources at the *cluster scope*:

the tuple *(kind, name)* needs to be unique

- List resource types again, and check the NAMESPACED column:
  ```bash
  kubectl api-resources
  ```

]

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Pre-existing namespaces

- If we deploy a cluster with `kubeadm`, we have three or four namespaces:

- `default` (for our applications)

- `kube-system` (for the control plane)

- `kube-public` (contains one ConfigMap for cluster discovery)

- `kube-node-lease` (in Kubernetes 1.14 and later; contains Lease objects)

- If we deploy differently, we may have different namespaces

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Creating namespaces

- Let's see two identical methods to create a namespace

- We can use `kubectl create namespace`:
  ```bash
  kubectl create namespace blue
  ```

- Or we can construct a very minimal YAML snippet:
 ```bash
	kubectl apply -f- <<EOF
	apiVersion: v1
	kind: Namespace
	metadata:
	 name: blue
	EOF
 ```

]

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Using namespaces

- We can pass a `-n` or `--namespace` flag to most `kubectl` commands:
  ```bash
  kubectl -n blue get svc
  ```

- We can also change our current *context*

- A context is a *(user, cluster, namespace)* tuple

- We can manipulate contexts with the `kubectl config` command

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Viewing existing contexts

- On our training environments, at this point, there should be only one context

- View existing contexts to see the cluster name and the current user:
  ```bash
  kubectl config get-contexts
  ```

]

- The current context (the only one!) is tagged with a `*`

- What are NAME, CLUSTER, AUTHINFO, and NAMESPACE?

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## What's in a context

- NAME is an arbitrary string to identify the context

- CLUSTER is a reference to a cluster

(i.e. API endpoint URL, and optional certificate)

- AUTHINFO is a reference to the authentication information to use

(i.e. a TLS client certificate, token, or otherwise)

- NAMESPACE is the namespace

(empty string = `default`)

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Switching contexts

- We want to use a different namespace

- Solution 1: update the current context

*This is appropriate if we need to change just one thing (e.g. namespace or authentication).*

- Solution 2: create a new context and switch to it

*This is appropriate if we need to change multiple things and switch back and forth.*

- Let's go with solution 1!

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Updating a context

- This is done through `kubectl config set-context`

- We can update a context by passing its name, or the current context with `--current`

- Update the current context to use the `blue` namespace:
  ```bash
  kubectl config set-context --current --namespace=blue
  ```

- Check the result:
  ```bash
  kubectl config get-contexts
  ```

]

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Using our new namespace

- Let's check that we are in our new namespace, then deploy a new copy of Dockercoins

- Verify that the new context is empty:
  ```bash
  kubectl get all
  ```

]

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Deploying DockerCoins with YAML files

- The GitHub repository `jpetazzo/kubercoins` contains everything we need!

- Clone the kubercoins repository:
  ```bash
  cd ~
  git clone https://github.com/jpetazzo/kubercoins
  ```

- Create all the DockerCoins resources:
  ```bash
  kubectl create -f kubercoins
  ```

]

If the argument behind `-f` is a directory, all the files in that directory are processed.

The subdirectories are *not* processed, unless we also add the `-R` flag.

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Viewing the deployed app

- Let's see if this worked correctly!

- Retrieve the port number allocated to the `webui` service:
  ```bash
  kubectl get svc webui
  ```

- Point our browser to http://X.X.X.X:3xxxx

]

If the graph shows up but stays at zero, give it a minute or two!

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Namespaces and isolation

- Namespaces *do not* provide isolation

- A pod in the `green` namespace can communicate with a pod in the `blue` namespace

- A pod in the `default` namespace can communicate with a pod in the `kube-system` namespace

- CoreDNS uses a different subdomain for each namespace

- Example: from any pod in the cluster, you can connect to the Kubernetes API with:

`https://kubernetes.default.svc.cluster.local:443/`

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Isolating pods

- Actual isolation is implemented with *network policies*

- Network policies are resources (like deployments, services, namespaces...)

- Network policies specify which flows are allowed:

- between pods

- from pods to the outside world

- and vice-versa

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Switch back to the default namespace

- Let's make sure that we don't run future exercises in the `blue` namespace

- Switch back to the original context:
  ```bash
  kubectl config set-context --current --namespace=
  ```

]

Note: we could have used `--namespace=default` for the same result.

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Switching namespaces more easily

- We can also use a little helper tool called `kubens`:

```bash
  # Switch to namespace foo
  kubens foo
  # Switch back to the previous namespace
  kubens -
  ```

- On our clusters, `kubens` is called `kns` instead

(so that it's even fewer keystrokes to switch namespaces)

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

##  `kubens` and `kubectx`

- With `kubens`, we can switch quickly between namespaces

- With `kubectx`, we can switch quickly between contexts

- Both tools are simple shell scripts available from https://github.com/ahmetb/kubectx

- On our clusters, they are installed as `kns` and `kctx`

(for brevity and to avoid completion clashes between `kubectx` and `kubectl`)

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## `kube-ps1`

- It's easy to lose track of our current cluster / context / namespace

- `kube-ps1` makes it easy to track these, by showing them in our shell prompt

- It is installed on our training clusters, and when using [shpod](https://github.com/jpetazzo/shpod)

- It gives us a prompt looking like this one:
  ```
  [123.45.67.89] `(kubernetes-admin@kubernetes:default)` docker@node1 ~
  ```
  (The highlighted part is `context:namespace`, managed by `kube-ps1`)

- Highly recommended if you work across multiple contexts or namespaces!

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

## Installing `kube-ps1`

- It's a simple shell script available from https://github.com/jonmosco/kube-ps1

- It needs to be [installed in our profile/rc files](https://github.com/jonmosco/kube-ps1#installing)

(instructions differ depending on platform, shell, etc.)

- Once installed, it defines aliases called `kube_ps1`, `kubeon`, `kubeoff`

(to selectively enable/disable it when needed)

- Pro-tip: install it on your machine during the next break!

.debug[[k8s/namespaces.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/namespaces.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/container-cranes.jpg)]

---

Controlling a Kubernetes cluster remotely

.nav[
[Previous section](#toc-namespaces)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-accessing-internal-services)
]

---
# Controlling a Kubernetes cluster remotely

- `kubectl` can be used either on cluster instances or outside the cluster

- Here, we are going to use `kubectl` from our local machine

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Requirements

- `kubectl` is officially available on Linux, macOS, Windows

(and unofficially anywhere we can build and run Go binaries)

- You may skip these exercises if you are following along from:

- a tablet or phone

- a web-based terminal

- an environment where you can't install and run new binaries

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Installing `kubectl`

- If you already have `kubectl` on your local machine, you can skip this

- Download the `kubectl` binary from one of these links:

[Linux](https://storage.googleapis.com/kubernetes-release/release/v1.15.3/bin/linux/amd64/kubectl)
  |
  [macOS](https://storage.googleapis.com/kubernetes-release/release/v1.15.3/bin/darwin/amd64/kubectl)
  |
  [Windows](https://storage.googleapis.com/kubernetes-release/release/v1.15.3/bin/windows/amd64/kubectl.exe)

- On Linux and macOS, make the binary executable with `chmod +x kubectl`

(And remember to run it with `./kubectl` or move it to your `$PATH`)

]

Note: if you are following along with a different platform (e.g. Linux on an architecture different from amd64, or with a phone or tablet), installing `kubectl` might be more complicated (or even impossible) so feel free to skip this section.

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Testing `kubectl`

- Check that `kubectl` works correctly

(before even trying to connect to a remote cluster!)

- Ask `kubectl` to show its version number:
  ```bash
  kubectl version --client
  ```

]

The output should look like this:
```
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0",
GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean",
BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc",
Platform:"darwin/amd64"}
```

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Preserving the existing `~/.kube/config`

- If you already have a `~/.kube/config` file, rename it

(we are going to overwrite it in the following slides!)

- If you never used `kubectl` on your machine before: nothing to do!

- Make a copy of `~/.kube/config`; if you are using macOS or Linux, you can do:
  ```bash
  cp ~/.kube/config ~/.kube/config.before.training
  ```

- If you are using Windows, you will need to adapt this command

]

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Copying the configuration file from `node1`

- The `~/.kube/config` file that is on `node1` contains all the credentials we need

- Let's copy it over!

- Copy the file from `node1`; if you are using macOS or Linux, you can do:
  ```
  scp `USER`@`X.X.X.X`:.kube/config ~/.kube/config
  # Make sure to replace X.X.X.X with the IP address of node1,
  # and USER with the user name used to log into node1!
  ```

- If you are using Windows, adapt these instructions to your SSH client

]

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Updating the server address

- There is a good chance that we need to update the server address

- To know if it is necessary, run `kubectl config view`

- Look for the `server:` address:

- if it matches the public IP address of `node1`, you're good!

- if it is anything else (especially a private IP address), update it!

- To update the server address, run:
  ```bash
  kubectl config set-cluster kubernetes --server=https://`X.X.X.X`:6443
  # Make sure to replace X.X.X.X with the IP address of node1!
  ```

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## What if we get a certificate error?

- Generally, the Kubernetes API uses a certificate that is valid for:

- `kubernetes`
  - `kubernetes.default`
  - `kubernetes.default.svc`
  - `kubernetes.default.svc.cluster.local`
  - the ClusterIP address of the `kubernetes` service
  - the hostname of the node hosting the control plane (e.g. `node1`)
  - the IP address of the node hosting the control plane

- On most clouds, the IP address of the node is an internal IP address

- ... And we are going to connect over the external IP address

- ... And that external IP address was not used when creating the certificate!

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Working around the certificate error

- We need to tell `kubectl` to skip TLS verification

(only do this with testing clusters, never in production!)

- The following command will do the trick:
  ```bash
  kubectl config set-cluster kubernetes --insecure-skip-tls-verify
  ```

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

## Checking that we can connect to the cluster

- We can now run a couple of trivial commands to check that all is well

- Check the versions of the local client and remote server:
  ```bash
  kubectl version
  ```

- View the nodes of the cluster:
  ```bash
  kubectl get nodes
  ```

]

We can now utilize the cluster exactly as if we're logged into a node, except that it's remote.

.debug[[k8s/localkubeconfig.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/localkubeconfig.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/container-housing.jpg)]

---

Accessing internal services

.nav[
[Previous section](#toc-controlling-a-kubernetes-cluster-remotely)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-testing)
]

---
# Accessing internal services

- When we are logged in on a cluster node, we can access internal services

(by virtue of the Kubernetes network model: all nodes can reach all pods and services)

- When we are accessing a remote cluster, things are different

(generally, our local machine won't have access to the cluster's internal subnet)

- How can we temporarily access a service without exposing it to everyone?

- `kubectl proxy`: gives us access to the API, which includes a proxy for HTTP resources

- `kubectl port-forward`: allows forwarding of TCP ports to arbitrary pods, services, ...

.debug[[k8s/accessinternal.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/accessinternal.md)]
---

## Suspension of disbelief

The exercises in this section assume that we have set up `kubectl` on our
local machine in order to access a remote cluster.

We will therefore show how to access services and pods of the remote cluster,
from our local machine.

You can also run these exercises directly on the cluster (if you haven't
installed and set up `kubectl` locally).

Running commands locally will be less useful
(since you could access services and pods directly),
but keep in mind that these commands will work anywhere as long as you have
installed and set up `kubectl` to communicate with your cluster.

.debug[[k8s/accessinternal.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/accessinternal.md)]
---

## `kubectl proxy` in theory

- Running `kubectl proxy` gives us access to the entire Kubernetes API

- The API includes routes to proxy HTTP traffic

- These routes look like the following:

`/api/v1/namespaces/<namespace>/services/<service>/proxy`

- We just add the URI to the end of the request, for instance:

`/api/v1/namespaces/<namespace>/services/<service>/proxy/index.html`

- We can access `services` and `pods` this way

.debug[[k8s/accessinternal.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/accessinternal.md)]
---

## `kubectl proxy` in practice

- Let's access the `webui` service through `kubectl proxy`

- Run an API proxy in the background:
  ```bash
  kubectl proxy &
  ```

- Access the `webui` service:
  ```bash
  curl localhost:8001/api/v1/namespaces/default/services/webui/proxy/index.html
  ```

- Terminate the proxy:
  ```bash
  kill %1
  ```

]

.debug[[k8s/accessinternal.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/accessinternal.md)]
---

## `kubectl port-forward` in theory

- What if we want to access a TCP service?

- We can use `kubectl port-forward` instead

- It will create a TCP relay to forward connections to a specific port

(of a pod, service, deployment...)

- The syntax is:

`kubectl port-forward service/name_of_service local_port:remote_port`

- If only one port number is specified, it is used for both local and remote ports

.debug[[k8s/accessinternal.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/accessinternal.md)]
---

## `kubectl port-forward` in practice

- Let's access our remote Redis server

- Forward connections from local port 10000 to remote port 6379:
  ```bash
  kubectl port-forward svc/redis 10000:6379 &
  ```

- Connect to the Redis server:
  ```bash
  telnet localhost 10000
  ```

- Issue a few commands, e.g. `INFO server` then `QUIT`

- Terminate the port forwarder:
  ```bash
  kill %1
  ```

]

.debug[[k8s/accessinternal.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/accessinternal.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/containers-by-the-water.jpg)]

---

Testing

.nav[
[Previous section](#toc-accessing-internal-services)
|
[Back to table of contents](#toc-chapter-1)
|
[Next section](#toc-managing-configuration)
]

---
# Testing

There are multiple levels of testing:

- unit testing (many small tests that run in isolation),

- integration testing (bigger tests involving multiple components),

- functional or end-to-end testing (even bigger tests involving the whole app).

In this section, we will focus on *unit testing*, where each test case
should (ideally) be completely isolated from other components and system
interaction: no real database, no real backend, *mocks* everywhere.

(For a good discussion on the merits of unit testing, we can read
[Just Say No to More End-to-End Tests](https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html).)

Unfortunately, this ideal scenario is easier said than done ...

.debug[[k8s/testing.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/testing.md)]
---

## Multi-stage build

```dockerfile
FROM <baseimage>
RUN <install dependencies>
COPY <code>
RUN <build code>
RUN <install test dependencies>
COPY <test data sets and fixtures>
RUN <unit tests>
FROM <baseimage>
RUN <install dependencies>
COPY <code>
RUN <build code>
CMD, EXPOSE ...
```

- This leverages the Docker cache: if the code doesn't change, the tests don't need to run

- If the tests require a database or other backend, we can use `docker build --network`

- If the tests fail, the build fails; and no image is generated

.debug[[k8s/testing.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/testing.md)]
---

## Docker Compose

```yaml
version: 3
service:
  project:
    image: my_image_name
    build:
      context: .
      target: dev

database:
    image: redis
  backend:
    image: backend

```
+

```shell
docker-compose build && docker-compose run project pytest -v
```

.debug[[k8s/testing.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/testing.md)]
---

## Skaffold/Container-structure-test

- The `test` field of the `skaffold.yaml` instructs skaffold to run test against your image.

- It uses the [container-structure-test](https://github.com/GoogleContainerTools/container-structure-test)

- It allows to run custom commands

- Unfortunately, nothing to run other Docker images

(to start a database or a backend that we need to run tests)

.debug[[k8s/testing.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/testing.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/distillery-containers.jpg)]

---

Managing configuration

.nav[
[Previous section](#toc-testing)
|
[Back to table of contents](#toc-chapter-2)
|
[Next section](#toc-sealed-secrets)
]

---
# Managing configuration

- Some applications need to be configured (obviously!)

- There are many ways for our code to pick up configuration:

- command-line arguments

- environment variables

- configuration files

- configuration servers (getting configuration from a database, an API...)

- ... and more (because programmers can be very creative!)

- How can we do these things with containers and Kubernetes?

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Passing configuration to containers

- There are many ways to pass configuration to code running in a container:

- baking it into a custom image

- command-line arguments

- environment variables

- injecting configuration files

- exposing it over the Kubernetes API

- configuration servers

- Let's review these different strategies!

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Baking custom images

- Put the configuration in the image

(it can be in a configuration file, but also `ENV` or `CMD` actions)

- It's easy! It's simple!

- Unfortunately, it also has downsides:

- multiplication of images

- different images for dev, staging, prod ...

- minor reconfigurations require a whole build/push/pull cycle

- Avoid doing it unless you don't have the time to figure out other options

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Command-line arguments

- Pass options to `args` array in the container specification

- Example ([source](https://github.com/coreos/pods/blob/master/kubernetes.yaml#L29)): 
  ```yaml
      args: 
        - "--data-dir=/var/lib/etcd"
        - "--advertise-client-urls=http://127.0.0.1:2379"
        - "--listen-client-urls=http://127.0.0.1:2379"
        - "--listen-peer-urls=http://127.0.0.1:2380"
        - "--name=etcd"
  ```

- The options can be passed directly to the program that we run ...

... or to a wrapper script that will use them to e.g. generate a config file

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Command-line arguments, pros & cons

- Works great when options are passed directly to the running program

(otherwise, a wrapper script can work around the issue)

- Works great when there aren't too many parameters

(to avoid a 20-lines `args` array)

- Requires documentation and/or understanding of the underlying program

("which parameters and flags do I need, again?")

- Well-suited for mandatory parameters (without default values)

- Not ideal when we need to pass a real configuration file anyway

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Environment variables

- Pass options through the `env` map in the container specification

- Example:
  ```yaml
      env:
      - name: ADMIN_PORT
        value: "8080"
      - name: ADMIN_AUTH
        value: Basic
      - name: ADMIN_CRED
        value: "admin:0pensesame!"
  ```

🤔 Why this weird `{name: xxx, value: yyy}` scheme? It will be revealed soon!

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## The downward API

- In the previous example, environment variables have fixed values

- We can also use a mechanism called the *downward API*

- The downward API allows exposing pod or container information

- either through special files (we won't show that for now)

- or through environment variables

- The value of these environment variables is computed when the container is started

- Remember: environment variables won't (can't) change after container start

- Let's see a few concrete examples!

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Exposing the pod's namespace

```yaml
    - name: MY_POD_NAMESPACE
      valueFrom:
        fieldRef:
          fieldPath: metadata.namespace
```

- Useful to generate FQDN of services

(in some contexts, a short name is not enough)

- For instance, the two commands should be equivalent:
  ```
  curl api-backend
  curl api-backend.$MY_POD_NAMESPACE.svc.cluster.local
  ```

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Exposing the pod's IP address

```yaml
    - name: MY_POD_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP
```

- Useful if we need to know our IP address

(we could also read it from `eth0`, but this is more solid)

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Exposing the container's resource limits

```yaml
    - name: MY_MEM_LIMIT
      valueFrom:
        resourceFieldRef:
          containerName: test-container
          resource: limits.memory
```

- Useful for runtimes where memory is garbage collected

- Example: the JVM

(the memory available to the JVM should be set with the `-Xmx ` flag)

- Best practice: set a memory limit, and pass it to the runtime

- Note: recent versions of the JVM can do this automatically

(see [JDK-8146115](https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8146115))
  and
  [this blog post](https://very-serio.us/2017/12/05/running-jvms-in-kubernetes/)
  for detailed examples)

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## More about the downward API

- [This documentation page](https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/) tells more about these environment variables

- And [this one](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/) explains the other way to use the downward API

(through files that get created in the container filesystem)

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Environment variables, pros and cons

- Works great when the running program expects these variables

- Works great for optional parameters with reasonable defaults

(since the container image can provide these defaults)

- Sort of auto-documented

(we can see which environment variables are defined in the image, and their values)

- Can be (ab)used with longer values ...

- ... You *can* put an entire Tomcat configuration file in an environment ...

- ... But *should* you?

(Do it if you really need to, we're not judging! But we'll see better ways.)

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Injecting configuration files

- Sometimes, there is no way around it: we need to inject a full config file

- Kubernetes provides a mechanism for that purpose: `configmaps`

- A configmap is a Kubernetes resource that exists in a namespace

- Conceptually, it's a key/value map

(values are arbitrary strings)

- We can think about them in (at least) two different ways:

- as holding entire configuration file(s)

- as holding individual configuration parameters

*Note: to hold sensitive information, we can use "Secrets", which
are another type of resource behaving very much like configmaps.
We'll cover them just after!*

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Configmaps storing entire files

- In this case, each key/value pair corresponds to a configuration file

- Key = name of the file

- Value = content of the file

- There can be one key/value pair, or as many as necessary

(for complex apps with multiple configuration files)

- Examples:
  ```
  # Create a configmap with a single key, "app.conf"
  kubectl create configmap my-app-config --from-file=app.conf
  # Create a configmap with a single key, "app.conf" but another file
  kubectl create configmap my-app-config --from-file=app.conf=app-prod.conf
  # Create a configmap with multiple keys (one per file in the config.d directory)
  kubectl create configmap my-app-config --from-file=config.d/
  ```

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Configmaps storing individual parameters

- In this case, each key/value pair corresponds to a parameter

- Key = name of the parameter

- Value = value of the parameter

- Examples:
  ```
  # Create a configmap with two keys
  kubectl create cm my-app-config \
      --from-literal=foreground=red \
      --from-literal=background=blue
  
  # Create a configmap from a file containing key=val pairs
  kubectl create cm my-app-config \
      --from-env-file=app.conf
  ```

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Exposing configmaps to containers

- Configmaps can be exposed as plain files in the filesystem of a container

- this is achieved by declaring a volume and mounting it in the container

- this is particularly effective for configmaps containing whole files

- Configmaps can be exposed as environment variables in the container

- this is achieved with the downward API

- this is particularly effective for configmaps containing individual parameters

- Let's see how to do both!

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Passing a configuration file with a configmap

- We will start a load balancer powered by HAProxy

- We will use the [official `haproxy` image](https://hub.docker.com/_/haproxy/)

- It expects to find its configuration in `/usr/local/etc/haproxy/haproxy.cfg`

- We will provide a simple HAproxy configuration, `k8s/haproxy.cfg`

- It listens on port 80, and load balances connections between IBM and Google

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Creating the configmap

- Go to the `k8s` directory in the repository:
  ```bash
  cd ~/container.training/k8s
  ```

- Create a configmap named `haproxy` and holding the configuration file:
  ```bash
  kubectl create configmap haproxy --from-file=haproxy.cfg
  ```

- Check what our configmap looks like:
  ```bash
  kubectl get configmap haproxy -o yaml
  ```

]

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Using the configmap

We are going to use the following pod definition:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: haproxy
spec:
  volumes:
  - name: config
    configMap:
      name: haproxy
  containers:
  - name: haproxy
    image: haproxy
    volumeMounts:
    - name: config
      mountPath: /usr/local/etc/haproxy/
```

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Using the configmap

- The resource definition from the previous slide is in `k8s/haproxy.yaml`

- Create the HAProxy pod:
  ```bash
  kubectl apply -f ~/container.training/k8s/haproxy.yaml
  ```

- Check the IP address allocated to the pod:
  ```bash
  kubectl get pod haproxy -o wide
  IP=$(kubectl get pod haproxy -o json | jq -r .status.podIP)
  ```

]

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Testing our load balancer

- The load balancer will send:

- half of the connections to Google

- the other half to IBM

- Access the load balancer a few times:
  ```bash
  curl $IP
  curl $IP
  curl $IP
  ```

]

We should see connections served by Google, and others served by IBM.
 
(Each server sends us a redirect page. Look at the URL that they send us to!)

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Exposing configmaps with the downward API

- We are going to run a Docker registry on a custom port

- By default, the registry listens on port 5000

- This can be changed by setting environment variable `REGISTRY_HTTP_ADDR`

- We are going to store the port number in a configmap

- Then we will expose that configmap as a container environment variable

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Creating the configmap

- Our configmap will have a single key, `http.addr`:
  ```bash
  kubectl create configmap registry --from-literal=http.addr=0.0.0.0:80
  ```

- Check our configmap:
  ```bash
  kubectl get configmap registry -o yaml
  ```

]

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Using the configmap

We are going to use the following pod definition:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: registry
spec:
  containers:
  - name: registry
    image: registry
    env:
    - name: REGISTRY_HTTP_ADDR
      valueFrom:
        configMapKeyRef:
          name: registry
          key: http.addr
```

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Using the configmap

- The resource definition from the previous slide is in `k8s/registry.yaml`

- Create the registry pod:
  ```bash
  kubectl apply -f ~/container.training/k8s/registry.yaml
  ```

- Check the IP address allocated to the pod:
  ```bash
  kubectl get pod registry -o wide
  IP=$(kubectl get pod registry -o json | jq -r .status.podIP)
  ```

- Confirm that the registry is available on port 80:
  ```bash
  curl $IP/v2/_catalog
  ```

]

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Passwords, tokens, sensitive information

- For sensitive information, there is another special resource: *Secrets*

- Secrets and Configmaps work almost the same way

(we'll expose the differences on the next slide)

- The *intent* is different, though:

*"You should use secrets for things which are actually secret like API keys, 
  credentials, etc., and use config map for not-secret configuration data."*

*"In the future there will likely be some differentiators for secrets like rotation or support for backing the secret API w/ HSMs, etc."*

(Source: [the author of both features](https://stackoverflow.com/a/36925553/580281
))

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

## Differences between configmaps and secrets
 
- Secrets are base64-encoded when shown with `kubectl get secrets -o yaml`

- keep in mind that this is just *encoding*, not *encryption*

- it is very easy to [automatically extract and decode secrets](https://medium.com/@mveritym/decoding-kubernetes-secrets-60deed7a96a3)

- [Secrets can be encrypted at rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/)

- With RBAC, we can authorize a user to access configmaps, but not secrets

(since they are two different kinds of resources)

.debug[[k8s/configuration.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/configuration.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/lots-of-containers.jpg)]

---

sealed-secrets

.nav[
[Previous section](#toc-managing-configuration)
|
[Back to table of contents](#toc-chapter-2)
|
[Next section](#toc-kustomize)
]

---
# sealed-secrets

- https://github.com/bitnami-labs/sealed-secrets

- has a server side (standard kubernetes deployment) and a client side *kubeseal* binary

- server-side start by generating a key pair, keep the private, expose the public.

- To create a sealed-secret, you only need access to public key

- You can enforce access with RBAC rules of kubernetes

.debug[[k8s/sealed-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/sealed-secrets.md)]
---

## sealed-secrets how to

- adding a secret: *kubeseal* will cipher it with the public key

- server side controller will re-create original secret, when the ciphered one are added to the cluster

- it makes it "safe" to add those secret to your source tree

- since version 0.9 key rotation are enable by default, so remember to backup private keys regularly.
 (or you won't be able to decrypt all you keys, in a case of *disaster recovery*)

.debug[[k8s/sealed-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/sealed-secrets.md)]
---

## First "sealed-secret"

.exercise[
- Install *kubeseal*
  ```bash
  wget https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.9.7/kubeseal-linux-amd64 -O kubeseal
  sudo install -m 755 kubeseal /usr/local/bin/kubeseal
  ```

- Install controller
  ```bash
  helm install -n kube-system sealed-secrets-controller stable/sealed-secrets
  ```

- Create a secret you don't want to leak
  ```bash
  kubectl create secret generic --from-literal=foo=bar my-secret -o yaml --dry-run \
    | kubeseal > mysecret.yaml
  kubectl apply -f mysecret.yaml
  ```
]

.debug[[k8s/sealed-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/sealed-secrets.md)]
---

## Alternative: sops / git crypt

- You can work a VCS level (ie totally abstracted from kubernetess)

- sops (https://github.com/mozilla/sops), VCS agnostic, encrypt portion of files

- git-crypt that work with git to transparently encrypt (some) files in git

.debug[[k8s/sealed-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/sealed-secrets.md)]
---

## Other alternative

- You can delegate secret management to another component like *hashicorp vault*

- Can work in multiple ways:

- encrypt secret from API-server (instead of the much secure *base64*)
   - encrypt secret before sending it in kubernetes (avoid git in plain text)
   - manager secret entirely in vault and expose to the container via volume

.debug[[k8s/sealed-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/sealed-secrets.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/plastic-containers.JPG)]

---

Kustomize

.nav[
[Previous section](#toc-sealed-secrets)
|
[Back to table of contents](#toc-chapter-2)
|
[Next section](#toc-managing-stacks-with-helm)
]

---
# Kustomize

- Kustomize lets us transform YAML files representing Kubernetes resources

- The original YAML files are valid resource files

(e.g. they can be loaded with `kubectl apply -f`)

- They are left untouched by Kustomize

- Kustomize lets us define *overlays* that extend or change the resource files

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Differences with Helm

- Helm charts use placeholders `{{ like.this }}`

- Kustomize "bases" are standard Kubernetes YAML

- It is possible to use an existing set of YAML as a Kustomize base

- As a result, writing a Helm chart is more work ...

- ... But Helm charts are also more powerful; e.g. they can:

- use flags to conditionally include resources or blocks

- check if a given Kubernetes API group is supported

- [and much more](https://helm.sh/docs/chart_template_guide/)

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Kustomize concepts

- Kustomize needs a `kustomization.yaml` file

- That file can be a *base* or a *variant*

- If it's a *base*:

- it lists YAML resource files to use

- If it's a *variant* (or *overlay*):

- it refers to (at least) one *base*

- and some *patches*

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## An easy way to get started with Kustomize

- We are going to use [Replicated Ship](https://www.replicated.com/ship/) to experiment with Kustomize

- The [Replicated Ship CLI](https://github.com/replicatedhq/ship/releases) has been installed on our clusters

- Replicated Ship has multiple workflows; here is what we will do:

- initialize a Kustomize overlay from a remote GitHub repository

- customize some values using the web UI provided by Ship

- look at the resulting files and apply them to the cluster

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Getting started with Ship

- We need to run `ship init` in a new directory

- `ship init` requires a URL to a remote repository containing Kubernetes YAML

- It will clone that repository and start a web UI

- Later, it can watch that repository and/or update from it

- We will use the [jpetazzo/kubercoins](https://github.com/jpetazzo/kubercoins) repository

(it contains all the DockerCoins resources as YAML files)

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## `ship init`

- Change to a new directory:
  ```bash
  mkdir ~/kustomcoins
  cd ~/kustomcoins
  ```

- Run `ship init` with the kustomcoins repository:
  ```bash
  ship init https://github.com/jpetazzo/kubercoins
  ```

]

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Access the web UI

- `ship init` tells us to connect on `localhost:8800`

- We need to replace `localhost` with the address of our node

(since we run on a remote machine)

- Follow the steps in the web UI, and change one parameter

(e.g. set the number of replicas in the worker Deployment)

- Complete the web workflow, and go back to the CLI

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Inspect the results

- Look at the content of our directory

- `base` contains the kubercoins repository + a `kustomization.yaml` file

- `overlays/ship` contains the Kustomize overlay referencing the base + our patch(es)

- `rendered.yaml` is a YAML bundle containing the patched application

- `.ship` contains a state file used by Ship

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Using the results

- We can `kubectl apply -f rendered.yaml`

(on any version of Kubernetes)

- Starting with Kubernetes 1.14, we can apply the overlay directly with:
  ```bash
  kubectl apply -k overlays/ship
  ```

- But let's not do that for now!

- We will create a new copy of DockerCoins in another namespace

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Deploy DockerCoins with Kustomize

- Create a new namespace:
  ```bash
  kubectl create namespace kustomcoins
  ```

- Deploy DockerCoins:
  ```bash
  kubectl apply -f rendered.yaml --namespace=kustomcoins
  ```

- Or, with Kubernetes 1.14, you can also do this:
  ```bash
  kubectl apply -k overlays/ship --namespace=kustomcoins
  ```

]

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

## Checking our new copy of DockerCoins

- We can check the worker logs, or the web UI

- Retrieve the NodePort number of the web UI:
  ```bash
  kubectl get service webui --namespace=kustomcoins
  ```

- Open it in a web browser

- Look at the worker logs:
  ```bash
  kubectl logs deploy/worker --tail=10 --follow --namespace=kustomcoins
  ```

]

Note: it might take a minute or two for the worker to start.

.debug[[k8s/kustomize.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kustomize.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/train-of-containers-1.jpg)]

---

Managing stacks with Helm

.nav[
[Previous section](#toc-kustomize)
|
[Back to table of contents](#toc-chapter-2)
|
[Next section](#toc-helm-chart-format)
]

---
# Managing stacks with Helm

- We created our first resources with `kubectl run`, `kubectl expose` ...

- We have also created resources by loading YAML files with `kubectl apply -f`

- For larger stacks, managing thousands of lines of YAML is unreasonable

- These YAML bundles need to be customized with variable parameters

(E.g.: number of replicas, image version to use ...)

- It would be nice to have an organized, versioned collection of bundles

- It would be nice to be able to upgrade/rollback these bundles carefully

- [Helm](https://helm.sh/) is an open source project offering all these things!

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Helm concepts

- `helm` is a CLI tool

- It is used to find, install, upgrade *charts*

- A chart is an archive containing templatized YAML bundles

- Charts are versioned

- Charts can be stored on private or public repositories

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Differences between charts and packages

- A package (deb, rpm...) contains binaries, libraries, etc.

- A chart contains YAML manifests

(the binaries, libraries, etc. are in the images referenced by the chart)

- On most distributions, a package can only be installed once

(installing another version replaces the installed one)

- A chart can be installed multiple times

- Each installation is called a *release*

- This allows to install e.g. 10 instances of MongoDB

(with potentially different versions and configurations)

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Wait a minute ...

*But, on my Debian system, I have Python 2 **and** Python 3.
 
Also, I have multiple versions of the Postgres database engine!*

Yes!

But they have different package names:

- `python2.7`, `python3.8`

- `postgresql-10`, `postgresql-11`

Good to know: the Postgres package in Debian includes
provisions to deploy multiple Postgres servers on the
same system, but it's an exception (and it's a lot of
work done by the package maintainer, not by the `dpkg`
or `apt` tools).

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Helm 2 vs Helm 3

- Helm 3 was released [November 13, 2019](https://helm.sh/blog/helm-3-released/)

- Charts remain compatible between Helm 2 and Helm 3

- The CLI is very similar (with minor changes to some commands)

- The main difference is that Helm 2 uses `tiller`, a server-side component

- Helm 3 doesn't use `tiller` at all, making it simpler (yay!)

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## With or without `tiller`

- With Helm 3:

- the `helm` CLI communicates directly with the Kubernetes API

- it creates resources (deployments, services...) with our credentials

- With Helm 2:

- the `helm` CLI communicates with `tiller`, telling `tiller` what to do

- `tiller` then communicates with the Kubernetes API, using its own credentials

- This indirect model caused significant permissions headaches

(`tiller` required very broad permissions to function)

- `tiller` was removed in Helm 3 to simplify the security aspects

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Installing Helm

- If the `helm` CLI is not installed in your environment, install it

- Check if `helm` is installed:
  ```bash
  helm
  ```

- If it's not installed, run the following command:
  ```bash
  curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get-helm-3 \
  | bash
  ```

]

(To install Helm 2, replace `get-helm-3` with `get`.)

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Only if using Helm 2 ...

- We need to install Tiller and give it some permissions

- Tiller is composed of a *service* and a *deployment* in the `kube-system` namespace

- They can be managed (installed, upgraded...) with the `helm` CLI

- Deploy Tiller:
  ```bash
  helm init
  ```

]

At the end of the install process, you will see:

```
Happy Helming!
```

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Only if using Helm 2 ...

- Tiller needs permissions to create Kubernetes resources

- In a more realistic deployment, you might create per-user or per-team
  service accounts, roles, and role bindings

- Grant `cluster-admin` role to `kube-system:default` service account:
  ```bash
  kubectl create clusterrolebinding add-on-cluster-admin \
      --clusterrole=cluster-admin --serviceaccount=kube-system:default
  ```

]

(Defining the exact roles and permissions on your cluster requires
a deeper knowledge of Kubernetes' RBAC model. The command above is
fine for personal and development clusters.)

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Charts and repositories

- A *repository* (or repo in short) is a collection of charts

- It's just a bunch of files

(they can be hosted by a static HTTP server, or on a local directory)

- We can add "repos" to Helm, giving them a nickname

- The nickname is used when referring to charts on that repo

(for instance, if we try to install `hello/world`, that
  means the chart `world` on the repo `hello`; and that repo
  `hello` might be something like https://blahblah.hello.io/charts/)

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Managing repositories

- Let's check what repositories we have, and add the `stable` repo

(the `stable` repo contains a set of official-ish charts)

- List our repos:
  ```bash
  helm repo list
  ```

- Add the `stable` repo:
  ```bash
  helm repo add stable https://kubernetes-charts.storage.googleapis.com/
  ```

]

Adding a repo can take a few seconds (it downloads the list of charts from the repo).

It's OK to add a repo that already exists (it will merely update it).

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Search available charts

- We can search available charts with `helm search`

- We need to specify where to search (only our repos, or Helm Hub)

- Let's search for all charts mentioning tomcat!

- Search for tomcat in the repo that we added earlier:
  ```bash
  helm search repo tomcat
  ```

- Search for tomcat on the Helm Hub:
  ```bash
  helm search hub tomcat
  ```

]

[Helm Hub](https://hub.helm.sh/) indexes many repos, using the [Monocular](https://github.com/helm/monocular) server.

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Charts and releases

- "Installing a chart" means creating a *release*

- We need to name that release

(or use the `--generate-name` to get Helm to generate one for us)

- Install the tomcat chart that we found earlier:
  ```bash
  helm install java4ever stable/tomcat
  ```

- List the releases:
  ```bash
  helm list
  ```

]

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Searching and installing with Helm 2

- Helm 2 doesn't have support for the Helm Hub

- The `helm search` command only takes a search string argument

(e.g. `helm search tomcat`)

- With Helm 2, the name is optional:

`helm install stable/tomcat` will automatically generate a name

`helm install --name java4ever stable/tomcat` will specify a name

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Viewing resources of a release

- This specific chart labels all its resources with a `release` label

- We can use a selector to see these resources

- List all the resources created by this release:
  ```bash
  kubectl get all --selector=release=java4ever
  ```

]

Note: this `release` label wasn't added automatically by Helm.
 
It is defined in that chart. In other words, not all charts will provide this label.

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Configuring a release

- By default, `stable/tomcat` creates a service of type `LoadBalancer`

- We would like to change that to a `NodePort`

- We could use `kubectl edit service java4ever-tomcat`, but ...

... our changes would get overwritten next time we update that chart!

- Instead, we are going to *set a value*

- Values are parameters that the chart can use to change its behavior

- Values have default values

- Each chart is free to define its own values and their defaults

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Checking possible values

- We can inspect a chart with `helm show` or `helm inspect`

- Look at the README for tomcat:
  ```bash
  helm show readme stable/tomcat
  ```

- Look at the values and their defaults:
  ```bash
  helm show values stable/tomcat
  ```

]

The `values` may or may not have useful comments.

The `readme` may or may not have (accurate) explanations for the values.

(If we're unlucky, there won't be any indication about how to use the values!)

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Setting values

- Values can be set when installing a chart, or when upgrading it

- We are going to update `java4ever` to change the type of the service

- Update `java4ever`:
  ```bash
  helm upgrade java4ever stable/tomcat --set service.type=NodePort
  ```

]

Note that we have to specify the chart that we use (`stable/tomcat`),
even if we just want to update some values.

We can set multiple values. If we want to set many values, we can use `-f`/`--values` and pass a YAML file with all the values.

All unspecified values will take the default values defined in the chart.

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

## Connecting to tomcat

- Let's check the tomcat server that we just installed

- Note: its readiness probe has a 60s delay

(so it will take 60s after the initial deployment before the service works)

- Check the node port allocated to the service:
  ```bash
  kubectl get service java4ever-tomcat
  PORT=$(kubectl get service java4ever-tomcat -o jsonpath={..nodePort})
  ```

- Connect to it, checking the demo app on `/sample/`:
  ```bash
  curl localhost:$PORT/sample/
  ```

]

.debug[[k8s/helm-intro.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-intro.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/train-of-containers-2.jpg)]

---

Helm chart format

.nav[
[Previous section](#toc-managing-stacks-with-helm)
|
[Back to table of contents](#toc-chapter-2)
|
[Next section](#toc-helm-secrets)
]

---
# Helm chart format

- What exactly is a chart?

- What's in it?

- What would be involved in creating a chart?

(we won't create a chart, but we'll see the required steps)

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## What is a chart

- A chart is a set of files

- Some of these files are mandatory for the chart to be viable

(more on that later)

- These files are typically packed in a tarball

- These tarballs are stored in "repos"

(which can be static HTTP servers)

- We can install from a repo, from a local tarball, or an unpacked tarball

(the latter option is preferred when developing a chart)

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## What's in a chart

- A chart must have at least:

- a `templates` directory, with YAML manifests for Kubernetes resources

- a `values.yaml` file, containing (tunable) parameters for the chart

- a `Chart.yaml` file, containing metadata (name, version, description ...)

- Let's look at a simple chart, `stable/tomcat`

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Downloading a chart

- We can use `helm pull` to download a chart from a repo

- Download the tarball for `stable/tomcat`:
  ```bash
  helm pull stable/tomcat
  ```
  (This will create a file named `tomcat-X.Y.Z.tgz`.)

- Or, download + untar `stable/tomcat`:
  ```bash
  helm pull stable/tomcat --untar
  ```
  (This will create a directory named `tomcat`.)

]

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Looking at the chart's content

- Let's look at the files and directories in the `tomcat` chart

- Display the tree structure of the chart we just downloaded:
  ```bash
  tree tomcat
  ```

]

We see the components mentioned above: `Chart.yaml`, `templates/`, `values.yaml`.

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Templates

- The `templates/` directory contains YAML manifests for Kubernetes resources

(Deployments, Services, etc.)

- These manifests can contain template tags

(using the standard Go template library)

- Look at the template file for the tomcat Service resource:
  ```bash
  cat tomcat/templates/appsrv-svc.yaml
  ```

]

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Analyzing the template file

- Tags are identified by `{{ ... }}`

- `{{ template "x.y" }}` expands a [named template](https://helm.sh/docs/chart_template_guide/named_templates/#declaring-and-using-templates-with-define-and-template)

(previously defined with `{{ define "x.y "}}...stuff...{{ end }}`)

- The `.` in `{{ template "x.y" . }}` is the *context* for that named template

(so that the named template block can access variables from the local context)

- `{{ .Release.xyz }}` refers to [built-in variables](https://helm.sh/docs/chart_template_guide/builtin_objects/) initialized by Helm

(indicating the chart name, version, whether we are installing or upgrading ...)

- `{{ .Values.xyz }}` refers to tunable/settable [values](https://helm.sh/docs/chart_template_guide/values_files/)

(more on that in a minute)

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Values

- Each chart comes with a
  [values file](https://helm.sh/docs/chart_template_guide/values_files/)

- It's a YAML file containing a set of default parameters for the chart

- The values can be accessed in templates with e.g. `{{ .Values.x.y }}`

(corresponding to field `y` in map `x` in the values file)

- The values can be set or overridden when installing or ugprading a chart:

- with `--set x.y=z` (can be used multiple times to set multiple values)

- with `--values some-yaml-file.yaml` (set a bunch of values from a file)

- Charts following best practices will have values following specific patterns

(e.g. having a `service` map allowing to set `service.type` etc.)

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Other useful tags

- `{{ if x }} y {{ end }}` allows to include `y` if `x` evaluates to `true`

(can be used for e.g. healthchecks, annotations, or even an entire resource)

- `{{ range x }} y {{ end }}` iterates over `x`, evaluating `y` each time

(the elements of `x` are assigned to `.` in the range scope)

- `{{- x }}`/`{{ x -}}` will remove whitespace on the left/right

- The whole [Sprig](http://masterminds.github.io/sprig/) library, with additions:

`lower` `upper` `quote` `trim` `default` `b64enc` `b64dec` `sha256sum` `indent` `toYaml` ...

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Pipelines

- `{{ quote blah }}` can also be expressed as `{{ blah | quote }}`

- With multiple arguments, `{{ x y z }}` can be expressed as `{{ z | x y }}`)

- Example: `{{ .Values.annotations | toYaml | indent 4 }}`

- transforms the map under `annotations` into a YAML string

- indents it with 4 spaces (to match the surrounding context)

- Pipelines are not specific to Helm, but a feature of Go templates

(check the [Go text/template documentation](https://golang.org/pkg/text/template/) for more details and examples)

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## README and NOTES.txt

- At the top-level of the chart, it's a good idea to have a README

- It will be viewable with e.g. `helm show readme stable/tomcat`

- In the `templates/` directory, we can also have a `NOTES.txt` file

- When the template is installed (or upgraded), `NOTES.txt` is processed too

(i.e. its `{{ ... }}` tags are evaluated)

- It gets displayed after the install or upgrade

- It's a great place to generate messages to tell the user:

- how to connect to the release they just deployed

- any passwords or other thing that we generated for them

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Additional files

- We can place arbitrary files in the chart (outside of the `templates/` directory)

- They can be accessed in templates with `.Files`

- They can be transformed into ConfigMaps or Secrets with `AsConfig` and `AsSecrets`

(see [this example](https://helm.sh/docs/chart_template_guide/accessing_files/#configmap-and-secrets-utility-functions) in the Helm docs)

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

## Hooks and tests

- We can define *hooks* in our templates

- Hooks are resources annotated with `"helm.sh/hook": NAME-OF-HOOK`

- Hook names include `pre-install`, `post-install`, `test`, [and much more](https://helm.sh/docs/topics/charts_hooks/#the-available-hooks)

- The resources defined in hooks are loaded at a specific time

- Hook execution is *synchronous*

(if the resource is a Job or Pod, Helm will wait for its completion)

- This can be use for database migrations, backups, notifications, smoke tests ...

- Hooks named `test` are executed only when running `helm test RELEASE-NAME`

.debug[[k8s/helm-chart-format.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-chart-format.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/two-containers-on-a-truck.jpg)]

---

Helm secrets

.nav[
[Previous section](#toc-helm-chart-format)
|
[Back to table of contents](#toc-chapter-2)
|
[Next section](#toc-shipping-images-with-a-registry)
]

---
# Helm secrets

- Helm can do *rollbacks*:

- to previously installed charts

- to previous sets of values

- How and where does it store the data needed to do that?

- Let's investigate!

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## We need a release

- We need to install something with Helm

- Let's use the `stable/tomcat` chart as an example

- Install a release called `tomcat` with the chart `stable/tomcat`:
  ```bash
  helm upgrade tomcat stable/tomcat --install
  ```

- Let's upgrade that release, and change a value:
  ```bash
  helm upgrade tomcat stable/tomcat --set ingress.enabled=true
  ```

]

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Release history

- Helm stores successive revisions of each release

- View the history for that release:
  ```bash
  helm history tomcat
  ```

]

Where does that come from?

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Investigate

- Possible options:

- local filesystem (no, because history is visible from other machines)

- persistent volumes (no, Helm works even without them)

- ConfigMaps, Secrets?

- Look for ConfigMaps and Secrets:
  ```bash
  kubectl get configmaps,secrets
  ```

]

We should see a number of secrets with TYPE `helm.sh/release.v1`.

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Unpacking a secret

- Let's find out what is in these Helm secrets

- Examine the secret corresponding to the second release of `tomcat`:
  ```bash
  kubectl describe secret sh.helm.release.v1.tomcat.v2
  ```
  (`v1` is the secret format; `v2` means revision 2 of the `tomcat` release)

]

There is a key named `release`.

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Unpacking the release data

- Let's see what's in this `release` thing!

- Dump the secret:
  ```bash
  kubectl get secret sh.helm.release.v1.tomcat.v2 \
      -o go-template='{{ .data.release }}'
  ```

]

Secrets are encoded in base64. We need to decode that!

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Decoding base64

- We can pipe the output through `base64 -d` or use go-template's `base64decode`

- Decode the secret:
  ```bash
  kubectl get secret sh.helm.release.v1.tomcat.v2 \
      -o go-template='{{ .data.release | base64decode }}'
  ```

]

... Wait, this *still* looks like base64. What's going on?

Let's try one more round of decoding!

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Decoding harder

- Just add one more base64 decode filter

- Decode it twice:
  ```bash
  kubectl get secret sh.helm.release.v1.tomcat.v2 \
      -o go-template='{{ .data.release | base64decode | base64decode }}'
  ```

]

... OK, that was *a lot* of binary data. What sould we do with it?

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Guessing data type

- We could use `file` to figure out the data type

- Pipe the decoded release through `file -`:
  ```bash
  kubectl get secret sh.helm.release.v1.tomcat.v2 \
      -o go-template='{{ .data.release | base64decode | base64decode }}' \
      | file -
  ```

]

Gzipped data! It can be decoded with `gunzip -c`.

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Uncompressing the data

- Let's uncompress the data and save it to a file

- Rerun the previous command, but with `| gunzip -c > release-info` :
  ```bash
  kubectl get secret sh.helm.release.v1.tomcat.v2 \
      -o go-template='{{ .data.release | base64decode | base64decode }}' \
      | gunzip -c > release-info
  ```

- Look at `release-info`:
  ```bash
  cat release-info
  ```

]

It's a bundle of ~~YAML~~ JSON.

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Looking at the JSON

If we inspect that JSON (e.g. with `jq keys release-info`), we see:

- `chart` (contains the entire chart used for that release)
- `config` (contains the values that we've set)
- `info` (date of deployment, status messages)
- `manifest` (YAML generated from the templates)
- `name` (name of the release, so `tomcat`)
- `namespace` (namespace where we deployed the release)
- `version` (revision number within that release; starts at 1)

The chart is in a structured format, but it's entirely captured in this JSON.

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

## Conclusions

- Helm stores each release information in a Secret in the namespace of the release

- The secret is JSON object (gzipped and encoded in base64)

- It contains the manifests generated for that release

- ... And everything needed to rebuild these manifests

(including the full source of the chart, and the values used)

- This allows arbitrary rollbacks, as well as tweaking values even without having access to the source of the chart (or the chart repo) used for deployment

.debug[[k8s/helm-secrets.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/helm-secrets.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/wall-of-containers.jpeg)]

---

Shipping images with a registry

.nav[
[Previous section](#toc-helm-secrets)
|
[Back to table of contents](#toc-chapter-3)
|
[Next section](#toc-registries)
]

---
# Shipping images with a registry

- Initially, our app was running on a single node

- We could *build* and *run* in the same place

- Therefore, we did not need to *ship* anything

- Now that we want to run on a cluster, things are different

- The easiest way to ship container images is to use a registry

.debug[[k8s/shippingimages.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/shippingimages.md)]
---

## How Docker registries work (a reminder)

- What happens when we execute `docker run alpine` ?

- If the Engine needs to pull the `alpine` image, it expands it into `library/alpine`

- `library/alpine` is expanded into `index.docker.io/library/alpine`

- The Engine communicates with `index.docker.io` to retrieve `library/alpine:latest`

- To use something else than `index.docker.io`, we specify it in the image name

- Examples:
  ```bash
  docker pull gcr.io/google-containers/alpine-with-bash:1.0

docker build -t registry.mycompany.io:5000/myimage:awesome .
  docker push registry.mycompany.io:5000/myimage:awesome
  ```

.debug[[k8s/shippingimages.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/shippingimages.md)]
---

## Running DockerCoins on Kubernetes

- Create one deployment for each component

(hasher, redis, rng, webui, worker)

- Expose deployments that need to accept connections

(hasher, redis, rng, webui)

- For redis, we can use the official redis image

- For the 4 others, we need to build images and push them to some registry

.debug[[k8s/shippingimages.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/shippingimages.md)]
---

## Building and shipping images

- There are *many* options!

- Manually:

- build locally (with `docker build` or otherwise)

- push to the registry

- Automatically:

- build and test locally

- when ready, commit and push a code repository

- the code repository notifies an automated build system

- that system gets the code, builds it, pushes the image to the registry

.debug[[k8s/shippingimages.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/shippingimages.md)]
---

## Which registry do we want to use?

- There are SAAS products like Docker Hub, Quay ...

- Each major cloud provider has an option as well

(ACR on Azure, ECR on AWS, GCR on Google Cloud...)

- There are also commercial products to run our own registry

(Docker EE, Quay...)

- And open source options, too!

- When picking a registry, pay attention to its build system

(when it has one)

.debug[[k8s/shippingimages.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/shippingimages.md)]
---

## Building on the fly

- Some services can build images on the fly from a repository

- Example: [ctr.run](https://ctr.run/)

- Use ctr.run to automatically build a container image and run it:
  ```bash
  docker run ctr.run/github.com/jpetazzo/container.training/dockercoins/hasher
  ```

]

There might be a long pause before the first layer is pulled,
because the API behind `docker pull` doesn't allow to stream build logs, and there is no feedback during the build.

It is possible to view the build logs by setting up an account on [ctr.run](https://ctr.run/).
.debug[[k8s/shippingimages.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/shippingimages.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/Container-Ship-Freighter-Navigation-Elbe-Romance-1782991.jpg)]

---

Registries

.nav[
[Previous section](#toc-shipping-images-with-a-registry)
|
[Back to table of contents](#toc-chapter-3)
|
[Next section](#toc-automation--cicd)
]

---
# Registries

- There are lots of options to ship our container images to a registry

- We can group them depending on some characteristics:

- SaaS or self-hosted

- with or without a build system

.debug[[k8s/registries.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/registries.md)]
---

## Docker registry

- Self-hosted and [open source](https://github.com/docker/distribution)

- Runs in a single Docker container

- Supports multiple storage backends

- Supports basic authentication out of the box

- [Other authentication schemes](https://docs.docker.com/registry/deploying/#more-advanced-authentication) through proxy or delegation

- No build system

- To run it with the Docker engine:

```shell
  docker run -d -p 5000:5000 --name registry registry:2
  ```

- Or use the dedicated plugin in minikube, microk8s, etc.

.debug[[k8s/registries.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/registries.md)]
---

## Harbor

- Self-hostend and [open source](https://github.com/goharbor/harbor)

- Supports both Docker images and Helm charts

- Advanced authentification mechanism

- Multi-site synchronisation

- Vulnerability scanning

- No build system

- To run it with Helm:
  ```shell
  helm repo add harbor https://helm.goharbor.io
  helm install my-release harbor/harbor
  ```

.debug[[k8s/registries.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/registries.md)]
---

## Gitlab

- Available both as a SaaS product and self-hosted

- SaaS product is free for open source projects; paid subscription otherwise

- Some parts are [open source](https://gitlab.com/gitlab-org/gitlab-foss/)

- Integrated CI

- No build system (but a custom build system can be hooked to the CI)

- To run it with Helm:
  ```shell
  helm repo add gitlab https://charts.gitlab.io/
  helm install gitlab gitlab/gitlab
  ```

.debug[[k8s/registries.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/registries.md)]
---

## Docker Hub

- SaaS product: [hub.docker.com](https://hub.docker.com)

- Free for public image; paid subscription for private ones

- Build system included

.debug[[k8s/registries.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/registries.md)]
---

## Quay

- Available both as a SaaS product (Quay) and self-hosted ([quay.io](https://quay.io))

- SaaS product is free for public repositories; paid subscription otherwise

- Some components of Quay and quay.io are open source

(see [Project Quay](https://www.projectquay.io/) and the [announcement](https://www.redhat.com/en/blog/red-hat-introduces-open-source-project-quay-container-registry))

- Build system included

.debug[[k8s/registries.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/registries.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/ShippingContainerSFBay.jpg)]

---

Automation && CI/CD

.nav[
[Previous section](#toc-registries)
|
[Back to table of contents](#toc-chapter-3)
|
[Next section](#toc-rolling-updates)
]

---
# Automation && CI/CD

What we've done so far:

- development of our application

- manual testing, and exploration of automated testing strategies

- packaging in a container image

- shipping that image to a registry

What still need to be done:

- deployment of our application

- automation of the whole build / ship / run cycle

.debug[[k8s/stop-manual.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/stop-manual.md)]
---
## Jenkins / Jenkins-X

- Multi-purpose CI

- Self-hosted CI for kubernetes

- create a namespace per commit and apply manifests in the namespace
 
 "A deploy per feature-branch"

.small[
```shell
curl -L "https://github.com/jenkins-x/jx/releases/download/v2.0.1103/jx-darwin-amd64.tar.gz" | tar xzv jx
./jx boot
```
]

.debug[[k8s/ci-cd.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/ci-cd.md)]
---

## GitLab

- Repository + registry + CI/CD integrated all-in-one

```shell
helm repo add gitlab https://charts.gitlab.io/
helm install gitlab gitlab/gitlab
```

.debug[[k8s/ci-cd.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/ci-cd.md)]
---

## ArgoCD / flux

- Watch a git repository and apply changes to kubernetes

- provide UI to see changes, rollback

.small[
```shell
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
```
]

.debug[[k8s/ci-cd.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/ci-cd.md)]
---

## Tekton / knative

- knative is serverless project from google

- Tekton leverages knative to run pipelines

- not really user friendly today, but stay tune for wrappers/products

.debug[[k8s/ci-cd.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/ci-cd.md)]
---
## Exercise - building with Kubernetes

- Let's go to https://github.com/enix/kubecoin

- Our goal is to follow the instructions and complete exercise #1

.debug[[k8s/exercise-ci-build.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/exercise-ci-build.md)]
---
## Privileged container

- Running privileged container could be really harmful for the node it run on.

- Getting control of a node could expose other containers in the cluster and the cluster itself

- It's even worse when it is docker that run in this privileged container

- `docker build` doesn't allow to run privileged container for building layer

- nothing forbid to run `docker run --privileged`

.debug[[k8s/kaniko.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kaniko.md)]
---
## Kaniko

- https://github.com/GoogleContainerTools/kaniko

- *kaniko doesn't depend on a Docker daemon and executes each command
within a Dockerfile completely in userspace*

- Kaniko is only a build system, there is no runtime like docker does

- generates OCI compatible image, so could be run on Docker or other CRI

- use a different cache system than Docker

.debug[[k8s/kaniko.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kaniko.md)]
---
## Rootless docker and rootless buildkit

- This is experimental

- Have a lot of requirement of kernel param, options to set

- But it exists

.debug[[k8s/kaniko.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kaniko.md)]
---
## Exercice - build with kaniko

Complete exercise #2, (again code at: https://github.com/enix/kubecoin )

.debug[[k8s/exercise-ci-kaniko.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/exercise-ci-kaniko.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/aerial-view-of-containers.jpg)]

---

Rolling updates

.nav[
[Previous section](#toc-automation--cicd)
|
[Back to table of contents](#toc-chapter-3)
|
[Next section](#toc-advanced-rollout)
]

---
# Rolling updates

- By default (without rolling updates), when a scaled resource is updated:

- new pods are created

- old pods are terminated

- ... all at the same time

- if something goes wrong, ¯\\\_(ツ)\_/¯

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## Rolling updates

- With rolling updates, when a Deployment is updated, it happens progressively

- The Deployment controls multiple Replica Sets

- Each Replica Set is a group of identical Pods

(with the same image, arguments, parameters ...)

- During the rolling update, we have at least two Replica Sets:

- the "new" set (corresponding to the "target" version)

- at least one "old" set

- We can have multiple "old" sets

(if we start another update before the first one is done)

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## Update strategy

- Two parameters determine the pace of the rollout: `maxUnavailable` and `maxSurge`

- They can be specified in absolute number of pods, or percentage of the `replicas` count

- At any given time ...

- there will always be at least `replicas`-`maxUnavailable` pods available

- there will never be more than `replicas`+`maxSurge` pods in total

- there will therefore be up to `maxUnavailable`+`maxSurge` pods being updated

- We have the possibility of rolling back to the previous version
 (if the update fails or is unsatisfactory in any way)

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## Checking current rollout parameters

- Recall how we build custom reports with `kubectl` and `jq`:

- Show the rollout plan for our deployments:
  ```bash
    kubectl get deploy -o json |
            jq ".items[] | {name:.metadata.name} + .spec.strategy.rollingUpdate"
  ```

]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## Rolling updates in practice

- As of Kubernetes 1.8, we can do rolling updates with:

`deployments`, `daemonsets`, `statefulsets`

- Editing one of these resources will automatically result in a rolling update

- Rolling updates can be monitored with the `kubectl rollout` subcommand

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Rolling out the new `worker` service

- Let's monitor what's going on by opening a few terminals, and run:
  ```bash
  kubectl get pods -w
  kubectl get replicasets -w
  kubectl get deployments -w
  ```

- Update `worker` either with `kubectl edit`, or by running:
  ```bash
  kubectl set image deploy worker worker=dockercoins/worker:v0.2
  ```

]

That rollout should be pretty quick. What shows in the web UI?

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Give it some time

- At first, it looks like nothing is happening (the graph remains at the same level)

- According to `kubectl get deploy -w`, the `deployment` was updated really quickly

- But `kubectl get pods -w` tells a different story

- The old `pods` are still here, and they stay in `Terminating` state for a while

- Eventually, they are terminated; and then the graph decreases significantly

- This delay is due to the fact that our worker doesn't handle signals

- Kubernetes sends a "polite" shutdown request to the worker, which ignores it

- After a grace period, Kubernetes gets impatient and kills the container

(The grace period is 30 seconds, but [can be changed](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods) if needed)

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Rolling out something invalid

- What happens if we make a mistake?

- Update `worker` by specifying a non-existent image:
  ```bash
  kubectl set image deploy worker worker=dockercoins/worker:v0.3
  ```

- Check what's going on:
  ```bash
  kubectl rollout status deploy worker
  ```

/

]

Our rollout is stuck. However, the app is not dead.

(After a minute, it will stabilize to be 20-25% slower.)

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## What's going on with our rollout?

- Why is our app a bit slower?

- Because `MaxUnavailable=25%`

... So the rollout terminated 2 replicas out of 10 available

- Okay, but why do we see 5 new replicas being rolled out?

- Because `MaxSurge=25%`

... So in addition to replacing 2 replicas, the rollout is also starting 3 more

- It rounded down the number of MaxUnavailable pods conservatively,
 
 but the total number of pods being rolled out is allowed to be 25+25=50%

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## The nitty-gritty details

- We start with 10 pods running for the `worker` deployment

- Current settings: MaxUnavailable=25% and MaxSurge=25%

- When we start the rollout:

- two replicas are taken down (as per MaxUnavailable=25%)
  - two others are created (with the new version) to replace them
  - three others are created (with the new version) per MaxSurge=25%)

- Now we have 8 replicas up and running, and 5 being deployed

- Our rollout is stuck at this point!

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Checking the dashboard during the bad rollout

If you didn't deploy the Kubernetes dashboard earlier, just skip this slide.

- Connect to the dashboard that we deployed earlier

- Check that we have failures in Deployments, Pods, and Replica Sets

- Can we see the reason for the failure?

]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Recovering from a bad rollout

- We could push some `v0.3` image

(the pod retry logic will eventually catch it and the rollout will proceed)

- Or we could invoke a manual rollback

- Cancel the deployment and wait for the dust to settle:
  ```bash
  kubectl rollout undo deploy worker
  kubectl rollout status deploy worker
  ```

]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Rolling back to an older version

- We reverted to `v0.2`

- But this version still has a performance problem

- How can we get back to the previous version?

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Multiple "undos"

- What happens if we try `kubectl rollout undo` again?

- Try it:
  ```bash
  kubectl rollout undo deployment worker
  ```

- Check the web UI, the list of pods ...

]

🤔 That didn't work.

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Multiple "undos" don't work

- If we see successive versions as a stack:

- `kubectl rollout undo` doesn't "pop" the last element from the stack

- it copies the N-1th element to the top

- Multiple "undos" just swap back and forth between the last two versions!

- Go back to v0.2 again:
  ```bash
  kubectl rollout undo deployment worker
  ```

]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## In this specific scenario

- Our version numbers are easy to guess

- What if we had used git hashes?

- What if we had changed other parameters in the Pod spec?

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## Listing versions

- We can list successive versions of a Deployment with `kubectl rollout history`

- Look at our successive versions:
  ```bash
  kubectl rollout history deployment worker
  ```

]

We don't see *all* revisions.

We might see something like 1, 4, 5.

(Depending on how many "undos" we did before.)

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Explaining deployment revisions

- These revisions correspond to our Replica Sets

- This information is stored in the Replica Set annotations

- Check the annotations for our replica sets:
  ```bash
  kubectl describe replicasets -l app=worker | grep -A3 ^Annotations
  ```

]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## What about the missing revisions?

- The missing revisions are stored in another annotation:

`deployment.kubernetes.io/revision-history`

- These are not shown in `kubectl rollout history`

- We could easily reconstruct the full list with a script

(if we wanted to!)

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---
class: hide-exercise

## Rolling back to an older version

- `kubectl rollout undo` can work with a revision number

- Roll back to the "known good" deployment version:
  ```bash
  kubectl rollout undo deployment worker --to-revision=1
  ```

- Check the web UI or the list of pods

]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## Changing rollout parameters

- We want to:

- revert to `v0.1`
  - be conservative on availability (always have desired number of available workers)
  - go slow on rollout speed (update only one pod at a time)
  - give some time to our workers to "warm up" before starting more

The corresponding changes can be expressed in the following YAML snippet:

.small[
```yaml
spec:
  template:
    spec:
      containers:
      - name: worker
        image: dockercoins/worker:v0.1
  strategy:
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  minReadySeconds: 10
```
]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

## Applying changes through a YAML patch

- We could use `kubectl edit deployment worker`

- But we could also use `kubectl patch` with the exact YAML shown before

- Apply all our changes and wait for them to take effect:
  ```bash
  kubectl patch deployment worker -p "
    spec:
      template:
        spec:
          containers:
          - name: worker
            image: dockercoins/worker:v0.1
      strategy:
        rollingUpdate:
          maxUnavailable: 0
          maxSurge: 1
      minReadySeconds: 10
    "
  kubectl rollout status deployment worker
  kubectl get deploy -o json worker |
          jq "{name:.metadata.name} + .spec.strategy.rollingUpdate"
  ```
  ]

]

.debug[[k8s/rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/rollout.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/blue-containers.jpg)]

---

Advanced Rollout

.nav[
[Previous section](#toc-rolling-updates)
|
[Back to table of contents](#toc-chapter-3)
|
[Next section](#toc-prometheus)
]

---
# Advanced Rollout

- In some cases the built-in mechanism of kubernetes is not enough.

- You want more control on the rollout, include a feedback of the monitoring, deploying
on multiple clusters, etc

- Two "main" strategies exist here:

- canary deployment
   - blue/green deployment

.debug[[k8s/advanced-rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/advanced-rollout.md)]
---
## Canary deployment

- focus on one component of the stack

- deploy a new version of the component close to the production
    - redirect some portion of prod traffic to new version
    - scale up new version, redirect more traffic, checking everything is ok
    - scale down old version
    - move component to component with the same procedure

- That's what kubernetes does by default, but does every components at the same time

- Could be paired with `kubectl wait --for` and applying component sequentially,
    for hand made canary deployement

.debug[[k8s/advanced-rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/advanced-rollout.md)]
---
## Blue/Green deployment

- focus on entire stack

- deploy a new stack
    - check the new stack work as espected
    - put traffic on new stack, rollback if any goes wrong
    - garbage collect the previous infra structure

- there is nothing like that by default in kubernetes

- helm chart with multiple releases is the closest one

- could be paired with ingress feature like `nginx.ingress.kubernetes.io/canary-*`

.debug[[k8s/advanced-rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/advanced-rollout.md)]
---
## Not hand-made ?

There is a few additionnal controllers that help achieving those kind of rollout behaviours

They leverage kubernetes API at different levels to achieve this goal.

.debug[[k8s/advanced-rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/advanced-rollout.md)]
---
## Spinnaker

- https://www.spinnaker.io

- Help to deploy the same app on multiple cluster.

- Is able to analyse rollout status (canary analysis) and correlate it to monitoring

- Rollback if anything goes wrong

- also support Blue/Green

- Configuration done via UI

.debug[[k8s/advanced-rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/advanced-rollout.md)]
---
## Argo-rollout

- https://github.com/argoproj/argo-rollouts

- Replace your deployments with CRD (Custom Resource Definition) "deployment-like"

- Full control via CRDs

- BlueGreen and Canary deployment

.debug[[k8s/advanced-rollout.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/advanced-rollout.md)]
---
## We are done, what else ?

We have seen what means developping an application on kubernetes.

There still few subjects to tackle that are not purely relevant for developers

They have *some involvement* for developers:

- Monitoring
- Security

.debug[[k8s/devs-and-ops-joined-topics.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/devs-and-ops-joined-topics.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/chinook-helicopter-container.jpg)]

---

Prometheus

.nav[
[Previous section](#toc-advanced-rollout)
|
[Back to table of contents](#toc-chapter-4)
|
[Next section](#toc-opentelemetry)
]

---
# Prometheus

Prometheus is a monitoring system with a small storage I/O footprint.

It's quite ubiquitous in the Kubernetes world.

This section is not an in-depth description of Prometheus.

*Note: More on Prometheus next day!*

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Prometheus exporter

- Prometheus *scrapes* (pulls) metrics from *exporters*

- A Prometheus exporter is an HTTP endpoint serving a response like this one:

```
 # HELP http_requests_total The total number of HTTP requests.
 # TYPE http_requests_total counter
 http_requests_total{method="post",code="200"} 1027 1395066363000
 http_requests_total{method="post",code="400"}    3 1395066363000

# Minimalistic line:
 metric_without_timestamp_and_labels 12.47
```

- Our goal, as a developer, will be to expose such an endpoint to Prometheus

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Implementing a Prometheus exporter

Multiple strategies can be used:

- Implement the exporter in the application itself

(especially if it's already an  HTTP server)

- Use building blocks that may already expose such an endpoint

(puma, uwsgi)

- Add a sidecar exporter that leverages and adapts an existing monitoring channel

(e.g. JMX for Java applications)

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Implementing a Prometheus exporter

- The Prometheus client libraries are often the easiest solution

- They offer multiple ways of integration, including:

- "I'm already running a web server, just add a monitoring route"

- "I don't have a web server (or I want another one), please run one in a thread"

- Client libraries for various languages:

- https://github.com/prometheus/client_python

- https://github.com/prometheus/client_ruby

- https://github.com/prometheus/client_golang

(Can you see the pattern?)

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Adding a sidecar exporter

- There are many exporters available already:

https://prometheus.io/docs/instrumenting/exporters/

- These are "translators" from one monitoring channel to another

- Writing your own is not complicated

(using the client libraries mentioned previously)

- Avoid exposing the internal monitoring channel more than enough

(the app and its sidecars run in the same network namespace,
 so they can communicate over `localhost`)

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Configuring the Prometheus server

- We need to tell the Prometheus server to *scrape* our exporter

- Prometheus has a very flexible "service discovery" mechanism

(to discover and enumerate the targets that it should scrape)

- Depending on how we installed Prometheus, various methods might be available

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Configuring Prometheus, option 1

- Edit `prometheus.conf`

- Always possible

(we should always have a Prometheus configuration file somewhere!)

- Dangerous and error-prone

(if we get it wrong, it is very easy to break Prometheus)

- Hard to maintain

(the file will grow over time, and might accumulate obsolete information)

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Configuring Prometheus, option 2

- Add *annotations* to the pods or services to monitor

- We can do that if Prometheus is installed with the official Helm chart

- Prometheus will detect these annotations and automatically start scraping

- Example:
  ```yaml
    annotations:
      prometheus.io/port: 9090
      prometheus.io/path: /metrics
  ```

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---

## Configuring Prometheus, option 3

- Create a ServiceMonitor custom resource

- We can do that if we are using the CoreOS Prometheus operator

- See the [Prometheus operator documentation](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#servicemonitor) for more details

.debug[[k8s/prometheus-endpoint.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/prometheus-endpoint.md)]
---
## Exercice - monitor with prometheus

Complete exercise #4, (again code at: https://github.com/enix/kubecoin )

*Note: Not all daemon are "ready" for prometheus, only `hasher` and `redis`

.debug[[k8s/exercise-prometheus.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/exercise-prometheus.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/container-cranes.jpg)]

---

OpenTelemetry

.nav[
[Previous section](#toc-prometheus)
|
[Back to table of contents](#toc-chapter-4)
|
[Next section](#toc-security-and-kubernetes)
]

---
# OpenTelemetry

*OpenTelemetry* is a "tracing" framework.

It's a fusion of two other frameworks:
*OpenTracing* and *OpenCensus*.

Its goal is to provide deep integration with programming languages and
application frameworks to enabled deep dive tracing of different events accross different components.

.debug[[k8s/opentelemetry.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/opentelemetry.md)]
---

## Span ! span ! span !

- A unit of tracing is called a *span*

- A span has: a start time, a stop time, and an ID

- It represents an action that took some time to complete

(e.g.: function call, database transaction, REST API call ...)

- A span can have a parent span, and can have multiple child spans

(e.g.: when calling function `B`, sub-calls to `C` and `D` were issued)

- Think of it as a "tree" of calls

.debug[[k8s/opentelemetry.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/opentelemetry.md)]
---

## Distributed tracing

- When two components interact, their spans can be connected together

- Example: microservice `A` sends a REST API call to microservice `B`

- `A` will have a span for the call to `B`

- `B` will have a span for the call from `A`
 (that normally starts shortly after, and finishes shortly before)

- the span of `A` will be the parent of the span of `B`

- they join the same "tree" of calls

details: `A` will send headers (depends of the protocol used) to tag the span ID,
so that `B` can generate child span and joining the same tree of call

.debug[[k8s/opentelemetry.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/opentelemetry.md)]
---

## Centrally stored

- What do we do with all these spans?

- We store them!

- In the previous exemple:

- `A` will send trace information to its local agent
    - `B` will do the same
    - every span will end up in the same DB
    - at a later point, we can reconstruct the "tree" of call and analyze it

- There are multiple implementations of this stack (agent + DB + web UI)

(the most famous open source ones are Zipkin and Jaeger)

.debug[[k8s/opentelemetry.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/opentelemetry.md)]
---

## Data sampling

- Do we store *all* the spans?

(it looks like this could need a lot of storage!)

- No, we can use *sampling*, to reduce storage and network requirements

- Smart sampling is applied directly in the application to save CPU if span is not needed

- It also insures that if a span is marked as sampled, all child span are sampled as well

(so that the tree of call is complete)

.debug[[k8s/opentelemetry.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/opentelemetry.md)]
---
## Exercice - monitor with opentelemetry

Complete exercise #5, (again code at: https://github.com/enix/kubecoin )

*Note: Not all daemon are "ready" for opentelemetry, only `rng` and `worker`

.debug[[k8s/exercise-opentelemetry.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/exercise-opentelemetry.md)]
---

.interstitial[![Image separating from the next chapter](https://gallant-turing-d0d520.netlify.com/containers/container-housing.jpg)]

---

Security and kubernetes

.nav[
[Previous section](#toc-opentelemetry)
|
[Back to table of contents](#toc-chapter-4)
|
[Next section](#toc-)
]

---
# Security and kubernetes

There are many mechanisms in kubernetes to ensure the security.
Obviously the more you constrain your app, the better.

There is also mechanism to forbid "unsafe" application to be launched on
kubernetes, but that's more for ops-guys 😈 (more on that next days)

Let's focus on what can we do on the developer latop, to make app
compatible with secure system, enforced or not (it's always a good practice)

.debug[[k8s/kubernetes-security.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kubernetes-security.md)]
---
## No container in privileged mode

- risks:
   - If one privileged container get compromised,
     we basically get full access to the node from within a container
     (not need to tamper auth logs, alter binary).

- Sniffing networks allow often to get access to the entire cluster.

- how to avoid:
   ```
   [...]
   spec:
     containers:
       - name: foo
         securityContext:
           privileged: false
   ```

Luckily that's the default !

.debug[[k8s/kubernetes-security.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kubernetes-security.md)]
---
## No container run as "root"

- risks:
 - bind mounting a directory like /usr/bin allow to change node system core
 ex: copy a tampered version of "ping", wait for an admin to login
 and to issue a ping command and bingo !

- how to avoid:

```
   [...]
   spec:
     containers:
       - name: foo
         securityContext:
           runAsUser: 1000
           runAsGroup: 100
   ```
   - The default is to use the image default

- If your writing your own Dockerfile, don't forget about the `USER` instruction
.debug[[k8s/kubernetes-security.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kubernetes-security.md)]
---
## Capabilities

- You can give capabilities one-by-one to a container

- It's useful if you need more capabilities (for some reason), but not grating 'root' privileged

- risks: no risks whatsoever, except by granting a big list of capabilities

- how to use:
   ```
   [...]
   spec:
     containers:
       - name: foo
         securityContext:
           capabilities:
             add: ["NET_ADMIN", "SYS_TIME"]
             drop: []
   ```
   The default use the container runtime defaults

- and we can also drop default capabilities granted by the container runtime !

.debug[[k8s/kubernetes-security.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/k8s/kubernetes-security.md)]
---
class: title, self-paced

Thank you!

.debug[[shared/thankyou.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/thankyou.md)]
---

That's all, folks! Questions?

![end](images/end.jpg)

.debug[[shared/thankyou.md](https://github.com/jpetazzo/container.training/tree/2020-02-enix/slides/shared/thankyou.md)]