Vagga

The Plan

• What is vagga?
• Challenges
• Rust + Containers
• Deployment Tools

What is Vagga?

vagga is...

venv/nvm/rvm on Steroids

containers:
  assets:
    setup:
    - !Ubuntu xenial
    - !Install [imagemagick, make]
    - !NpmDependencies "package.json"

commands:
  webpack: !Command
    container: assets
    run: [webpack]
  run: !Command
    container: python
    environ: { FLASK_APP: "myapp.py" }
    run: "python -m flask run"

$ git clone git://github.com/.../foobar
$ cd foobar
$ vagga
Available commands:
    build       Build static files
    run         Run nginx+app+redis
    build-docs  Build docs
$ vagga build

$ git pull
$ vagga run

vagga is...

The Higher Level Package Manager

nginx:
  setup:
  - !Alpine v3.5
  - !Py3Requirements "requirements.txt"
  - !NpmDependencies "package.json"
  - !GemBundle
  - !ComposerDependencies

nginx:
  setup:
  - !Alpine v3.5
  - !Install [nginx]
  - !Build
    container: jsstatic
    source: /var/javascripts
    path: /srv/www

run: !Command
  container: rust
  prerequisites: [make-bin, make-js]
  run: "./target/debug/app"

run: !Supervise
  description: Run full server stack
  children:
    redis: !Command
      container: redis
      run: [redis-server, --daemonize, no]
    nginx: !Command
      container: nginx
      run: [nginx, -c, /work/config/nginx.conf]
    foobar: !Command
      run: [python, -m, foobar]

vagga is...

A Containerization Tool Without Daemons

\-+= vagga run
  |-+= python -m foobar
  |-+= redis-server --daemonize --no
  \-+= nginx -c /work/config/nginx.conf

# docker tree
-+= 00001 root systemd --system
 |-+- 10771 root docker -d
 | \--= 32029 root bash   << our process
 \-+= 30029 pc tmux
   \-+= 10718 pc -zsh     << our shell
     \--= 32021 pc docker run -it --rm bash

# vagga tree
-+= 00001 root systemd --system
 \-+= 30029 pc tmux
   \-+= 10358 pc -zsh        << our shell
     \-+= 00940 pc vagga bash
       \-+- 00941 pc vagga bash
         \--= 00942 pc bash  << our process

Vagga

• simple YAML config (+versioning)
• user namespaces (no root/setuid)
• multiple process monitoring
• only for dev.env.

(written in rust)

Challenges

After Clone

• No memory allocations

After Clone

let x = HashMap::new();
// ...
cmd.before_exec(|| {
    for y in x.iter() {
        // crashes in debug build
        // works in release build
    }
});

Cloexec

• Cloexec by default
• Clean the flag after fork, before exec

PID1

• KILL
• Signals
• Reparenting

(remember tini?)

OS Issues

• Mostly undocumented user namespaces
• Limited error codes

Linux Distro Issues

• Can't install as unprivileged (mknod, audit)
• Large images (containing kernel)
• Occassionally break
• Unreliable mirrors

(the major pain!)

Rust

Unshare Crate

Command::new("sh")
.arg("-c").arg("echo hello")
.status().unwrap()

Command::new("sh")
.arg("-c").arg("echo hello")
.unshare(&[Net, User, Uts, Mount])
.chroot_dir("/container")
.set_id_maps(...)
.status().unwrap()

for event in reap_zombies() {
    match event {
        Death(pid, result) =>
        Stop(..) => {}
        Continue(..) => {}
    }
}

Command::new("postfix")
.allow_daemonize()
// the latter disables
// .set_parent_death_signal(SIGKILL)

Libmount Crate

Tmpfs::new("/tmp")
.size_bytes(1_048_576)
.mount()

let src = "/x";
let dest = "/y";
Bind::new(&src, &dest)
.bare_mount()
.map_err(|e| format!(
    "bind mount {:?} -> {:?}: {}",
    src, dest, e))?

Fatal error: Can't mount bind /x to /y:
    No such file or directory (os error 2)

Bind::new("/x", "/y")
    .mount()?

Fatal error: recursive bind mount "/x" -> "/y":
    No such file or directory (os error 2)
    (source: exists, target: missing, superuser)

To Do

Get rid of busybox:

• tar/unzip
• Use Tokio to download files
• ip
• iptables
• brctl

Deployment

• lithos -- containers
• cantal -- monitoring
• verwalter -- orchestration

All three written in rust

Lithos

• Supervisor with containers support
• Uses unshare + libmount crates
• Super simple, < 5k LoC

Lithos

Written with security in mind

Cantal

Cantal

Decentralized monitoring (metrics)

Cantal

• 2 second interval
• 3k-10k metrics in 10-40ms
• 1h in 15-35MiB
• Debug build

Cantal: Network

• Peer discovery
• Aggregated stats

Verwalter

Cluster-wide Scheduling

Verwalter

Scriptable with Lua

Questions?