- commit
- e01b3567022e5623b7b15824a8560f5db497e5d1
- parent
- c7e82b5b26bf110355e1ed9a63b3177302f6d042
- Author
- Tobias Bengfort <tobias.bengfort@posteo.de>
- Date
- 2022-06-18 00:26
init
Diffstat
| A | PKGBUILD | 11 | +++++++++++ |
| A | README.md | 102 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | d-pull | 69 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | d-run | 99 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
4 files changed, 281 insertions, 0 deletions
diff --git a/PKGBUILD b/PKGBUILD
@@ -0,0 +1,11 @@
-1 1 pkgname='d-utils'
-1 2 pkgver="0.0.0"
-1 3 pkgdesc='utils to run docker images without docker'
-1 4 arch=('all')
-1 5 license='MIT'
-1 6 depends=('curl' 'tar' 'jq' 'bubblewrap' 'python3')
-1 7
-1 8 package() {
-1 9 install -m 755 -D d-pull "$pkgdir/usr/bin/d-pull"
-1 10 install -m 755 -D d-run "$pkgdir/usr/bin/d-run"
-1 11 }
diff --git a/README.md b/README.md
@@ -0,0 +1,102 @@ -1 1 # d-utils -1 2 -1 3 `d-utils` is a set of simple utils to use docker images without docker. -1 4 -1 5 - `d-pull NAME[@TAG] [DIR]` will download a docker image and save it to `DIR`. -1 6 It is saved as a single `rootfs` folder along with `config.json`. -1 7 - `d-run DIR [CMD]` will execute `CMD` in the container give by `DIR`. Volumes -1 8 are created in `DIR/volumes/`. -1 9 -1 10 # Config -1 11 -1 12 This is the full list of values from docker's `config.json` that are actually -1 13 used by `d-run`: -1 14 -1 15 - `Hostname` -1 16 - `WorkingDir` -1 17 - `Env` -1 18 - `Volumes` -1 19 - `Entrypoint` -1 20 - `Cmd` -1 21 -1 22 `d-run` uses the following additional values: -1 23 -1 24 - `net` (bool) - enable networking (default: false) -1 25 - `fakeroot` (bool) - map UID and GID to 0 (default: false) -1 26 -1 27 You are encouraged to modify this file, e.g. to add a volume or change the -1 28 default command. -1 29 -1 30 You can also modify the rootfs, both from a running container and from the host -1 31 system. If you need a new container based on the same image you can just run -1 32 `d-pull` again. The layers are cached in `~/.cache/d-utils/` for 30 days. -1 33 -1 34 # Motivation -1 35 -1 36 > That (Linux) Containers are a userspace fiction is a well-known dictum -1 37 > nowadays. […] This is achieved by combining a multitude of Linux kernel -1 38 > features. -1 39 > -- [Christian Brauner](https://people.kernel.org/brauner/the-seccomp-notifier-new-frontiers-in-unprivileged-container-development) -1 40 -1 41 I (think I) can remember when cgroups and namespaces were added to linux. Back -1 42 then they were announced as low level features that were not supposed to be -1 43 used directly, but that could enable exciting new high-level tools. -1 44 -1 45 And boy did that make waves. Nowadays flatpak and snap use these features to -1 46 containerize desktop applications, systemd uses them to isolate system -1 47 services, and docker seems to have thoroughly conquered deployment. -1 48 -1 49 My trouble is: None of these tools feel like they have nailed the "high level" -1 50 aspect of this. `systemd-analyze security` for example lists 80 (!) different -1 51 settings. The user experience is just not that great. -1 52 -1 53 Docker is the worst offender in my opinion. I still have to check the docs -1 54 every time I want to start a container, but worse than that it hides the -1 55 images, containers, and volumes in some impenetrable folder structure. On top -1 56 of that, it executes all containers as root which is a massive security issue. -1 57 -1 58 So let's start from scratch. -1 59 -1 60 As far as I understand, containerization has two goals: Bundle an application -1 61 with libraries and configuration so it becomes self-contained, and then isolate -1 62 the whole thing so it cannot mess up the host system. -1 63 -1 64 To run such a container you would basically just need a chroot, ideally -1 65 unprivileged. Namespaces can then help to further isolate the container, which -1 66 is good but not essential. `bwrap` (also used in flatpak) provides all of -1 67 that and actually has good UX, so we are up to a promising start. -1 68 -1 69 But then you also need the container itself. And this is where docker makes a -1 70 comeback: The ideas of images, containers, layers, volumes as well as -1 71 Dockerfiles and an online registry are seriously great and probably a big part -1 72 why docker blew up. -1 73 -1 74 So with this project I tried to combine docker images with bwrap. The guiding -1 75 principles are: -1 76 -1 77 - Simple is better than complex -1 78 - Less than 1000 lines of code -1 79 - Completeness can be sacrificed in favor of simplicity -1 80 ([worse is better](https://www.jwz.org/doc/worse-is-better.html)) -1 81 - Use established tools for the complicated bits -1 82 - The filesystem is stored in a single folder, no layerfs/aufs/overlayfs -1 83 - Linux only -1 84 - Explicit is better than implicit -1 85 - Containers have a simple folder structure -1 86 - Users can identify containers by their (manually chosen) path -1 87 - Everything is unprivileged -1 88 -1 89 # Limitations -1 90 -1 91 - This approach will use more disk space because it does not share -1 92 layers/images between containers. -1 93 - It is currently not possible to do any network configuration (e.g. map ports) -1 94 other than sharing network or not sharing network. -1 95 - Some tools are known to cause issues when not running as root: -1 96 - dpkg ([workaround](https://github.com/opencontainers/runc/issues/2517#issuecomment-1030859646)) -1 97 -1 98 # Prior Art -1 99 -1 100 - https://github.com/containers/bubblewrap -1 101 - https://github.com/NotGlop/docker-drag -1 102 - https://github.com/twosigma/debootwrap
diff --git a/d-pull b/d-pull
@@ -0,0 +1,69 @@
-1 1 #!/bin/sh -e
-1 2
-1 3 REGISTRY='https://registry-1.docker.io'
-1 4
-1 5 CACHE_DIR="$HOME/.cache/d-utils/"
-1 6 mkdir -p "$CACHE_DIR"
-1 7
-1 8 if [ $# -lt 1 -o $1 = '-h' ]; then
-1 9 echo "usage: d-pull [-h] name[@tag] [dir]"
-1 10 exit 1
-1 11 fi
-1 12
-1 13 if echo "$1" | grep -q '@'; then
-1 14 tag=$(echo "$1" | cut -f2 -d@)
-1 15 img=$(echo "$1" | cut -f1 -d@)
-1 16 else
-1 17 img=$1
-1 18 tag='latest'
-1 19 fi
-1 20 if [ $# -gt 1 ]; then
-1 21 dir=$(realpath -m "$2")
-1 22 else
-1 23 dir=$(realpath -m "$1")
-1 24 fi
-1 25 if echo "$img" | grep -q -v '/'; then
-1 26 img="library/$img"
-1 27 fi
-1 28
-1 29 if [ -e "$dir" ]; then
-1 30 echo "$dir already exists"
-1 31 exit 1
-1 32 fi
-1 33
-1 34 echo "pulling $img@$tag to $dir"
-1 35
-1 36 echo " fetching token"
-1 37 auth_url=$(curl -s -I "$REGISTRY/v2/" | grep -i www-authenticate | sed 's/.*realm="\(.*\)",service="\(.*\)".*/\1?service=\2/')
-1 38 auth_token=$(curl -s "$auth_url&scope=repository:$img:pull" | jq -r '.token')
-1 39 auth="Authorization: Bearer $auth_token"
-1 40
-1 41 # fail if server reports error
-1 42 curl --head -f --no-progress-meter -o /dev/null -H "$auth" "$REGISTRY/v2/$img/manifests/$tag"
-1 43
-1 44 mkdir -p "$dir/rootfs"
-1 45 echo "$img@$tag" > "$dir/image.txt"
-1 46
-1 47 echo " fetching manifest"
-1 48 curl -s -H "$auth" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" "$REGISTRY/v2/$img/manifests/$tag" -o "$dir/manifest.json"
-1 49
-1 50 echo " fetching config"
-1 51 config=$(jq -r '.config.digest' "$dir/manifest.json")
-1 52 curl -s -L -H "$auth" "$REGISTRY/v2/$img/blobs/$config" | jq '.config' > "$dir/config.json"
-1 53
-1 54 echo " fetching layers"
-1 55 jq -r '.layers|map(.digest)|.[]' "$dir/manifest.json" | while read -r blob; do
-1 56 echo " $blob"
-1 57 if [ -e "$CACHE_DIR/$blob" ]; then
-1 58 touch "$CACHE_DIR/$blob"
-1 59 else
-1 60 curl -s -L -H "$auth" "$REGISTRY/v2/$img/blobs/$blob" -o "$CACHE_DIR/$blob"
-1 61 fi
-1 62 tar -C "$dir/rootfs" -xf "$CACHE_DIR/$blob"
-1 63 done
-1 64
-1 65 echo " cleanup"
-1 66 rm "$dir/manifest.json"
-1 67 find "$CACHE_DIR" -type f -mtime +30 -exec rm -f {} +
-1 68
-1 69 echo "successfully pulled $img@$tag to $dir"
diff --git a/d-run b/d-run
@@ -0,0 +1,99 @@
-1 1 #!/bin/env python3
-1 2
-1 3 import os
-1 4 import json
-1 5 import argparse
-1 6
-1 7
-1 8 def make_volume(path, dir):
-1 9 if ':' in path:
-1 10 hostpath, path = path.split(':', 1)
-1 11 else:
-1 12 hostpath = os.path.join(dir, 'volumes', path.lstrip('/'))
-1 13 os.makedirs(hostpath, exist_ok=True)
-1 14
-1 15 if ':' in path:
-1 16 path, option = path.rsplit(':', 1)
-1 17 else:
-1 18 option = 'rw'
-1 19
-1 20 op = '--ro-bind' if option == 'ro' else '--bind'
-1 21
-1 22 return [op, hostpath, path]
-1 23
-1 24
-1 25 def build_cmd(dir, config):
-1 26 cmd = [
-1 27 'bwrap',
-1 28 '--bind', os.path.join(dir, 'rootfs'), '/',
-1 29 '--tmpfs', '/tmp',
-1 30 '--dev', '/dev',
-1 31 '--proc', '/proc',
-1 32 # '--clear-env', # bwrap >= 0.5
-1 33 '--unshare-all',
-1 34 '--die-with-parent',
-1 35 ]
-1 36
-1 37 if config.get('Hostname'):
-1 38 cmd += ['--hostname', config['Hostname']]
-1 39 if config.get('WorkingDir'):
-1 40 cmd += ['--chdir', config['WorkingDir']]
-1 41
-1 42 for entry in config['Env']:
-1 43 key, value = entry.split('=', 1)
-1 44 cmd += ['--setenv', key, value]
-1 45
-1 46 if config.get('Volumes'):
-1 47 for volume in config['Volumes']:
-1 48 cmd += make_volume(volume, dir)
-1 49
-1 50 if config.get('net'):
-1 51 cmd += [
-1 52 '--ro-bind', '/etc/resolv.conf', '/etc/resolv.conf',
-1 53 '--share-net',
-1 54 ]
-1 55
-1 56 if config.get('fakeroot'):
-1 57 cmd += [
-1 58 '--uid', '0',
-1 59 '--gid', '0',
-1 60 ]
-1 61
-1 62 if config.get('Entrypoint'):
-1 63 cmd += config['Entrypoint']
-1 64
-1 65 cmd += config['Cmd']
-1 66
-1 67 return cmd
-1 68
-1 69
-1 70 def parse_args():
-1 71 parser = argparse.ArgumentParser()
-1 72 parser.add_argument('dir')
-1 73 parser.add_argument('-v', '--volume', action='append')
-1 74 parser.add_argument('-n', '--net', action='store_true')
-1 75 parser.add_argument('-r', '--fakeroot', action='store_true')
-1 76 parser.add_argument('cmd', nargs='*')
-1 77 return parser.parse_args()
-1 78
-1 79
-1 80 if __name__ == '__main__':
-1 81 args = parse_args()
-1 82
-1 83 with open(os.path.join(args.dir, 'config.json')) as fh:
-1 84 config = json.load(fh)
-1 85
-1 86 if args.cmd:
-1 87 config['Cmd'] = args.cmd
-1 88 if args.net:
-1 89 config['net'] = True
-1 90 if args.fakeroot:
-1 91 config['fakeroot'] = True
-1 92 if args.volume:
-1 93 if not config.get('Volumes'):
-1 94 config['Volumes'] = {}
-1 95 for volume in args.volume:
-1 96 config['Volumes'][volume] = {}
-1 97
-1 98 cmd = build_cmd(args.dir, config)
-1 99 os.execvp(cmd[0], cmd)