SlideShare a Scribd company logo

Universal Userland

•
Sean Chittenden, profile picture
Sean Chittenden

The document discusses the concept of operating systems (OS), focusing on various aspects such as the definitions of a kernel and userland, and their relevance across different contexts like embedded systems and hosting providers. It introduces tools like Packer for bootstrapping images and explores the ideological underpinnings of OS development and community dynamics. Additionally, it presents a series of thoughts on progress in software engineering, usability, and performance, highlighting the importance of a collaborative approach to development.

Internet
1 of 55
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
Universal Userland Why do we need to be different just like everyone else? Engineering seanc@joyent.com Sean Chittenden @SeanChittenden
Getting Started • Packer - packer.io • Image building utility for Vagrant (VMware, Virtualbox, bhyve), AWS, Triton, GCE, Azure • An appreciation for version controlled software • Pick your poison: git, hg, fossil 2
Agenda 1. Why Part 1 2. Bootstrap an image using Packer 3. Iterate using go-coreutils 4. Why Part 2 5. Iterate using rust-coreutils 6. Parting Thoughts 3
Why? Part 1: What is an OS? • Is it a kernel? • Is it a userland? • Is it the applications? • Is it a release? • Is it a distribution? • Is it an ethos and shared set of values? • Is it a community? 4
What is an OS, Kernel Edition • CDNs? Yes. "The network stack is everything." • Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door." • Embedded systems? Yes. "Userland is never installed." • Hosting Providers? Yes. "Application containment." • Network Security Deployments? Yes. "Hardened network stack." • Users? If it's stable, "what's a kernel?" • Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged." • Developers? "That's why we're here (or to support the kernel)." • Management? Maybe. 5
What is an OS, Kernel Edition 6 • CDNs? Yes. "The network stack is everything." • Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door." • Embedded systems? Yes. "Userland is never installed." • Hosting Providers? Yes. "Application containment." • Network Security Deployments? Yes. "Hardened network stack." • Users? If it's stable, "what's a kernel?" • Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged." • Developers? "That's why we're here (or to support the kernel)." • Management? Maybe. YES
What is an OS, Userland Edition 7 • CDNs? Partially. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? Yes. "Familiarity for customers is important." • Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code." • Users? "I like my ls(1) and top(1)*." * if [ "`uname -s`" = "SunOS" ]; then alias top="prstat"; fi
What is an OS, Userland Edition 8 • CDNs? Partially. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? Yes. "Familiarity for customers is important." • Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code." • Users? "I like my ls(1) and top(1)." • Operators? Yes. "This is how I deploy, maintain, investigate, and manipulate the kernel." • Developers? It depends on who you ask. "Team effort!" • Management? "Productivity? Effectiveness?"
Solving science with an Oracle 9
Solving science with an Oracle 10 Cloudy with a chance of Linux admins. Management Human Resources Recruiting College Grads
Solving science with an Oracle 11
What is an OS, Applications Edition 12 • CDNs? Partially. "We use nginx and/or varnish, so yeah." • Appliance vendors? No. "Everything that matters is BSDL or proprietary already." • Embedded systems? No. "What's a userland application?" • Hosting Providers? Yes. "Customers can't be bothered to build 3rd party apps." • Network Security Deployments? Yes. "Wireshark is an app, but tcpdump is kernel, right?" • Users? "Give me X11, emacs/vi, and a webserver. Or give me death." • Operators? Yes. This is how I deploy, maintain, investigate, and manipulate the kernel. • Developers? "Well duh, how else would someone install Chromium?" • Management? "Productivity? Effectiveness?"
Solving science with an Oracle 13
What is an OS, Release Engineering Edition 14 • Is an OS the release and stability guarantee? • CDNs? No. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? Yes. "IPU and upgrades are important: every customer is a snowflake." • Network Security Deployments? No. • Users? Yes. "IPU and upgrades on my laptop is important." • Operators? Depends: legacy thinking, yes. Modern: not really. • Developers? "We need to protect our users from themselves, so yes." • Management? "Productivity?"
What is an OS, Distribution Edition 15 • Is an OS the distribution? • CDNs? No. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? No. • Network Security Deployments? No. • Users? Yes. "IPU and upgrades on my laptop is important." • Operators? Depends: legacy thinking, yes. Modern: not really. • Developers? "We need to protect our users from themselves, so yes." • Management? "We don't have to do that work anymore, right?"
Bootstrap an Image 16 $ git clone -q https://github.com/sean-/BSDCan-2017.git $ cd BSDCan-2017 $ make install-cfgt go get -u github.com/sean-/cfgt $ which packer envchain /Users/seanc/go/bin/packer /opt/local/bin/envchain $ make packer-build TEMPLATE=freebsd-userland-go.json EXTRA_ARGS=-on-error=abort cfgt -i freebsd-userland-go.json5 -o freebsd-userland-go.json envchain tf_triton_test packer build -var "image_name=my-freebsd-image" -var "image_version=1.0.0" -on-error=abort freebsd-userland-go.json triton output will be in this color. ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Waiting for SSH to become available... ==> triton: Connected to SSH! ==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363 triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
Bootstrap an Image 17 triton output will be in this color. ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Waiting for SSH to become available... ==> triton: Connected to SSH! ==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363 triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 triton: Checking integrity... done (0 conflicting) [snip]
Bootstrap an Image 18 triton: ------------------------------------------------------------------------ triton: install: /bin/cat -> /bin/cat.base triton: install: /root/go/bin/cat -> /bin/cat triton: install: /bin/date -> /bin/date.base triton: install: /root/go/bin/date -> /bin/date triton: install: /bin/echo -> /bin/echo.base triton: install: /root/go/bin/echo -> /bin/echo triton: install: /bin/expr -> /bin/expr.base triton: install: /root/go/bin/expr -> /bin/expr triton: install: /bin/mkdir -> /bin/mkdir.base triton: install: /root/go/bin/mkdir -> /bin/mkdir triton: install: /bin/mv -> /bin/mv.base triton: install: /root/go/bin/mv -> /bin/mv triton: install: /bin/pwd -> /bin/pwd.base triton: install: /root/go/bin/pwd -> /bin/pwd triton: install: /bin/rm -> /bin/rm.base triton: install: /root/go/bin/rm -> /bin/rm triton: install: /bin/rmdir -> /bin/rmdir.base triton: install: /root/go/bin/rmdir -> /bin/rmdir triton: install: /bin/sleep -> /bin/sleep.base triton: install: /root/go/bin/sleep -> /bin/sleep triton: install: /bin/sync -> /bin/sync.base triton: install: /root/go/bin/sync -> /bin/sync triton: install: /sbin/md5 -> /sbin/md5.base triton: install: /root/go/bin/md5sum -> /sbin/md5 triton: install: /sbin/sha1 -> /sbin/sha1.base
Bootstrap an Image 19 triton: install: /sbin/sha1 -> /sbin/sha1.base triton: install: /root/go/bin/sha1sum -> /sbin/sha1 triton: install: /root/go/bin/sha224sum -> /sbin/sha224 triton: install: /sbin/sha256 -> /sbin/sha256.base triton: install: /root/go/bin/sha256sum -> /sbin/sha256 triton: install: /sbin/sha384 -> /sbin/sha384.base triton: install: /root/go/bin/sha384sum -> /sbin/sha384 triton: install: /sbin/sha512 -> /sbin/sha512.base triton: install: /root/go/bin/sha512sum -> /sbin/sha512 triton: install: /usr/bin/basename -> /usr/bin/basename.base triton: install: /root/go/bin/basename -> /usr/bin/basename triton: install: /usr/bin/dirname -> /usr/bin/dirname.base triton: install: /root/go/bin/dirname -> /usr/bin/dirname triton: install: /usr/bin/env -> /usr/bin/env.base triton: install: /root/go/bin/env -> /usr/bin/env triton: install: /usr/bin/factor -> /usr/bin/factor.base triton: install: /root/go/bin/factor -> /usr/bin/factor triton: install: /usr/bin/false -> /usr/bin/false.base triton: install: /root/go/bin/false -> /usr/bin/false triton: install: /usr/bin/groups -> /usr/bin/groups.base triton: install: /root/go/bin/groups -> /usr/bin/groups triton: install: /usr/bin/head -> /usr/bin/head.base triton: install: /root/go/bin/head -> /usr/bin/head triton: install: /usr/bin/logname -> /usr/bin/logname.base triton: install: /root/go/bin/logname -> /usr/bin/logname triton: install: /usr/bin/tail -> /usr/bin/tail.base
Bootstrap an Image 20 triton: install: /root/go/bin/tail -> /usr/bin/tail triton: install: /usr/bin/tee -> /usr/bin/tee.base triton: install: /root/go/bin/tee -> /usr/bin/tee triton: install: /usr/bin/touch -> /usr/bin/touch.base triton: install: /root/go/bin/touch -> /usr/bin/touch triton: install: /usr/bin/true -> /usr/bin/true.base triton: install: /root/go/bin/true -> /usr/bin/true triton: install: /usr/bin/tsort -> /usr/bin/tsort.base triton: install: /root/go/bin/tsort -> /usr/bin/tsort triton: install: /usr/bin/wc -> /usr/bin/wc.base triton: install: /root/go/bin/wc -> /usr/bin/wc triton: install: /usr/bin/whoami -> /usr/bin/whoami.base triton: install: /root/go/bin/whoami -> /usr/bin/whoami triton: install: /usr/bin/yes -> /usr/bin/yes.base triton: install: /root/go/bin/yes -> /usr/bin/yes
go-coreutils: Initial thoughts? 21 triton: triton: Rebuilding locate database: triton: flag provided but not defined: -128 triton: Usage of head: triton: -bytes int triton: output the first K bytes triton: -c int triton: output the first K bytes triton: -help triton: triton: Usage: head [OPTION]... [FILE]... triton: triton: Print the first 10 lines of each FILE to standard output. With more than one FILE, precede each with a header giving the file name. With no FILE, or when FILE is -, read standard input. triton: triton: triton: Mandatory arguments to long options are mandatory for short options too. triton: triton: -help display this help and exit triton: -version output version information and exit triton: triton: -c, --bytes=K triton: output the first K bytes; or use -n +K to output starting with the Kth byte.
go-coreutils: Initial thoughts? 22 triton: -n, -lines=K triton: output the first K lines; or use -n +K to output starting with triton: the Kth triton: triton: -q, -quiet, -silent triton: never output headers giving file names triton: triton: -lines int triton: output the first K lines (default 10) triton: -n int triton: output the first K lines (default 10) triton: -qnever output headers giving file names triton: -quiet triton: never output headers giving file names triton: -silent triton: never output headers giving file names triton: -version triton: triton: head (go-coreutils) 0.1 triton: triton: Copyright (C) 2014, The GO-Coreutils Developers. triton: This program comes with ABSOLUTELY NO WARRANTY; for deprintTails see triton: LICENSE. This is free software, and you are welcome to redistribute triton: it under certain conditions in LICENSE. triton:
go-coreutils: Initial thoughts? 23 • Needs to bake more • Probably not going anywhere • GPLv3ed code • Suspicion: No commercial incentive or expectation of quality
Why? Part 2: What is Progress? 24
Science: A lens through which we learn • Whereas research has shown that to a considerable extent children learn by imitating the behavior of adults, especially those they are dependent upon; and the use of corporal punishment by adults having authority over children is likely to train children to use physical violence to control behavior rather than rational persuasion, education, and intelligent forms of both positive and negative reinforcement; • Whereas research has shown that the effective use of punishment in eliminating undesirable behavior requires precision in timing, duration, intensity, and specificity, as well as considerable sophistication in controlling a variety of relevant environmental and cognitive factors, such that punishment administered in institutional settings, without attention to all these factors, is likely to instill hostility, rage, and a sense of powerlessness without reducing the undesirable behavior; • Therefore, be it resolved that the American Psychological Association opposes the use of corporal punishment in schools, juvenile facilities, child care nurseries, and all other institutions, public or private, where children are cared for or educated (Conger, 1975). American Psychological Association
 http://www.apa.org/about/policy/corporal-punishment.aspx 25
Software? 26
Software Performance? Performance? Find regression, fix regression. https://github.com/freebsd/freebsd/commit/89c59fbb17d89068626d6214477d40c81d43a025 27
Software Usability? 28 $ echo baz | tee foo > bar $ echo foo >> foo $ echo bar >> bar $ diff foo bar 1c1 < foo --- > bar $ date Sat May 10 15:49:02 UTC 2017 $ diff -u foo bar --- foo2017-06-10 15:48:47.261401000 +0000 +++ bar2017-06-10 15:48:51.157525000 +0000 @@ -1,2 +1,2 @@ baz -foo +bar
Could we have learned something? 29
Software Usability? 30 The UNIX system family tree: Research and BSD --------------------------------------------- First Edition (V1) | Second Edition (V2) | Third Edition (V3) | Fourth Edition (V4) | Fifth Edition (V5) | Sixth Edition (V6) -----* | | | Seventh Edition (V7) | | 1BSD 32V | 2BSD---------------* / | / | / | 3BSD | | | 4.0BSD 2.79BSD | | 4.1BSD --------------> 2.8BSD | | 4.1aBSD ----------- | | | 4.1bBSD | | | *------ 4.1cBSD --------------> 2.9BSD
Software Usability? 31 / | | Eighth Edition | 2.9BSD-Seismo | | | +----<--- 4.2BSD 2.9.1BSD | | | +----<--- 4.3BSD -------------> 2.10BSD | | / | Ninth Edition | / 2.10.1BSD | 4.3BSD Tahoe-----+ | | | | | | | v | 2.11BSD Tenth Edition | | | 2.11BSD rev #430 4.3BSD NET/1 | | v 4.3BSD Reno | *---------- 4.3BSD NET/2 -------------------+-------------* | | | | 386BSD 0.0 | | BSD/386 ALPHA | | | | 386BSD 0.1 ------------>+ | BSD/386 0.3.[13] | | 4.4BSD Alpha | | 386BSD 1.0 | | BSD/386 0.9.[34] | | 4.4BSD | | | / | | | | 4.4BSD-Encumbered | | | -NetBSD 0.8 | BSD/386 1.0 | / | | | FreeBSD 1.0 <-----' NetBSD 0.9 | BSD/386 1.1 | | .----- 4.4BSD Lite | FreeBSD 1.1 | / / | | | | / / | | FreeBSD 1.1.5 .---|--------' / | | | / | / | |
Software Usability? 32 FreeBSD 1.1.5.1 / | / | | | / NetBSD 1.0 <-' | | | / | | | FreeBSD 2.0 <--' | | BSD/OS 2.0 | | | FreeBSD 2.0.5 | BSD/OS 2.0.1 | .------------------ 4.4BSD Lite2 | | | | | | | | | | .-----|------Rhapsody | | | | | | | | NetBSD 1.3 | | | | | | | OpenBSD 2.3 | | | | | | BSD/OS 3.0 | FreeBSD 2.1 | | | | | | | | NetBSD 1.1 ------. BSD/OS 2.1 | FreeBSD 2.1.5 | | | | | | | | NetBSD 1.2 BSD/OS 3.0 | FreeBSD 2.1.6 | | | OpenBSD 2.0 | | | | | | | | | FreeBSD 2.1.6.1 | | | | | | | | | | | | | FreeBSD 2.1.7 | | | | | | | | | | | NetBSD 1.2.1 | | | FreeBSD 2.1.7.1 | | | | | | | | | | | | | | | | | *-FreeBSD 2.2 | | | | | | | | | | | | FreeBSD 2.2.1 | | | | | | | | | | | | | FreeBSD 2.2.2 | | | OpenBSD 2.1 | | | | | | | | | FreeBSD 2.2.5 | | | | | | | | | | OpenBSD 2.2 | | | | | NetBSD 1.3 | | | FreeBSD 2.2.6 | | | | | | | | | | | NetBSD 1.3.1 | BSD/OS 3.1
Software Usability? 33 | | | | | | OpenBSD 2.3 | | | | | | NetBSD 1.3.2 | | | FreeBSD 2.2.7 | | | | | | | | | | | | | BSD/OS 4.0 | FreeBSD 2.2.8 | | | | | | | | | | | | | | | v | | | | OpenBSD 2.4 | | FreeBSD 2.2.9 | | | | | | | | | | | | | FreeBSD 3.0 <--------* | | v | | | | | NetBSD 1.3.3 | | *---FreeBSD 3.1 | | | | | | | | | BSD/OS 4.0.1 | FreeBSD 3.2----* | NetBSD 1.4 OpenBSD 2.5 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FreeBSD 3.3 | | | | NetBSD 1.4.1 | | | | | | | | | OpenBSD 2.6 | | FreeBSD 3.4 | | | | | | | | | | | | | | | BSD/OS 4.1 FreeBSD 4.0 | | | | | NetBSD 1.4.2 | | | | | | | | | | | | | | | | | | | | | FreeBSD 3.5 | | | | | OpenBSD 2.7 | | | | | | | | | | | FreeBSD 3.5.1 | | | | | | | | | | | | | | | *---FreeBSD 4.1 | | | | | | | | | | | (?) | | | | | FreeBSD 4.1.1 | | / | | | | | | | | / | | | | | FreeBSD 4.2 Darwin/ | NetBSD 1.4.3 | | | | Mac OS X | OpenBSD 2.8 BSD/OS 4.2 | | | | | | | | | | | |
Software Usability? 34 | | 10.0 NetBSD 1.5 | | | FreeBSD 4.3 | | | | | | | | | | OpenBSD 2.9 | | | | | NetBSD 1.5.1 | | | | | | | | | | FreeBSD 4.4-. | | NetBSD 1.5.2 | | | | | Mac OS X | | | | | | | 10.1 | | OpenBSD 3.0 | | FreeBSD 4.5 | | | | | | | | | | | | BSD/OS 4.3 | FreeBSD 4.6 | | | OpenBSD 3.1 | | | | | NetBSD 1.5.3 | | | FreeBSD 4.6.2 Mac OS X | | | | | 10.2 | | | | FreeBSD 4.7 | | | | | | | NetBSD 1.6 OpenBSD 3.2 | | FreeBSD 4.8 | | | | | | | | | NetBSD 1.6.1 | | | |--------. | | | OpenBSD 3.3 BSD/OS 5.0 | | | | | | | | FreeBSD 4.9 | | | | OpenBSD 3.4 BSD/OS 5.1 ISE | | | | | | | | | | | | NetBSD 1.6.2 | | | | | | | | | | | | | | OpenBSD 3.5 | | | | | v | | FreeBSD 4.10 | | | | | | | | | | | FreeBSD 4.11 | | | | | | | | | | `-|------|-----------------|---------------------. | | | | FreeBSD 5.0 | | | | | | | | | FreeBSD 5.1 | | | DragonFly 1.0 | | | | |
Software Usability? 35 | ----- Mac OS X | | | | 10.3 | | | FreeBSD 5.2 | | | | | | | | | | | FreeBSD 5.2.1 | | | | | | | | | *-------FreeBSD 5.3 | | | | | | | | OpenBSD 3.6 | | | | NetBSD 2.0 | | | | | | | | | DragonFly 1.2.0 | | Mac OS X | | NetBSD 2.0.2 | | | | 10.4 | | | | | | FreeBSD 5.4 | | | | | | | | | | | | OpenBSD 3.7 | | | | | | NetBSD 2.0.3 | | | | | | | | | | *--FreeBSD | | | | v OpenBSD 3.8 | | 6.0 | | | | | | | | | | | | | | | | | | NetBSD 2.1 | | | | | | | | | | | | | NetBSD 3.0 | | | | | | | | | | DragonFly 1.4.0 | | | | | | | OpenBSD 3.9 | | FreeBSD | | | | | | | | 6.1 | | | | | | | | | FreeBSD 5.5 | | | | | | | | | | | NetBSD 3.0.1 | DragonFly 1.6.0 | | | | | | | | | | | | | | OpenBSD 4.0 | | | | | | NetBSD 3.0.2 | | | | | | NetBSD 3.1 | | | FreeBSD 6.2 | | | | | | | | | DragonFly 1.8.0 | | | | OpenBSD 4.1 | | | | | | DragonFly 1.10.0
Software Usability? 36 | | Mac OS X | | | | | 10.5 | | | | | | | OpenBSD 4.2 | | | | NetBSD 4.0 | | | FreeBSD 6.3 | | | | | | | | | | | *--FreeBSD | | | | | DragonFly 1.12.0 | 7.0 | | | | | | | | | | | | OpenBSD 4.3 | | | | | | NetBSD | DragonFly 2.0.0 | | FreeBSD | | 4.0.1 OpenBSD 4.4 | | | 6.4 | | | | | | | | | | | FreeBSD 7.1 | | | | | | | | | DragonFly 2.2.0 | FreeBSD 7.2 | NetBSD 5.0 OpenBSD 4.5 | | | | | | | | | Mac OS X | | | | | | 10.6 | | | | | | | | | NetBSD | DragonFly 2.4.0 | | | | | 5.0.1 OpenBSD 4.6 | | | | | | | | | *--FreeBSD | | | | | | | | 8.0 | | | | | | | | | FreeBSD | | | NetBSD | | | | 7.3 | | | 5.0.2 | DragonFly 2.6.0 | | | | | | OpenBSD 4.7 | | FreeBSD | | | | | | | 8.1 | | | | | | | | | | | | | DragonFly 2.8.2 | | | | | | OpenBSD 4.8 | | | | | | *--NetBSD | | | FreeBSD FreeBSD | | | 5.1 | | | 8.2 7.4 | | | | | DragonFly 2.10.1 | | | | | | OpenBSD 4.9 | | `-----. Mac OS X | | | | |
Software Usability? 37 | 10.7 | | | | | | | | | | | OpenBSD 5.0 | *--FreeBSD | | | | | | | | 9.0 | | | | NetBSD | DragonFly 3.0.1 | | FreeBSD | | | 5.1.2 | | | | 8.3 | | | | | | | | | | | | NetBSD | | | | | | | | 5.1.3 | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 5.1.4 | | | | | | | | OpenBSD 5.1 | | | | Mac OS X | `----. | | | | | 10.8 | | | | | | | NetBSD 6.0 | | | | | | | | | | | OpenBSD 5.2 DragonFly 3.2.1 | FreeBSD | | | | | NetBSD | | | 9.1 | | | | | 5.2 | | | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | | 5.2.1 | | | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | | 5.2.2 | | | | | | | | | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 6.0.1 | | | | | | | | | OpenBSD 5.3 DragonFly 3.4.1 | | | | | | NetBSD | | | | | | | | 6.0.2 | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 6.0.3 | | | | | | | | | | | | | | | | | NetBSD | |
Software Usability? 38 | | | | | | 6.0.4 | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 6.0.5 | | | | | | | | | | | | | | | |`-NetBSD 6.1 | | | | FreeBSD | | | | | | | 8.4 | | NetBSD 6.1.1 | | | | | | | | | | FreeBSD | | NetBSD 6.1.2 | | | 9.2 Mac OS X | | | | | | 10.9 | | OpenBSD 5.4 | | `-----. | | | | DragonFly 3.6.0 | | | | | | *--FreeBSD | | | NetBSD 6.1.3 | | | 10.0 | | | | | | | | | | | | | DragonFly 3.6.1 | | | | | | | | | | | | | | | | | | | | | | | DragonFly 3.6.2 | | | | | NetBSD 6.1.4 | | | | | | | | | | | | | | | | OpenBSD 5.5 | | | | | | | | | | | | | | | | DragonFly 3.8.0 | | | | | | | | | | | | | | | | | | | | | | | DragonFly 3.8.1 | | | | | | | | | | | | | | | | | | | | | | | DragonFly 3.6.3 | | | | | | | | | | FreeBSD | | | | | | | 9.3 | | | | | | | | | NetBSD 6.1.5 | DragonFly 3.8.2 | | Mac OS X | | |
Software Usability? 39 | | 10.10 | | | | | | | OpenBSD 5.6 | | FreeBSD | | | | | 10.1 | | | DragonFly 4.0.1 | | | | | | | | | | | DragonFly 4.0.2 | | | | | | | | | | | DragonFly 4.0.3 | | | | | | | | | | | DragonFly 4.0.4 | | | | | | | | | | | DragonFly 4.0.5 | | | | | | | | | | OpenBSD 5.7 | | | | | | DragonFly 4.2.0 | FreeBSD | | | | | 10.2 | | | | | | OS X NetBSD 7.0 | | | | 10.11 | OpenBSD 5.8 | | | | | | DragonFly 4.4.1 | FreeBSD | | OpenBSD 5.9 | | 10.3 | | | | | | NetBSD 7.0.1 | | | | | | DragonFly 4.6.0 | | | | | | | | | | *--FreeBSD macOS | OpenBSD 6.0 | | 11.0 10.12 | | | | | NetBSD 7.1 | | | | | | DragonFly 4.8.0 | | | OpenBSD 6.1 | | | | | | FreeBSD 12 -current | NetBSD -current OpenBSD -current DragonFly -current | | | | | v v v v v
Software Usability? 40 Time ---------------- Time tolerance +/- 6 months, depending on which book/article you read; if it was the announcement in Usenet or if it was available as tape. [44B] McKusick, Marshall Kirk, Keith Bostic, Michael J Karels, and John Quarterman. The Design and Implementation of the 4.4BSD Operating System. [APL] Apple website [http://www.apple.com/macosx/] [BSDI] Berkeley Software Design, Inc. [DFB] DragonFlyBSD Project, The. [DOC] README, COPYRIGHT on tape. [FBD] FreeBSD Project, The. [KB] Keith Bostic. BSD2.10 available from Usenix. comp.unix.sources, Volume 11, Info 4, April, 1987. [KKK] Mike Karels, Kirk McKusick, and Keith Bostic. tahoe announcement. comp.bugs.4bsd.ucb-fixes, June 15, 1988. [KSJ] Michael J. Karels, Carl F. Smith, and William F. Jolitz. Changes in the Kernel in 2.9BSD. Second Berkeley Software Distribution UNIX Version 2.9, July, 1983. [NBD] NetBSD Project, The. [OBD] OpenBSD Project, The. [QCU] Salus, Peter H. A quarter century of UNIX. [SMS] Steven M. Schultz. 2.11BSD, UNIX for the PDP-11. [TUHS] The Unix Historical Society. http://minnie.tuhs.org/Unix_History/. [USE] Usenet announcement. [WRS] Wind River Systems, Inc. [dmr] Dennis Ritchie, via E-Mail Multics 1965 UNIX Summer 1969 DEC PDP-7 First Edition 1971-11-03 [QCU] DEC PDP-11/20, Assembler Second Edition 1972-06-12 [QCU] 1965
Software Usability? 41 Multics 1965 UNIX Summer 1969 DEC PDP-7 [snip] FreeBSD 11.0 2016-10-10 [FBD] NetBSD 7.1 2017-03-11 [NBD] DragonFly 4.8.0 2017-03-27 [DFB] OpenBSD 6.1 2017-04-11 [OBD] 2017
Software Usability? 42 Multics 1965 UNIX Summer 1969 DEC PDP-7 [snip] FreeBSD 11.0 2016-10-10 [FBD] NetBSD 7.1 2017-03-11 [NBD] DragonFly 4.8.0 2017-03-27 [DFB] OpenBSD 6.1 2017-04-11 [OBD] 2017-1965=??
Software Usability? 43 Multics 1965 UNIX Summer 1969 DEC PDP-7 [snip] FreeBSD 11.0 2016-10-10 [FBD] NetBSD 7.1 2017-03-11 [NBD] DragonFly 4.8.0 2017-03-27 [DFB] OpenBSD 6.1 2017-04-11 [OBD] 2017-1965=52
Oracle: Could We Have Learned Something? 44 Yes definitely
Why? Part 2: What is Progress? 45
Why? Part 2: Why the quagmire? • Pluralistic Base Existed • Ended up with a base that is only made up of self-bootstrapped bits • Deprecated everything not C or sh(1) • Deorbiting base to ports • Why are base utilities limited to C and sh(1) again if we're building base with poudriere(1)? 46
Automation to the Rescue • Building releases image is now "easy" by comparison • Debian GNU/kFreeBSD was a big effort • Gentoo FreeBSD was a big effort • Rust/FreeBSD? Pretty trivial 47
Bootstrap an Image 48 $ make packer-build TEMPLATE=freebsd-userland-rust.json EXTRA_ARGS=-on-error=abort cfgt -i freebsd-userland-rust.json5 -o freebsd-userland-rust.json envchain tf_triton_test packer build -var "image_name=my-freebsd-image" -var "image_version=1.0.0" -on-error=abort freebsd-userland-rust.json triton output will be in this color. ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Waiting for SSH to become available... ==> triton: Connected to SSH! [snip] triton: gmake: Entering directory '/root/coreutils' triton: cargo build --features "arch base32 base64 basename cat chgrp chmod chown chroot cksum comm cp cut dircolors dirname du echo env expand expr factor false fmt fold groups hashsum head hostid hostname id install kill link ln logname mkdir mkfifo mknod mktemp more mv nice nl nohup nproc od paste pathchk pinky printenv printf ptx pwd readlink realpath relpath rm rmdir seq shred shuf sleep sort split stdbuf sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users wc whoami yes" --release --no-default-features triton: Updating registry `https://github.com/rust-lang/crates.io-index` triton: Updating git repository `https://github.com/rust-lang/libc.git` triton: Updating registry `https://github.com/rust-lang/crates.io-index`
Bootstrap an Image 49 triton: Finished release [optimized] target(s) in 0.66 secs triton: cargo build --release -p unexpand triton: Compiling unexpand v0.0.1 (file:///root/coreutils/src/unexpand) triton: Finished release [optimized] target(s) in 0.57 secs triton: cargo build --release -p uniq triton: Compiling uniq v0.0.1 (file:///root/coreutils/src/uniq) triton: Finished release [optimized] target(s) in 0.56 secs triton: cargo build --release -p unlink triton: Compiling unlink v0.0.1 (file:///root/coreutils/src/unlink) triton: Finished release [optimized] target(s) in 0.57 secs triton: cargo build --release -p uptime triton: Compiling uptime v0.0.1 (file:///root/coreutils/src/uptime) triton: Finished release [optimized] target(s) in 0.56 secs triton: cargo build --release -p users triton: Compiling users v0.0.1 (file:///root/coreutils/src/users) triton: Finished release [optimized] target(s) in 0.56 secs triton: cargo build --release -p wc triton: Compiling wc v0.0.1 (file:///root/coreutils/src/wc) triton: Finished release [optimized] target(s) in 0.63 secs triton: cargo build --release -p whoami triton: Compiling whoami v0.0.1 (file:///root/coreutils/src/whoami) triton: Finished release [optimized] target(s) in 0.57 secs triton: cargo build --release -p yes triton: Compiling yes v0.0.1 (file:///root/coreutils/src/yes) triton: Finished release [optimized] target(s) in 0.61 secs triton: mkdir -p /usr/obj/bin
Bootstrap an Image 50 triton: install /root/coreutils/target/release/arch /usr/obj/bin/arch; install /root/coreutils/target/release/base32 /usr/obj/bin/base32; install /root/coreutils/target/release/base64 /usr/obj/bin/base64; install /root/coreutils/target/ release/basename /usr/obj/bin/basename; install /root/coreutils/target/release/cat /usr/obj/bin/cat; install /root/ coreutils/target/release/chgrp /usr/obj/bin/chgrp; install /root/coreutils/target/release/chmod /usr/obj/bin/chmod; install /root/coreutils/target/release/chown /usr/obj/bin/chown; install /root/coreutils/target/release/chroot /usr/obj/ bin/chroot; install /root/coreutils/target/release/cksum /usr/obj/bin/cksum; install /root/coreutils/target/release/ comm /usr/obj/bin/comm; install /root/coreutils/target/release/cp /usr/obj/bin/cp; install /root/coreutils/target/ release/cut /usr/obj/bin/cut; install /root/coreutils/target/release/dircolors /usr/obj/bin/dircolors; install /root/ coreutils/target/release/dirname /usr/obj/bin/dirname; install /root/coreutils/target/release/du /usr/obj/bin/du; install /root/coreutils/target/release/echo /usr/obj/bin/echo; install /root/coreutils/target/release/env /usr/obj/bin/ env; install /root/coreutils/target/release/expand /usr/obj/bin/expand; install /root/coreutils/target/release/expr / usr/obj/bin/expr; install /root/coreutils/target/release/factor /usr/obj/bin/factor; install /root/coreutils/target/ release/false /usr/obj/bin/false; install /root/coreutils/target/release/fmt /usr/obj/bin/fmt; install /root/coreutils/ target/release/fold /usr/obj/bin/fold; install /root/coreutils/target/release/groups /usr/obj/bin/groups; install / root/coreutils/target/release/hashsum /usr/obj/bin/hashsum; install /root/coreutils/target/release/head /usr/obj/bin/ head; install /root/coreutils/target/release/hostid /usr/obj/bin/hostid; install /root/coreutils/target/release/ hostname /usr/obj/bin/hostname; install /root/coreutils/target/release/id /usr/obj/bin/id; install /root/coreutils/ target/release/install /usr/obj/bin/install; install /root/coreutils/target/release/kill /usr/obj/bin/kill; install / root/coreutils/target/release/link /usr/obj/bin/link; install /root/coreutils/target/release/ln /usr/obj/bin/ln; install /root/coreutils/target/release/logname /usr/obj/bin/logname; install /root/coreutils/target/release/mkdir /usr/ obj/bin/mkdir; install /root/coreutils/target/release/mkfifo /usr/obj/bin/mkfifo; install /root/coreutils/target/ release/mknod /usr/obj/bin/mknod; install /root/coreutils/target/release/mktemp /usr/obj/bin/mktemp; install /root/ coreutils/target/release/more /usr/obj/bin/more; install /root/coreutils/target/release/mv /usr/obj/bin/mv; install / root/coreutils/target/release/nice /usr/obj/bin/nice; install /root/coreutils/target/release/nl /usr/obj/bin/nl; install /root/coreutils/target/release/nohup /usr/obj/bin/nohup; install /root/coreutils/target/release/nproc /usr/obj/ bin/nproc; install /root/coreutils/target/release/od /usr/obj/bin/od; install /root/coreutils/target/release/paste /
Bootstrap an Image 51 triton: triton: Rebuilding locate database: $ echo It worked ls(1), stat(1), and who(1) don't work and install(1) is missing options. Plus more WIP problems.
Parting Thoughts • Swapping out syscall(2) tables is somehow okay with us • Do we package each binary? • Can we version binaries? We do shared objects and APIs • Squatting on the filesystem is not versioning or an API • What history? Nuke and pave. No instance up older than 3 years. • Need tight coupling? Depend on a specific package. • How can we embrace plurality (read: R&D) • SmartOS and OmniOS uses different userlands, all slightly different • What history? Nuke and pave. No instance up older than 3 years. 52
Parting Thoughts • Writing BSD licensed software gives you access to the guild of like-minded commercially motivated individuals who require: • quality • debugability • observability • performance • shipability 53
Parting Thoughts • Exporting an OSes base utilities takes excessive effort as the modern programming languages have shown • Embrace the future • If we could start over knowing we had 10M LoC and >1K contributors, what and how would we build an OS? • If the ideal is different than what we're doing now, let's begin to plan how to get to an ideal situation Post Talk Comment: "This is like introducing the metric system to the UNIX countries of the world." 54 This resonated with me
Engineering seanc@joyent.com Sean Chittenden @SeanChittenden Thank you! Questions? https://github.com/sean-/BSDCan-2017 https://www.slideshare.net/SeanChittenden/universal-userland

More Related Content

PDF
Life Cycle of Metrics, Alerting, and Performance Monitoring in Microservices
Sean Chittenden
 
PDF
LinuxCon_2013_NA_Eckermann_Filesystems_btrfs.pdf
degarden
 
PDF
How to Root 10 Million Phones with One Exploit
Jiahong Fang
 
PDF
The New Systems Performance
Brendan Gregg
 
PDF
Linux Systems Performance 2016
Brendan Gregg
 
PDF
Stateless Hypervisors at Scale
Antony Messerl
 
PDF
Kernel Recipes 2019 - Kernel documentation: past, present, and future
Anne Nicolas
 
PDF
Embedded Recipes 2018 - Finding sources of Latency In your system - Steven Ro...
Anne Nicolas
 
Life Cycle of Metrics, Alerting, and Performance Monitoring in Microservices
Sean Chittenden
 
LinuxCon_2013_NA_Eckermann_Filesystems_btrfs.pdf
degarden
 
How to Root 10 Million Phones with One Exploit
Jiahong Fang
 
The New Systems Performance
Brendan Gregg
 
Linux Systems Performance 2016
Brendan Gregg
 
Stateless Hypervisors at Scale
Antony Messerl
 
Kernel Recipes 2019 - Kernel documentation: past, present, and future
Anne Nicolas
 
Embedded Recipes 2018 - Finding sources of Latency In your system - Steven Ro...
Anne Nicolas
 

What's hot (20)

PDF
EuroBSDcon 2017 System Performance Analysis Methodologies
Brendan Gregg
 
ODP
High Availability in 37 Easy Steps
Tim Serong
 
PPTX
Backups
Payal Singh
 
PDF
Kernel Recipes 2015 - So you want to write a Linux driver framework
Anne Nicolas
 
ODP
SystemV vs systemd
All Things Open
 
PDF
USENIX ATC 2017 Performance Superpowers with Enhanced BPF
Brendan Gregg
 
PDF
Kernel Recipes 2015 - Porting Linux to a new processor architecture
Anne Nicolas
 
PPTX
Automated out-of-band management with Ansible and Redfish
Jose De La Rosa
 
PDF
Hands on Virtualization with Ganeti (part 1) - LinuxCon 2012
Lance Albertson
 
PDF
Kdump and the kernel crash dump analysis
Buland Singh
 
PPT
Hadoop Installation
mrinalsingh385
 
PDF
Kernel Recipes 2019 - CVEs are dead, long live the CVE!
Anne Nicolas
 
PDF
Linux kernel-rootkit-dev - Wonokaerun
idsecconf
 
PDF
Systems Performance: Enterprise and the Cloud
Brendan Gregg
 
PDF
Kdump-FUDcon-2015-Session
Buland Singh
 
PDF
Docker - container and lightweight virtualization
Sim Janghoon
 
PDF
High Availability With DRBD & Heartbeat
Chris Barber
 
PDF
Single node hadoop cluster installation
Mahantesh Angadi
 
PDF
LISA2010 visualizations
Brendan Gregg
 
ODP
Speeding up ps and top
Kirill Kolyshkin
 
EuroBSDcon 2017 System Performance Analysis Methodologies
Brendan Gregg
 
High Availability in 37 Easy Steps
Tim Serong
 
Backups
Payal Singh
 
Kernel Recipes 2015 - So you want to write a Linux driver framework
Anne Nicolas
 
SystemV vs systemd
All Things Open
 
USENIX ATC 2017 Performance Superpowers with Enhanced BPF
Brendan Gregg
 
Kernel Recipes 2015 - Porting Linux to a new processor architecture
Anne Nicolas
 
Automated out-of-band management with Ansible and Redfish
Jose De La Rosa
 
Hands on Virtualization with Ganeti (part 1) - LinuxCon 2012
Lance Albertson
 
Kdump and the kernel crash dump analysis
Buland Singh
 
Hadoop Installation
mrinalsingh385
 
Kernel Recipes 2019 - CVEs are dead, long live the CVE!
Anne Nicolas
 
Linux kernel-rootkit-dev - Wonokaerun
idsecconf
 
Systems Performance: Enterprise and the Cloud
Brendan Gregg
 
Kdump-FUDcon-2015-Session
Buland Singh
 
Docker - container and lightweight virtualization
Sim Janghoon
 
High Availability With DRBD & Heartbeat
Chris Barber
 
Single node hadoop cluster installation
Mahantesh Angadi
 
LISA2010 visualizations
Brendan Gregg
 
Speeding up ps and top
Kirill Kolyshkin
 
Ad

Similar to Universal Userland (20)

PDF
Django dev-env-my-way
Robert Lujo
 
PDF
Containers for Science and High-Performance Computing
Dmitry Spodarets
 
PDF
Deploying software at Scale
Kris Buytaert
 
PDF
Containers with systemd-nspawn
Gábor Nyers
 
PDF
OpenStack Murano introduction
Victor Zhang
 
PDF
Janus Workshop @ ClueCon 2020
Lorenzo Miniero
 
PPTX
drupal ci cd concept cornel univercity.pptx
rukuntravel
 
PDF
Docker presentasjon java bin
Olve Hansen
 
PDF
Pentesting iOS Apps
Herman Duarte
 
PDF
Slides Aquarium Paris 2008
julien.ponge
 
PDF
Metasploitation part-1 (murtuja)
ClubHack
 
PDF
Android Internals
Opersys inc.
 
PDF
My "Perfect" Toolchain Setup for Grails Projects
GR8Conf
 
PDF
Explore Android Internals
National Cheng Kung University
 
PDF
Iz Pack
Inria
 
PDF
IzPack at LyonJUG'11
julien.ponge
 
PDF
Getting started with developing Nodejs
Phil Hawksworth
 
PDF
So. many. vulnerabilities. Why are containers such a mess and what to do abou...
Eric Smalling
 
PDF
Using and Customizing the Android Framework / part 4 of Embedded Android Work...
Opersys inc.
 
ODP
Repositories as Code
Kris Buytaert
 
Django dev-env-my-way
Robert Lujo
 
Containers for Science and High-Performance Computing
Dmitry Spodarets
 
Deploying software at Scale
Kris Buytaert
 
Containers with systemd-nspawn
Gábor Nyers
 
OpenStack Murano introduction
Victor Zhang
 
Janus Workshop @ ClueCon 2020
Lorenzo Miniero
 
drupal ci cd concept cornel univercity.pptx
rukuntravel
 
Docker presentasjon java bin
Olve Hansen
 
Pentesting iOS Apps
Herman Duarte
 
Slides Aquarium Paris 2008
julien.ponge
 
Metasploitation part-1 (murtuja)
ClubHack
 
Android Internals
Opersys inc.
 
My "Perfect" Toolchain Setup for Grails Projects
GR8Conf
 
Explore Android Internals
National Cheng Kung University
 
Iz Pack
Inria
 
IzPack at LyonJUG'11
julien.ponge
 
Getting started with developing Nodejs
Phil Hawksworth
 
So. many. vulnerabilities. Why are containers such a mess and what to do abou...
Eric Smalling
 
Using and Customizing the Android Framework / part 4 of Embedded Android Work...
Opersys inc.
 
Repositories as Code
Kris Buytaert
 
Ad

More from Sean Chittenden (13)

PDF
BSDCan '19 Core Update
Sean Chittenden
 
PDF
pg_prefaulter: Scaling WAL Performance
Sean Chittenden
 
PDF
FreeBSD VPC Introduction
Sean Chittenden
 
PDF
Codified PostgreSQL Schema
Sean Chittenden
 
PDF
PostgreSQL + ZFS best practices
Sean Chittenden
 
PDF
Incrementalism: An Industrial Strategy For Adopting Modern Automation
Sean Chittenden
 
PDF
Production Readiness Strategies in an Automated World
Sean Chittenden
 
PDF
FreeBSD: Dev to Prod
Sean Chittenden
 
PDF
PostgreSQL on ZFS Lightning Talk
Sean Chittenden
 
PDF
Dynamic Database Credentials: Security Contingency Planning
Sean Chittenden
 
PDF
PostgreSQL High-Availability and Geographic Locality using consul
Sean Chittenden
 
PDF
Modern tooling to assist with developing applications on FreeBSD
Sean Chittenden
 
PDF
Creating PostgreSQL-as-a-Service at Scale
Sean Chittenden
 
BSDCan '19 Core Update
Sean Chittenden
 
pg_prefaulter: Scaling WAL Performance
Sean Chittenden
 
FreeBSD VPC Introduction
Sean Chittenden
 
Codified PostgreSQL Schema
Sean Chittenden
 
PostgreSQL + ZFS best practices
Sean Chittenden
 
Incrementalism: An Industrial Strategy For Adopting Modern Automation
Sean Chittenden
 
Production Readiness Strategies in an Automated World
Sean Chittenden
 
FreeBSD: Dev to Prod
Sean Chittenden
 
PostgreSQL on ZFS Lightning Talk
Sean Chittenden
 
Dynamic Database Credentials: Security Contingency Planning
Sean Chittenden
 
PostgreSQL High-Availability and Geographic Locality using consul
Sean Chittenden
 
Modern tooling to assist with developing applications on FreeBSD
Sean Chittenden
 
Creating PostgreSQL-as-a-Service at Scale
Sean Chittenden
 

Recently uploaded (20)

PDF
UI/UX Developer Guide: Tools, Trends, and Tips for 2025
Penguin peak
 
PPTX
Parallel & Concurrent ...
yashpavasiya892
 
PPT
1965 INDO PAK WAR which Pak will never forget.ppt
sanjaychief112
 
PDF
The Internet of Things (IoT) refers to a vast network of interconnected devic...
chethana8182
 
PPTX
Unlocking Hope : How Crypto Recovery Services Can Reclaim Your Lost Funds
lionsgate network
 
PDF
LOGENVIDAD DANNYFGRETRRTTRRRTRRRRRRRRR.pdf
juan456ytpro
 
PDF
Latest Scam Shocking the USA in 2025.pdf
onlinescamreport4
 
PPT
Transformaciones de las funciones elementales.ppt
rirosel211
 
PPTX
B2B_Ecommerce_Internship_Simranpreet.pptx
LipakshiJindal
 
PPTX
dns domain name system history work.pptx
MUHAMMADKAVISHSHABAN
 
PPTX
谢尔丹学院毕业证购买|Sheridan文凭不见了怎么办谢尔丹学院成绩单
mookxk3
 
PPTX
Slides Powerpoint: Eco Economic Epochs.pptx
Steven McGee
 
PPTX
How tech helps people in the modern era.
upadhyayaryan154
 
PDF
Cybersecurity Awareness Presentation ppt.
banodhaharshita
 
PDF
BGP Security Best Practices that Matter, presented at PHNOG 2025
APNIC
 
PDF
APNIC Update, presented at PHNOG 2025 by Shane Hermoso
APNIC
 
PPTX
Artificial-Intelligence-in-Daily-Life (2).pptx
nidhigoswami335
 
PDF
DNSSEC Made Easy, presented at PHNOG 2025
APNIC
 
PPTX
The Internet of Things (IoT) refers to a vast network of interconnected devic...
chethana8182
 
PPTX
AI ad its imp i military life read it ag
ShwetaBharti31
 
UI/UX Developer Guide: Tools, Trends, and Tips for 2025
Penguin peak
 
Parallel & Concurrent ...
yashpavasiya892
 
1965 INDO PAK WAR which Pak will never forget.ppt
sanjaychief112
 
The Internet of Things (IoT) refers to a vast network of interconnected devic...
chethana8182
 
Unlocking Hope : How Crypto Recovery Services Can Reclaim Your Lost Funds
lionsgate network
 
LOGENVIDAD DANNYFGRETRRTTRRRTRRRRRRRRR.pdf
juan456ytpro
 
Latest Scam Shocking the USA in 2025.pdf
onlinescamreport4
 
Transformaciones de las funciones elementales.ppt
rirosel211
 
B2B_Ecommerce_Internship_Simranpreet.pptx
LipakshiJindal
 
dns domain name system history work.pptx
MUHAMMADKAVISHSHABAN
 
谢尔丹学院毕业证购买|Sheridan文凭不见了怎么办谢尔丹学院成绩单
mookxk3
 
Slides Powerpoint: Eco Economic Epochs.pptx
Steven McGee
 
How tech helps people in the modern era.
upadhyayaryan154
 
Cybersecurity Awareness Presentation ppt.
banodhaharshita
 
BGP Security Best Practices that Matter, presented at PHNOG 2025
APNIC
 
APNIC Update, presented at PHNOG 2025 by Shane Hermoso
APNIC
 
Artificial-Intelligence-in-Daily-Life (2).pptx
nidhigoswami335
 
DNSSEC Made Easy, presented at PHNOG 2025
APNIC
 
The Internet of Things (IoT) refers to a vast network of interconnected devic...
chethana8182
 
AI ad its imp i military life read it ag
ShwetaBharti31
 

Universal Userland

  • 1. Universal Userland Why do we need to be different just like everyone else? Engineering [email protected] Sean Chittenden @SeanChittenden
  • 2. Getting Started • Packer - packer.io • Image building utility for Vagrant (VMware, Virtualbox, bhyve), AWS, Triton, GCE, Azure • An appreciation for version controlled software • Pick your poison: git, hg, fossil 2
  • 3. Agenda 1. Why Part 1 2. Bootstrap an image using Packer 3. Iterate using go-coreutils 4. Why Part 2 5. Iterate using rust-coreutils 6. Parting Thoughts 3
  • 4. Why? Part 1: What is an OS? • Is it a kernel? • Is it a userland? • Is it the applications? • Is it a release? • Is it a distribution? • Is it an ethos and shared set of values? • Is it a community? 4
  • 5. What is an OS, Kernel Edition • CDNs? Yes. "The network stack is everything." • Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door." • Embedded systems? Yes. "Userland is never installed." • Hosting Providers? Yes. "Application containment." • Network Security Deployments? Yes. "Hardened network stack." • Users? If it's stable, "what's a kernel?" • Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged." • Developers? "That's why we're here (or to support the kernel)." • Management? Maybe. 5
  • 6. What is an OS, Kernel Edition 6 • CDNs? Yes. "The network stack is everything." • Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door." • Embedded systems? Yes. "Userland is never installed." • Hosting Providers? Yes. "Application containment." • Network Security Deployments? Yes. "Hardened network stack." • Users? If it's stable, "what's a kernel?" • Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged." • Developers? "That's why we're here (or to support the kernel)." • Management? Maybe. YES
  • 7. What is an OS, Userland Edition 7 • CDNs? Partially. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? Yes. "Familiarity for customers is important." • Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code." • Users? "I like my ls(1) and top(1)*." * if [ "`uname -s`" = "SunOS" ]; then alias top="prstat"; fi
  • 8. What is an OS, Userland Edition 8 • CDNs? Partially. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? Yes. "Familiarity for customers is important." • Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code." • Users? "I like my ls(1) and top(1)." • Operators? Yes. "This is how I deploy, maintain, investigate, and manipulate the kernel." • Developers? It depends on who you ask. "Team effort!" • Management? "Productivity? Effectiveness?"
  • 9. Solving science with an Oracle 9
  • 10. Solving science with an Oracle 10 Cloudy with a chance of Linux admins. Management Human Resources Recruiting College Grads
  • 11. Solving science with an Oracle 11
  • 12. What is an OS, Applications Edition 12 • CDNs? Partially. "We use nginx and/or varnish, so yeah." • Appliance vendors? No. "Everything that matters is BSDL or proprietary already." • Embedded systems? No. "What's a userland application?" • Hosting Providers? Yes. "Customers can't be bothered to build 3rd party apps." • Network Security Deployments? Yes. "Wireshark is an app, but tcpdump is kernel, right?" • Users? "Give me X11, emacs/vi, and a webserver. Or give me death." • Operators? Yes. This is how I deploy, maintain, investigate, and manipulate the kernel. • Developers? "Well duh, how else would someone install Chromium?" • Management? "Productivity? Effectiveness?"
  • 13. Solving science with an Oracle 13
  • 14. What is an OS, Release Engineering Edition 14 • Is an OS the release and stability guarantee? • CDNs? No. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? Yes. "IPU and upgrades are important: every customer is a snowflake." • Network Security Deployments? No. • Users? Yes. "IPU and upgrades on my laptop is important." • Operators? Depends: legacy thinking, yes. Modern: not really. • Developers? "We need to protect our users from themselves, so yes." • Management? "Productivity?"
  • 15. What is an OS, Distribution Edition 15 • Is an OS the distribution? • CDNs? No. • Appliance vendors? No. • Embedded systems? No. • Hosting Providers? No. • Network Security Deployments? No. • Users? Yes. "IPU and upgrades on my laptop is important." • Operators? Depends: legacy thinking, yes. Modern: not really. • Developers? "We need to protect our users from themselves, so yes." • Management? "We don't have to do that work anymore, right?"
  • 16. Bootstrap an Image 16 $ git clone -q https://github.com/sean-/BSDCan-2017.git $ cd BSDCan-2017 $ make install-cfgt go get -u github.com/sean-/cfgt $ which packer envchain /Users/seanc/go/bin/packer /opt/local/bin/envchain $ make packer-build TEMPLATE=freebsd-userland-go.json EXTRA_ARGS=-on-error=abort cfgt -i freebsd-userland-go.json5 -o freebsd-userland-go.json envchain tf_triton_test packer build -var "image_name=my-freebsd-image" -var "image_version=1.0.0" -on-error=abort freebsd-userland-go.json triton output will be in this color. ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Waiting for SSH to become available... ==> triton: Connected to SSH! ==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363 triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017 [email protected]:/usr/obj/usr/src/sys/GENERIC amd64
  • 17. Bootstrap an Image 17 triton output will be in this color. ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Waiting for SSH to become available... ==> triton: Connected to SSH! ==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363 triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017 [email protected]:/usr/obj/usr/src/sys/GENERIC amd64 triton: Checking integrity... done (0 conflicting) [snip]
  • 18. Bootstrap an Image 18 triton: ------------------------------------------------------------------------ triton: install: /bin/cat -> /bin/cat.base triton: install: /root/go/bin/cat -> /bin/cat triton: install: /bin/date -> /bin/date.base triton: install: /root/go/bin/date -> /bin/date triton: install: /bin/echo -> /bin/echo.base triton: install: /root/go/bin/echo -> /bin/echo triton: install: /bin/expr -> /bin/expr.base triton: install: /root/go/bin/expr -> /bin/expr triton: install: /bin/mkdir -> /bin/mkdir.base triton: install: /root/go/bin/mkdir -> /bin/mkdir triton: install: /bin/mv -> /bin/mv.base triton: install: /root/go/bin/mv -> /bin/mv triton: install: /bin/pwd -> /bin/pwd.base triton: install: /root/go/bin/pwd -> /bin/pwd triton: install: /bin/rm -> /bin/rm.base triton: install: /root/go/bin/rm -> /bin/rm triton: install: /bin/rmdir -> /bin/rmdir.base triton: install: /root/go/bin/rmdir -> /bin/rmdir triton: install: /bin/sleep -> /bin/sleep.base triton: install: /root/go/bin/sleep -> /bin/sleep triton: install: /bin/sync -> /bin/sync.base triton: install: /root/go/bin/sync -> /bin/sync triton: install: /sbin/md5 -> /sbin/md5.base triton: install: /root/go/bin/md5sum -> /sbin/md5 triton: install: /sbin/sha1 -> /sbin/sha1.base
  • 19. Bootstrap an Image 19 triton: install: /sbin/sha1 -> /sbin/sha1.base triton: install: /root/go/bin/sha1sum -> /sbin/sha1 triton: install: /root/go/bin/sha224sum -> /sbin/sha224 triton: install: /sbin/sha256 -> /sbin/sha256.base triton: install: /root/go/bin/sha256sum -> /sbin/sha256 triton: install: /sbin/sha384 -> /sbin/sha384.base triton: install: /root/go/bin/sha384sum -> /sbin/sha384 triton: install: /sbin/sha512 -> /sbin/sha512.base triton: install: /root/go/bin/sha512sum -> /sbin/sha512 triton: install: /usr/bin/basename -> /usr/bin/basename.base triton: install: /root/go/bin/basename -> /usr/bin/basename triton: install: /usr/bin/dirname -> /usr/bin/dirname.base triton: install: /root/go/bin/dirname -> /usr/bin/dirname triton: install: /usr/bin/env -> /usr/bin/env.base triton: install: /root/go/bin/env -> /usr/bin/env triton: install: /usr/bin/factor -> /usr/bin/factor.base triton: install: /root/go/bin/factor -> /usr/bin/factor triton: install: /usr/bin/false -> /usr/bin/false.base triton: install: /root/go/bin/false -> /usr/bin/false triton: install: /usr/bin/groups -> /usr/bin/groups.base triton: install: /root/go/bin/groups -> /usr/bin/groups triton: install: /usr/bin/head -> /usr/bin/head.base triton: install: /root/go/bin/head -> /usr/bin/head triton: install: /usr/bin/logname -> /usr/bin/logname.base triton: install: /root/go/bin/logname -> /usr/bin/logname triton: install: /usr/bin/tail -> /usr/bin/tail.base
  • 20. Bootstrap an Image 20 triton: install: /root/go/bin/tail -> /usr/bin/tail triton: install: /usr/bin/tee -> /usr/bin/tee.base triton: install: /root/go/bin/tee -> /usr/bin/tee triton: install: /usr/bin/touch -> /usr/bin/touch.base triton: install: /root/go/bin/touch -> /usr/bin/touch triton: install: /usr/bin/true -> /usr/bin/true.base triton: install: /root/go/bin/true -> /usr/bin/true triton: install: /usr/bin/tsort -> /usr/bin/tsort.base triton: install: /root/go/bin/tsort -> /usr/bin/tsort triton: install: /usr/bin/wc -> /usr/bin/wc.base triton: install: /root/go/bin/wc -> /usr/bin/wc triton: install: /usr/bin/whoami -> /usr/bin/whoami.base triton: install: /root/go/bin/whoami -> /usr/bin/whoami triton: install: /usr/bin/yes -> /usr/bin/yes.base triton: install: /root/go/bin/yes -> /usr/bin/yes
  • 21. go-coreutils: Initial thoughts? 21 triton: triton: Rebuilding locate database: triton: flag provided but not defined: -128 triton: Usage of head: triton: -bytes int triton: output the first K bytes triton: -c int triton: output the first K bytes triton: -help triton: triton: Usage: head [OPTION]... [FILE]... triton: triton: Print the first 10 lines of each FILE to standard output. With more than one FILE, precede each with a header giving the file name. With no FILE, or when FILE is -, read standard input. triton: triton: triton: Mandatory arguments to long options are mandatory for short options too. triton: triton: -help display this help and exit triton: -version output version information and exit triton: triton: -c, --bytes=K triton: output the first K bytes; or use -n +K to output starting with the Kth byte.
  • 22. go-coreutils: Initial thoughts? 22 triton: -n, -lines=K triton: output the first K lines; or use -n +K to output starting with triton: the Kth triton: triton: -q, -quiet, -silent triton: never output headers giving file names triton: triton: -lines int triton: output the first K lines (default 10) triton: -n int triton: output the first K lines (default 10) triton: -qnever output headers giving file names triton: -quiet triton: never output headers giving file names triton: -silent triton: never output headers giving file names triton: -version triton: triton: head (go-coreutils) 0.1 triton: triton: Copyright (C) 2014, The GO-Coreutils Developers. triton: This program comes with ABSOLUTELY NO WARRANTY; for deprintTails see triton: LICENSE. This is free software, and you are welcome to redistribute triton: it under certain conditions in LICENSE. triton:
  • 23. go-coreutils: Initial thoughts? 23 • Needs to bake more • Probably not going anywhere • GPLv3ed code • Suspicion: No commercial incentive or expectation of quality
  • 24. Why? Part 2: What is Progress? 24
  • 25. Science: A lens through which we learn • Whereas research has shown that to a considerable extent children learn by imitating the behavior of adults, especially those they are dependent upon; and the use of corporal punishment by adults having authority over children is likely to train children to use physical violence to control behavior rather than rational persuasion, education, and intelligent forms of both positive and negative reinforcement; • Whereas research has shown that the effective use of punishment in eliminating undesirable behavior requires precision in timing, duration, intensity, and specificity, as well as considerable sophistication in controlling a variety of relevant environmental and cognitive factors, such that punishment administered in institutional settings, without attention to all these factors, is likely to instill hostility, rage, and a sense of powerlessness without reducing the undesirable behavior; • Therefore, be it resolved that the American Psychological Association opposes the use of corporal punishment in schools, juvenile facilities, child care nurseries, and all other institutions, public or private, where children are cared for or educated (Conger, 1975). American Psychological Association
 http://www.apa.org/about/policy/corporal-punishment.aspx 25
  • 27. Software Performance? Performance? Find regression, fix regression. https://github.com/freebsd/freebsd/commit/89c59fbb17d89068626d6214477d40c81d43a025 27
  • 28. Software Usability? 28 $ echo baz | tee foo > bar $ echo foo >> foo $ echo bar >> bar $ diff foo bar 1c1 < foo --- > bar $ date Sat May 10 15:49:02 UTC 2017 $ diff -u foo bar --- foo2017-06-10 15:48:47.261401000 +0000 +++ bar2017-06-10 15:48:51.157525000 +0000 @@ -1,2 +1,2 @@ baz -foo +bar
  • 29. Could we have learned something? 29
  • 30. Software Usability? 30 The UNIX system family tree: Research and BSD --------------------------------------------- First Edition (V1) | Second Edition (V2) | Third Edition (V3) | Fourth Edition (V4) | Fifth Edition (V5) | Sixth Edition (V6) -----* | | | Seventh Edition (V7) | | 1BSD 32V | 2BSD---------------* / | / | / | 3BSD | | | 4.0BSD 2.79BSD | | 4.1BSD --------------> 2.8BSD | | 4.1aBSD ----------- | | | 4.1bBSD | | | *------ 4.1cBSD --------------> 2.9BSD
  • 31. Software Usability? 31 / | | Eighth Edition | 2.9BSD-Seismo | | | +----<--- 4.2BSD 2.9.1BSD | | | +----<--- 4.3BSD -------------> 2.10BSD | | / | Ninth Edition | / 2.10.1BSD | 4.3BSD Tahoe-----+ | | | | | | | v | 2.11BSD Tenth Edition | | | 2.11BSD rev #430 4.3BSD NET/1 | | v 4.3BSD Reno | *---------- 4.3BSD NET/2 -------------------+-------------* | | | | 386BSD 0.0 | | BSD/386 ALPHA | | | | 386BSD 0.1 ------------>+ | BSD/386 0.3.[13] | | 4.4BSD Alpha | | 386BSD 1.0 | | BSD/386 0.9.[34] | | 4.4BSD | | | / | | | | 4.4BSD-Encumbered | | | -NetBSD 0.8 | BSD/386 1.0 | / | | | FreeBSD 1.0 <-----' NetBSD 0.9 | BSD/386 1.1 | | .----- 4.4BSD Lite | FreeBSD 1.1 | / / | | | | / / | | FreeBSD 1.1.5 .---|--------' / | | | / | / | |
  • 32. Software Usability? 32 FreeBSD 1.1.5.1 / | / | | | / NetBSD 1.0 <-' | | | / | | | FreeBSD 2.0 <--' | | BSD/OS 2.0 | | | FreeBSD 2.0.5 | BSD/OS 2.0.1 | .------------------ 4.4BSD Lite2 | | | | | | | | | | .-----|------Rhapsody | | | | | | | | NetBSD 1.3 | | | | | | | OpenBSD 2.3 | | | | | | BSD/OS 3.0 | FreeBSD 2.1 | | | | | | | | NetBSD 1.1 ------. BSD/OS 2.1 | FreeBSD 2.1.5 | | | | | | | | NetBSD 1.2 BSD/OS 3.0 | FreeBSD 2.1.6 | | | OpenBSD 2.0 | | | | | | | | | FreeBSD 2.1.6.1 | | | | | | | | | | | | | FreeBSD 2.1.7 | | | | | | | | | | | NetBSD 1.2.1 | | | FreeBSD 2.1.7.1 | | | | | | | | | | | | | | | | | *-FreeBSD 2.2 | | | | | | | | | | | | FreeBSD 2.2.1 | | | | | | | | | | | | | FreeBSD 2.2.2 | | | OpenBSD 2.1 | | | | | | | | | FreeBSD 2.2.5 | | | | | | | | | | OpenBSD 2.2 | | | | | NetBSD 1.3 | | | FreeBSD 2.2.6 | | | | | | | | | | | NetBSD 1.3.1 | BSD/OS 3.1
  • 33. Software Usability? 33 | | | | | | OpenBSD 2.3 | | | | | | NetBSD 1.3.2 | | | FreeBSD 2.2.7 | | | | | | | | | | | | | BSD/OS 4.0 | FreeBSD 2.2.8 | | | | | | | | | | | | | | | v | | | | OpenBSD 2.4 | | FreeBSD 2.2.9 | | | | | | | | | | | | | FreeBSD 3.0 <--------* | | v | | | | | NetBSD 1.3.3 | | *---FreeBSD 3.1 | | | | | | | | | BSD/OS 4.0.1 | FreeBSD 3.2----* | NetBSD 1.4 OpenBSD 2.5 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FreeBSD 3.3 | | | | NetBSD 1.4.1 | | | | | | | | | OpenBSD 2.6 | | FreeBSD 3.4 | | | | | | | | | | | | | | | BSD/OS 4.1 FreeBSD 4.0 | | | | | NetBSD 1.4.2 | | | | | | | | | | | | | | | | | | | | | FreeBSD 3.5 | | | | | OpenBSD 2.7 | | | | | | | | | | | FreeBSD 3.5.1 | | | | | | | | | | | | | | | *---FreeBSD 4.1 | | | | | | | | | | | (?) | | | | | FreeBSD 4.1.1 | | / | | | | | | | | / | | | | | FreeBSD 4.2 Darwin/ | NetBSD 1.4.3 | | | | Mac OS X | OpenBSD 2.8 BSD/OS 4.2 | | | | | | | | | | | |
  • 34. Software Usability? 34 | | 10.0 NetBSD 1.5 | | | FreeBSD 4.3 | | | | | | | | | | OpenBSD 2.9 | | | | | NetBSD 1.5.1 | | | | | | | | | | FreeBSD 4.4-. | | NetBSD 1.5.2 | | | | | Mac OS X | | | | | | | 10.1 | | OpenBSD 3.0 | | FreeBSD 4.5 | | | | | | | | | | | | BSD/OS 4.3 | FreeBSD 4.6 | | | OpenBSD 3.1 | | | | | NetBSD 1.5.3 | | | FreeBSD 4.6.2 Mac OS X | | | | | 10.2 | | | | FreeBSD 4.7 | | | | | | | NetBSD 1.6 OpenBSD 3.2 | | FreeBSD 4.8 | | | | | | | | | NetBSD 1.6.1 | | | |--------. | | | OpenBSD 3.3 BSD/OS 5.0 | | | | | | | | FreeBSD 4.9 | | | | OpenBSD 3.4 BSD/OS 5.1 ISE | | | | | | | | | | | | NetBSD 1.6.2 | | | | | | | | | | | | | | OpenBSD 3.5 | | | | | v | | FreeBSD 4.10 | | | | | | | | | | | FreeBSD 4.11 | | | | | | | | | | `-|------|-----------------|---------------------. | | | | FreeBSD 5.0 | | | | | | | | | FreeBSD 5.1 | | | DragonFly 1.0 | | | | |
  • 35. Software Usability? 35 | ----- Mac OS X | | | | 10.3 | | | FreeBSD 5.2 | | | | | | | | | | | FreeBSD 5.2.1 | | | | | | | | | *-------FreeBSD 5.3 | | | | | | | | OpenBSD 3.6 | | | | NetBSD 2.0 | | | | | | | | | DragonFly 1.2.0 | | Mac OS X | | NetBSD 2.0.2 | | | | 10.4 | | | | | | FreeBSD 5.4 | | | | | | | | | | | | OpenBSD 3.7 | | | | | | NetBSD 2.0.3 | | | | | | | | | | *--FreeBSD | | | | v OpenBSD 3.8 | | 6.0 | | | | | | | | | | | | | | | | | | NetBSD 2.1 | | | | | | | | | | | | | NetBSD 3.0 | | | | | | | | | | DragonFly 1.4.0 | | | | | | | OpenBSD 3.9 | | FreeBSD | | | | | | | | 6.1 | | | | | | | | | FreeBSD 5.5 | | | | | | | | | | | NetBSD 3.0.1 | DragonFly 1.6.0 | | | | | | | | | | | | | | OpenBSD 4.0 | | | | | | NetBSD 3.0.2 | | | | | | NetBSD 3.1 | | | FreeBSD 6.2 | | | | | | | | | DragonFly 1.8.0 | | | | OpenBSD 4.1 | | | | | | DragonFly 1.10.0
  • 36. Software Usability? 36 | | Mac OS X | | | | | 10.5 | | | | | | | OpenBSD 4.2 | | | | NetBSD 4.0 | | | FreeBSD 6.3 | | | | | | | | | | | *--FreeBSD | | | | | DragonFly 1.12.0 | 7.0 | | | | | | | | | | | | OpenBSD 4.3 | | | | | | NetBSD | DragonFly 2.0.0 | | FreeBSD | | 4.0.1 OpenBSD 4.4 | | | 6.4 | | | | | | | | | | | FreeBSD 7.1 | | | | | | | | | DragonFly 2.2.0 | FreeBSD 7.2 | NetBSD 5.0 OpenBSD 4.5 | | | | | | | | | Mac OS X | | | | | | 10.6 | | | | | | | | | NetBSD | DragonFly 2.4.0 | | | | | 5.0.1 OpenBSD 4.6 | | | | | | | | | *--FreeBSD | | | | | | | | 8.0 | | | | | | | | | FreeBSD | | | NetBSD | | | | 7.3 | | | 5.0.2 | DragonFly 2.6.0 | | | | | | OpenBSD 4.7 | | FreeBSD | | | | | | | 8.1 | | | | | | | | | | | | | DragonFly 2.8.2 | | | | | | OpenBSD 4.8 | | | | | | *--NetBSD | | | FreeBSD FreeBSD | | | 5.1 | | | 8.2 7.4 | | | | | DragonFly 2.10.1 | | | | | | OpenBSD 4.9 | | `-----. Mac OS X | | | | |
  • 37. Software Usability? 37 | 10.7 | | | | | | | | | | | OpenBSD 5.0 | *--FreeBSD | | | | | | | | 9.0 | | | | NetBSD | DragonFly 3.0.1 | | FreeBSD | | | 5.1.2 | | | | 8.3 | | | | | | | | | | | | NetBSD | | | | | | | | 5.1.3 | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 5.1.4 | | | | | | | | OpenBSD 5.1 | | | | Mac OS X | `----. | | | | | 10.8 | | | | | | | NetBSD 6.0 | | | | | | | | | | | OpenBSD 5.2 DragonFly 3.2.1 | FreeBSD | | | | | NetBSD | | | 9.1 | | | | | 5.2 | | | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | | 5.2.1 | | | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | | 5.2.2 | | | | | | | | | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 6.0.1 | | | | | | | | | OpenBSD 5.3 DragonFly 3.4.1 | | | | | | NetBSD | | | | | | | | 6.0.2 | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 6.0.3 | | | | | | | | | | | | | | | | | NetBSD | |
  • 38. Software Usability? 38 | | | | | | 6.0.4 | | | | | | | | | | | | | | | | | NetBSD | | | | | | | | 6.0.5 | | | | | | | | | | | | | | | |`-NetBSD 6.1 | | | | FreeBSD | | | | | | | 8.4 | | NetBSD 6.1.1 | | | | | | | | | | FreeBSD | | NetBSD 6.1.2 | | | 9.2 Mac OS X | | | | | | 10.9 | | OpenBSD 5.4 | | `-----. | | | | DragonFly 3.6.0 | | | | | | *--FreeBSD | | | NetBSD 6.1.3 | | | 10.0 | | | | | | | | | | | | | DragonFly 3.6.1 | | | | | | | | | | | | | | | | | | | | | | | DragonFly 3.6.2 | | | | | NetBSD 6.1.4 | | | | | | | | | | | | | | | | OpenBSD 5.5 | | | | | | | | | | | | | | | | DragonFly 3.8.0 | | | | | | | | | | | | | | | | | | | | | | | DragonFly 3.8.1 | | | | | | | | | | | | | | | | | | | | | | | DragonFly 3.6.3 | | | | | | | | | | FreeBSD | | | | | | | 9.3 | | | | | | | | | NetBSD 6.1.5 | DragonFly 3.8.2 | | Mac OS X | | |
  • 39. Software Usability? 39 | | 10.10 | | | | | | | OpenBSD 5.6 | | FreeBSD | | | | | 10.1 | | | DragonFly 4.0.1 | | | | | | | | | | | DragonFly 4.0.2 | | | | | | | | | | | DragonFly 4.0.3 | | | | | | | | | | | DragonFly 4.0.4 | | | | | | | | | | | DragonFly 4.0.5 | | | | | | | | | | OpenBSD 5.7 | | | | | | DragonFly 4.2.0 | FreeBSD | | | | | 10.2 | | | | | | OS X NetBSD 7.0 | | | | 10.11 | OpenBSD 5.8 | | | | | | DragonFly 4.4.1 | FreeBSD | | OpenBSD 5.9 | | 10.3 | | | | | | NetBSD 7.0.1 | | | | | | DragonFly 4.6.0 | | | | | | | | | | *--FreeBSD macOS | OpenBSD 6.0 | | 11.0 10.12 | | | | | NetBSD 7.1 | | | | | | DragonFly 4.8.0 | | | OpenBSD 6.1 | | | | | | FreeBSD 12 -current | NetBSD -current OpenBSD -current DragonFly -current | | | | | v v v v v
  • 40. Software Usability? 40 Time ---------------- Time tolerance +/- 6 months, depending on which book/article you read; if it was the announcement in Usenet or if it was available as tape. [44B] McKusick, Marshall Kirk, Keith Bostic, Michael J Karels, and John Quarterman. The Design and Implementation of the 4.4BSD Operating System. [APL] Apple website [http://www.apple.com/macosx/] [BSDI] Berkeley Software Design, Inc. [DFB] DragonFlyBSD Project, The. [DOC] README, COPYRIGHT on tape. [FBD] FreeBSD Project, The. [KB] Keith Bostic. BSD2.10 available from Usenix. comp.unix.sources, Volume 11, Info 4, April, 1987. [KKK] Mike Karels, Kirk McKusick, and Keith Bostic. tahoe announcement. comp.bugs.4bsd.ucb-fixes, June 15, 1988. [KSJ] Michael J. Karels, Carl F. Smith, and William F. Jolitz. Changes in the Kernel in 2.9BSD. Second Berkeley Software Distribution UNIX Version 2.9, July, 1983. [NBD] NetBSD Project, The. [OBD] OpenBSD Project, The. [QCU] Salus, Peter H. A quarter century of UNIX. [SMS] Steven M. Schultz. 2.11BSD, UNIX for the PDP-11. [TUHS] The Unix Historical Society. http://minnie.tuhs.org/Unix_History/. [USE] Usenet announcement. [WRS] Wind River Systems, Inc. [dmr] Dennis Ritchie, via E-Mail Multics 1965 UNIX Summer 1969 DEC PDP-7 First Edition 1971-11-03 [QCU] DEC PDP-11/20, Assembler Second Edition 1972-06-12 [QCU] 1965
  • 41. Software Usability? 41 Multics 1965 UNIX Summer 1969 DEC PDP-7 [snip] FreeBSD 11.0 2016-10-10 [FBD] NetBSD 7.1 2017-03-11 [NBD] DragonFly 4.8.0 2017-03-27 [DFB] OpenBSD 6.1 2017-04-11 [OBD] 2017
  • 42. Software Usability? 42 Multics 1965 UNIX Summer 1969 DEC PDP-7 [snip] FreeBSD 11.0 2016-10-10 [FBD] NetBSD 7.1 2017-03-11 [NBD] DragonFly 4.8.0 2017-03-27 [DFB] OpenBSD 6.1 2017-04-11 [OBD] 2017-1965=??
  • 43. Software Usability? 43 Multics 1965 UNIX Summer 1969 DEC PDP-7 [snip] FreeBSD 11.0 2016-10-10 [FBD] NetBSD 7.1 2017-03-11 [NBD] DragonFly 4.8.0 2017-03-27 [DFB] OpenBSD 6.1 2017-04-11 [OBD] 2017-1965=52
  • 44. Oracle: Could We Have Learned Something? 44 Yes definitely
  • 45. Why? Part 2: What is Progress? 45
  • 46. Why? Part 2: Why the quagmire? • Pluralistic Base Existed • Ended up with a base that is only made up of self-bootstrapped bits • Deprecated everything not C or sh(1) • Deorbiting base to ports • Why are base utilities limited to C and sh(1) again if we're building base with poudriere(1)? 46
  • 47. Automation to the Rescue • Building releases image is now "easy" by comparison • Debian GNU/kFreeBSD was a big effort • Gentoo FreeBSD was a big effort • Rust/FreeBSD? Pretty trivial 47
  • 48. Bootstrap an Image 48 $ make packer-build TEMPLATE=freebsd-userland-rust.json EXTRA_ARGS=-on-error=abort cfgt -i freebsd-userland-rust.json5 -o freebsd-userland-rust.json envchain tf_triton_test packer build -var "image_name=my-freebsd-image" -var "image_version=1.0.0" -on-error=abort freebsd-userland-rust.json triton output will be in this color. ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Creating source machine... ==> triton: Waiting for source machine to become available... ==> triton: Waiting for SSH to become available... ==> triton: Connected to SSH! [snip] triton: gmake: Entering directory '/root/coreutils' triton: cargo build --features "arch base32 base64 basename cat chgrp chmod chown chroot cksum comm cp cut dircolors dirname du echo env expand expr factor false fmt fold groups hashsum head hostid hostname id install kill link ln logname mkdir mkfifo mknod mktemp more mv nice nl nohup nproc od paste pathchk pinky printenv printf ptx pwd readlink realpath relpath rm rmdir seq shred shuf sleep sort split stdbuf sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users wc whoami yes" --release --no-default-features triton: Updating registry `https://github.com/rust-lang/crates.io-index` triton: Updating git repository `https://github.com/rust-lang/libc.git` triton: Updating registry `https://github.com/rust-lang/crates.io-index`
  • 49. Bootstrap an Image 49 triton: Finished release [optimized] target(s) in 0.66 secs triton: cargo build --release -p unexpand triton: Compiling unexpand v0.0.1 (file:///root/coreutils/src/unexpand) triton: Finished release [optimized] target(s) in 0.57 secs triton: cargo build --release -p uniq triton: Compiling uniq v0.0.1 (file:///root/coreutils/src/uniq) triton: Finished release [optimized] target(s) in 0.56 secs triton: cargo build --release -p unlink triton: Compiling unlink v0.0.1 (file:///root/coreutils/src/unlink) triton: Finished release [optimized] target(s) in 0.57 secs triton: cargo build --release -p uptime triton: Compiling uptime v0.0.1 (file:///root/coreutils/src/uptime) triton: Finished release [optimized] target(s) in 0.56 secs triton: cargo build --release -p users triton: Compiling users v0.0.1 (file:///root/coreutils/src/users) triton: Finished release [optimized] target(s) in 0.56 secs triton: cargo build --release -p wc triton: Compiling wc v0.0.1 (file:///root/coreutils/src/wc) triton: Finished release [optimized] target(s) in 0.63 secs triton: cargo build --release -p whoami triton: Compiling whoami v0.0.1 (file:///root/coreutils/src/whoami) triton: Finished release [optimized] target(s) in 0.57 secs triton: cargo build --release -p yes triton: Compiling yes v0.0.1 (file:///root/coreutils/src/yes) triton: Finished release [optimized] target(s) in 0.61 secs triton: mkdir -p /usr/obj/bin
  • 50. Bootstrap an Image 50 triton: install /root/coreutils/target/release/arch /usr/obj/bin/arch; install /root/coreutils/target/release/base32 /usr/obj/bin/base32; install /root/coreutils/target/release/base64 /usr/obj/bin/base64; install /root/coreutils/target/ release/basename /usr/obj/bin/basename; install /root/coreutils/target/release/cat /usr/obj/bin/cat; install /root/ coreutils/target/release/chgrp /usr/obj/bin/chgrp; install /root/coreutils/target/release/chmod /usr/obj/bin/chmod; install /root/coreutils/target/release/chown /usr/obj/bin/chown; install /root/coreutils/target/release/chroot /usr/obj/ bin/chroot; install /root/coreutils/target/release/cksum /usr/obj/bin/cksum; install /root/coreutils/target/release/ comm /usr/obj/bin/comm; install /root/coreutils/target/release/cp /usr/obj/bin/cp; install /root/coreutils/target/ release/cut /usr/obj/bin/cut; install /root/coreutils/target/release/dircolors /usr/obj/bin/dircolors; install /root/ coreutils/target/release/dirname /usr/obj/bin/dirname; install /root/coreutils/target/release/du /usr/obj/bin/du; install /root/coreutils/target/release/echo /usr/obj/bin/echo; install /root/coreutils/target/release/env /usr/obj/bin/ env; install /root/coreutils/target/release/expand /usr/obj/bin/expand; install /root/coreutils/target/release/expr / usr/obj/bin/expr; install /root/coreutils/target/release/factor /usr/obj/bin/factor; install /root/coreutils/target/ release/false /usr/obj/bin/false; install /root/coreutils/target/release/fmt /usr/obj/bin/fmt; install /root/coreutils/ target/release/fold /usr/obj/bin/fold; install /root/coreutils/target/release/groups /usr/obj/bin/groups; install / root/coreutils/target/release/hashsum /usr/obj/bin/hashsum; install /root/coreutils/target/release/head /usr/obj/bin/ head; install /root/coreutils/target/release/hostid /usr/obj/bin/hostid; install /root/coreutils/target/release/ hostname /usr/obj/bin/hostname; install /root/coreutils/target/release/id /usr/obj/bin/id; install /root/coreutils/ target/release/install /usr/obj/bin/install; install /root/coreutils/target/release/kill /usr/obj/bin/kill; install / root/coreutils/target/release/link /usr/obj/bin/link; install /root/coreutils/target/release/ln /usr/obj/bin/ln; install /root/coreutils/target/release/logname /usr/obj/bin/logname; install /root/coreutils/target/release/mkdir /usr/ obj/bin/mkdir; install /root/coreutils/target/release/mkfifo /usr/obj/bin/mkfifo; install /root/coreutils/target/ release/mknod /usr/obj/bin/mknod; install /root/coreutils/target/release/mktemp /usr/obj/bin/mktemp; install /root/ coreutils/target/release/more /usr/obj/bin/more; install /root/coreutils/target/release/mv /usr/obj/bin/mv; install / root/coreutils/target/release/nice /usr/obj/bin/nice; install /root/coreutils/target/release/nl /usr/obj/bin/nl; install /root/coreutils/target/release/nohup /usr/obj/bin/nohup; install /root/coreutils/target/release/nproc /usr/obj/ bin/nproc; install /root/coreutils/target/release/od /usr/obj/bin/od; install /root/coreutils/target/release/paste /
  • 51. Bootstrap an Image 51 triton: triton: Rebuilding locate database: $ echo It worked ls(1), stat(1), and who(1) don't work and install(1) is missing options. Plus more WIP problems.
  • 52. Parting Thoughts • Swapping out syscall(2) tables is somehow okay with us • Do we package each binary? • Can we version binaries? We do shared objects and APIs • Squatting on the filesystem is not versioning or an API • What history? Nuke and pave. No instance up older than 3 years. • Need tight coupling? Depend on a specific package. • How can we embrace plurality (read: R&D) • SmartOS and OmniOS uses different userlands, all slightly different • What history? Nuke and pave. No instance up older than 3 years. 52
  • 53. Parting Thoughts • Writing BSD licensed software gives you access to the guild of like-minded commercially motivated individuals who require: • quality • debugability • observability • performance • shipability 53
  • 54. Parting Thoughts • Exporting an OSes base utilities takes excessive effort as the modern programming languages have shown • Embrace the future • If we could start over knowing we had 10M LoC and >1K contributors, what and how would we build an OS? • If the ideal is different than what we're doing now, let's begin to plan how to get to an ideal situation Post Talk Comment: "This is like introducing the metric system to the UNIX countries of the world." 54 This resonated with me
  • 55. Engineering [email protected] Sean Chittenden @SeanChittenden Thank you! Questions? https://github.com/sean-/BSDCan-2017 https://www.slideshare.net/SeanChittenden/universal-userland