Unix in the Cloud — Ignorance, Stagnation, Obsolescence
0 likes447 views
This document discusses the limitations of traditional operating systems in large cloud computing environments and proposes incremental steps to address these limitations. It argues that the concept of the operating system is becoming obsolete as distributed applications take on more OS-like responsibilities. Some specific problems mentioned include a lack of scalability, fault tolerance, distributed configuration management, and storage solutions for large multi-node environments. The document advocates for a mindful approach focused on immediate practical improvements while keeping a long-term perspective.
Unix in the Cloud — Ignorance, Stagnation, Obsolescence
- 1. Unix in the Cloud Ignorance, Stagnation, Obsolescence
- 2. Synopsis ▪ cloud in the broad sense of ideology ▪ not quite about running BSD on EC2 ▪ very limited to skills and experience of yours humbly
- 3. Multi-core ▪ installation? ▪ configuration management? ▪ load balancing?
- 4. Multi-node ▪ installation? ▪ configuration management? ▪ load balancing? ▪ why multi-node?
- 5. Large Computing Needs ▪ Facebook, Google, ... ▪ more than any OS can provide
- 6. Happy Hardware Vendor Law The amount of nodes needed to solve a given task doubles every now and again.
- 7. OS Scalability Limit ▪ 1 node only ▪ multi-socket and stacks approaching NUMA ▪ E25K, z10, etc — fail for most purposes
- 8. Operating System — ? ▪ traditional definition no more relevant ▪ the notion itself on the brink of obsolescence ▪ field heavily eroded by current distributed apps
- 9. Distributed Applications ▪ forced to be an OS unto themselves ▪ huge overlap ▪ huge opportunity for sharing and consolidation
- 10. Anti-Patterns ▪ virtualization ▪ chefs and puppets ▪ thick abstraction
- 11. Attempts ▪ z/OS ▪ Plan 9, Inferno ▪ Clustrx, E1, DYSEAC, ... ▪ OpenStack (~~)
- 12. Species Survival Plan Freeze the bodies and leave them for future generations to fix.
- 13. Don't Panic: Incremental ▪ perfection v. done ▪ still a decade or more till a good AI ▪ no practical need for POSIX over a cloud
- 14. Mindful Approach ▪ immediate practicality ▪ long-term perspective ▪ sustained, integrally rich effect
- 15. Operating System ▪ major abstraction repository ▪ overlapping code distillery ▪ pre-production architecture research
- 16. Increments
- 17. Machine Generated Data ▪ logs, error messages, status monitors ▪ meant for humans... no more ▪ rethinking for better aggregation and analysis
- 18. Identity and Authentication ▪ YP, LDAP outdated and poorly supported ▪ no distributed model ▪ passwd in git as a first stab
- 19. Remote Procedure Call ▪ ssh losing relevance, HPN or not ▪ all-mighty agent daemon worse than rsh ▪ capabilities, RBAC, WoT
- 20. Hardware Failures ▪ no culture for low-level fault-tolerance ▪ watchdogd as state-of-the-art self-healing ▪ focus on self-diagnostics: disk error counters, etc
- 21. Distributed Configuration ▪ current anti-patterns worsen the problem ▪ role-aware configuration ▪ / in git as a second stab
- 22. Storage ▪ intra-node redundancy irrelevant ▪ no appropriate local multi-disk FS ▪ no fast path for data exchange ▪ nginx + curl + dispatcher
- 23. Error Handling ▪ cf MGD and hardware failures ▪ software is 10x more prone to failures ▪ serious problem at scale
- 24. ☺