Modern Software Infrastructures Revisited
- 1. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Modern Software Infrastructures Revisited Igor Bogicevic ([email protected]) January 29, 2011 Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 2. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Introduction and Axiomatics Complexity of a software does not reduce over a time. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 3. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Introduction and Axiomatics Complexity of a software does not reduce over a time. Features, robustness of code, scalable architecture are indeed increasing the quality of software, but are also increasing complexity. Refactoring, as a good practice, does not reduce complexity, but it merely pays for a technical debt. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 4. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Introduction and Axiomatics Complexity of a software does not reduce over a time. Features, robustness of code, scalable architecture are indeed increasing the quality of software, but are also increasing complexity. Refactoring, as a good practice, does not reduce complexity, but it merely pays for a technical debt. In environment where multiple components mutually depend complexity can explode. Moreover, complexity is not related to a sandbox of mutual dependencies, but also on the production environment as well. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 5. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Introduction and Axiomatics Complexity of a software does not reduce over a time. Features, robustness of code, scalable architecture are indeed increasing the quality of software, but are also increasing complexity. Refactoring, as a good practice, does not reduce complexity, but it merely pays for a technical debt. In environment where multiple components mutually depend complexity can explode. Moreover, complexity is not related to a sandbox of mutual dependencies, but also on the production environment as well. Embracing and controlling the complexity might be a better way. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 6. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Introduction and Axiomatics Complexity of a software does not reduce over a time. Features, robustness of code, scalable architecture are indeed increasing the quality of software, but are also increasing complexity. Refactoring, as a good practice, does not reduce complexity, but it merely pays for a technical debt. In environment where multiple components mutually depend complexity can explode. Moreover, complexity is not related to a sandbox of mutual dependencies, but also on the production environment as well. Embracing and controlling the complexity might be a better way. Integration complexity != Code complexity Integration complexity Code complexity Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 7. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Introduction and Axiomatics Complexity of a software does not reduce over a time. Features, robustness of code, scalable architecture are indeed increasing the quality of software, but are also increasing complexity. Refactoring, as a good practice, does not reduce complexity, but it merely pays for a technical debt. In environment where multiple components mutually depend complexity can explode. Moreover, complexity is not related to a sandbox of mutual dependencies, but also on the production environment as well. Embracing and controlling the complexity might be a better way. Integration complexity != Code complexity Integration complexity Code complexity Infrastructure complexity != Code complexity Infrastructure complexity Code complexity Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 8. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! What is the ”Software Infrastructure” infrastructure - noun: the basic, underlying framework or features of a system or organization. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 9. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! What is the ”Software Infrastructure” infrastructure - noun: the basic, underlying framework or features of a system or organization. software - noun: ... Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 10. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! What is the ”Software Infrastructure” infrastructure - noun: the basic, underlying framework or features of a system or organization. software - noun: ... Yes, I made up this one... Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 11. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! What is the ”Software Infrastructure” infrastructure - noun: the basic, underlying framework or features of a system or organization. software - noun: ... Yes, I made up this one... In my defense, I needed a glue term that will somehow connect software architecture, infrastructure, build and deploy process Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 12. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! What is the ”Software Infrastructure” infrastructure - noun: the basic, underlying framework or features of a system or organization. software - noun: ... Yes, I made up this one... In my defense, I needed a glue term that will somehow connect software architecture, infrastructure, build and deploy process ... configuration distribution, monitoring/alerting Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 13. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! What is the ”Software Infrastructure” infrastructure - noun: the basic, underlying framework or features of a system or organization. software - noun: ... Yes, I made up this one... In my defense, I needed a glue term that will somehow connect software architecture, infrastructure, build and deploy process ... configuration distribution, monitoring/alerting ... dispatching of system updates, server orchestration Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 14. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! What is the ”Software Infrastructure” infrastructure - noun: the basic, underlying framework or features of a system or organization. software - noun: ... Yes, I made up this one... In my defense, I needed a glue term that will somehow connect software architecture, infrastructure, build and deploy process ... configuration distribution, monitoring/alerting ... dispatching of system updates, server orchestration No, that’s not ”Information System”. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 15. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 16. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 17. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. What was sometimes a privilege of the few, became approachable to a larger audience. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 18. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. What was sometimes a privilege of the few, became approachable to a larger audience. An organic transformation - with more traffic and content, there was more data available to use. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 19. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. What was sometimes a privilege of the few, became approachable to a larger audience. An organic transformation - with more traffic and content, there was more data available to use. After certain point, you need to re-think how are you storing and processing your data. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 20. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. What was sometimes a privilege of the few, became approachable to a larger audience. An organic transformation - with more traffic and content, there was more data available to use. After certain point, you need to re-think how are you storing and processing your data. Once you get to the ”internet scale” of data, this becomes inevitable. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 21. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. What was sometimes a privilege of the few, became approachable to a larger audience. An organic transformation - with more traffic and content, there was more data available to use. After certain point, you need to re-think how are you storing and processing your data. Once you get to the ”internet scale” of data, this becomes inevitable. Scaling up the data processing heavily depends on the both nature and operations on top of data. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 22. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. What was sometimes a privilege of the few, became approachable to a larger audience. An organic transformation - with more traffic and content, there was more data available to use. After certain point, you need to re-think how are you storing and processing your data. Once you get to the ”internet scale” of data, this becomes inevitable. Scaling up the data processing heavily depends on the both nature and operations on top of data. That also applies to information retrieval, search on top of data, etc. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 23. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scope Of The Problem Hardware didn’t became cheap, however it became cheap enough. Operating system became a commodity. What was sometimes a privilege of the few, became approachable to a larger audience. An organic transformation - with more traffic and content, there was more data available to use. After certain point, you need to re-think how are you storing and processing your data. Once you get to the ”internet scale” of data, this becomes inevitable. Scaling up the data processing heavily depends on the both nature and operations on top of data. That also applies to information retrieval, search on top of data, etc. This problem was not new, and concepts and solutions existed on the market - but not as a commodity. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 24. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 25. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. New resources are driving both new patterns and paradigm shift. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 26. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. New resources are driving both new patterns and paradigm shift. Instead of infrastructure being driven by business requirements, in last couple of years we have an opposite situation: Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 27. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. New resources are driving both new patterns and paradigm shift. Instead of infrastructure being driven by business requirements, in last couple of years we have an opposite situation: Products and business are starting to be driven by available capacity. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 28. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. New resources are driving both new patterns and paradigm shift. Instead of infrastructure being driven by business requirements, in last couple of years we have an opposite situation: Products and business are starting to be driven by available capacity. On-demand infrastructure, in the matter of minutes. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 29. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. New resources are driving both new patterns and paradigm shift. Instead of infrastructure being driven by business requirements, in last couple of years we have an opposite situation: Products and business are starting to be driven by available capacity. On-demand infrastructure, in the matter of minutes. Dynamic allocation of resources - AWS, Google App Engine, Rackspace Cloud, Joyent SDC, etc. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 30. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. New resources are driving both new patterns and paradigm shift. Instead of infrastructure being driven by business requirements, in last couple of years we have an opposite situation: Products and business are starting to be driven by available capacity. On-demand infrastructure, in the matter of minutes. Dynamic allocation of resources - AWS, Google App Engine, Rackspace Cloud, Joyent SDC, etc. As proven again - when you have a hammer, everything looks like a nail. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 31. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Paradigm Shift Complexity moved from the code and software architecture to the infrastructure as a whole. New resources are driving both new patterns and paradigm shift. Instead of infrastructure being driven by business requirements, in last couple of years we have an opposite situation: Products and business are starting to be driven by available capacity. On-demand infrastructure, in the matter of minutes. Dynamic allocation of resources - AWS, Google App Engine, Rackspace Cloud, Joyent SDC, etc. As proven again - when you have a hammer, everything looks like a nail. Paradigm shift also leads to a new set of pointy-hair semantic and yugioh grade conversations with appropriate keywords (Cloud, SaaS, PaaS, ...). Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 32. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Comic Relief Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 33. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 34. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 35. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Many of them are open. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 36. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Many of them are open. To number some: distributed map/reduce, distributed stream computing, distributed analytical search, distributed graph databases, distributed synchronization. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 37. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Many of them are open. To number some: distributed map/reduce, distributed stream computing, distributed analytical search, distributed graph databases, distributed synchronization. Distributed everything? Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 38. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Many of them are open. To number some: distributed map/reduce, distributed stream computing, distributed analytical search, distributed graph databases, distributed synchronization. Distributed everything? Maybe you don’t need to distribute your problem after all. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 39. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Many of them are open. To number some: distributed map/reduce, distributed stream computing, distributed analytical search, distributed graph databases, distributed synchronization. Distributed everything? Maybe you don’t need to distribute your problem after all. In reality we’re struggling with infrastructure design as we’re pioneering good practices in the various scenarios. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 40. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Many of them are open. To number some: distributed map/reduce, distributed stream computing, distributed analytical search, distributed graph databases, distributed synchronization. Distributed everything? Maybe you don’t need to distribute your problem after all. In reality we’re struggling with infrastructure design as we’re pioneering good practices in the various scenarios. This means we have various tools, platforms, paradigms at our disposal, yet we’re hitting various nails with various hammers. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 41. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Infrastructure Specialization We’ve got to the point where we have multitude of paradigms to choose from. Many theoretical CS paradigms in distributed computing got to a stage of transformation to a production ready software. Many of them are open. To number some: distributed map/reduce, distributed stream computing, distributed analytical search, distributed graph databases, distributed synchronization. Distributed everything? Maybe you don’t need to distribute your problem after all. In reality we’re struggling with infrastructure design as we’re pioneering good practices in the various scenarios. This means we have various tools, platforms, paradigms at our disposal, yet we’re hitting various nails with various hammers. And this is a call for infrastructure specialization. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 42. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 43. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 44. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. ... however that scales only up to certain level. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 45. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. ... however that scales only up to certain level. Horizontal scaling imposes a new set of problems. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 46. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. ... however that scales only up to certain level. Horizontal scaling imposes a new set of problems. Horizontal scaling is completely driven by the domain of the problem and data. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 47. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. ... however that scales only up to certain level. Horizontal scaling imposes a new set of problems. Horizontal scaling is completely driven by the domain of the problem and data. Some of the problems and datasets are trivial to scale (sharding), most of them are not - especially from the standpoint of fault tolerance. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 48. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. ... however that scales only up to certain level. Horizontal scaling imposes a new set of problems. Horizontal scaling is completely driven by the domain of the problem and data. Some of the problems and datasets are trivial to scale (sharding), most of them are not - especially from the standpoint of fault tolerance. We are living victims of CAP theorem... Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 49. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. ... however that scales only up to certain level. Horizontal scaling imposes a new set of problems. Horizontal scaling is completely driven by the domain of the problem and data. Some of the problems and datasets are trivial to scale (sharding), most of them are not - especially from the standpoint of fault tolerance. We are living victims of CAP theorem... And large corpus of papers on distributed computing in general. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 50. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Scaling Up Scaling should be avoided if possible (as premature optimization). ... and in ideal world you would be able to ”throw some money on it” and solve the problem with vertical scaling. ... however that scales only up to certain level. Horizontal scaling imposes a new set of problems. Horizontal scaling is completely driven by the domain of the problem and data. Some of the problems and datasets are trivial to scale (sharding), most of them are not - especially from the standpoint of fault tolerance. We are living victims of CAP theorem... And large corpus of papers on distributed computing in general. Design for a failure - you’ll appreciate when you see your system running. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 51. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 52. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 53. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Product Management should move back to where it started - driving business requirements and articulating BizDev requests. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 54. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Product Management should move back to where it started - driving business requirements and articulating BizDev requests. ”We can’t hire sufficiently good engineers” is a lousy excuse. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 55. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Product Management should move back to where it started - driving business requirements and articulating BizDev requests. ”We can’t hire sufficiently good engineers” is a lousy excuse. Either you can’t attract great engineers, or you can’t build them. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 56. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Product Management should move back to where it started - driving business requirements and articulating BizDev requests. ”We can’t hire sufficiently good engineers” is a lousy excuse. Either you can’t attract great engineers, or you can’t build them. Problem is always in you - figure out what you’re doing wrong. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 57. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Product Management should move back to where it started - driving business requirements and articulating BizDev requests. ”We can’t hire sufficiently good engineers” is a lousy excuse. Either you can’t attract great engineers, or you can’t build them. Problem is always in you - figure out what you’re doing wrong. No-Ops - BOFH syndrome has no place in an environment with complex infrastructures. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 58. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Product Management should move back to where it started - driving business requirements and articulating BizDev requests. ”We can’t hire sufficiently good engineers” is a lousy excuse. Either you can’t attract great engineers, or you can’t build them. Problem is always in you - figure out what you’re doing wrong. No-Ops - BOFH syndrome has no place in an environment with complex infrastructures. Same standards should apply for System Engineers/DevOps as for Engineers. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 59. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! Organizational Changes No-Product, this is a red area for Product Management. Engineering should be empowered to drive the core product. Product Management should move back to where it started - driving business requirements and articulating BizDev requests. ”We can’t hire sufficiently good engineers” is a lousy excuse. Either you can’t attract great engineers, or you can’t build them. Problem is always in you - figure out what you’re doing wrong. No-Ops - BOFH syndrome has no place in an environment with complex infrastructures. Same standards should apply for System Engineers/DevOps as for Engineers. What is traditionally considered to be Engineering and Operations should be one organizational group. Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 60. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! We are hiring! We are fresh and well funded startup from Boston, MA. We are building a first open on-demand scalable platform for distributed genomic data processing. We’re in a BigData business (10GB-2Tb per genome). We have an ambition of becoming a de-facto platform for Bioinformatic data processing. We are looking for a talent across several categories: - Core Engineering - Frontend Application Engineers (python/django) - Frontend Developers (html5/js/css) - System Engineers/DevOps Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited
- 61. Outline Introduction and Axiomatics What is the ”Software Infrastructure” Scope Of The Problem Paradigm Shift Comic Relief Infrastructure Specialization Scaling Up Organizational Changes We are hiring! We are hiring! We are fresh and well funded startup from Boston, MA. We are building a first open on-demand scalable platform for distributed genomic data processing. We’re in a BigData business (10GB-2Tb per genome). We have an ambition of becoming a de-facto platform for Bioinformatic data processing. We are looking for a talent across several categories: - Core Engineering - Frontend Application Engineers (python/django) - Frontend Developers (html5/js/css) - System Engineers/DevOps If you’d like to work on a problems of large scale genomics data processing, please shoot me an email: [email protected] Igor Bogicevic ([email protected]) Modern Software Infrastructures Revisited