kubernetes security with falco & falco talon

Editor's Notes

• #1: Credits: this deck was created by Loris. The script was added by ewj@, after listening to Loris deliver it! Hi everyone, I'm $NAME. I work for Sysdig, where our mission is to make every cloud deployment secure and reliable. <very brief credential about you as a speaker> e.g. for Edd, "I've spent my career working with open source, from web development to big data and analytics, and from startups to Google. I'm really excited to be focusing my attention on modern security and to be able to talk to you today." Today I will be giving an introduction to run-time threat detection in the cloud, why we need it, and how Falco provides an open source solution to the problem. <audience interaction ideas> who here has heard of Falco? is anyone running it? who's running Kubernetes? VMs? in a Cloud? On-prem? what are you hoping to get out of today's session?
• #2: The exact problem we just talked about was the inspiration for creating the Falco project. "It's 10pm, do you know what your containers are doing?" Falco was created by Loris Degioanni, borne from his experience in network security gained as a lead developer of Wireshark. Network operations and security are impossible without deep visibility into hosts and traffic, so why not bring this same mentality to monitoring computing endpoints? Falco aims to be the security camera over hosts, containers and Kubernetes, and cloud services. It is an open source project, hosted under the aegis of the Cloud Native Computing Foundation (CNCF). Loris and Sysdig created and contributed the project to the CNCF, and it now has contributors from many other vendors. If you’re not familiar with the CNCF, it is an industry foundation created as part of the open-sourcing of Kubernetes by Google, and it is a home to many projects that are part of the modern application infrastructure stack, such as Kubernetes itself, Prometheus and gRPC.
• #3: So what is Falco? It’s a runtime security engine: that means it provides real-time threat detection from within your infrastructure. It supports observability into the endpoints you have: whether Linux hosts, virtual machines, or a container and Kubernetes based infrastructure. Additionally, as we’ll see later, it can monitor cloud infrastructure more broadly. Falco is built on eBPF, which means it provides a safe and standard way to add kernel-level observability capabilities. This gives Falco the highest degree of visibility possible inside the kernel — down to the syscall. The syscall, or system call, is a very important concept in an operating system kernel, mediating facilities such as file access, network operations, managing processes, and changing permissions. If something is happening, there’s going to be a syscall involved. (Of course, eBPF isn't everywhere yet, so Falco provides other methods such as kernel modules to provide the same functionality.) And as is appropriate for a modern security solution, Falco is deeply integrated with Kubernetes. You get an efficient deployment pattern, and added the context you need for understanding exactly where security events are happening inside your clusters. Think about it – your container may only live for a few minutes – it's vital that your security solution can still be effective even though the compute instance could be remarkably short-lived.
• #4: To give you a nuts and bolts idea of what Falco can do, take a look at a couple of these sample alerts, with different priority levels. You can see that you can track things such as whether an interactive shell session was created inside a container, or whether a program was spawned in such a way as to enable potentially harmful behavior, in this case remote code execution with netcat. You can also see that the alert has been augmented with other pertinent information such as the user, full command line, and container identity. You configure Falco with a series of rules to detect events like these: it receives the raw events from the kernel, and then matches to see if harmful situations are occuring. And of course, Falco ships with a collection of rules that address well-known threats.
• #5: Let’s have a look at the fundamental concept that Falco uses to drive these alerts, a Sensor. In much the same way that a contemporary physical security environment deploys a range of sensors in important places, so does Falco. The input to each sensor is twofold: the raw events that are occurring in the environment — such as Linux syscalls for example — and a set of rules that tell the sensor when it ought to generate an alert. The rule language is the engine room of Falco. Under the hood it's a YAML file that contains many instructions to counter common security threats. We already looked at shell spawning, or malicious processes. Other conditions that the default rule set can pick up on include: adding or changing users installation of software, or writes to a /bin/ directory SSH connections to unapproved hosts addition of cron jobs attempts to rewrite configuration for well known software such as nginx One important thing to note here is that the intelligence is pushed to the edge. There is a staggering amount of information generated by any particular computing endpoint, making real time threat detection extremely costly and difficult if you wanted to first collect and pool all the raw data. Instead, Falco is designed to recognize what is interesting as a threat, and forward only those alerts. These sensors execute on the edge, one per kernel: so in Kubernetes, that’s per-node (not per container or pod), or once in every host or VM.
• #6: So if we’re putting together a Falco deployment in practice, what does it look like? Here we can see a variety of sensors that may emit alerts: some from computing nodes, and others from cloud services or Kubernetes. Where those alerts go to next is up to you as the user. That's the role of the "collector" here. By default, Falco can send alerts to files, a syslog, a web service or gRPC endpoint, or another program. One common collector that is part of the project is called Falco Sidekick. Sidekick offers you a wide variety of integrations enabling you to forward events. So, you can direct them to realtime services such as Slack, or PagerDuty, forward them into a message queue on AWS, or collect them in an S3 bucket, for example. Falco Sidekick's outputs include chat services, metrics or observability destinations such as Datadog or Prometheus, alerting services, log aggregators, message queues, or even execute serverless functions such as AWS Lambda, Knative or Google Cloud Run. Being open source and flexible, you can integrate Falco into your broader security and monitoring infrastructure pretty easily. And if you want to share with others, it's not hard to join and contribute to a project such as Falco Sidekick: many others have. Obviously there's a lot beyond whichever "collector" you implement. So, why and where do we see people installing Falco? Here are some examples: Falco can provide the intrusion detection system that many compliance standards require, such as SOC2 To provides a base level of visibility into activity in Kubernetes clusters For fulfilling auditing use cases over network, process, file, activity Allows easy customization of detections to send events into SIEM/centralized security logging
• #8: On the previous slide, you saw events coming not just from operating system calls, but cloud infrastructure too. You’re probably wondering how they got there? Early in 2022, Falco introduced the concept of plugins. A plugin enables you to write an adapter for any event source, which will direct the input to Falco and enable you to write rules to take appropriate responses. Why are these useful? Let’s go back to our original analogy of the cloud being an amusement park – a collection of multiple diverse services and platforms. The attack surface is complex and spans multiple layers of concern. You can’t pick up a change in a cloud-configured role access via Linux syscall, for example. It's not just about the data in the host kernels anymore, even when you're running Kubernetes. Falco plug-ins enable us to address this through being able to integrate many external event sources. All you need is a plugin to deliver the events. One particularly powerful thing to note is that you can use your cloud’s logging service as an input – so that can route many data sources that you're interested in, as long as it can write to the logging service. As an example of this cloud capability, let’s think about GitHub. Many many modern enterprises are now using GitHub for source control, and unfortunately not every developer is as careful as we‘d like. One common anti pattern is a developer accidentally checking in a secret credential to the code base. With Falco’s GitHub plugin you can monitor every commit and immediately alert if something like this happens, or for instance, if a repository is changed from being private to public. It can also monitor GitHub actions, providing protection against increasingly common attacks such as cryptojacking, where a GitHub action can be compromised into (expensively) mining cryptocurrency. So, with plugins, Falco is a security monitoring approach that lets you secure the runtime defense of not just your application workloads but your cloud SaaS usage as well.
• #9: On the previous slide, you saw events coming not just from operating system calls, but cloud infrastructure too. You’re probably wondering how they got there? Early in 2022, Falco introduced the concept of plugins. A plugin enables you to write an adapter for any event source, which will direct the input to Falco and enable you to write rules to take appropriate responses. Why are these useful? Let’s go back to our original analogy of the cloud being an amusement park – a collection of multiple diverse services and platforms. The attack surface is complex and spans multiple layers of concern. You can’t pick up a change in a cloud-configured role access via Linux syscall, for example. It's not just about the data in the host kernels anymore, even when you're running Kubernetes. Falco plug-ins enable us to address this through being able to integrate many external event sources. All you need is a plugin to deliver the events. One particularly powerful thing to note is that you can use your cloud’s logging service as an input – so that can route many data sources that you're interested in, as long as it can write to the logging service. As an example of this cloud capability, let’s think about GitHub. Many many modern enterprises are now using GitHub for source control, and unfortunately not every developer is as careful as we‘d like. One common anti pattern is a developer accidentally checking in a secret credential to the code base. With Falco’s GitHub plugin you can monitor every commit and immediately alert if something like this happens, or for instance, if a repository is changed from being private to public. It can also monitor GitHub actions, providing protection against increasingly common attacks such as cryptojacking, where a GitHub action can be compromised into (expensively) mining cryptocurrency. So, with plugins, Falco is a security monitoring approach that lets you secure the runtime defense of not just your application workloads but your cloud SaaS usage as well.
• #11: On the previous slide, you saw events coming not just from operating system calls, but cloud infrastructure too. You’re probably wondering how they got there? Early in 2022, Falco introduced the concept of plugins. A plugin enables you to write an adapter for any event source, which will direct the input to Falco and enable you to write rules to take appropriate responses. Why are these useful? Let’s go back to our original analogy of the cloud being an amusement park – a collection of multiple diverse services and platforms. The attack surface is complex and spans multiple layers of concern. You can’t pick up a change in a cloud-configured role access via Linux syscall, for example. It's not just about the data in the host kernels anymore, even when you're running Kubernetes. Falco plug-ins enable us to address this through being able to integrate many external event sources. All you need is a plugin to deliver the events. One particularly powerful thing to note is that you can use your cloud’s logging service as an input – so that can route many data sources that you're interested in, as long as it can write to the logging service. As an example of this cloud capability, let’s think about GitHub. Many many modern enterprises are now using GitHub for source control, and unfortunately not every developer is as careful as we‘d like. One common anti pattern is a developer accidentally checking in a secret credential to the code base. With Falco’s GitHub plugin you can monitor every commit and immediately alert if something like this happens, or for instance, if a repository is changed from being private to public. It can also monitor GitHub actions, providing protection against increasingly common attacks such as cryptojacking, where a GitHub action can be compromised into (expensively) mining cryptocurrency. So, with plugins, Falco is a security monitoring approach that lets you secure the runtime defense of not just your application workloads but your cloud SaaS usage as well.
• #12: On the previous slide, you saw events coming not just from operating system calls, but cloud infrastructure too. You’re probably wondering how they got there? Early in 2022, Falco introduced the concept of plugins. A plugin enables you to write an adapter for any event source, which will direct the input to Falco and enable you to write rules to take appropriate responses. Why are these useful? Let’s go back to our original analogy of the cloud being an amusement park – a collection of multiple diverse services and platforms. The attack surface is complex and spans multiple layers of concern. You can’t pick up a change in a cloud-configured role access via Linux syscall, for example. It's not just about the data in the host kernels anymore, even when you're running Kubernetes. Falco plug-ins enable us to address this through being able to integrate many external event sources. All you need is a plugin to deliver the events. One particularly powerful thing to note is that you can use your cloud’s logging service as an input – so that can route many data sources that you're interested in, as long as it can write to the logging service. As an example of this cloud capability, let’s think about GitHub. Many many modern enterprises are now using GitHub for source control, and unfortunately not every developer is as careful as we‘d like. One common anti pattern is a developer accidentally checking in a secret credential to the code base. With Falco’s GitHub plugin you can monitor every commit and immediately alert if something like this happens, or for instance, if a repository is changed from being private to public. It can also monitor GitHub actions, providing protection against increasingly common attacks such as cryptojacking, where a GitHub action can be compromised into (expensively) mining cryptocurrency. So, with plugins, Falco is a security monitoring approach that lets you secure the runtime defense of not just your application workloads but your cloud SaaS usage as well.
• #13: On the previous slide, you saw events coming not just from operating system calls, but cloud infrastructure too. You’re probably wondering how they got there? Early in 2022, Falco introduced the concept of plugins. A plugin enables you to write an adapter for any event source, which will direct the input to Falco and enable you to write rules to take appropriate responses. Why are these useful? Let’s go back to our original analogy of the cloud being an amusement park – a collection of multiple diverse services and platforms. The attack surface is complex and spans multiple layers of concern. You can’t pick up a change in a cloud-configured role access via Linux syscall, for example. It's not just about the data in the host kernels anymore, even when you're running Kubernetes. Falco plug-ins enable us to address this through being able to integrate many external event sources. All you need is a plugin to deliver the events. One particularly powerful thing to note is that you can use your cloud’s logging service as an input – so that can route many data sources that you're interested in, as long as it can write to the logging service. As an example of this cloud capability, let’s think about GitHub. Many many modern enterprises are now using GitHub for source control, and unfortunately not every developer is as careful as we‘d like. One common anti pattern is a developer accidentally checking in a secret credential to the code base. With Falco’s GitHub plugin you can monitor every commit and immediately alert if something like this happens, or for instance, if a repository is changed from being private to public. It can also monitor GitHub actions, providing protection against increasingly common attacks such as cryptojacking, where a GitHub action can be compromised into (expensively) mining cryptocurrency. So, with plugins, Falco is a security monitoring approach that lets you secure the runtime defense of not just your application workloads but your cloud SaaS usage as well.
• #14: On the previous slide, you saw events coming not just from operating system calls, but cloud infrastructure too. You’re probably wondering how they got there? Early in 2022, Falco introduced the concept of plugins. A plugin enables you to write an adapter for any event source, which will direct the input to Falco and enable you to write rules to take appropriate responses. Why are these useful? Let’s go back to our original analogy of the cloud being an amusement park – a collection of multiple diverse services and platforms. The attack surface is complex and spans multiple layers of concern. You can’t pick up a change in a cloud-configured role access via Linux syscall, for example. It's not just about the data in the host kernels anymore, even when you're running Kubernetes. Falco plug-ins enable us to address this through being able to integrate many external event sources. All you need is a plugin to deliver the events. One particularly powerful thing to note is that you can use your cloud’s logging service as an input – so that can route many data sources that you're interested in, as long as it can write to the logging service. As an example of this cloud capability, let’s think about GitHub. Many many modern enterprises are now using GitHub for source control, and unfortunately not every developer is as careful as we‘d like. One common anti pattern is a developer accidentally checking in a secret credential to the code base. With Falco’s GitHub plugin you can monitor every commit and immediately alert if something like this happens, or for instance, if a repository is changed from being private to public. It can also monitor GitHub actions, providing protection against increasingly common attacks such as cryptojacking, where a GitHub action can be compromised into (expensively) mining cryptocurrency. So, with plugins, Falco is a security monitoring approach that lets you secure the runtime defense of not just your application workloads but your cloud SaaS usage as well.
• #15: So, you like the idea of Falco, and want to try it out. But you're worried about supportability and sustainability – who else is using Falco, and who is contributing to its maintenance? Here are just a few of the organizations that are using Falco. In addition to companies who have Falco deployed in their own infrastructure, there are several vendors with security solutions in the market that either ship Falco as part of their product or embed it somehow. Being open source, Falco encourages this, and works to integrate with many popular platforms and technologies. For example On AWS, Falco can be used for Fargate runtime security We worked to integrate Falco into Google's gVisor application sandbox for providing serverless security IBM's Sysflow telemetry and security solution uses Falco, and Red Hat's Stackroxx uses Falco libraries for data collection Sumologic uses Falco for Kubernetes anomaly detection At Sysdig it won't surprise you to know we use Falco too, having created it! We leverage Falco for the runtime component of our Secure product, and augment its capabilities with enterprise management and the intelligence from our Threat Research team.
• #19: So if you’d like to know more, what next? Short answer: try it! That's how open source works. Documentation and downloads for Falco can be found on Falco.org, and you can take a look at the code itself. The Falco user and contributor community hangs out on Slack, where you can get help if you’re stuck, or get to know how to join the community and help be a builder of Falco. At Sysdig, we've also created workshops to help you learn Falco. Look out for us at an industry conference near you: we'll keep you posted about new opportunities, and for large teams could schedule a custom workshop. And of course there’s the O’Reilly book, Practical Cloud Native Security with Falco, which we’ll be sending to you all as a follow-up from this session, as well as all the links on this slide. Thank you so much for joining us, and good luck in your journey in runtime threat detection!