Last updated at Wed, 06 Dec 2017 17:50:35 GMT

Sysdig Falco is a new open source endpoint security monitoring tool for Linux released by Sysdig.  It allows you to create simple rules to alert on behaviors you find suspicious. Sysdig falco also supports containers, so you can easily alert on container activity.

Deploying Sysdig Falco

Sysdig Falco deploys on Linux servers.  Sysdig provides several mechanisms for deploying, including a one-liner sudo bash script and package management repos.

To get started, you can deploy Sysdig using a bash one-liner. Note, this method is useful for testing and experimentation, but due to security concerns you should probably not be using this method for production deployment.

# curl -s https://s3.amazonaws.com/download.draios.com/stable/install-falco | sudo bash
  • Deployment note:  Sysdig Falco requires deployment of a kernel module to your host servers, so make sure confirm this is ok with your deployment policies, and are whitelisting the behavior (if you are doing any monitoring or restrictions of kernel modules).

Let’s try running falco to confirm everything was installed correctly.  Passing the –help flag indicates several options for launching falco:

# falco --help 
Usage: falco [options]

Options: 
-h, --help Print this page 
-c Configuration file (default /mnt/workspace/falco-build-stable/label/builder-agent-64/falco/falco.yaml, /etc/falco.yaml) 
-o, --option <key>=<val> Set the value of option <key> to <val>. Overrides values in configuration file. <key> can be a two-part <key>.<subkey> 
-d, --daemon Run as a daemon 
-p, --pidfile <pid_file> When run as a daemon, write pid to specified file 
-e <events_file> Read the events from <events_file> (in .scap format) instead of tapping into live. 
-r <rules_file> Rules file (defaults to value set in configuration file, or /etc/falco_rules.yaml). 
-L Show the name and description of all rules and exit. 
-l <rule> Show the name and description of the rule with name <rule> and exit.

Configuring Falco

Sysdig Falco looks for its configuration using two yaml files which are installed by default in /etc/falco.yaml and /etc/falco_rules.yaml

The paths can be dynamically configured using the flags

falco -c <config file> -r <rules file>

Let’s look at /etc/falco.yaml, which controls several logging and high-level configuration items:

# cat /etc/falco.yaml 
# File containing Falco rules, loaded at startup. 
rules_file: /etc/falco_rules.yaml

# Whether to output events in json or text 
json_output: false

# Send information logs to stderr and/or syslog Note these are *not* security 
# notification logs! These are just Falco lifecycle (and possibly error) logs. 
log_stderr: true 
log_syslog: true 
# Where security notifications should go. 
# Multiple outputs can be enabled.

syslog_output: 
enabled: true

file_output: 
enabled: false 
filename: ./events.txt 
stdout_output: 
enabled: true

As you can see, there are several options for configuring security event output (to a file, syslog, or stdout) along with two options for formats (text vs JSON).   Let’s create a rule, and look at the example output for both.

Creating A Simple Alert Rule

Sysdig Falco comes with a simple rules filtering language for defining alerts.  Using this mechanism, you can alert on a number of things regarding behavior on a system (or ‘inside’ of a container, as we’ll look at below).   The rules by default are stored in /etc/falco_rules.yaml, and Sysdig has provided some default rules for you to use.

In /etc/falco_rules.yaml

look for a rule for system_binaries_network_activity

Let’s modify that rule slightly to alert on any and all network traffic:

 - rule: system_binaries_network_activity 
     desc: any network activity performed by system binaries that are not expected to send or receive any network traffic 
     condition: ((inbound or outbound) and (fd.sockfamily = ip)) and fd.name != '' 
     output: "Suspicious binary sent/received network traffic (user=%user.name command=%proc.cmdline connection=%fd.name type=%evt.type)" 
     priority: WARNING

As you can see, there are a few elements to building a rule.  Let’s look at these fields in detail.

Now, let’s try running falco

# falco 
Mon Jun 6 06:46:16 2016: Falco initialized with configuration file /etc/falco.yaml 
Mon Jun 6 06:46:16 2016: Parsed rules from file /etc/falco_rules.yaml

Try pinging a hostname, or opening a browser and going to a website.  You’ll see right away alert messages being logged to stdout.  By default, they should be in the text format, but you can try switching the json_output: true in /etc/falco.yaml to see examples of the JSON format.  I have included examples of both below:

Text log example:

 06:29:38.911653567: Warning Suspicious binary sent/received network traffic (user=root command=Chrome_IOThread connection=10.0.1.10:34165->74.125.192.189:443 type=connect)

JSON log example:

[{"evt.time":1465208656129019404,"evt.type":"connect","fd.name":"127.0.0.1:56184->127.0.0.1:24866","proc.cmdline":"Chrome_IOThread    ","user.name":"root"}

Notes:

  • The JSON format is nice for parsing, but at the current time, seems to lose some of the ‘alerting’ information you’d like (e.g. the output message).
  • The rule identifier is unfortunately not included in the alert message itself.
  • The priority defined in the rule is not included in the message itself (but is used by the syslog facility)

Monitoring Containers

Sysdig Falco can also be used to monitor activity inside of containers.  The recommended deployment strategy is to deploy on the host of the container, which will allow for full visibility throughout all of the containers running on the host –  although some limited functionality can be gleaned by running Sysdig Falco inside a container itself.

Let’s look at a simple alert rule specific to some container activity to see the container monitoring in action!

First, in one terminal window, let’s pull the container we are going to used to test, and run it:

# docker pull ubuntu:14.04 
# docker run --rm -it ubuntu:14.04 /bin/bash 
$

You should have a bash prompt inside of the container which we can use for testing.

On the host, let’s also modify the rule we created in /etc/falco_rules.yml to look for network activity **inside of a container **and record the the container id in our output message:

 - rule: system_binaries_network_activity_container 
    desc: any network activity performed by system binaries that are not expected to send or receive any network traffic in a container 
    condition: ((inbound or outbound) and (fd.sockfamily = ip)) and fd.name != '' and container 
    output: "Suspicious binary sent/received network traffic from container=%container.id (user=%user.name command=%proc.cmdlin 
    e connection=%fd.name type=%evt.type)" 
    priority: WARNING

Now let’s try running falco, and then typing ping google.com in our container!

# falco 
Mon Jun 6 06:57:56 2016: Falco initialized with configuration file /etc/falco.yaml 
Mon Jun 6 06:57:56 2016: Parsed rules from file /etc/falco_rules.yaml 
06:57:58.143196226: Warning Suspicious binary sent/received network traffic from container=9a96cb629b9a (user=root command=ping google.com connection=172.17.0.2:54523-&gt;8.8.8.8:53 type=connect) 
06:57:58.177714266: Warning Suspicious binary sent/received network traffic from container=9a96cb629b9a (user=root command=ping google.com connection=172.17.0.2:34597-&gt;146.115.8.93:1025 type=connect) 
06:57:58.196316749: Warning Suspicious binary sent/received network traffic from container=9a96cb629b9a (user=root command=ping google.com connection=172.17.0.2:38879-&gt;8.8.8.8:53 type=connect)

You should now see the container activity, along with the container id!

Limitations

Sysdig Falco appears to be a promising new tool in a defender’s toolkit, but there are some limitations you should be aware of as you are planning your deployments.

  • No centralized collection facilities. Sysdig Falco is only an endpoint tool; it does not come with integrations for any centralized collection or aggregation facilities. You can integrate it with Syslog or collect the individual logs from files as described above, and send to your Splunk, Logstash, or SIEM, but this will require some programming lift for your security engineers.

  • No built-in “signature management.” Signatures for Sysdig Falco are stored in a file, which again lacks any centralized management so to deploy any signature changes, you must integrate with a local configuration management system (e.g. Chef, Puppet, Ansible) to manage updates to these files.

  • No event aggregation. As observed above, often, you’ll get multiple events that are really related to the same activity.  To avoid ‘alert fatigue’ it would be nice if these events are aggregated into a single alert for viewing/testing purposes, but right now you must accomplish this within a SIEM or log management system.

  • Beware of any potential performance hit. Sysdig works by using a kernel module to hook various kernel activity so beware of any potential performance hit on a system that requires high-performance, such as a distributed database.  The product is mature and has undergone a lot of testing, but as with any system that is introducing a lot of logging, it’s probably a good idea to do some benchmarks using the rulesets you’d like to deploy before you deploy Sysdig Falco on a machine where performance is critical.