Implements a v2 Lambda Output with AssumeRole

[Ryxias]

to: @ryandeivert @blakemotl @chunyong-lin
cc: @airbnb/streamalert-maintainers

Background

We needed to be able to invoke Lambda across AWS accounts. The problem is the existing Lambda output seems to be.. wrong. It doesn't derive configs from the right place.

Instead of breaking reverse compatibility, I just added a v2 that is easier to set up and is more consistent with existing implementations. This v2 also has the ability to make sts:AssumeRole calls, in order to make Lambda invocations across accounts. However, it assumes that you already set up the IAM Policies correctly.

Testing

Tested on stage; built a test lambda and was able to invoke it.

[coveralls]

Coverage decreased (-0.007%) to 95.425% when pulling 95e36a6 on dw--lambuh into f4c343a on release-3-2-0.

[ryandeivert]

IIRC, this is not actually how this info will be written to the outputs.json config file for AWSOutput types .. see here:

streamalert/streamalert/alert_processor/outputs/aws.py

Lines 50 to 69 in 41da6b5

	def format_output_config(cls, service_config, values):
	"""Format the output configuration for this AWS service to be written to disk
	AWS services are stored as a dictionary within the config instead of a list so
	we have access to the AWS value (arn/bucket name/etc) for Terraform
	Args:
	service_config (dict): The actual outputs config that has been read in
	values (OrderedDict): Contains all the OutputProperty items for this service
	Returns:
	dict{<string>: <string>}: Updated dictionary of descriptors and
	values for this AWS service needed for the output configuration
	NOTE: S3 requires the bucket name, not an arn, for this value.
	Instead of implementing this differently in subclasses, all AWSOutput
	subclasses should use a generic 'aws_value' to store the value for the
	descriptor used in configuration
	"""
	return dict(service_config.get(cls.__service__, {}),
	**{values['descriptor'].value: values['aws_value'].value})

did you confirm that this actually works as expected with the manage.py outputs new ... command?

[Ryxias]

Oh wow, no this is totally not gonna work. Good catch. No I didn't confirm it as I'm still testing it, but it's good to know that it's definitely wrong.

I think the easy fix is to just not inherit from this base class and just use the OutputDispatcher base class like any other sane output.

[ryandeivert]

agreed - we can trend away from the AWSOutput class usage for future aws outputs now that ssm is a thing

[ryandeivert]

lgtm with one curiosity/question

[Ryxias]

I got this to work with Terraform configurations that look like:

# Module for cross-account assume role policies
resource "aws_iam_role_policy" "assume_role_policy_on_alert" {
  name   = "AssumeRoleOnTestLambdaAccount"
  role   = module.alert_processor_lambda.role_id
  policy = data.aws_iam_policy_document.allow_assume_role.json
}

data "aws_iam_policy_document" "allow_assume_role" {
  statement {
    effect = "Allow"

    actions = [
      "sts:AssumeRole"
    ]

    resources = ["arn:aws:iam::999999999999:role/service-role/derek_test_func-role-k29ynxjk"]
  }
}

# Role on cross account 999999999999
resource "aws_iam_role" "test_role" {
  name   = "derek_test_func-role-k29ynxjk"
  path   = "/service-role/"
  assume_role_policy = data.aws_iam_policy_document.instance-assume-role-policy.json
}

data "aws_iam_policy_document" "instance-assume-role-policy" {
  statement {
    effect = "Allow"
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
  }

  statement {
    effect = "Allow"
    actions = ["sts:AssumeRole"]

    principals {
      type = "AWS"

      # For some reason this doesn't work? You gotta grant to the **assumed** role?
      # identifiers = ["arn:aws:iam::123456789012:role/streamalert/derek20200406_streamalert_alert_processor_role"]
      identifiers = ["arn:aws:sts::123456789012:assumed-role/derek20200406_streamalert_alert_processor_role/derek20200406_streamalert_alert_processor"]
    }
  }
}

resource "aws_iam_role_policy" "invoke_test_func" {
  name   = "AssumeRoleOnTestLambdaAccount"
  role   = aws_iam_role.test_role.id
  policy = data.aws_iam_policy_document.invoke_policy.json
}

data "aws_iam_policy_document" "invoke_policy" {
  statement {
    effect = "Allow"
    actions = [
      "lambda:InvokeFunction"
    ]
    resources = [
      "arn:aws:lambda:us-east-1:123456789012:function:derek_test_func"
    ]
  }
}