This module is used to check that the infrastructure rules are being used and provides automated notifications for compliance violations.

The enforcement tools support multiple notification methods:

• GitHub Issues: Automatically create GitHub issues with detailed compliance reports
• Email Notifications: Send email alerts via SMTP for compliance violations
• Console Output: Print detailed reports to console for manual review

The enforcement is done by validating the IAM policies against the defined policies. The tool monitors and enforces compliance for user permissions, service account roles, and group memberships across your GCP project.

You can specify the action either through the configuration file (config.yml) or via command-line arguments:

# Check compliance and report issues (default)
python iam.py --action check

# Create/update GitHub issue and send email if compliance violations are found
python iam.py --action announce

# Print announcement details for testing purposes (no actual issue created)
python iam.py --action print

# Generate new compliance file based on current IAM policy
python iam.py --action generate

• check: Validates IAM policies against defined policies and reports any differences (default behavior)
• announce: Creates or updates a GitHub issue and sends an email notification when IAM policies differ from the defined ones. If no open issue exists, creates a new one; if an open issue exists, updates the issue body with current violations
• print: Prints announcement details for testing purposes without creating actual GitHub issues or sending emails
• generate: Updates the compliance file to match the current GCP IAM policy, creating a new baseline from existing permissions

The IAM Policy enforcement tool provides the following capabilities:

• Comprehensive Policy Export: Automatically exports all IAM bindings and roles from the GCP project
• Member Type Recognition: Handles users, service accounts, and groups with proper parsing and identification
• Permission Comparison: Detailed comparison between expected and actual permissions for each user
• Conditional Role Filtering: Automatically excludes conditional roles (roles with conditions) from compliance checks
• Sorted Output: Provides consistent, sorted output for easy comparison and review
• Detailed Reporting: Comprehensive reporting of permission differences with clear before/after comparisons
• GitHub Integration: Automatic issue creation with detailed compliance violation reports
• Email Notifications: Optional email notifications for compliance issues via SMTP
• Issue Management: Smart issue handling - creates new issues when none exist, updates existing open issues with current violations
• Testing Support: Print action allows testing notification content without actually sending

The config.yml file supports the following parameters for IAM policies:

• project_id: GCP project ID to check (default: apache-beam-testing)
• users_file: Path to the YAML file containing expected IAM policies (default: ../iam/users.yml)
• action: Default action to perform (check, announce, print, or generate)
• logging: Logging configuration (level and format)

When using the announce action, the following environment variables are required:

• GITHUB_TOKEN: GitHub personal access token for creating issues
• GITHUB_REPOSITORY: Repository in format owner/repo (default: apache/beam)
• SMTP_SERVER: SMTP server for email notifications
• SMTP_PORT: SMTP port (default: 587)
• EMAIL_ADDRESS: Email address for sending notifications
• EMAIL_PASSWORD: Email password for authentication
• EMAIL_RECIPIENT: Email address to receive notifications

The IAM policy file should follow this YAML structure:

- username: john.doe
  email: john.doe@example.com
  permissions:
    - role: roles/viewer
    - role: roles/storage.objectViewer
- username: service-account-name
  email: service-account-name@project-id.iam.gserviceaccount.com
  permissions:
    - role: roles/compute.instanceAdmin
    - role: roles/iam.serviceAccountUser

Each user entry includes:

• username: The derived username (typically the part before @ in email addresses)
• email: The full email address of the user or service account
• permissions: List of IAM roles assigned to this member
  • role: The full GCP IAM role name (e.g., roles/viewer, roles/editor)

1. Policy Extraction: Retrieves current IAM policy from the GCP project
2. Member Parsing: Parses all IAM members and extracts usernames, emails, and types
3. Role Processing: Processes all roles while filtering out conditional bindings
4. Comparison: Compares current permissions with expected permissions from the policy file
5. Reporting: Generates detailed reports of any discrepancies found
6. Notification: Sends notifications via GitHub issues and/or email when using announce action

The print action can be used for testing notification content without actually creating GitHub issues or sending emails.

Command-line arguments take precedence over configuration file settings.

The enforcement tools are integrated with GitHub Actions to provide automated compliance monitoring. The workflow is configured to run weekly and automatically create GitHub issues and send email notifications for any policy violations.

The repository includes workflows for different security domains:

• IAM Policy Enforcer (.github/workflows/beam_Infrastructure_PolicyEnforcer.yml): Runs weekly on Mondays at 9:00 AM UTC.
• Unmanaged Keys Audit (.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml): Runs daily at 00:00 UTC. It manages the continuous execution of the account_keys.py script to swiftly detect rogue service account keys generated outside the official rotation system.
• Manual trigger: Can be triggered manually via workflow_dispatch
• Actions: Runs both IAM and Account Keys enforcement with the announce action

Note:

• The email service is configured to use gmail
• The recipient email is set to dev@beam.apache.org for Apache Beam project notifications
• The GITHUB_TOKEN is automatically provided by GitHub Actions and doesn't need to be configured manually

The enforcement is also done by validating service account keys and their access permissions against the defined policies. The tool supports three different actions when discrepancies are found:

You can specify the action either through the configuration file (config.yml) or via command-line arguments:

# Check compliance and report issues (default)
python account_keys.py --action check

# Create/update GitHub issue and send email if compliance violations are found
python account_keys.py --action announce

# Print announcement details for testing purposes (no actual issue created)
python account_keys.py --action print

# Generate new compliance file based on current service account keys policy
python account_keys.py --action generate

• check: Validates service account keys and their permissions against defined policies and reports any differences (default behavior)
• announce: Creates or updates a GitHub issue and sends an email notification when service account keys policies differ from the defined ones.
  • For general configuration errors, it updates the main compliance issue.
  • For unmanaged/rogue keys (Security Alerts), it consolidates alerts into a dedicated [SECURITY] issue acting as a live dashboard. It updates the issue by placing the newest audit report at the top and moving the previous reports into a collapsed <details> history section. If the keys are revoked and the infrastructure becomes healthy, the system automatically resolves and closes the issue.
• print: Prints announcement details for testing purposes without creating actual GitHub issues or sending emails
• generate: Updates the compliance file to match the current GCP service account keys and Secret Manager permissions

The Account Keys enforcement tool provides the following capabilities:

• Service Account Discovery: Automatically discovers all active (non-disabled) service accounts in the project
• Secret Manager Integration: Monitors secrets created by the beam-infra-secret-manager service
• Permission Validation: Ensures that Secret Manager permissions match the declared authorized users
• Compliance Reporting: Identifies missing service accounts, undeclared managed secrets, and permission mismatches
• Automatic Remediation: Can automatically update the compliance file to match current infrastructure state
• Unmanaged Key Detection: Identifies physical keys in IAM that were created outside the automated rotation system by comparing them against the legal, managed keys registered in Google Cloud Secret Manager.
• Stateful Security Alerts: Consolidates security violations into a single, dynamically updated GitHub issue, keeping a collapsed <details> history of past scans to prevent alert fatigue.
• Self-Healing Resolution: Automatically closes the security GitHub issue when all unmanaged keys have been successfully revoked.

The config.yml file supports the following parameters for account keys:

• project_id: GCP project ID to check
• service_account_keys_file: Path to the YAML file containing expected service account keys policies (default: ../keys/keys.yaml)
• action: Default action to perform (check, announce, print, or generate)
• logging: Logging configuration (level and format)

When using the announce action, the following environment variables are required:

• GITHUB_TOKEN: GitHub personal access token for creating issues
• GITHUB_REPOSITORY: Repository in format owner/repo (default: apache/beam)
• SMTP_SERVER: SMTP server for email notifications
• SMTP_PORT: SMTP port (default: 587)
• EMAIL_ADDRESS: Email address for sending notifications
• EMAIL_PASSWORD: Email password for authentication
• EMAIL_RECIPIENT: Email address to receive notifications

The service account keys file should follow this YAML structure:

service_accounts:
- account_id: example-service-account
  display_name: example-service-account@project-id.iam.gserviceaccount.com
  authorized_users:
  - email: user1@example.com
  - email: user2@example.com

Each service account entry includes:

• account_id: The unique identifier for the service account (without the full email domain)
• display_name: The full service account email address or any custom display name
• authorized_users: List of users who should have access to the service account's secrets