Securing AWS Systems Manager Session Manager Port Forwarding

August 18, 2024    AWS SSM

Using the SSM Session Manager port forwarding feature is straightforward and requires minimal setup if you already manage your EC2 instances with AWS SSM. However, using it out-of-the-box without addressing the security and access controls can pose a risk. Once the SSM agent is installed and running on your EC2 instances, and connected to AWS SSM service, you can easily access any instance within your AWS account.

This is a continuation of my previous blog post AWS Systems Manager Session Manager Port Forwarding. In my previous post, I talked about AWS Systems Manager Session Manger port forwarding and demonstrated the feature. In this post, I will share a solution to enhance the security of AWS Systems Manager Session Manager port forwarding.

Solution Overview

There are several ways to establish a secure and scalable approach for using the port forwarding feature while reducing the attack surface.

To secure the feature, you will need to enforce restrictions using IAM policy for SSM API access and the SSM port forwarding document to ensure only authorised user can establish the SSM session.

  1. Create a custom SSM document to restrict which ports are exposed on the target instance.

    By default, SSM Session Manager can forward any port on your instance, which could lead to unintended exposure of sensitive services. To mitigate this risk, you can create a custom SSM document that only allows port forwarding to specific port(s).

    This custom document ensures that port forwarding can only be done on the specified port, providing a layer of security that prevents unauthorised access to other services on the instance.

  2. Set up an IAM policy that grants SSM access only to the custom SSM document.

    To further secure your setup, you will want to create an IAM policy that allows an IAM principal to use the custom SSM document without granting broader SSM permissions.

    The IAM principal (user or role) has the ability to start a session using the specific document but doesn’t have unrestricted SSM access. The policy limits the scope of what the principal can do with SSM, aligning with the principle of least privilege.

  3. Use tags to manage which documents can run and which EC2 instances they apply to.

    As your infrastructure scales, you may need to manage access to many instances. Rather than manually updating IAM policies, you can use tags to dynamically control which instances can be targeted with the port forwarding session.

    This tag-based approach is a scalable solution as it allows you to control which instances can be accessed via port forwarding without having to update the IAM policy every time a new instance is created.

Technical Walkthrough

I have coded the solution demo using Terraform. It’s an extension (v1.1) of the demo environment that I used in my previous blog post. You can check out the Terraform code in my GitHub repo.

Important note: Deploying the demo environment using the Terraform code will incur some cost in your AWS account even if you’re on free tier.

This is the high-level architectural diagram of the solution:

Secure SSM Port Forwarding Demo Architecture

The Terraform code secure-ssm.tf will create the following resources:

  • An SSM document for restricted port forwarding.
resource "aws_ssm_document" "restricted_forwarding" {
  name          = "RestrictedPortForwardingSession"
  document_type = "Session"

  content = jsonencode({
    schemaVersion = "1.0",
    description = "Document for initiating port forwarding sessions via Session Manager",
    sessionType = "Port",
    parameters = {
      portNumber = {
        type = "String",
        description = "Port number to expose from the instance (optional)",
        allowedPattern = "^(80|443|22|3389)$",
        default = "80"
      },
      localPortNumber = {
        type = "String",
        description = "Local machine port number for traffic forwarding (optional)",
        allowedPattern = "^([0-9]|[1-9][0-9]{1,3}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$",
        default = "0"
      }
    },
    properties = {
      portNumber = "{{ portNumber }}",
      type = "LocalPortForwarding",
      localPortNumber = "{{ localPortNumber }}"
    }
  })
}

In the parameter portNumber, I set the attribute allowPattern value to only accept port number 80, 443, 22, or 3389.

  • An IAM policy to allow starting SSM port forwarding sessions to specific ports on EC2 instances based on tagging.
resource "aws_iam_policy" "ssm_port_forwarding" {
  name = "SSMPortForwarding"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      # Permissions to list and describe EC2 instances and SSM sessions
      {
        Action   = [
          "ssm:DescribeSessions",
          "ssm:ListDocuments",
          "ssm:GetConnectionStatus",
          "ssm:DescribeInstanceInformation",
          "ec2:DescribeInstances"
        ]
        Effect   = "Allow"
        Resource = "*"
      },
      # Permission to initiate SSM sessions on EC instances with a specific tag
      {
        Action   = [
          "ssm:StartSession",
        ]
        Effect   = "Allow"
        Resource = [
          "arn:aws:ec2:*:${data.aws_caller_identity.current.account_id}:instance/*",
        ]
        Condition = {
          StringEquals = {
            "aws:resourceTag/PortForward" = "true"
          }
        }
      },
      # Permission to allow starting specific SSM document sessions
      {
        Action   = [
          "ssm:StartSession",
        ]
        Effect   = "Allow"
        Resource = [
          "arn:aws:ssm:ap-southeast-2::document/RestrictedPortForwardingSession",
        ]
        Condition = {
          BoolIfExists = {
            "ssm:SessionDocumentAccessCheck" = "true"
          }
        }
      },
      # Deny usage of default SSM port forwarding documents
      {
        Action   = [
          "ssm:StartSession",
        ]
        Effect   = "Deny"
        Resource = [
          "arn:aws:ssm:*:*:document/AWS-StartPortForwardingSession*",
        ]
      },
      # Permissions to terminate and resume SSM sessions initiated by the role
      {
        Action   = [
          "ssm:TerminateSession",
          "ssm:ResumeSession"
        ]
        Effect   = "Allow"
        Resource = "arn:aws:ssm:*:*:session/$${aws:userid}-*"
      }
    ]
  })
}

In the IAM policy, condition elements “aws:resourceTag/PortForward” = “true” and “ssm:SessionDocumentAccessCheck” = “true” are set for SSM start session permission. An SSM session can only be initiated on EC2 instances with the specified tag and the SSM document name RestrictedPortForwardingSession.

With the SessionDocumentAccessCheck condition set to true, the document-name parameter must be specified when starting a session using the AWS CLI. Otherwise, the request will fail.

The policy also denies the default AWS SSM port forwarding documents from being used.

  • An IAM role to be used for the solution demo.
resource "aws_iam_role" "ssm_port_forwarding" {
  name                = "SSMPortForwarding"
  managed_policy_arns = [
    aws_iam_policy.ssm_port_forwarding.arn
  ]

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Sid    = ""
        Principal = {
          AWS = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:root"
        }
      }]
  })
}

Let’s now test the solution.

Quick Demo

Note that the EC2 instances have been tagged accordingly with Terraform. The Linux EC2 instance has the tag “PortForward” set to “true” and the Windows EC2 instance has the tag “PortForward” set to “false”.

Step 1: Configure AWS credentials

I need an AWS user credentials with the IAM permission to assume role. I simply going to use my Administrator access keys from IAM Identity Center.

AWS SSO Access Keys

Copy and paste the access keys into the terminal.

Step 2: Assume the restricted SSM role

I created this IAM role SSMPortForwarding with the IAM policy attached through Terraform. I will use this role for the demo.

(Note: the AWS account ID has been redacted)

resp=$(aws sts assume-role --role-arn arn:aws:iam::123456789012:role/SSMPortForwarding --role-session-name SSMPortForwardingDemo)
export AWS_ACCESS_KEY_ID=$(echo $resp | jq -r '.Credentials.AccessKeyId')
export AWS_SECRET_ACCESS_KEY=$(echo $resp | jq -r '.Credentials.SecretAccessKey')
export AWS_SESSION_TOKEN=$(echo $resp | jq -r '.Credentials.SessionToken')

Step 3: Initiate SSM session for authorised port numbers

Run the AWS CLI command to start the SSM session on the Linux EC2 instance.

INSTANCE_ID=$(aws ec2 describe-instances \
  --filter "Name=tag:Name,Values=ssm-demo-linux" \
  --query "Reservations[].Instances[?State.Name == 'running'].InstanceId[]" --output text)
aws ssm start-session --target $INSTANCE_ID \
  --document-name RestrictedPortForwardingSession \
  --parameters '{"portNumber":["22"],"localPortNumber":["8022"]}'

The port forwarding session is connected successfully and waiting for connections.

Starting session with SessionId: SSMPortForwardingDemo-c3aejch27iopbpyuka6pc5b7kq
Port 8022 opened for sessionId SSMPortForwardingDemo-c3aejch27iopbpyuka6pc5b7kq.
Waiting for connections...

On a different terminal window, I run the shell command to create a SSH connection over the port forwarding tunnel and login.

$ ssh -p 8022 -o "StrictHostKeyChecking no" ssm-demo@localhost
Warning: Permanently added '[localhost]:8022' (ED25519) to the list of known hosts.
ssm-demo@localhost's password:
   ,     #_
   ~\_  ####_        Amazon Linux 2
  ~~  \_#####\
  ~~     \###|       AL2 End of Life is 2025-06-30.
  ~~       \#/ ___
   ~~       V~' '->
    ~~~         /    A newer version of Amazon Linux is available!
      ~~._.   _/
         _/ _/       Amazon Linux 2023, GA and supported until 2028-03-15.
       _/m/'           https://aws.amazon.com/linux/amazon-linux-2023/

[ssm-demo@ip-10-0-1-47 ~]$

I’m able to log onto the Amazon Linux EC2 instance successfully.

Let’s try port 443 connection.

$ aws ssm start-session --target $INSTANCE_ID \
>   --document-name RestrictedPortForwardingSession \
>   --parameters '{"portNumber":["443"],"localPortNumber":["8443"]}'

Starting session with SessionId: SSMPortForwardingDemo-hjhsq77ohwnueawqv3oy2mn3zm
Port 8443 opened for sessionId SSMPortForwardingDemo-hjhsq77ohwnueawqv3oy2mn3zm.
Waiting for connections...

The port forwarding tunnel is established successfully. As there’s no web service running on this demo instance, I won’t establish a HTTPS connection through the tunnel.

Step 4: Initiate SSM session for unauthorised port number

Now let’s try to use an unauthorised port number, eg. 8080.

An error occurred (InvalidParameters) when calling the StartSession operation: Parameter "portNumber" has value "8080" not matching "^(80|443|22|3389)$"

As expected, port 8080 is not allowed and the session request failed.

Step 5: Initiate SSM session for unauthorised EC2 instance

For this step, I will use the Windows EC2 instance. Run the AWS CLI command to start the SSM session.

INSTANCE_ID=$(aws ec2 describe-instances \
  --filter "Name=tag:Name,Values=ssm-demo-windows" \
  --query "Reservations[].Instances[?State.Name == 'running'].InstanceId[]" --output text)
aws ssm start-session --target $INSTANCE_ID \
  --document-name RestrictedPortForwardingSession \
  --parameters '{"portNumber":["3389"],"localPortNumber":["13389"]}'

Looks like I’m unable to create the session.

An error occurred (AccessDeniedException) when calling the StartSession operation: User: arn:aws:sts::123456789012:assumed-role/SSMPortForwarding/SSMPortForwardingDemo is not authorized to perform: ssm:StartSession on resource: arn:aws:ec2:ap-southeast-2:123456789012:instance/i-0a22e6982e121c7c8 because no identity-based policy allows the ssm:StartSession action

This is due to the IAM policy only allows the SSM session to be started with correctly tagged EC2 instance as the target. The Windows EC2 tag PortForward was set to false.

Step 6: Initiate SSM session using the default SSM document

What if I try to use the default AWS SSM document AWS-StartPortForwardingSession?

$ aws ssm start-session --target $INSTANCE_ID \
>   --document-name AWS-StartPortForwardingSession \
>   --parameters '{"portNumber":["3389"],"localPortNumber":["13389"]}'

I get an access denied error.

An error occurred (AccessDeniedException) when calling the StartSession operation: User: arn:aws:sts::123456789012:assumed-role/SSMPortForwarding/SSMPortForwardingDemo is not authorized to perform: ssm:StartSession on resource: arn:aws:ssm:ap-southeast-2::document/AWS-StartPortForwardingSession with an explicit deny in an identity-based policy

So I’m unable start a session using the default AWS SSM document AWS-StartPortForwardingSession because of the policy restriction.

The IAM policy is working in a way I wanted. 👍

Summary

By creating custom SSM documents with strict IAM policies and using tags for access control, you can significantly improve the security around AWS SSM Session Manager port forwarding feature. This approach minimises the risk of exposing unnecessary ports and ensures that access is tightly controlled, adhering to the best practices of least privilege and resource tagging. These measures will help maintain a secure and manageable environment as you scale your infrastructure.

Reference