Jump Your First Box

WHAT

You can hear it by many names: jump box/jump server/bastion. I will use jump box or jumpbox from now on.

No, not this box, engineer is not physically healthy enough to jump this box

Imagine that you are a ultra rich man, who have no trust on the goverment and banking institutions, so you want your assets & property within your sight and your security team. You live in a large mansion which have fences all around the premises. The only way in and out is via the main gate where security guards always check for anyone's identity card, only authorized people allow to get inside the premises and granted with a special access card. From that, they can only access to rooms that match their access card.

The main gate/jumpbox is where your first state your identity before entering the premises/infrastructure, only authorized people/privileged account allowed to access the rooms/resources inside. For every group of people/account, there is a card/rule to define what they can do and where they can go.

In short:

A jump server is a server that is used to access other servers. It is also known as a bastion host. It is a server that is accessible from the internet and is used to access other servers that are not accessible from the internet.

Ref: https://roadmap.sh/guides/how-to-setup-a-jump-server

WHY

Like the way you want to protect your mansion from outsider, noone should freely take a look and make changes to your precious cloud infrastructure. A jumpbox acts as an intermediate gate between the private network and the internet. Only an authorized engineer/administrator is allowed to access to jumpbox and further access to private resources via private network. This help to add a barrier to prevent unauthorized accesses. Moreover, from a jumpbox, we can supervise any access by implementing monitoring and logging on a central channel. And even auditing after incident/breach in worst case scenarios.

If you have already read Part 1 , you may already know that I use devbox for every project I have worked on. A devbox is not any different than a jumpbox, but it is equiped with development tools that provide a remote workspace so that I can work from any laptop that can connect to my devbox via SSH from trusted network. Which make jumpbox/devbox an essential setup step for CloudOps.

Now you know what a jumpbox is, it is time to make one. This is a great starting point to practice what we learnt on the last post

HOW

🚨 READ THIS BEFORE MAKING ROOKIE MISTAKES 🚨

• A jumpbox is a public facing instance, so you have to configure your security group to allow access from a trusted source only
• A misconfigured jumpbox means a security risk for your infrastructure
• If this is your first time working on any cloud infrastructure, try to get a student account or free-tier account
• This article is for the sole purpose of sharing knowledge, so the example codes might and will not be maintained properly. But hey, a wise engineer once said: "It works on my machine"

Jump your first box:

Manual way is the trusted way, every ec2 launch requires a specific set of requirements:

• Your OS: Linux or Windows, with Linux comes a wide range of distritions. Here I chose Amazon Linux 2023.
• Networking: as I said, this is a devbox/jumpbox hence it will be public facing and need to be attached with a public IP (IPv4 in this case, which is explained later)
• Networking (cont.): a firewall rule aka security group to allow/deny inbound and outbound traffic. Protect your box against anonymous access
• Sizing: Nothing special, t3a.micro suit my need
• Instance profile (optional): You can skip this bit if it is your first time. Otherwise, only allow what you need in instance profile's policy

IaC speaking, you can refer to the example at https://github.com/tienvu461/CloudOps-with-a-touch-of-automation/tree/main/01-jump-your-first-box/terraform/my-ground-breaking-application

Copycopy code to clipboard
1data "cloudinit_config" "jumpbox_cloudinit" {
2  gzip          = true
3  base64_encode = true
4  part {
5    content_type = "text/cloud-config"
6    content      = file("path/to/scripts/jumpbox_cloudinit.yaml")
7  }
8}
9
10resource "aws_instance" "jumpbox" {
11  ami                         = "ami-abcd1234"
12  availability_zone           = "ap-southeast-1a"
13  ebs_optimized               = false
14  instance_type               = "t3a.micro"
15  monitoring                  = false
16  key_name                    = "jumpbox"
17  subnet_id                   = "subnet-abcd1234"
18  vpc_security_group_ids      = [sg-abcd1234]
19  associate_public_ip_address = true
20  source_dest_check           = true
21  iam_instance_profile        = "your_instance_profile_name"
22
23  root_block_device {
24    volume_type           = "gp3"
25    volume_size           = 30
26    delete_on_termination = true
27  }
28
29  user_data = data.cloudinit_config.jumpbox_cloudinit.rendered
30  tags = {
31    "Name"          = "your-jumpbox"
32    "AUTO_DNS_ZONE" = "YOURZONEID"
33    "AUTO_DNS_NAME" = "jumpbox.example.com"
34  }
35}

🚨 EXTRA CONTENT 🚨

You might have been noticing the weird tag keys AUTO_DNS_NAME, AUTO_DNS_ZONE above. At the time I was working the content of this blogpost, I read the news of AWS will begin to charge on public IPv4 with a rate of $0.005/hr from Feb, 2024 :evil: and with the initial of cost saving in mind, I decided to pump this tutorial one level higher.

• Any jumpbox should have the ability of start/stop on schedule and no need to use EIP
• And no, I dont want IPv6, I dont want EC2 Instance Connect or Session Manager as well

This is the plan:

• Since I want to be able to SSH to my jumpbox during my work, it should be running from 9-5 or 9-6 depends on which capitalism country you are living in.
• Outside of working hour, my jumpbox should be offline for both instance hour cost saving and IPv4 cost saving
• However, if my instance is start/stop everyday, its IPv4 will be dynamically assigned unless I associate it wil a Elastic IP which will also cost me $0.005/hr either used or idle. I want to have a A record to resolve to which ever IP is currently assigned to my jumpbox.
• Hence, everytime my instance is started at 9:00, the A record must be updated accordingly (which is a problem I cannot solve)
• But luckily, thanks John Rotenstein, he has a great article on how to tackle that problem

Copycopy code to clipboard
1#!/bin/bash
2# Extract information about the Instance
3echo "Updating jumpbox record"
4status_code=$(curl -s -o /dev/null -w "%{http_code}" http://169.254.169.254/latest/meta-data/)
5if [[ "$status_code" -eq 200 ]]
6then
7    INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id/)
8    AZ=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone/)
9    MY_IP=$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4/)
10else
11    TOKEN=`curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
12    INSTANCE_ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id/)
13    AZ=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone/)
14    MY_IP=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/public-ipv4/)
15fi
16# Extract tags associated with instance
17ZONE_TAG=$(aws ec2 describe-tags --region ${AZ::-1} --filters "Name=resource-id,Values=${INSTANCE_ID}" --query 'Tags[?Key==`AUTO_DNS_ZONE`].Value' --output text)
18NAME_TAG=$(aws ec2 describe-tags --region ${AZ::-1} --filters "Name=resource-id,Values=${INSTANCE_ID}" --query 'Tags[?Key==`AUTO_DNS_NAME`].Value' --output text)
19
20# Update Route 53 Record Set based on the Name tag to the current Public IP address of the Instance
21aws route53 change-resource-record-sets --hosted-zone-id $ZONE_TAG --change-batch '{"Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"'$NAME_TAG'","Type":"A","TTL":300,"ResourceRecords":[{"Value":"'$MY_IP'"}]}}]}'

All of the mentioned settings are implemented into the example terraform code with a small modification to make sure it run per-boot, because dynamic public IPv4 is re-assigned everytime the machine start. Although this is a very good solution, but it poses some security risks with assigned iam role, there are rooms for optimisation.

Now simply click "Launch" if you are using AWS console, or run 'terraform apply' if your setup is terraform. Bada bing, bada boom, your devbox is now in the room.

What's next

In the next article, I will write some python code to handle events from sources (EventBridge, Cloudwatch, ...) and using lambda, I can forward those notification to webhook (Slack, Discord, MS Teams), api, ... And I call it SNS Integration Forwarder Function - SNIFF