User Data in EC2 for CloudFormation
- We can have user data at EC2 instance launch through the console
- We can also include it in CloudFormation
- The important thing to pass is the entire script through the function
Fn::Base64
- Good to know: user data script log is in
/var/log/cloud-init-output.log
Let’s see how to do this in CloudFormation. We have such an yml file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
--- Parameters: SSHKey: Type: AWS::EC2::KeyPair::KeyName Description: Name of an existing EC2 KeyPair to enable SSH access to the instance Resources: MyInstance: Type: AWS::EC2::Instance Properties: AvailabilityZone: eu-central-1a ImageId: ami-00a205cb8e06c3c4e InstanceType: t2.micro KeyName: !Ref SSHKey SecurityGroups: - !Ref SSHSecurityGroup # we install our web server with user data UserData: Fn::Base64: | #!/bin/bash -xe yum update -y yum install -y httpd systemctl start httpd systemctl enable httpd echo "Hello World from user data" > /var/www/html/index.html # our EC2 security group SSHSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: SSH and HTTP SecurityGroupIngress: - CidrIp: 0.0.0.0/0 FromPort: 22 IpProtocol: tcp ToPort: 22 - CidrIp: 0.0.0.0/0 FromPort: 80 IpProtocol: tcp ToPort: 80 |
Next->
Next -> Create stack
After the EC2 instance has been created we can check the public DNS address of the EC2 and go to that adress in the browser:
We can connect to the EC2 through SSH and check the output of user data:
1 |
cat /var/log/cloud-init-output.log |
cfn init
AWS::CloudFormation::Init
must be in the Metadata of a resource- With the cfn-init script, it helps make complex EC2 configurations readable
- The EC2 instance will query the CloudFormation service to get init data
- Logs go to
/var/log/cfn-init.log
Consider such a tamplate:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
--- Parameters: SSHKey: Type: AWS::EC2::KeyPair::KeyName Description: Name of an existing EC2 KeyPair to enable SSH access to the instance Resources: MyInstance: Type: AWS::EC2::Instance Properties: AvailabilityZone: eu-central-1a ImageId: ami-00a205cb8e06c3c4e InstanceType: t2.micro KeyName: !Ref SSHKey SecurityGroups: - !Ref SSHSecurityGroup # we install our web server with user data UserData: Fn::Base64: !Sub | #!/bin/bash -xe # Get the latest CloudFormation package yum update -y aws-cfn-bootstrap # Start cfn-init /opt/aws/bin/cfn-init -s ${AWS::StackId} -r MyInstance --region ${AWS::Region} || error_exit 'Failed to run cfn-init' Metadata: Comment: Install a simple Apache HTTP page AWS::CloudFormation::Init: config: packages: yum: httpd: [] files: "/var/www/html/index.html": content: | <h1>Hello World from EC2 instance!</h1> <p>This was created using cfn-init</p> mode: '000644' commands: hello: command: "echo 'hello world'" services: sysvinit: httpd: enabled: 'true' ensureRunning: 'true' # our EC2 security group SSHSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: SSH and HTTP SecurityGroupIngress: - CidrIp: 0.0.0.0/0 FromPort: 22 IpProtocol: tcp ToPort: 22 - CidrIp: 0.0.0.0/0 FromPort: 80 IpProtocol: tcp ToPort: 80 |
Cfn init is more readable way to install packages and modify the files on the system than user data way.
Let’s create a CfnInitExample stack:
Next ->
Next -> Create stack
Now we can go to the url of the new EC2 instance:
Now let’s ssh to the EC2 instance:
1 |
sudo cat /var/log/cloud-init-output.log |
And
1 2 3 4 5 6 7 8 9 10 11 |
[ec2-user@ip-172-31-24-120 ~]$ sudo cat /var/log/cfn-init.log 2021-06-01 12:50:36,880 [INFO] -----------------------Starting build----------------------- 2021-06-01 12:50:36,881 [INFO] Running configSets: default 2021-06-01 12:50:36,883 [INFO] Running configSet default 2021-06-01 12:50:36,884 [INFO] Running config config 2021-06-01 12:50:48,881 [INFO] Yum installed ['httpd'] 2021-06-01 12:50:48,888 [INFO] Command hello succeeded 2021-06-01 12:50:49,012 [INFO] enabled service httpd 2021-06-01 12:50:49,188 [INFO] Started httpd successfully 2021-06-01 12:50:49,189 [INFO] ConfigSets completed 2021-06-01 12:50:49,189 [INFO] -----------------------Build complete----------------------- |
All the logs:
1 2 3 |
[ec2-user@ip-172-31-24-120 ~]$ ls /var/log amazon boot.log cfn-init-cmd.log cfn-wire.log cloud-init.log cron grubby_prune_debug journal maillog sa spooler wtmp audit btmp cfn-init.log chrony cloud-init-output.log dmesg httpd lastlog messages secure tallylog yum.log |
If we need troubleshot:
1 2 3 4 5 6 7 8 9 10 11 |
[ec2-user@ip-172-31-24-120 ~]$ sudo cat /var/log/cfn-init.log 2021-06-01 12:50:36,880 [INFO] -----------------------Starting build----------------------- 2021-06-01 12:50:36,881 [INFO] Running configSets: default 2021-06-01 12:50:36,883 [INFO] Running configSet default 2021-06-01 12:50:36,884 [INFO] Running config config 2021-06-01 12:50:48,881 [INFO] Yum installed ['httpd'] 2021-06-01 12:50:48,888 [INFO] Command hello succeeded 2021-06-01 12:50:49,012 [INFO] enabled service httpd 2021-06-01 12:50:49,188 [INFO] Started httpd successfully 2021-06-01 12:50:49,189 [INFO] ConfigSets completed 2021-06-01 12:50:49,189 [INFO] -----------------------Build complete----------------------- |
cfn signal & wait conditions
- We still don’t know how to tell CloudFormation that the EC2 instance got properly configured after a cfn-init
- For this, we can use the cfn-signal script!
- We run cfn-signal right after cfn-init
- Tell CloudFormation service to keep on going or fail
- We need to define WaitCondition:
- Block the template until it receives a signal from cfn-signal
- We attach a CreationPolicy (also works on EC2, ASG)
Consider such a tamplate:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
--- Parameters: SSHKey: Type: AWS::EC2::KeyPair::KeyName Description: Name of an existing EC2 KeyPair to enable SSH access to the instance Resources: MyInstance: Type: AWS::EC2::Instance Properties: AvailabilityZone: eu-central-1a ImageId: ami-00a205cb8e06c3c4e InstanceType: t2.micro KeyName: !Ref SSHKey SecurityGroups: - !Ref SSHSecurityGroup # we install our web server with user data UserData: Fn::Base64: !Sub | #!/bin/bash -xe # Get the latest CloudFormation package yum update -y aws-cfn-bootstrap # Start cfn-init /opt/aws/bin/cfn-init -s ${AWS::StackId} -r MyInstance --region ${AWS::Region} # Start cfn-signal to the wait condition /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource SampleWaitCondition --region ${AWS::Region} Metadata: Comment: Install a simple Apache HTTP page AWS::CloudFormation::Init: config: packages: yum: httpd: [] files: "/var/www/html/index.html": content: | <h1>Hello World from EC2 instance!</h1> <p>This was created using cfn-init</p> mode: '000644' commands: hello: command: "echo 'hello world'" services: sysvinit: httpd: enabled: 'true' ensureRunning: 'true' SampleWaitCondition: CreationPolicy: ResourceSignal: Timeout: PT2M Count: 1 Type: AWS::CloudFormation::WaitCondition # our EC2 security group SSHSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: SSH and HTTP SecurityGroupIngress: - CidrIp: 0.0.0.0/0 FromPort: 22 IpProtocol: tcp ToPort: 22 - CidrIp: 0.0.0.0/0 FromPort: 80 IpProtocol: tcp ToPort: 80 |
Using this template let’s create a stack:
Next -> Create stack
cfn-signal failures troubleshooting
Wait Condition Didn’t Receive the Required Number of Signals from an Amazon EC2 Instance
- Ensure that the AMI you’re using has the AWS CloudFormation helper scripts installed. If the AMI doesn’t include the helper scripts, you can also cownload them to your instance.
- Verify that the
cfn-init
&cfn-signal
command was successfully run on the instance. You can view logs, such as/var/log/cloud-init.log
or/var/log/cfn-init.log
, to help you debug the instance launch. - You can retrieve the logs by logging in to your instance, but you must disable rollback on failure or else AWS-CloudFormation deletes the instance after your stack fails to create.
- Verify that the instance has a connection to the Internet. If the instance is in a VPC, the instance should be able to connect to the Internet through a NAT device if it’s is in a private subnet or through an Internet gateway if it’s in a public subnet
- For example, run:
curl -I https://aws.amazon.com
Consider a following yml tamplate:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
--- Parameters: SSHKey: Type: AWS::EC2::KeyPair::KeyName Description: Name of an existing EC2 KeyPair to enable SSH access to the instance Resources: MyInstance: Type: AWS::EC2::Instance Properties: AvailabilityZone: eu-central-1a ImageId: ami-00a205cb8e06c3c4e InstanceType: t2.micro KeyName: !Ref SSHKey SecurityGroups: - !Ref SSHSecurityGroup # we install our web server with user data UserData: Fn::Base64: !Sub | #!/bin/bash -xe # Get the latest CloudFormation package yum update -y aws-cfn-bootstrap # Start cfn-init /opt/aws/bin/cfn-init -s ${AWS::StackId} -r MyInstance --region ${AWS::Region} # Start cfn-signal to the wait condition /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource SampleWaitCondition --region ${AWS::Region} Metadata: Comment: Install a simple Apache HTTP page AWS::CloudFormation::Init: config: packages: yum: httpd: [] files: "/var/www/html/index.html": content: | <h1>Hello World from EC2 instance!</h1> <p>This was created using cfn-init</p> mode: '000644' commands: hello: command: "echo 'boom' && exit 1" services: sysvinit: httpd: enabled: 'true' ensureRunning: 'true' SampleWaitCondition: CreationPolicy: ResourceSignal: Timeout: PT1M Count: 1 Type: AWS::CloudFormation::WaitCondition # our EC2 security group SSHSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: SSH and HTTP SecurityGroupIngress: - CidrIp: 0.0.0.0/0 FromPort: 22 IpProtocol: tcp ToPort: 22 - CidrIp: 0.0.0.0/0 FromPort: 80 IpProtocol: tcp ToPort: 80 |
Command
1 |
hello: command: "echo 'boom' && exit 1" |
will exit with error 1. 1 is bad exit code, good is 0. This will directly trigger cfn-init failure.
Let’s create a stack:
Next
Next -> Create stack
And we see “Failed to receive 1 resource signal(s) within the specified duration”
As the result all the stack has been roll backed.
cfn-hup & cfn-metadata
We have such a yaml template:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
AWSTemplateFormatVersion: '2010-09-09' Description: AWS CloudFormation Sample Template for CFN Init Parameters: KeyName: Description: Name of an existing EC2 KeyPair to enable SSH access to the instances Type: AWS::EC2::KeyPair::KeyName ConstraintDescription: must be the name of an existing EC2 KeyPair. LatestLinuxAmiId: Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>' Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2' WelcomeMessage: Type: String Default: "Hello World" Resources: WebServerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Enable HTTP access via port 80 and SSH access via port 22 SecurityGroupIngress: - IpProtocol: tcp FromPort: '80' ToPort: '80' CidrIp: 0.0.0.0/0 - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: 0.0.0.0/0 WebServerHost: Type: AWS::EC2::Instance Metadata: Comment: Install a simple PHP application AWS::CloudFormation::Init: config: packages: yum: httpd: [] php: [] groups: apache: {} users: "apache": groups: - "apache" sources: "/home/ec2-user/aws-cli": "https://github.com/aws/aws-cli/tarball/master" files: "/var/www/html/index.html": content: !Sub | <h1>${WelcomeMessage} from ${AWS::StackName}</h1> mode: '000644' owner: apache group: apache # The cfn-hup.conf file stores the name of the stack and the AWS credentials that the cfn-hup daemon targets. "/etc/cfn/cfn-hup.conf": content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} # The interval used to check for changes to the resource metadata in minutes. Default is 15 interval=2 mode: "000400" owner: "root" group: "root" # The user actions that the cfn-hup daemon calls periodically are defined in the hooks.conf configuration file. # To support composition of several applications deploying change notification hooks, cfn-hup supports a directory named hooks.d that is located in the hooks configuration directory. You can place one or more additional hooks configuration files in the hooks.d directory. The additional hooks files must use the same layout as the hooks.conf file. "/etc/cfn/hooks.d/cfn-auto-reloader.conf": content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.WebServerHost.Metadata.AWS::CloudFormation::Init action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource WebServerHost --region ${AWS::Region} mode: "000400" owner: "root" group: "root" services: sysvinit: httpd: enabled: 'true' ensureRunning: 'true' CreationPolicy: ResourceSignal: Timeout: PT5M Properties: ImageId: !Ref LatestLinuxAmiId KeyName: Ref: KeyName InstanceType: t2.micro SecurityGroups: - Ref: WebServerSecurityGroup UserData: "Fn::Base64": !Sub | #!/bin/bash -xe # Get the latest CloudFormation package yum update -y aws-cfn-bootstrap # Start cfn-init /opt/aws/bin/cfn-init -s ${AWS::StackId} -r WebServerHost --region ${AWS::Region} || error_exit 'Failed to run cfn-init' # Start up the cfn-hup daemon to listen for changes to the EC2 instance metadata /opt/aws/bin/cfn-hup || error_exit 'Failed to start cfn-hup' # All done so signal success /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource WebServerHost --region ${AWS::Region} Outputs: InstanceId: Description: The instance ID of the web server Value: Ref: WebServerHost WebsiteURL: Value: !Sub 'http://${WebServerHost.PublicDnsName}' Description: URL for newly created LAMP stack PublicIP: Description: Public IP address of the web server Value: !GetAtt WebServerHost.PublicIp # Get metadata (change the region accordingly) # /opt/aws/bin/cfn-get-metadata --stack CfnHupDemo --resource WebServerHost --region eu-west-1 |
Let’s create a stack:
Next ->
Next -> Create stack
The stack has successfully been created
The http server on EC2 instance is up:
To get metatdata from EC2 instance let’s connect to it and run command:
1 |
/opt/aws/bin/cfn-get-metadata --stack CfnHupDemo --resource WebServerHost --region eu-central-1 |
The output:
Now let’s update the parameter of stack:
Next -> Next -> Update stack
The meadata of EC2 instance has been changed:
But the content of index.html bas been changed after 2 minutues: