Episode 5 – Understanding Heat (An OpenStack Series)

What is Heat? And what is the big deal about it? It is the “orchestration” service in OS, it automates configuration and deployment and it is “blah blah blah.” Yes all of us including myself have heard this a thousand times. But what is it actually good for? The idea of this episode is to go through a practical use case with heat and see how we can utilize it for reducing our manual work, get improved quality by standardizing the work we do, and “begin” to understand how one would like to manage ‘tenants’ (guests) in a cloud at scale.

Recap for Episode 4:

  • We were able to host our first simple guest with OS.
  • We are getting ready to throw a party!!

I am feeling a bit less corny in the morning today. Which is good news. Although that might change at any time for which I apologize in advance. So what are we looking to do today. For this episode we will pick up a simple example of automating the creation of a tenant (hosting of a guest) in OS. You might say, well wait a minute, whats the big deal about that? I can just use the OS command line to launch a cloud instance, right!! Well not quite. There is a lot more work that goes in, in creating a tenant and the various components that form the complete tenant service. Take a look at Episode 4 – Entertaining Guests. We configured the following, after the installation of OS components had been completed:

  • Public Network / and Subnet
  • Glance Image (cirros)
  • Flavor (m1.tiny)
  • New Project (Tenant with project user and rights)
  • Private Network / and Subnet
  • Neutron Router and connectivity
  • Security Group
  • SSH Key
  • Floating IP
  • Nova Instance (Guest Image based on Cirros)

You notice above that I have split the list in two parts. The first three items need not be configured for each tenant and can be created once (depending on requirement) to be used by all tenants. The remaining seven items are what need to be configured for each new tenant (guest) in the OS cloud. For the course of this episode we will tackle automating the configuration of these seven items using heat.

1. Create the new Project (Tenant)

Let us begin with the creation of a new project (tenant.) There are several ways to interact with Heat. We will begin with horizon, because that is the simplest. We will later try to squeeze in the command line if possible. So in order to create the New Project (tenant), login in to horizon using the admin credentials. Make sure you are in the admin project.

In the menu go to Project -> Orchestration -> Stacks. You will see something similar to what you see below:

ose5-1

Before we do anything create two files on your Desktop. Call them:

  • create-tenant.yaml
  • create-tenant-env.yaml

Use your favorite text editor to edit the two files as shown below:

create-tenant.yaml

heat_template_version: 2016-04-08 
 
description: Tenant init for GBM Cloud 
 
parameters: 
 
  project_name: 
        type: string 
        description: Name of Project (Tenant) 
   
  admin_name: 
        type: string 
        description: Username of Project Admin  
 
  admin_password: 
        type: string 
        description: Password for Project Admin  
        hidden: true 
 
resources: 
 
  tenant: 
        type: OS::Keystone::Project 
        properties: 
          name: { get_param: project_name } 
          domain: default 
 
  tenant_admin: 
        type: OS::Keystone::User 
        properties: 
          name: { get_param: admin_name } 
          password: { get_param: admin_password } 
 
  tenant_admin_role: 
        type: OS::Keystone::UserRoleAssignment 
        properties: 
          user: { get_resource: tenant_admin } 
          roles:  
            - {project: {get_resource: tenant}, role: user} 
 
  default_admin_role: 
        type: OS::Keystone::UserRoleAssignment 
        properties: 
          user: admin 
          roles:  
          - {project: {get_resource: tenant}, role: admin}

Woah!! What in the world is that. Don’t worry its not complicated at all I promise. What you see above is an example of a Heat Orchestration Template (AKA HOT.) It is simply a set of instructions that we are passing to the Heat service for execution. Lets break it down.

  • heat_template_version: 2016-04-08 Defines the template version. NOTE: This is NEITHER any random date NOR the current date. This is a specific date that defines the version of HOT you are defining. If you want to know what are the supported versions for your OS, then go to Project -> Orchestration -> Template Versions. This will give you a list. I generally like to use the latest version for HOT.
  • description: This is well just the description of what this HOT does.
  • parameters: Here we are defining the user input variables. This is the information that the HOT template will expect from the user. In this example we are taking the project name, admin name and password as inputs. Each parameter has a very simple format. Look at one example below:
admin_password: (name of the parameter can be anything) 
        type: string (variable type of the parameter) 
        description: Password for Project Admin (description of the parameter) 
        hidden: true (Optional parameter options. In this example we are making this field hidden, meaning the interface will not show the text in the password field when the user is typing the input.)
  • Look at the other parameters admin_name and project_name. They both follow the     same format, except that they don’t have any optional parameter options.
  • resources: Here we define the OS resources that need to be created/configured. These can be interpreted as instructions that are passed on to heat. Just browse through this section. Immediately you will see that we are trying to create a tenant, a tenant admin, and then assigning some roles. Let us look at a sample resource:
tenant_admin: (name of the resource, can be anything) 
        type: OS::Keystone::User (type of the resource) 
        properties: (resource properties) 
              name: { get_param: admin_name } (property1) 
              password: { get_param: admin_password } (property2)
  • A few things to notice here.
    • The type defines the kind of resource you are trying to create. Here we are trying to creating an OS Keystone User, hence OS::Keystone::User. A full reference of resource types and their respective parameters can be accessed in the OpenStack heat documentation, or Project -> Orchestration -> Resource Types. You can look into each individual Resource type and it will tell you the details on how to define each of the parameters.
    • get_param: admin_name: Tells HOT to take the value from the user input variable named “admin_name” in the parameters section an replace it here. So this equates to name = admin_name, where admin name is provided by the user.
    • In your create-tenant.yaml file look for “get_resource: tenant_admin.” get_resource is similar to get_param, with one significant difference. It looks for the value of a resource created previously in this HOT template (rather than a parameter) and replaces its value here. In tenant_admin_role we are equating the user to the user created in the previous step by the resource tenant_admin.
  • This file defines three user inputs: project_name, admin_name and admin_password.
  • In the resources section we are creating 1 tenant, 1 user for the tenant, giving user level rights to this new user on the newly created tenant and giving admin rights to the admin user on this new tenant.

create-tenant-env.yaml

parameter_defaults:  
  project_name: TNT82CL3 
  admin_name: tnt82cl3user 
  admin_password: password

This second file is nothing but an environment file that defines the default values for the user input variables (parameters) that we defined in the create-tenant.yaml file.

Time to have some fun. If not already there make sure you are logged in as admin and are in the admin project.

  • Go to Project -> Orchestration -> Stack and click on Launch Stack.
  • For Template source select the create-tenant.yaml file from the Desktop
  • For Environment source select the create-tenant-env.yaml file from the Desktop and click Next.

Word of Caution: Please take extra care about the indentation in the .yaml files. If the indentation is wrong the system will keep throwing weird errors at you even though all the syntax is correct.

ose5-2

  • For Stack Name give any name. I used create_tenant
  • For Password for user “admin,” enter the OS admin user’s password

Note the three user input fields admin_name, admin_password and project_name. These three fields are created by the parameters section of the create-tenant.yaml file. Also note that these fields are pre-populated. These default values are defined in the “parameter_defaults” section in the create-tenant-env.yaml file.

ose5-3

  • Click Launch and wait for the Status of create_tenant stack to change to “Create Complete.”
  • Click on create_tenant stack name to explore the details of the deployed stack. In particular look at the Topology and Events section to see all the steps that were carried out.
  • You can also check Identity -> Projects and Identity -> Users to ensure that the new tenant was created and the respective user / user roles were created and assigned.

If all has gone well you should be able to see the newly created Tenant named TNT82CL3 in horizon project list for the admin user as shown below:

ose5-5

Go ahead and make the switch. You are now inside the newly created Tenant in OS.

2. Create Tenant Networking

At this point I hope the .yaml files are not as scary as before and you are starting to get the hang of how they work. Ensure that you are logged in as admin and the project (tenant) selected is the new tenant. In my case it is TNT82CL3.

Now we will start configuring the tenant network. Lets create another two files on your Desktop:

  • init-tenant.yaml
  • init-tenant-env.yaml

I am calling them init, since we are going to initialize the tenant. Use your favorite text editor to edit the two files as shown below:

init-tenant.yaml

heat_template_version: 2016-04-08 
 
description: Tenant init for GBM Cloud 
 
parameters: 
 
  private_net_name: 
        type: string 
        description: Name of Private Tenant Network 
   
  private_net_vlan: 
        type: string 
        description: VLAN Number for Private Tenant Network 
 
  private_subnet_name: 
        type: string 
        description: Name of Private Subnet 
 
  private_subnet_cidr: 
        type: string 
        description: CIDR of Private Subnet 
   
  private_subnet_gateway: 
        type: string 
        description: Gateway of Private Subnet 
 
  public_net_name: 
        type: string 
        description: Name of Private Network 
 
  router_name: 
        type: string 
        description: Name of Router 
   
resources: 
 
  private_net: 
        type: OS::Neutron::ProviderNet 
        properties: 
          name: { get_param: private_net_name } 
          network_type: vlan 
          physical_network: vlan 
          segmentation_id: { get_param: private_net_vlan } 
          shared: false 
 
  private_subnet: 
        type: OS::Neutron::Subnet 
        properties: 
          name: { get_param: private_subnet_name } 
          network_id: { get_resource: private_net } 
          cidr: { get_param: private_subnet_cidr } 
          gateway_ip: { get_param: private_subnet_gateway } 
 
 
  router1: 
        type: OS::Neutron::Router 
        properties: 
          name: { get_param: router_name } 
          external_gateway_info: 
                network: { get_param: public_net_name } 
 
  router1_interface1: 
        type: OS::Neutron::RouterInterface 
        properties: 
          router_id: { get_resource: router1 } 
          subnet_id: { get_resource: private_subnet }

Do not get intimidated by the looooong text. There is nothing new here.

  • Look at the parameters section. We are asking the user for the following inputs:
    • Private Network Name
    • Private Network Vlan
    • Private Subnet Name
    • Private Subnet CIDR
    • Private Subnet Gateway
    • Public Network Name
    • Router Name

Note that the above are inputs that we used on the OS command line in the previous     episode, when we were creating the tenant network and router.

  • physical_network: vlan points to the name of the physical network provider which is vlan. This is the alias we set for the vlan bridge br-vlan in Episode 2 – Getting to know her better.
  • Look at the following extract from the file:
router1: 
        type: OS::Neutron::Router 
        properties: 
          name: { get_param: router_name } 
          external_gateway_info: 
                network: { get_param: public_net_name } 
 
router1_interface1: 
        type: OS::Neutron::RouterInterface 
        properties: 
          router_id: { get_resource: router1 } 
          subnet_id: { get_resource: private_subnet }
  • The external_gateway_info defines the router gateway. This is done by passing a sub parameter called network to it.
  • The router1_interface1 is a separate resource that is defined to attach a particular interface to a router. This could have been part of the router1 configuration as well, but this is how heat likes to do things.

init-tenant-env.yaml

parameter_defaults:  
  private_net_name: PriNet1TNT82CL3 
  private_net_vlan: 1382 
  private_subnet_name: PriSub1TNT82CL3 
  private_subnet_cidr: 10.103.82.0/24 
  private_subnet_gateway: 10.103.82.1 
  public_net_name: PublicNet 
  router_name: RTR1TNT82CL3

These are the input value defaults for the parameters in the init-tenant.yaml file. Note that we are passing public_net_name: PublicNet as a default value, however this network is assumed to already exist and is not being created by our HOT template. We are using the name to refer to it, and using it to define the default gateway for router1 using the external_gateway_info property.

Remember to always adjust the env files as per your environment configuration.

If not already there make sure you are logged in as admin and are in the TNT82CL3 project.

  • Go to Project -> Orchestration -> Stack and click on Launch Stack.
  • For Template source select the init-tenant.yaml file from the Desktop
  • For Environment source select the init-tenant-env.yaml file from the Desktop and click Next
  • For Stack Name give any name. I used init_tenant
  • For Password for user “admin,” enter the openstack admin user’s password

Review all the input fields and compare them to the parameters section of the     init-tenant.yaml file.

  • Click Launch and wait for the Status of init_tenant stack to change to “Create Complete.” If you run in to errors click on the stack name and look under Events. If you don’t find anything there then go to /var/log/heat/ and tail the file heat-engine.log to look for errors.  
  • Click on init_tenant stack name to explore the details of the deployed stack. In particular look at the Topology and Events section to see all the steps that were carried out.
  • Also check Network -> Network Topology to ensure that the new tenant was initiated and the respective networking has been configured.

    ose5-6
    Network Topology for TNT82CL3

3. Configure Tenant Security and Launch Instance

Now we come to the last part of this encounter. Before you proceed:

  • Go to Project -> Orchestration -> Stack and select init_tenant. Make sure you are in the TNT82CL3 project.
  • Click on Delete Stacks and confirm. Don’t worry we will create it again.

Watch how magically the stack disappears. If you go and see your network topology the private network and the router would have disappeared. You just did a successful roll-back in heat. This is a very useful feature that allows you to roll-back automatically everything that you did with this stack no matter how complex.

Now open the below two files again in your favorite text editor:

  • init-tenant.yaml
  • init-tenant-env.yaml

We are going to add to these files. The additions are shown in red below:

init-tenant.yaml

heat_template_version: 2016-04-08 
 
description: Tenant init for GBM Cloud 
 
parameters: 
 
  private_net_name: 
        type: string 
        description: Name of Private Network 
   
  private_net_vlan: 
        type: string 
        description: VLAN Number for Private Network 
 
  private_subnet_name: 
        type: string 
        description: Name of Private Subnet 
 
  private_subnet_cidr: 
        type: string 
        description: CIDR of Private Subnet 
   
  private_subnet_gateway: 
        type: string 
        description: Gateway of Private Subnet 
 
  public_net_name: 
        type: string 
        description: Name of Private Network 
 
  router_name: 
        type: string 
        description: Name of Router 
 
  VM1_name: 
        type: string 
        description: Name of Virtual Machine 
   
  key_name: 
        type: string 
        description: Name of SSH Key 
 
  image_name: 
        type: string 
        description: Name of Image to use for the VM instance (Should exist before hand) 
 
  flavor_name: 
        type: string 
        description: Name of flavor to use for the VM instance (Should exist before hand) 
 
  sec_group_name: 
        type: string 
        description: Name of Security Group 
 
resources: 
 
  private_net: 
        type: OS::Neutron::ProviderNet 
        properties: 
          name: { get_param: private_net_name } 
          network_type: vlan 
          physical_network: vlan 
          segmentation_id: { get_param: private_net_vlan } 
          shared: false 
 
  private_subnet: 
        type: OS::Neutron::Subnet 
        properties: 
          name: { get_param: private_subnet_name } 
          network_id: { get_resource: private_net } 
          cidr: { get_param: private_subnet_cidr } 
          gateway_ip: { get_param: private_subnet_gateway } 
 
 
  router1: 
        type: OS::Neutron::Router 
        properties: 
          name: { get_param: router_name } 
          external_gateway_info: 
                network: { get_param: public_net_name } 
 
  router1_interface1: 
        type: OS::Neutron::RouterInterface 
        properties: 
          router_id: { get_resource: router1 } 
          subnet_id: { get_resource: private_subnet } 
           
  server_security_group: 
        type: OS::Neutron::SecurityGroup 
        properties: 
             description: Security Group for VM 
             name: { get_param: sec_group_name } 
             rules: 
              - remote_ip_prefix: 0.0.0.0/0 
                protocol: tcp 
                port_range_min: 22 
                port_range_max: 22 
              - remote_ip_prefix: 0.0.0.0/0 
                protocol: icmp 
 
  server_ssh_key: 
        type: OS::Nova::KeyPair 
        properties: 
             name: { get_param: key_name } 
             save_private_key: true 
 
  floating_ip: 
    depends_on: [private_subnet,green_port] 
    type: OS::Neutron::FloatingIP 
    properties: 
      floating_network_id: { get_param: public_net_name } 
      port_id: { get_resource: green_port } 
 
  green_port: 
    depends_on: [private_subnet] 
    type: OS::Neutron::Port 
    properties: 
      network_id: { get_resource: private_net} 
      security_groups: [{ get_resource: server_security_group }] 
 
 
  server1: 
    depends_on: [server_security_group, server_ssh_key,private_subnet,green_port] 
    type: OS::Nova::Server 
    properties: 
      name: { get_param: VM1_name } 
      key_name: { get_resource: server_ssh_key } 
      image: { get_param: image_name } 
      flavor: { get_param: flavor_name } 
      networks: 
      - port: { get_resource: green_port } 
 
outputs: 
  tenant_ssh_key: 
    description: Private Key for Tenant Machine  
    value: { get_attr: [server_ssh_key, private_key] }

Make sure that your new file looks like the one shown above. Lets look at the new additions now shall we:

  • We are configuring the following additional items:
    • Security Group
    • SSH Key for instances
    • Floating IP
    • Assignment of security group and floating IP to a network port
    • Creation of an instance and assigning the network port and ssh key to the instance
  • In the security group we are creating 2 rules. One for ssh and the other for icmp. See if you find them.
  • green_port is a neutron port that is created separately and then a security group is assigned to it.
  • A floating ip is also assigned to the green_port
  • server1 is simply launching an instance for the tenant with the provided image and flavor values, and assigning the green_port and ssh key.
  • Look at the sample below:
server1: 
    depends_on: [server_security_group,server_ssh_key,private_subnet,green_port]

depends_on defines what resources is this resource dependent on. For the above     example heat is instructed not to provision server1 until server_security_group,     server_ssh_key,private_subnet and green_port have all been provisioned. Hence provisioning of server1 is triggered at the very end.

  • In server_ssh_key locate save_private_key: true. This saves the private key to a variable that is accessible by heat. If you remember previous episodes when we create ssh keys, the horizon interface and the command line both output the private key for us to save. This is the key that we use to log in to our cloud instances. Heat does not show the private key by default. Once it is saved we need to access it. Look at the last section of the file:
outputs: 
  tenant_ssh_key: 
    description: Private Key for Tenant Machine  
    value: { get_attr: [server_ssh_key, private_key] }

The outputs section, outputs the given values to the stack output. We will see this shortly. The get_attr: [server_ssh_key, private_key] gets the attribute value for private_key from the resource named server_ssh_key. Note that this value will be blank if you do not set save_private_key: true in the init-tenant.yaml file for the server_ssh_key.

init-tenant-env.yaml

parameter_defaults:  
  private_net_name: PriNet1TNT82CL3 
  private_net_vlan: 1382 
  private_subnet_name: PriSub1TNT82CL3 
  private_subnet_cidr: 10.103.82.0/24 
  private_subnet_gateway: 10.103.82.1 
  public_net_name: PublicNet 
  router_name: RTR1TNT82CL3 
  VM1_name: Service82CL3 
  key_name: KP1_TNT82_CL3 
  image_name: cirros 
  flavor_name:     m1.tiny 
  sec_group_name: SG1_TNT82_CL3

Note that the public network, image and flavor are assumed to already exist.

If not already there make sure you are logged in as admin and are in the TNT82CL3 project.

  • Go to Project -> Orchestration -> Stack and click on Launch Stack.
  • For Template source select the init-tenant.yaml file from the Desktop
  • For Environment source select the init-tenant-env.yaml file from the Desktop and click Next
  • For Stack Name give any name. I used init_tenant
  • For Password for user “admin,” enter the openstack admin user’s password
  • Click Launch and wait for the Status of init_tenant stack to change to “Create Complete.” Again if you run in to errors click on the stack name and look under Events. If you don’t find anything there then go to /var/log/heat/ and tail the file heat-engine.log to look for errors.
  • Click on init_tenant stack name to explore the details of the deployed stack. In particular look at the Overview section. Under Outputs (in the Overview section) you should be able to see the private ssh key. Yippie!! Save it for logging in to your cirros instance later.

If you want you can now check Network -> Network Topology to ensure that the new tenant was initiated and the respective networking has been configured. There should also be a cirros instance. Use the horizon UI to login in to the cirros machine. Once logged in run a ping to 8.8.8.8 to ensure that the network connectivity is working as expected. Also verify that your security group is applied. Try logging in to the instance using the floating ip and ssh key to verify these were applied too. Your network topology should look similar to the one below:

OSE5-7.png
Final Tenant Network Topology

If you want to use the command line then login in to your controller and copy over the Stack file and the environment file on to the controller.

#Source as the admin 
$ source ~/keystone_admin 
 
# ensure you have copied the HOT (Heat Stack) and the environment file on the controller 
$ heat stack-create -f <Stack.yaml> -e <Environment.yaml>

Replace Stack.yaml with your heat stack yaml file and the Environment.yaml with your environment variable file. The above command will let you run your stacks from command line. Now use your imagination and linux shell scripting skills to figure out if you can create a small script to create 10 tenants. Hint: Try using a loop 🙂

As you can see clearly you can use heat to automate a large number of operations, both custom and in bulk. Think of ways you could use this as a cloud service provider. I think throwing a party for 50 guests should not be a big deal for you now.

RECAP:
In this Episode we learned:

  • The basics of heat
  • How to use heat to automate some operational tasks in OS

Thank you for reading. I hope the series is fun to read and useful to at least some of you. If you have any questions or comments please feel free to share below in the comments. Stay tuned for more Episodes soon!!!

For my latest posts please visit WhatCloud.

 

Advertisements

7 thoughts on “Episode 5 – Understanding Heat (An OpenStack Series)

Add yours

  1. Hi @NOORUDDIN ALI,

    Congrats by your excellent Openstack series. It is an excellent material, with great diagrams.
    Do you have intention to write something about “Upgrade Openstack System” !? It would be a great material 🙂

    Congrats!

    1. Hi Carlos,

      I am very glad that you liked the material, and thank you so much for your kind appreciation. Upgrading is actually something I have been wanting to do for some time. It is a great idea! Let me see if I can do something. Will keep you posted.

      1. Nice Nooruddin, but I believe that before it, you could write a new serie to add new nodes in this infrastructure, to after it, do an upgrade/maintenance with a minimum or none “Down Service”
        Only indeas, suggestions based to production env 🙂

        Thank you by your feedback.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Powered by WordPress.com.

Up ↑

%d bloggers like this: