Creating Autoscale group with Monitoring apache connection check

In this article we will go over how to create a Auto-scale Group with a policy associating with a monitoring apache connection check.
Auto-scale will create or remove Cloud Servers based on the amount of apache connections (idle_workers) are being used.

Required setup and Optional Tools :

This is assuming you are using a Mac or Linux terminal.
You will need the following for this example (Required):

  • Cloud Server with Cloud Monitoring agent installed
  • Cloud Load Balancer with your Cloud Server as a node
  • A saved, working image of your Cloud Server

*** If you have any questions about the required items please feel free to jump into Chat support or Call in.***
Optional Tools but recommended (These tools are used in this article to access the API)

API Ussage

API URL

All items in curl brackets { } will need to be replaced with your values.
http://{datacenter}.autoscale.api.rackspacecloud.com/v1.0/{tenantId}/groups/{groupID}/policies/{policyID}/webhooks/{webhookID}

API required items

  • Account (Tennant) ID – Found in the Control Panel on the upper right hand corner.
  • API Key – Found in the Control Panel
  • API Token – Can be obtained through this command:
    curl -X POST https://identity.api.rackspacecloud.com/v2.0/tokens -d
    '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"theUserName", "apiKey":"YOUR_API_KEY" } } }' -H "Content-type: application/json

     

Variables

Setting variables will help save you time. I will place the option to set a variable throughout this article. I suggest using the exact variable I use in the examples for ease of use.
In this article we will use the IAD datacenter as an example.
Please use your own datacenter in the api url: iad, dfw, ord, syd, hkg
To Create a variable in Linux/Mac terminal type in the key=value (example: myname=Jim)

  • TENANTID variable TENANTID=xxxx  (Replace x with your account number.)
  • TOKEN varaible TOKEN=xxxx (Replace X with Token)

Now that we have the needed information and tools we can set up the Auto-Scale and Monitoring Check.

Auto Scale Configuration

Create auto scaling group. This can be done from the control panel. Setup the image, Cloud Server flavor size, Cloud Load Balancer and networks to be used on servers that are scaled up.

If you wanted a scaling policy based on a schedule you can also do that from the control panel, but since we want to scale triggered by an event we’ll need to use the API.

    We need to grab the group ID we created.
http get https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/ X-Auth-Token:$TOKEN
   
Next we will create the scaling UP policy webhook. I like to put JSON in a file then pass this file to our API call. Create create_policy_up.json and paste this JSON data, edit as needed. Change is how many Cloud Servers to scale up with a cooldown of 900s.

        [
            {
                “name”: “Scale UP”,
                “change”: 1,
                “cooldown”: 900,
                “type”: “webhook”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies X-Auth-Token:$TOKEN < create_policy_up.json
   
We make another file with our JSON data and create a scaling DOWN policy webhook. Create create_policy_down.json and paste this JSON data, edit as needed. Change is how many Cloud Servers to scale down (delete) with a cooldown of 600s.

        [
            {
                “name”: “Scale DOWN”,
                “change”: -1,
                “cooldown”: 600,
                “type”: “webhook”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies X-Auth-Token:$TOKEN < create_policy_down.json
   
Now we create the webhook URLs that Cloud Monitoring (Nagios, New Relic) can use to trigger the scaling events. Create create_webhook_up.json and create_webhook_down.json with this JSON data, edit as needed.

        [
            {
                “metadata”: {
                    “notes”: “Cloud Monitoring will fire this webhook and scale UP”
                },
                “name”: “Cloud Monitoring Webhook – UP”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies/$POLICYUPID/webhooks X-Auth-Token:$TOKEN < create_webhook_up.json
        [
            {
                “metadata”: {
                    “notes”: “Cloud Monitoring will fire this webhook and scale DOWN”
                },
                “name”: “Cloud Monitoring Webhook – DOWN”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies/$POLICYDOWNID/webhooks X-Auth-Token:$TOKEN < create_webhook_down.json

Your Auto Scale group, policy and webhooks should now be set. Next, we need to configure Cloud Monitoring to use these webhooks we created to then trigger our scaling up or down policy.

Cloud Monitoring configuration

We first need to setup a notification for up, down and email. Use the webhook URLs we created for the up and down notifications.

$ raxmon-notifications-create –label=autoscaleUP-notification –type=webhook –details=url=https://iad.autoscale.api.rackspacecloud.com/v1.0/execute/1/46260ec2ee8fa1f1dcc6404c5ef01ad6fb1ad5a4f17c8cf3b521edc1cedbb7fd/
Resource created. ID: nteacRqyed
$ raxmon-notifications-create –label=autoscaleDOWN-notification –type=webhook –details=url=https://iad.autoscale.api.rackspacecloud.com/v1.0/execute/1/452ab958f503dec27dca502b3a829c5784db1c84ff7ad76c2e50446301f386b2/
Resource created. ID: ntVzEPokKs

This is optional but when our scaling up/down policies trigger I also want an email for these critical/ok alerts.

$ raxmon-notifications-create –label “autoscale-email” –type=email –details=address=youremail@domain.com
Resource created. ID: ntsUwpwLg6

This next command tells Cloud Monitoring to create a new notification plan “Autoscale-Notification-Plan”, and when this plan is selected as the notification on Cloud Monitoring checks, trigger the scaling up policy on critical alerts and trigger the scaling down policy when the check recovers to OK. We add the email resource ID ntsUwpwLg6 so it will also send an email with the alerts.

$ raxmon-notification-plans-create –label “Autoscale-Notification-Plan” –critical-state nteacRqyed,ntsUwpwLg6 –ok-state ntVzEPokKs,ntsUwpwLg6
Resource created. ID: npP1qUz46q
Create a variable NOTIFICATION-PLAN=npP1qUz46q

Grab the entity id for your server. Find the Label for your server and grab the ID.
raxmon-entities-list –username=Your-User –api-key=Your-api-Key –details
You will be given a chunk of results look for the ID,  For this example we have enx8cLq6kp .
Create a variable: ENTITY=enx8cLq6kp

Next we create the check using the agent.apache check type with the entity id we retrieved above.  Keep in mind that this check uses apache fullstatu and this will need to be configured on your webserver.
raxmon-checks-create –username=Your-Username –api-key=Your-api-key –entity-id=enx8cLq6kp –label=apache-connections –period=90 –timeout=60 –type=agent.apache
Resource created. ID: chrWcInOr1

Create a variable CHECK=chrWcInOr1
Then you will need to create an Alarm on the check that uses the Notification plan Autoscale-Notification-Plan.
raxmon-alarms-create –check-id=$CHECK –entity-id=$ENTITY –criteria “if (metric[‘tidle_workers’] > 45) {return new AlarmStatus(CRITICAL, ‘Connections are exceeding 45′);}” –notification-plan-id=’$NOTIFICATION-PLAN’ –label=”Apache-Idle-Workers”
You can adjust the metric idle_workers to greater than 45. I suggest two/ hirds of your max-connections Directive in your apache conf file. Modify the metric as needed.
You can adjust the metric variable in your control panel. Click on Servers > Your Server name > Click your check > Click gear next to alarms and click edit criteria.
Be sure to test your AutoScale by changing the metric idle_workers to something low like 5.
This should spin up servers immediately. Once a this succeeds. Return the metric to the proper number.

In this article we will go over how to create a Auto-scale Group with a policy associating with a monitoring apache connection check.
Auto-scale will create or remove Cloud Servers based on the amount of apache connections (idle_workers) are being used.

Required setup and Optional Tools :

This is assuming you are using a Mac or Linux terminal.
You will need the following for this example (Required):

  • Cloud Server with Cloud Monitoring agent installed
  • Cloud Load Balancer with your Cloud Server as a node
  • A saved, working image of your Cloud Server

*** If you have any questions about the required items please feel free to jump into Chat support or Call in.***
Optional Tools but recommended (These tools are used in this article to access the API)

API Ussage

API URL

All items in curl brackets { } will need to be replaced with your values.
http://{datacenter}.autoscale.api.rackspacecloud.com/v1.0/{tenantId}/groups/{groupID}/policies/{policyID}/webhooks/{webhookID}

API required items

  • Account (Tennant) ID – Found in the Control Panel on the upper right hand corner.
  • API Key – Found in the Control Panel
  • API Token – Can be obtained through this command:
    curl -X POST https://identity.api.rackspacecloud.com/v2.0/tokens -d
    '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"theUserName", "apiKey":"YOUR_API_KEY" } } }' -H "Content-type: application/json

     

Variables

Setting variables will help save you time. I will place the option to set a variable throughout this article. I suggest using the exact variable I use in the examples for ease of use.
In this article we will use the IAD datacenter as an example.
Please use your own datacenter in the api url: iad, dfw, ord, syd, hkg
To Create a variable in Linux/Mac terminal type in the key=value (example: myname=Jim)

  • TENANTID variable TENANTID=xxxx  (Replace x with your account number.)
  • TOKEN varaible TOKEN=xxxx (Replace X with Token)

Now that we have the needed information and tools we can set up the Auto-Scale and Monitoring Check.

Auto Scale Configuration

Create auto scaling group. This can be done from the control panel. Setup the image, Cloud Server flavor size, Cloud Load Balancer and networks to be used on servers that are scaled up.

If you wanted a scaling policy based on a schedule you can also do that from the control panel, but since we want to scale triggered by an event we’ll need to use the API.

    We need to grab the group ID we created.
http get https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/ X-Auth-Token:$TOKEN
   
Next we will create the scaling UP policy webhook. I like to put JSON in a file then pass this file to our API call. Create create_policy_up.json and paste this JSON data, edit as needed. Change is how many Cloud Servers to scale up with a cooldown of 900s.

        [
            {
                “name”: “Scale UP”,
                “change”: 1,
                “cooldown”: 900,
                “type”: “webhook”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies X-Auth-Token:$TOKEN < create_policy_up.json
   
We make another file with our JSON data and create a scaling DOWN policy webhook. Create create_policy_down.json and paste this JSON data, edit as needed. Change is how many Cloud Servers to scale down (delete) with a cooldown of 600s.

        [
            {
                “name”: “Scale DOWN”,
                “change”: -1,
                “cooldown”: 600,
                “type”: “webhook”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies X-Auth-Token:$TOKEN < create_policy_down.json
   
Now we create the webhook URLs that Cloud Monitoring (Nagios, New Relic) can use to trigger the scaling events. Create create_webhook_up.json and create_webhook_down.json with this JSON data, edit as needed.

        [
            {
                “metadata”: {
                    “notes”: “Cloud Monitoring will fire this webhook and scale UP”
                },
                “name”: “Cloud Monitoring Webhook – UP”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies/$POLICYUPID/webhooks X-Auth-Token:$TOKEN < create_webhook_up.json
        [
            {
                “metadata”: {
                    “notes”: “Cloud Monitoring will fire this webhook and scale DOWN”
                },
                “name”: “Cloud Monitoring Webhook – DOWN”
            }
        ]
http post https://iad.autoscale.api.rackspacecloud.com/v1.0/$TENANTID/groups/$GROUPID/policies/$POLICYDOWNID/webhooks X-Auth-Token:$TOKEN < create_webhook_down.json

Your Auto Scale group, policy and webhooks should now be set. Next, we need to configure Cloud Monitoring to use these webhooks we created to then trigger our scaling up or down policy.

Cloud Monitoring configuration

We first need to setup a notification for up, down and email. Use the webhook URLs we created for the up and down notifications.

$ raxmon-notifications-create –label=autoscaleUP-notification –type=webhook –details=url=https://iad.autoscale.api.rackspacecloud.com/v1.0/execute/1/46260ec2ee8fa1f1dcc6404c5ef01ad6fb1ad5a4f17c8cf3b521edc1cedbb7fd/
Resource created. ID: nteacRqyed
$ raxmon-notifications-create –label=autoscaleDOWN-notification –type=webhook –details=url=https://iad.autoscale.api.rackspacecloud.com/v1.0/execute/1/452ab958f503dec27dca502b3a829c5784db1c84ff7ad76c2e50446301f386b2/
Resource created. ID: ntVzEPokKs

This is optional but when our scaling up/down policies trigger I also want an email for these critical/ok alerts.

$ raxmon-notifications-create –label “autoscale-email” –type=email –details=address=youremail@domain.com
Resource created. ID: ntsUwpwLg6

This next command tells Cloud Monitoring to create a new notification plan “Autoscale-Notification-Plan”, and when this plan is selected as the notification on Cloud Monitoring checks, trigger the scaling up policy on critical alerts and trigger the scaling down policy when the check recovers to OK. We add the email resource ID ntsUwpwLg6 so it will also send an email with the alerts.

$ raxmon-notification-plans-create –label “Autoscale-Notification-Plan” –critical-state nteacRqyed,ntsUwpwLg6 –ok-state ntVzEPokKs,ntsUwpwLg6
Resource created. ID: npP1qUz46q
Create a variable NOTIFICATION-PLAN=npP1qUz46q

Grab the entity id for your server. Find the Label for your server and grab the ID.
raxmon-entities-list –username=Your-User –api-key=Your-api-Key –details
You will be given a chunk of results look for the ID,  For this example we have enx8cLq6kp .
Create a variable: ENTITY=enx8cLq6kp

Next we create the check using the agent.apache check type with the entity id we retrieved above.  Keep in mind that this check uses apache fullstatu and this will need to be configured on your webserver.
raxmon-checks-create –username=Your-Username –api-key=Your-api-key –entity-id=enx8cLq6kp –label=apache-connections –period=90 –timeout=60 –type=agent.apache
Resource created. ID: chrWcInOr1

Create a variable CHECK=chrWcInOr1
Then you will need to create an Alarm on the check that uses the Notification plan Autoscale-Notification-Plan.
raxmon-alarms-create –check-id=$CHECK –entity-id=$ENTITY –criteria “if (metric[‘tidle_workers’] > 45) {return new AlarmStatus(CRITICAL, ‘Connections are exceeding 45′);}” –notification-plan-id=’$NOTIFICATION-PLAN’ –label=”Apache-Idle-Workers”
You can adjust the metric idle_workers to greater than 45. I suggest two/ hirds of your max-connections Directive in your apache conf file. Modify the metric as needed.
You can adjust the metric variable in your control panel. Click on Servers > Your Server name > Click your check > Click gear next to alarms and click edit criteria.
Be sure to test your AutoScale by changing the metric idle_workers to something low like 5.
This should spin up servers immediately. Once a this succeeds. Return the metric to the proper number.

jim has written 83 articles

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>