Article freely adapted from this nice one
Alerts & monitoring are necessary in any IT system, however the old school way to perform it by sending email is just wrong. How many of your colleagues has their inbox splitted in different subfolders with a bunch of unread alerts ?
Moreover, in case of handing over the action, how to communicate and synchronize with the other people who might have get the alert ? often you don't know who received the alert and communicating by emails on issue resolution is complex. Last but not least, imagine other protagonist getting 50 non-linear emails next morning and getting really hard time to understand the full story.
We are using hipchat in our company for remote teams & contractors synchronization and we are using Azure Application Insights for monitoring.
Application Insights provide as well a really simple & easy to configure alerting system which covers majority of needs.
In our situation, we have some Azure Service Bus queues & subscribers (including deadletters) which are monitored as metrics in Insights through a nodejs azure webjob.
In case of
- normal queue over a threshold, the reading job is not doing the job and might be down
- deadletter queue over a threshold, the reading job is running but not able to proceed the message in 10 attempts pushing automatically the message into deadletter queue.
It's then easy to track
However, per default, email alerts only goes to some people and we end up in the same exact initial situation. Let's use the webhook functionality.
The goal is somehow improving the communication flow
at work or home
on mobile
As stated in the blog article referenced at the beginning, send a notification message to hipchat is simple by creating a token and calling a simple web service.
and play with postman for testing
but we can only specify in Insights webhook an url and no json transformation, input sent is like this one
{
"status": "Activated",
"context": {
"id": "xxxxxxx",
"name": "YOSEMITESAM ERROR",
"description": "",
"conditionType": "Metric",
"condition": {
"metricName": "opportunitycrm.dead",
"metricUnit": "",
"metricValue": "0",
"threshold": "50",
"timeAggregation": "Average",
"operator": "GreatherThan",
"windowSize": "15"
},
"subscriptionId": "xxxxxxx",
"resourceGroupName": "xxxxxxx",
"timestamp": "10/14/2016 23:34:01",
"resourceName": "xxxxxxxx",
"resourceType": "components",
"resourceId": "xxxxxxxx",
"resourceRegion": "centralus",
"portalLink": "xxxxxxx"
}
}
which can't be directly sent to hipchat. we will need an Azure Function App for doing the translation for us.
triggers
- HTTP (req)
- Mode standard
- Authorization level function
outputs
- http (res)
in developer area, you can create any nodejs code for transforming input json into something readable by hipchat.
var request = require('request');
module.exports = function(context, req) {
var room = 'ACME';
var color = req.body.status == 'Activated' ? 'red' : 'green';
var context = req.body.context;
var condition = context.condition;
var options = { method: 'POST',
url: 'https://api.hipchat.com/v2/room/' + room + '/notification',
qs: { auth_token: 'YOUR_TOKEN' },
headers:
{ 'content-type': 'application/json' },
body: {
'message': context.name + '<br />' + context.description + '<br />metric <strong>' + condition.metricName + '</strong> ' + condition.operator + ' ' + condition.threshold + ' over ' + condition.windowSize + ' minutes',
'color': color,
'notify' : false },
json: true };
request(options, function (error, response, body) {
if (error) throw new Error(error);
context.done();
});
};
However, this code won't be working at first run, because for simplicity we used the module 'request' which isn't available per default. You need to navigate to the underlying service app service
Right click on function app & settings
First go to 'App Service Editor' in order to edit package.json and add request dependency
Finally got to console, navigate to you Function App and type npm install
Finally, copy your function url into Aure Insights webhook and you are done !
