HTTP monitoring rules

, by DnC

NSS allows to regularly send an HTTP request, waits for a response and interprets it by applying rules. If an error situation is detected, NSS generates an alert.

The HTTP monitoring rules offered by NSS Lite allow you to not only verify that a website is functioning and has not been hijacked, but also to monitor the response of any HTTP device: DNS, Proxy, Web service etc.

NSS Lite allows in particular to test a response in JSON format or the data-xxx values of HTML5.

HTTP request

NSS Lite accepts an HTTP request limited to the scheme, host, path, query and fragment components.

An HTTP request is entered in the "Request" field of the task in the format:

<scheme (http: | https:)>//<host>/ [<path>] [? <query>] [# <fragment>]

Examples:

https://buy.dnc.global/-Nos-produits-.html
https://www.ossec.dnc.global/web/spip.php?article1

"Rules" field empty

The response expected by NSS following an HTTP request appears in the "Rules" field of the task. This field is optional.

When the "Rules" field is not filled in, NSS takes into account the HTTP code returned in the header of the response, regardless of the payload of the response:

 When the HTTP response code is 500 or more (server error), an event is created with a level equal to 6 (Alert).

 When the HTTP code of the response is 400 or more and less than 500 (web client error), an event is created with a level equal to 5 (Critical).

 When the HTTP code of the response is 300 or more and less than 400 (redirect), NSS tries to follow the redirect. If successful, the HTTP code is 200.

 When the process has ended without error (HTTP code <400), NSS applies a set of standard rules.

"Rules" field not empty

Analysis of the response

An HTTP response includes a "response header" and a "response payload" also referred to as the "body" of the response.

NSS first processes the HTTP response as described above. When the process has finished without error (HTTP code <400), NSS processes the response by applying the rules indicated in the "Rules" field of the task.

If the test fails, an event is created with a level equal to 4, or the level specified in the task definition.

There are two types of rules: "general rules" which do not take into account the payload of the response, and those which apply to that payload.

General rules


 TIME< number of ms: Checks that the server returns the response within a time less than the value indicated in ms.

 MEAN< number of ms: Checks that the server returns the response within an average delay less than the value indicated in ms.

 SAME LOCATION: Check that the URL of the response is identical to the request (there was no redirection).

 SRCIP = nnn.nnn.nnn.nnn: Check that the host of the real response has the indicated address for IPv4 (source IP). In particular, it is possible to check that a redirection does not leave the machine targeted by the request.

 SAME HOST: checks that the domain of the response is identical to that of the request. In particular, it is possible to check that a redirection does not leave the initial domain.

 HTTP = nnn: the HTTP code of the response must have the value indicated.

Response payload analysis rules

 NULL: the body of the response must have zero length.

Note: NSS Lite does not load more than 20,000 bytes from the response body. This must be taken into account for the following rules.

 LENGTH> integer: the body of the response must have a length greater than or equal to the specified value. Note that this is the HTML response, not the section of the HTML page.

 STRING = character string: the body of the response must be equal to the specified string.

 CONTAINS character string: the body of the response must contain the specified string.

 MATCH character string: this rule allows you to check the presence of any string in the response. The string can be a regular expression, so it is possible to perform any check in the response.

 JSON array [’a’] [’b’] ... = value, JSON array [’a’] [’b’] ...> value: The response must be a associative array in JSON format. The rule tests the value of the array element [’a’] [’b’] ...
The number of index dimensions (recursion depth) is limited to 10.
Unlike other rules, the names of the array and indexes are case sensitive.

 DATA-XXX = value, DATA-XXX> value: The response must contain the HTML5 data-xxx attribute. NSS compares the value of the attribute to the value specified in the rule. Note that if the response has multiple data-xxxs, only the first occurrence will be tested.

Rule inversion

We can precede the payload analysis rules with NOT to test the reverse condition (it is often more interesting), the rules are thus valid: NOT NULL, NOT HTTP =, NOT LENGTH =, NOT STRING =, NOT CONTAINS etc ..
However, MATCH cannot be reversed.

Examples of rules

SRCIP = 52.128.11.2 Checks that the response comes from the indicated IP.

CONTAINS <body> checks that the returned document contains a body (without prejudging its content).

LENGTH> 10000 checks that the response has a length> 10000. Used to detect a page generation error if its content (including header) is normally greater than the value indicated.

NOT CONTAINS tep stop The response must not contain the string ’tep stop’ (which is a fatal error returned by osCommerce).

NOT JSON Values ['temperature'] ['output']> 55 The response must be a JSON array ’Values’ whose element [’temperature’] [’output’ ] must be less than or equal to 55.

DATA-price> 50 The price indicated in the data-price field must be greater than 50.

Combination of several rules for the same task

Several rules can be indicated in the "Rules" field, one per line.
They are combined with the logical operator AND, ie the task is in error if one of the rules is verified.

The number of rules is limited only by the total length of the "Rules" field, limited to 1024 characters.

Here is an example of DNS monitoring for a domain [1], In which the rules are applied to the response from a DNS test site such as testdns.fr:

CONTAINS ns102.ovh.net
CONTAINS dns102.ovh.net
CONTAINS 51.178.18.44

This other example shows how to verify that a page is non-empty, doesn’t show a MySQL error, hasn’t been hijacked, and how to track response time:

SRCIP = NNN.NNN; NNN; NNN
LENGTH> 10,000: 6
CONTAINS TheTitleOfThePageByExample: 6
NOT CONTAINS MySQL error
NOT CONTAINS forbidden
TIME <1000: 2
TIME <3000: 3
TIME <10000: 4

Please note that rules such as "NOT CONTAINS MySQL error" or "NOT CONTAINS forbidden" are not universal: they depend on the application or the firewall.

Standard rule set applied by default

When the rules field is empty, NSS applies the following standard rule set (not requiring the loading of the response body):

TIME <2000: 2
TIME <5000: 3 "
TIME <10000: 4

Response time

If no rule has generated an error (including the TIME and MEAN rules described above), NSS compares the response time against the average response time observed for the task. An alert is generated with a level equal to 2 (Notice), or the level indicated in the definition of the task, if the response time exceeds 4 times the average time.

For more details on this topic, see: Surveillance du temps de réponse.

Alert level

Definition

NSS classifies events according to their severity into 8 levels, described in the following table:

Level Severity Elevation
7 Emergency: the system cannot be used NA
6 Alert: action must be taken immediately 7 if number exceeds 5
5 Critical condition 6 if number exceeds 5
4 Error 5 if number exceeds 60
3 Attention NA
2 Significant information 3 if number exceeds 60
1 Information NA
0 Debug Info NA

Alert level assignment

In the event of an error, NSS assigns an alert level of 4 by default.

This alert level can be modified for a given task by entering a different value in the "Error level" field of the task creation / edition form.

Thus defined, the level of error applies regardless of the rule in error. It is possible to define a particular level of error for the rule by following it with ’: N’ where N is the desired level, as in the following example:
LENGTH> 10000: 6

Of course, the defined error levels must be consistent with the "Minimum alert level for sending" defined for notifications (see below).

Compaction of events

When identical events follow one another, NSS does not generate an alert each time, but counts the number of repetitions. Only one line appears in the Alerts table.
As indicated in the "Elevation" column, the alert level is raised by one when the same error is repeated.
Thus, the Alerts table remains synthetic, and the rate of notifications reasonable.

Notification

The Manage notifications form allows you to define the destination addresses and the minimum level triggering a notification.

By default, the alert level is 5 (send "critical conditions" and above).
This level can be modified in the "Minimum alert level for sending" field, indicating a number from 0 (send all alerts) to 7 (send only "unusable system").

NSS issues a notification message at the start of an alert series and when the level rises.