3. Graph Data Request API

What This Chapter Does

It queries metric data. This is the core of the guide, and the values used in the graphs on the console screen can be received exactly as they are.

How the Request Address Is Built

The address is a matter of writing what, from which target, and how to aggregate into the path in turn.

Writing several columns receives several metrics in one request. It reduces the number of requests when drawing several graphs on one screen.

The query period and interval are given as query parameters. They appear in the request examples of each section.

Below are the queries used most often, organized by purpose. Look up only the one you need.

Requesting Statistics for an Application Group

These are the values seen with the whole group treated as one. When a group has several instances, the values come combined by the aggregate function -- with mean, for example, it is the average across the instances.

To see them broken down by instance, use Requesting Statistics for an Instance in the next section.

The following is the request format common to every metric query. Actual query examples are organized by purpose later in this chapter, so learn the format here and look up the example you need.

Item	Value
URL	/monitoring/api/metrics/apps/{appName}/{ns}/{name}/{columns}/{aggregate}
HTTP METHOD	GET
Content-Type	application/json

Path parameters

Position	Description
{appName}	The application group name (obtained in chapter 2)
{ns}	The metric namespace
{name}	The metric name
{columns}	The column names to receive. Separate with : to request several at once
{aggregate}	The aggregate function -- mean, sum, max, min. Separate with : and give as many as there are columns; giving only one applies the same function to every column

Query parameters

Name	Default	Description
start	Required	The query start time. Unix epoch milliseconds (from 1970-01-01 00:00:00 UTC)
end	Required	The query end time. Same format
interval	—	The data interval (milliseconds). 5000 (5 seconds) is usual
fill	—	Fills gaps with no value with null
merge	—	Merges several values into one series
persec	—	Calculates per second rather than per interval. Use only for metrics counted per second, such as TPS (dashboard only)
history	false	Use when the data is history data
clientTime	false	The default is server time. Set it to true to use the times written in the request as they are
downGrade	false	Set it to true to pull from raw data rather than statistics data
exclusion	true	Looks for the group in the list with hidden agents removed. If every agent in the group is hidden or stopped, that group disappears from the list and the query fails -- set it to false in that case

If a query by group name returns an error

When the server cannot find the group given in {appName}, HTTP 500 comes back. Besides a typo in the name, the group being dropped from the list because of the exclusion default (true) is common -- that happens when every agent in the group is hidden or stopped.

Check whether the name exists first with GET /monitoring/api/metrics/APP?exclusion=true, and if it does not, add &exclusion=false to the query request.

Requesting Statistics for an Instance

These are the values for a single instance. Apart from the host IP and instance name added to the address, it is the same as above.

Putting * in {ipAddr} and {instanceId} means every target, and the results come back split by instance in one response. To choose several targets, join them with : -- for example, 192.168.0.1:192.168.0.2.

Item	Value
URL	/monitoring/api/metrics/graphs/{ipAddr}/{agentType}/{instanceId}/{ns}/{name}/{columns}/{aggregate}
HTTP METHOD	GET
Content-Type	application/json

Path parameters

Position	Description
{ipAddr}	The target server IP. Several can be written, separated with :
{agentType}	The agent type -- WAS (a WAS instance), WEB (a web server), SYS (a system)
{instanceId}	The instance ID. Several can be written, separated with :
{ns}	The metric namespace
{name}	The metric name
{columns}	The column names to receive. Separate with : to request several at once
{aggregate}	The aggregate function -- mean, sum, max, min. Separate with : and give as many as there are columns; giving only one applies the same function to every column

Query parameters

Name	Default	Description
start	Required	The query start time. Unix epoch milliseconds (from 1970-01-01 00:00:00 UTC)
end	Required	The query end time. Same format
interval	—	The data interval (milliseconds). 5000 (5 seconds) is usual
fill	—	Fills gaps with no value with null
merge	—	Merges several values into one series
persec	—	Calculates per second rather than per interval. Use only for metrics counted per second, such as TPS (dashboard only)
history	false	Use when the data is history data
clientTime	false	The default is server time. Set it to true to use the times written in the request as they are
downGrade	false	Set it to true to pull from raw data rather than statistics data

Statistics Response

Key
Response Body	The content of the response JSON string is as follows.
Statistics	[{"name":"*___*___*___*___apdex___maxRT:avgRT:minRT:count","start":1493457870000,"end":1493457880000, "interval":"5s","columns":["time","maxRT","avgRT","minRT","count"], "points":[[1493457870000,1493457875000],[7877.0,5752.0],[4436.5,2851.667],[0.0,0.0],[7.0,5.0]]}]

• Explanation of the JSON data items

[
  {
    "name": "*___*___*___*___apdex___maxRT:avgRT:minRT:count", ← the name
    "start": 1493457870000, ← the start time (milliseconds)
    "end": 1493457880000, ← the end time (milliseconds)
    "interval": "5s", ← the data interval
    "columns": [ ← the data column names
      "time",
      "maxRT",
      "avgRT",
      "minRT",
      "count"
    ],
    "points": [ ← the data items
      [ ← time column data (milliseconds)
        1493457870000,
        1493457875000
      ],
      [ ← first column, maxRT data
        7877,
        5752
      ],
      [ ← second column, avgRT data
        4436.5,
        2851.667
      ],
      [ ← third column, minRT data
        0,
        0
      ],
      [ ← fourth column, count data
        7,
        5
      ]
    ]
  }
]

Average Response Time and Call Count per Application Group

This is the first value to look at when a report of slowness comes in. The maximum (maxRT), average (avgRT), and minimum (minRT) response times and the call count (count) are received together in one request.

All four have to be read together to narrow the cause. If the average is unchanged but only the maximum spikes, only some requests are slow (a particular screen or a particular query condition); if the average rises too, the whole thing has slowed. If the call count rose with it, it is load; if the call count is unchanged and it is still slow, look at the application or the systems it calls.

Request

Item	Description
URL	/monitoring/api/metrics/apps/{appName}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/apps/front/NIL/apdex/maxRT:avgRT:minRT:count/max:mean:min:sum?start=1493457870000&end=1493457875000&interval=5000&fill=true&merge=true&persec=false
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":"*___*___*___*___apdex___maxRT:avgRT:minRT:count","start":1493457870000,"end":1493457880000,\
    "interval":"5s","columns":["time","maxRT","avgRT","minRT","count"],"points":[[1493457870000,1493457875000],\
      [7877.0,5752.0],[4436.5,2851.667],[0.0,0.0],[7.0,5.0]]}]

Average Response Time and Call Count per Instance

The same values seen per single instance. It is used to narrow down which machine is the cause when the group figures worsen.

If only one is slow, it is a problem specific to that instance (garbage collection, disk, a neighbouring process), so restarting that one or taking it out of the load is enough. If every one is evenly slow, the cause is shared (the database, an external integration, the network) and adding instances will not help.

Request

Item	Description
URL	/monitoring/api/metrics/graphs/{ipAddr}/{agentType}/{instanceId}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/graphs/192.168.23.113/WAS/front31/NIL/apdex/maxRT:avgRT:minRT:count/max:mean:min:sum?start=1493458160000&end=1493458165000&interval=5000&fill=true&merge=false
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":"NIL___192.168.23.113___WAS___front31___NIL___apdex","start":1493458160000,"end":1493458170000,\
    "interval":"5s","columns":["time","maxRT","avgRT","minRT","count"],"points":[[1493458160000,1493458165000],\
      [6290,7498],[4411,3706.5],[0,0],[4,5]]}]

Active User Count per Application Group

This is the number of users on the service right now. It has to be read alongside response time to mean anything -- if it got slower as users grew, it is a capacity problem; if users are unchanged and it got slower, it is an application problem.

It is a value that can be put straight on an in-house status board or daily report, and it is also used to record the actual concurrent user count during an event or campaign.

Request

Item	Description
URL	/monitoring/api/metrics/apps/{appName}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/apps/front/users/active/activeUser/mean?start=1493458395000&end=1493458400000&interval=5000&fill=true&merge=false&persec=false
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":"NIL___192.168.23.112___WAS___front21___users___active","start":1493458395000,"end":1493458405000,\
    "interval":"5s","columns":["time","activeUser"],"points":[[1493458395000,1493458400000],[0,0]]},\
  {"name":"NIL___192.168.23.113___WAS___front31___users___active","start":1493458395000,"end":1493458405000,\
    "interval":"5s","columns":["time","activeUser"],"points":[[1493458395000,1493458400000],[6,6]]}]

APDEX per Application Group

APDEX (Application Performance Index) divides response time into three bands -- satisfied, tolerating, and frustrated -- and converts it into a score from 0 to 100. Higher is better.

Reporting response time in milliseconds means arguing every time about whether 1.2 seconds is acceptable, whereas APDEX makes a target such as "keep it above 90" easy to set, so it is often used as a service level objective (SLO) metric. It is a good shape for management reports and cross-department agreements.

Request

Item	Description
URL	/monitoring/api/metrics/apps/{appName}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/apps/front/NIL/apdex/apdex/mean?start=1493458395000&end=1493458400000&interval=5000&fill=true&merge=false&persec=false
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":"*___*___*___*___apdex___apdex","start":1496382935000,"end":1496382945000,"interval":"5s",\
    "columns":["time","apdex"],"points":[[1496382935000,1496382940000],[null,null]]}]

Active User Count per Instance

The number of users a single instance is handling. It is used to check whether the load balancer is spreading traffic evenly.

If users are piled on one machine, check the distribution algorithm or the sticky session setting. Conversely, if only one machine sits at 0, it may have been dropped by the health check.

Request

Item	Description
URL	/monitoring/api/metrics/graphs/{ipAddr}/{agentType}/{instanceId}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/graphs/192.168.23.113/WAS/front31/users/active/activeUser/mean?start=1493458305000&end=1493458310000&interval=5000&fill=true&merge=false
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":"NIL___192.168.23.113___WAS___front31___users___active","start":1493458300000,"end":1493458310000,\
    "interval":"5s","columns":["time","activeUser"],"points":[[1493458300000,1493458305000],[6,6]]}]

Error Rate per Application Group

The proportion of all requests whose HTTP response code is in the 400s or 500s. If the success rate is what you need, calculate it as 100 - error rate.

This is the value looked at most right after a deployment. Querying the periods before and after the deployment time and comparing whether the error rate rose catches a bad deployment before a user reports it, and lets it be rolled back. Some in-house deployment pipelines check this value and stop the deployment automatically when it passes a threshold.

Request

Item	Description
URL	/monitoring/api/metrics/apps/{appName}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/apps/front/**/status/errorRate/mean?start=1493458515000&end=1493458520000&interval=5000&fill=true&merge=true&persec=false&sort=true
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":"*___*___*___*___status___errorRate","start":1493458515000,"end":1493458525000,"interval":"5s",\
    "columns":["time","errorRate"],"points":[[1493458515000,1493458520000],[0.0,0.0]]}]

Error Rate per Instance

The same value seen per single instance. It is calculated the same way as above.

It is particularly useful during a rolling update. If only the instances that received the new version show a rising error rate, the rollout can be stopped before the rest. If every one rises together, look at a shared cause (the database, an external integration) rather than the deployment.

Request

Item	Description
URL	/monitoring/api/metrics/graphs/{ipAddr}/{agentType}/{instanceId}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/graphs/192.168.23.112/WAS/front21/**/status/errorRate/mean?start=1493463050000&end=1493463055000&interval=5000&fill=true&merge=false&sort=true
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":".*___(192.168.23.112)___(WAS)___(front21)___*___status","start":1493463050000,"end":1493463060000,\
    "interval":"5s","columns":["time","errorRate"],"points":[[1493463050000,1493463055000],[null,null]]}]

APDEX per Instance

The APDEX score seen per single instance. It is used to find the instance dragging the score down when the group APDEX falls below target.

If only one is low, taking that one out of the load restores the group score straight away. For automated action, query this value periodically and send an alert when it drops below a threshold.

Request

Item	Description
URL	/monitoring/api/metrics/graphs/{ipAddr}/{agentType}/{instanceId}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/graphs/192.168.23.112/WAS/front21/NIL/apdex/apdex/mean?start=1493463050000&end=1493463055000&interval=5000&fill=true&merge=false
HTTP METHOD	GET
Content-Type	application/json

Response

[{"name":".*___(192.168.23.11)___(WAS)___(t1server11)___NIL___apdex","start":1496383025000,"end":1496383035000,\
    "interval":"5s","columns":["time","apdex"],"points":[[1496383025000,1496383030000],[null,null]]}]

Calling APDEX, Average Response Time, and Call Count for Every Instance at Once

This is how to reduce the number of requests in an environment with many instances. Querying machine by machine means 50 requests for 50 instances, whereas this way ends in one. The difference matters where calls are frequent -- a dashboard redrawing every 5 seconds, or a collection batch running periodically.

The response comes split into one array element per instance, so divide it by the name value after receiving it.

To call for several IPs or instances at once, use '*' in the {ipAddr} or {instanceId} item and the information for every instance comes back in one call.

To call for particular IPs and instance IDs only, enter several IPs and instance IDs separated by ':'. (For example, …/graphs/192.168.0.1:192.168.0.2/WAS/test11:test12:test21:test22/…)

The default of clientTime is false; using true uses the start and end times given in the URL.

As below, the error rate and the user count can also be called for every instance at once.

Request

Item	Description
URL	/monitoring/api/metrics/graphs/{ipAddr}/{agentType}/{instanceId}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/graphs/**/WAS/**/NIL/apdex/apdex:avgRT:count/mean:mean:sum?start=1493463050000&end=1493463055000&interval=5000&fill=true&merge=false&clientTime=true
HTTP METHOD	GET
Content-Type	application/json

For example:

Item	Description
Error rate example	/monitoring/api/metrics/graphs/**/WAS/**/NIL/status/errorRate/mean?start=1493463050000&end=1493463055000&interval=5000&fill=true&merge=false&clientTime=true
Active user count example	/monitoring/api/metrics/graphs/**/WAS/**/users/active/activeUser/mean?start=1493463050000&end=1493463055000&interval=5000&fill=true&merge=false&clientTime=true

Response

[{"name":"NIL___192.168.23.11___WAS___server11___NIL___apdex","start":1502154380000,"end":1502154390000,\
    "interval":"5s","columns":["time","apdex","avgRT","count"],"points":[[1502154380000,1502154385000],[100,100],\
      [1,1],[6,7]]},{"name":"NIL___192.168.23.11___WAS___server12___NIL___apdex","start":1502154380000,\
    "end":1502154390000,"interval":"5s","columns":["time","apdex","avgRT","count"],"points":[[1502154380000,\
        1502154385000],[0,0],[10082,14994],[2,1]]}]

One instance is one array element. Read which instance it is from the ___-joined values in name; the order of points matches the order of columns.

Column	Meaning
time	The data time (Unix epoch milliseconds)
apdex	The APDEX score
avgRT	The average response time (milliseconds)
count	The call count

Daily Hourly Call Count Statistics per Application Group

Call count statistics per application group, grouped by hour (1h) or by day (1d).

Where the 5-second metrics of the earlier sections show the state right now, this API shows the trend. It is used for the hourly traffic graph in a weekly report, choosing a quiet window for batch jobs, and as the basis for next quarter's capacity planning.

Putting a regular expression in the url parameter picks out particular screens -- for example, pulling only the call count of the payment screen and lining it up against sales figures.

Request

Item	Value
URL	/monitoring/api/stat/url/graph/{appName}
Example request URL	/monitoring/api/stat/url/graph/front?url=*&interval=1h&minX=1493478000000&maxX=1493564400000&fixedRange=true
HTTP METHOD	GET
Content-Type	application/json

Path parameters

Position	Description
{appName}	The application group name (obtained in chapter 2)

Query parameters

Name	Default	Description
url	—	The called URL name. A regular expression picks out particular URLs
interval	—	The data interval (milliseconds). 5000 (5 seconds) is usual
minX	Required	The query start time. Unix epoch milliseconds
maxX	Required	The query end time. Unix epoch milliseconds
fixedRange	—	With true, the server does not readjust the times and uses the requested range as it is
exclusion	true	Looks for the group in the list with hidden agents removed. If every agent in the group is hidden or stopped, that group disappears from the list and the query fails -- set it to false in that case

Response

{"status":200,"result":{"name":"NIL___192.168.23.112:192.168.23.113___WAS___front21:front31___NIL___apdex",\
    "start":1493478000000,"end":1493564400000,"interval":"1h","columns":["counts","apdex"],\
    "points":[[1493478000000,1493481600000,1493485200000,1493488800000,1493492400000,1493496000000,1493499600000,\
        1493503200000,1493506800000,1493510400000,1493514000000,1493517600000,1493521200000,1493524800000,1493528400000,\
        1493532000000,1493535600000,1493539200000,1493542800000,1493546400000,1493550000000,1493553600000,1493557200000,\
        1493560800000],[3172,3036,3190,3144,3141,3122,3084,3108,3099,3151,3094,3137,3129,3137,3132,3186,3110,3099,2938,\
        3128,3118,3080,3082,3049],[64.757,69.976,66.282,70.0,69.286,69.231,68.0,63.306,90.0,68.571,71.111,68.834,65.385,\
        63.043,69.111,65.686,65.0,58.333,71.154,71.429,71.053,68.519,68.333,72.857]],"names":null}}

Daily Hourly Call Count Statistics per Instance

The same statistics seen per single instance.

It is used to check on a daily basis whether traffic actually spread after instances were added or removed. Querying before and after side by side shows straight away whether the newly added machine is taking its share.

Request

Item	Value
URL	/monitoring/api/stat/url/graph/{ipAddr}/{instanceId}
Example request URL	/monitoring/api/stat/url/graph/192.168.23.113/front31?url=*&interval=1h&minX=1493478000000&maxX=1493564400000&fixedRange=true
HTTP METHOD	GET
Content-Type	application/json

Path parameters

Position	Description
{ipAddr}	The target server IP. Several can be written, separated with :
{instanceId}	The instance ID. Several can be written, separated with :

Query parameters

Name	Default	Description
url	—	The called URL name. A regular expression picks out particular URLs
interval	—	The data interval (milliseconds). 5000 (5 seconds) is usual
minX	Required	The query start time. Unix epoch milliseconds
maxX	Required	The query end time. Unix epoch milliseconds
fixedRange	—	With true, the server does not readjust the times and uses the requested range as it is

Response

{"status":200,"result":{"name":"NIL___192.168.23.113___WAS___front31___NIL___apdex","start":1493564400000,\
    "end":1493650800000,"interval":"1h","columns":["counts","apdex"],"points":[[1493564400000,1493568000000,\
        1493571600000,1493575200000,1493578800000,1493582400000,1493586000000,1493589600000,1493593200000,1493596800000\
        ,1493600400000,1493604000000,1493607600000,1493611200000,1493614800000,1493618400000,1493622000000,1493625600000\
        ,1493629200000,1493632800000,1493636400000,1493640000000,1493643600000,1493647200000],[2898,2929,3034,3015,2994,\
        3001,1871,1809,1829,1838,1823,1814,1808,1070,null,null,null,null,null,null,null,null,null,null],[75.862,72.0,\
        61.111,65.789,70.0,63.889,71.429,65.0,70.0,72.222,57.692,72.545,60.0,67.5,null,null,null,null,null,null,null\
        ,null,null,null]],"names":null}}

Events of the Last 10 Minutes by APM Server Time

The list of events that occurred in the last 10 minutes, by OPENMARU APM server time.

It is used for showing recent alerts on an in-house status board or a messenger bot. It has no parameters, so it is the simplest to call.

Watch the call interval

The query range is fixed at 10 minutes. Calling at an interval longer than 10 minutes misses the events that occurred in between. If the interval exceeds 10 minutes, or the integration must not miss a single one, use the start method in the next section.

Request

Item	Description
URL	/monitoring/api/events/recents
Example request URL	/monitoring/api/events/recents
HTTP METHOD	GET
Content-Type	application/json

Response

{"status": 200,"result": [{"uuid": "FD82A912-85B2-4D06-872E-C3CE80B5F017",\
      "created": 1494387531070,"eventId": 5000,"name": "EVENT_FORECAST_CRITICAL_THRESHOLD_REACH",\
      "level": "FORECAST","ipAddress": "192.168.23.144","agentType": "SYS","instanceId": "ens192",\
      "type": "Network Error %","readableMessage": "'Network Error %' (current value: 9.31) is expected to reach\
      the CRITICAL threshold '10' at a forecast value of '13.24' around '2017-05-10 12:43:51'.","readableWhere": "SYS Agent > \
      192.168.23.144 > Network Error % > ens192","param": {"current": 9.31,"field": "errorRate",\
        "channels": ["email"],"recipients": ["omadm"],"nextX": 300000,\
        "threshold": 10,"forecast": 13.24,"nextTimestamp": 1494387831070,\
        "type": "Network Error %","stderr": 3.43,"streamName": "NetworkErrorStream"},\
      "instanceIdAsNil": "ens192","instanceIdOrg": "ens192"}, … ]

JSON Data Format

{
  "status": 200, ← the response code
  "result": [ ← the event list (array)
    {
      "uuid": "17E1F5C5-BEAC-481B-B1CF-E724678CEE23", ← the event UUID
      "created": 1494388083491, ← the event creation time, Unix epoch
      "eventId": 1100, ← the event ID
      "name": "EVENT_WARN_THRESHOLD_", ← the event name
      "level": "WARN", ← the event level, one of INFO, WARN, FORECAST, CRITICAL
      "ipAddress": "192.168.23.144", ← the IP address of the agent where the event occurred
      "agentType": "WAS", ← the type of agent where the event occurred, one of WAS, SYS, WEB
      "instanceId": "testap-8-7rh9q", ← the ID of the instance where the event occurred
      "type": "Heap Usage %", ← the event type
      "readableMessage": "'Heap Usage %' (average: 85.02) has passed the WARN threshold '85'.", ← a readable description of the event
      "readableWhere": "WAS Agent > 10.130.0.35 > testap-8-7rh9q > Heap Usage %",
      ← a readable indication of where the event occurred
      "param": { ← statistics for the information the event was raised on
        "average": 85.02, ← the average
        "current": 83.62, ← the current value
        "field": "heapUsedPercent",
        "channels": [ ← the channels the event is sent through
          "email"
        ],
        "recipients": [ ← the IDs of the event recipients
          "omadm"
        ],
        "timewindow": 180000, ← the statistics window for the event data
        "threshold": 85, ← the configured threshold that raises the event
        "type": "Heap Usage %",
        "streamName": "HeapStream"
      },
      "instanceIdAsNil": "testap-8-7rh9q",
      "instanceIdOrg": "testap-8-7rh9q"
    },
    ……
  }
]
}

Recent Events from a Given Time to Now (Server Time)

The list of events from the time given in start to now (server time).

This is how to receive events without missing one. Passing the created value of the most recent event in this response as the start of the next request fetches every event in between, even when the call interval is irregular or the integration program paused and resumed.

It is used for integrations where a gap is a problem, such as automatic ticket creation and audit log ingestion. The response format is the same as the previous section.

Request

Item	Value
URL	/monitoring/api/events/recents?start={start}
Example request URL	/monitoring/api/events/recents?start=1494387225255
HTTP METHOD	GET
Content-Type	application/json

Query parameters

Name	Default	Description
start	Required	The query start time. Unix epoch milliseconds (from 1970-01-01 00:00:00 UTC)

Response

{"status": 200,"result": [{"uuid": "FD82A912-85B2-4D06-872E-C3CE80B5F017",\
      "created": 1494387531070,"eventId": 5000,"name": "EVENT_FORECAST_CRITICAL_THRESHOLD_REACH",\
      "level": "FORECAST","ipAddress": "192.168.23.144","agentType": "SYS","instanceId": "ens192",\
      "type": "Network Error %","readableMessage": "'Network Error %' (current value: 9.31) is expected to reach\
      the CRITICAL threshold '10' at a forecast value of '13.24' around '2017-05-10 12:43:51'.","readableWhere": "SYS Agent > \
      192.168.23.144 > Network Error % > ens192","param": {"current": 9.31,"field": "errorRate",\
        "channels": ["email"],"recipients": ["omadm"],"nextX": 300000,\
        "threshold": 10,"forecast": 13.24,"nextTimestamp": 1494387831070,"type": "Network Error %",\
        "stderr": 3.43,"streamName": "NetworkErrorStream"},"instanceIdAsNil": "ens192",\
      "instanceIdOrg": "ens192"}, … ]

Comparing System CPU Utilization

It receives the resource utilization of several servers at once and compares them. The response comes split into one array element per server, so sorting by value after receiving it makes the loaded server obvious straight away.

It is used for deciding on adding or removing servers, checking whether one particular server is unusually high, and building the resource usage table in a periodic inspection report.

The idle in the response is the idle rate. If you need the utilization, calculate it as 100 - idle.

Putting * in place of {appName} means every target, and agentType=SYS leaves only the system agents. Changing just the metric names in the path ({ns}/{name}/{columns}) compares memory the same way -- NIL/memory/usedPercent/mean.

Request

Item	Value
URL	/monitoring/api/metrics/apps/{appName}/{ns}/{name}/{columns}/{aggregate}
Example request URL	/monitoring/api/metrics/apps/*/NIL/cpu/idle/mean?start=1786065150000&end=1786065160000&interval=5000&fill=true&agentType=SYS
HTTP METHOD	GET
Content-Type	application/json

Query parameters

Name	Default	Description
start	Required	The query start time. Unix epoch milliseconds (from 1970-01-01 00:00:00 UTC)
end	Required	The query end time. Same format
interval	—	The data interval (milliseconds). 5000 (5 seconds) is usual
fill	—	Fills gaps with no value with null
exclusion	—	Specifies whether to exclude hidden agents
agentType	—	The agent type -- WAS, WEB, SYS, CUB
clientTime	false	The default is server time. Set it to true to use the times written in the request as they are

Response

It comes split into one array element per server. The second field of name is the server IP.

[
  {
    "name": "NIL___192.168.11.2___SYS___NIL___NIL___cpu",
    "start": 1786065150000,
    "end": 1786065160000,
    "interval": "5s",
    "columns": [
      "time",
      "idle"
    ],
    "points": [
      [
        1786065150000,
        1786065155000
      ],
      [
        97.095,
        97.033
      ]
    ]
  },
  {
    "name": "NIL___192.168.11.4___SYS___NIL___NIL___cpu",
    "start": 1786065150000,
    "end": 1786065160000,
    "interval": "5s",
    "columns": [
      "time",
      "idle"
    ],
    "points": [
      [
        1786065150000,
        1786065155000
      ],
      [
        83.203,
        84.185
      ]
    ]
  }
]