9.1. Settings

Managing a project's connection status, API keys, inspection conditions, application groups, alert channels and user permissions in one place.

Overview

The Settings page manages the configuration of an OPENMARU Observability project as a whole. Open it from the Settings menu in the left sidebar, and switch between areas with the tabs at the top.

Tab	Contents
System settings	Server connection status and API key management
Inspection conditions	The thresholds at which incidents are raised
Application categories	Application grouping rules and custom application settings
Alert channels	Slack, Teams, email (SMTP) and webhook integrations
Users and permissions	User account management and RBAC (role-based access control)
Security settings	IP whitelist management

Screen layout

The Settings page is made up of the following.

• Page header: the settings icon and the page title
• Tab bar: switches between the six tabs (system settings, inspection conditions, application categories, alert channels, users and permissions, security settings)
• Tab content area: the settings for the selected tab

---

Main features

System settings

The System settings tab shows the project's server connection status and manages API keys.

Node status

The Node status section shows the connection status of the components the project depends on.

Item	Description	Action
VictoriaMetrics	The metric store's connection status	A Configure link appears where it is not connected
openmaru-node-agent	Whether the node agent is installed, and how many nodes were found	An Install button appears
kube-state-metrics	The Kubernetes state metrics collector's connection status, and how many applications were found	—

Each item's state is shown by a colour indicator.

• Green: connected normally
• Red: connection failed, or not installed
• Grey: state unknown

Where openmaru-node-agent is not installed, clicking Install opens the agent installation guide.

API keys

An API key is the credential agents and external applications use to send telemetry to this project.

Note: API key management is available to users with the Admin role only. Users who are not Admin do not see the key values.

Viewing API keys

1. Open the System settings tab.
2. Read each key's description and masked value in the table in the API keys section.
3. Click the eye icon to reveal the full key value. Click it again to mask it.
4. Click the copy button to copy the API key to the clipboard.

Creating an API key

1. Click Create API key.
2. Enter what the key is for in the Description field (for example "production-agent", "otel-collector").
3. Click Create.
4. The new API key is added to the list.

Note: the API key you create is used when installing the agent. Keep the key value somewhere safe.

Editing an API key

1. Click the edit button (the pencil icon) on the key you want to change.
2. Edit the Description field and click Save.

Deleting an API key

1. Click the delete button (the bin icon) on the key you want to remove.
2. A confirmation dialog opens.
3. Click Delete to confirm.

Caution: once an API key is deleted, agents and applications using it can no longer send telemetry. Check that no agent is still using the key before deleting it.

Cost settings

On projects that are not in non-Kubernetes (host) mode, a Cost settings section also appears. Set the unit price of resource usage and it becomes the basis for cost calculations in the resource status and forecast reports. For details, see the SRE report chapter.

User estimation

The User estimation settings section configures how unique users are identified from HTTP traffic. Changes reach every node agent after about a minute. For details, see the user estimation chapter.

---

Configuring inspection conditions

An inspection is a threshold condition that monitors the state of your applications and infrastructure continuously. When a threshold is passed, an incident is raised automatically.

The inspection list

The Inspection conditions tab lists the inspections as a table.

Column	Description
Inspection	The inspection's category and name (availability, response time, node CPU usage and so on)
Condition	The default threshold condition currently in force
Project-level override	A custom threshold applied across this whole project
Application-level override	A custom threshold applied to an individual application

The main inspections:

• Availability: the request success rate threshold
• Latency: the request handling time threshold
• Instance availability: whether instances are running normally
• Node CPU usage: the node CPU usage threshold
• Container CPU usage: the container CPU usage threshold
• Out of memory (OOM): detects out-of-memory kills
• Memory leak: detects continuously rising memory use
• Disk I/O load: the disk input/output load threshold
• Restarts: the container restart count threshold
• Deployment status: detects deployment anomalies
• Errors: the error rate threshold
• Redis availability / response time: thresholds for the Redis service
• PostgreSQL availability / response time / replication lag: thresholds for the PostgreSQL service
• MongoDB availability / replication lag: thresholds for the MongoDB service
• MySQL availability / replication lag: thresholds for the MySQL service
• Memcached availability: the Memcached availability threshold
• DNS response time / errors: DNS thresholds
• Log errors: the threshold for the number of ERROR and CRITICAL log messages
• JVM availability: the JVM instance availability threshold
• Network connections: upstream service RTT and availability thresholds

Overriding a threshold at project level

To change an inspection's threshold across the whole project:

1. Click the Override link in the Project-level override column of the inspection you want to change.
2. Enter the threshold you want in the dialog that opens.
   • The dialog shows the global default, the project-level override and, where relevant, an application-level override row.
   • Where no project-level override is set, the global default applies, and you can click Override to enter a custom value.
3. Click Save.

Once set, the override appears in that column. To change it, click the edit icon beside the value.

Note: the SLO availability and SLO response time items do not support project-level overrides. Set those individually on the application detail page.

Overriding a threshold per application

To apply a different threshold to one application only:

1. Click the edit icon beside that application's name in the Application-level override column.
2. Enter the threshold to apply to that application.
3. Click Save.

Tip: adjusting thresholds per application to suit each service's characteristics reduces unnecessary incidents and gives you more accurate alerts.

Configuring SLO inspections

The SLO availability and SLO response time items are set on the SLO tab of the application detail page. The inspection dialog lets you configure the following.

SLO availability

• Metrics: use the built-in inbound requests, or specify your own PromQL query. Choosing custom lets you enter the total request query and the failed request query separately.
• Objective: tick the checkbox to enable SLO tracking and set the request success rate target as a percentage (for example 99% of requests must not fail).

SLO response time

• Metrics: use the built-in inbound requests, or specify your own histogram query.
• Objective: set the target proportion of requests that must be handled within a given response time (for example 99% of requests must be handled within 500ms).
• Target bucket: choose from 5 ms, 10 ms, 25 ms, 50 ms, 100 ms, 250 ms, 500 ms, 1 s, 2 s, 3 s, 4 s, 5 s, 6 s, 7 s, 8 s, 9 s and 10 s (in one-second steps from 1 s).

Alerting

The bottom of the SLO dialog shows the state of the alert channels currently connected. Where none is configured, click Configure alert channels to go to the alert channels tab.

To remove an SLO inspection override, click the delete icon in the dialog.

---

Application categories

Groups applications logically. Define a category and you can then use the category filter to separate applications on the dashboard, the topology map, the application list and other screens.

Configuring categories

The category list shows the categories currently defined.

Column	Description
Category	The category's name
Pattern	The glob pattern that matches the applications in this category
Deployment alerts	Whether to receive alerts when this category's applications are deployed (on/off)
Actions	The edit and delete buttons

Note: the default category holds the applications that fall into no other category. It cannot be deleted. Built-in categories' names and built-in patterns cannot be changed; you can only add custom patterns to them.

Adding a category

1. Click Add category.
2. Enter the category's name in the Name field.
3. Enter the patterns for the applications in this category in the Custom patterns field.
   • Patterns take the form namespace/application (for example staging/*, test-*/*).
   • Separate several patterns with spaces.
   • Glob pattern syntax is supported.
4. Set the Deployment alerts option. When enabled, an alert goes to the configured alert channels whenever an application in this category is deployed.
   • Where no alert channel is configured, "No alert channel is configured" appears, and you can click Configure alert channels to go to the alert channels tab.
5. Click Save.

Editing and deleting a category

• Edit: click the edit button (the pencil icon) on the category in the list to change its settings.
• Delete: click the delete button (the bin icon). Built-in categories cannot be deleted.

Note: for Kubernetes applications, you can also define the category by putting an annotation on the Kubernetes object.

Custom applications

OPENMARU Observability groups containers into applications automatically, as follows.

• Kubernetes metadata: pods are grouped by their Deployment, StatefulSet and so on.
• Non-Kubernetes containers: Docker containers and systemd units are grouped by name. A mysql service running on several servers, for instance, is gathered into one mysql application.

Where this default does not suit, custom application settings let you define instance name patterns to gather the containers you want into one application.

Column	Description
Application name	The custom application's name
Instance patterns	The glob pattern that matches the instances in this application
Actions	The edit and delete buttons

Adding a custom application

1. Click Add application.
2. Enter the application's name in the Name field.
3. Enter the instance name patterns to include in the Instance patterns field.
   • Matching is against instance names (for example mysql@node1, cassandra@cass-node*).
   • Separate several patterns with spaces.
4. Click Save.

Note: custom application settings apply to non-Kubernetes containers (Docker containers, systemd units and so on). They do not apply to Kubernetes workloads.

Namespace rules

For non-Kubernetes workloads (Docker, systemd, host processes), matches container_id against a glob pattern and assigns a namespace of your choosing. Rules are evaluated in order (first match wins), and user estimation and security attack detection are then aggregated under the namespace you assigned.

Hidden applications

Manages the applications hidden from every screen (the list, the topology, health, incidents). Hiding is reversible and the data is kept, so you can restore an application by showing it again at any time.

---

Alert channels

Configures notifications to external channels when incidents and deployment events occur.

The base URL

The URL the links in alert messages are built from. Enter the URL at which this system can be reached from outside.

1. Enter the URL in the Base URL field.
2. Click Save.

Note: where no base URL is set, the URL of the browser you are currently using is set automatically.

The alert channel list

The supported alert channels appear in a table.

Column	Description
Type	The kind of channel (Slack, MS Teams, email (SMTP), webhook)
Incident alerts	Whether incidents are sent to this channel
Deployment alerts	Whether deployments are sent to this channel
Alerts	Whether firing alert rules are sent to this channel
Actions	The configure, edit and delete buttons

Channels not yet configured carry a Configure button; those already configured carry edit (pencil) and delete (bin) buttons.

Slack

To receive incident and deployment alerts in a Slack channel:

1. Click Configure on the Slack entry.
2. Create a Slack app: click Create Slack app to create the app in Slack. Once created, click Install to workspace in Slack to authorise it.
3. Slack bot user OAuth token: copy the Bot User OAuth Token from the app's OAuth & Permissions page and paste it in.
4. Slack channel name: create a public channel in Slack, then enter its name after the #.
5. Alert types: use the Incidents and Deployments checkboxes to choose which alerts to receive.
6. Click Send test alert to check the configuration is correct.
7. Click Save.

Microsoft Teams

To receive alerts in a Microsoft Teams channel:

1. Choose the target channel in Teams (or create a new one).
2. Click the three dots (...) in the navigation menu at the top and choose Connectors.
3. Search for Incoming Webhook and press Configure.
4. Enter a name for the webhook and press Create.
5. Copy the webhook URL it produces.
6. Click Configure on the Teams entry of the Settings page.
7. Paste the URL into the Webhook URL field.
8. Alert types: use the Incidents and Deployments checkboxes to choose which alerts to receive.
9. Click Send test alert to check the configuration is correct.
10. Click Save.

Webhooks

A custom webhook lets you integrate freely with an external system.

1. Click Configure on the webhook entry.
2. Enter the Webhook URL.
3. Set the following advanced options where needed.
   • Skip TLS verification: enable this where a self-signed certificate is in use (available for HTTPS URLs only)
   • HTTP basic authentication: enter a username and password
   • Custom HTTP headers: add header names and values. Several headers can be added.
4. Alert types: use the Incidents and Deployments checkboxes to choose which alerts to receive.
5. Incident template: sets the message format sent when an incident occurs.
6. Deployment template: sets the message format sent when a deployment occurs.
7. Click Send test alert to check the configuration is correct.
8. Click Save.

Email (SMTP)

Receive alerts through your own mail server. In an air-gapped network that cannot reach an external SaaS, this is the channel that works.

1. Click Configure on the Email (SMTP) row.
2. Enter the mail server host and port.
3. Choose the encryption. Choosing a method also sets the default port, but a port you typed yourself is left alone.

Method	Default port	Description
STARTTLS	587	Connects in plain text, then upgrades. Fails if the server does not support it
TLS	465	Encrypted from the first byte
None	25	No encryption. Authentication is refused on an unencrypted connection

4. Turn on Skip TLS certificate verification if the server uses a certificate from a private CA.

Caution: with this on, any certificate is accepted. Use it only where you trust the private CA.

5. Enter the authentication username and password. Leave them empty for a relay that does not require authentication.
6. Enter the addresses: the from address and from name, and the recipients (comma separated for several).
7. Set the batching interval in minutes. Zero sends each notification immediately.

Caution: batching does not treat severity differently. Incident alerts are also delayed by up to that long. Leave it at zero on a channel where an urgent alert must not wait.

8. Choose the notification language (Korean / English).
9. Under notify of, pick which of incidents, deployments and alerts to receive.

Note: turning deployments on sends one notification covering deployments from the last 24 hours. With batching on, they arrive as a single message.

10. Press Send test alert to check the settings.
11. Click Save.

What the failures look like

Mail settings rarely come out right first time. When a test send fails, the cause is shown on screen as it came back.

Text in the message	Cause
connection refused or i/o timeout	Wrong host or port, or blocked by a firewall
certificate or x509	The certificate is not trusted. For a private certificate, turn on skip verification
Must issue a STARTTLS command first	The server requires encryption but None was selected
Authentication credentials invalid	Wrong username or password

Report attachments

Take the scheduled reports from SRE reports over SMTP and the PDF and Excel files arrive as attachments. Where an attachment exceeds the limit (20MB by default), it is left out and the reason is written into the body.

Editing and deleting a channel

• Edit: click the edit button (the pencil icon) on a configured channel to change its settings.
• Delete: click the delete button (the bin icon) to remove the integration.

---

Users and permissions

Manages who can reach the project, and sets each user's permissions through RBAC.

Managing users

The registered users appear as a table.

Column	Description
Email (login)	The user's login email address
Name	The user's name
Role	The role assigned (Admin, Editor, Viewer)
Actions	The edit and delete buttons

Adding a user

1. Click Add user.
2. Fill in the following in the dialog that opens.
   • Email (login): the user's login email address
   • Name: the user's name
   • Role: the role to grant (Admin, Editor, Viewer)
   • Password: the initial password
3. Click Create.

Editing a user

1. Click the edit button (the pencil icon) on the user you want to change.
2. Change the email, name, role or password and click Save.

Note: leaving the password field empty keeps the existing password.

Deleting a user

1. Click the delete button (the bin icon) on the user you want to remove.
2. Click Delete in the confirmation dialog.

Note: users marked read-only cannot be edited or deleted.

Role-based access control (RBAC)

RBAC governs what each role may do.

OPENMARU Observability provides three built-in roles.

Role	Description
Admin	Full access to everything. Can manage users, change settings and read all data.
Editor	Can use most monitoring features. Some administrative functions (user management and so on) are restricted.
Viewer	Can read monitoring data only. Cannot change settings.

The table in the Role-based access control (RBAC) section shows what each role may do.

• A check mark (green): the role is allowed this action
• A list icon (green): the action is allowed for particular objects only (hover over the icon for a tooltip listing them)
• An X (red): the role is not allowed this action

The main actions are as follows.

Action	Admin	Editor	Viewer
Manage users	Allowed	Denied	Denied
Manage roles	Allowed	Denied	Denied
Change project settings	Allowed	Denied	Denied
Configure alert channels	Allowed	Denied	Denied
Configure categories	Allowed	Allowed	Denied
Configure custom applications	Allowed	Allowed	Denied
Configure inspection conditions	Allowed	Allowed	Denied
View distributed traces	Allowed	Allowed	Allowed
View risks	Allowed	Allowed	Allowed
Edit risks	Allowed	Denied	Denied
View applications	Allowed	Allowed	Allowed
View servers	Allowed	Allowed	Allowed

Note: where users with custom roles exist, those roles appear in the RBAC table too. Click a custom role's edit button (the pencil icon) to see its fine-grained permission policy by scope, action and object.

---

Security settings

The Security settings tab manages the IP whitelist. Register an IP address or CIDR range and traffic from that source is excluded from security attack detection. Use it to stop legitimate traffic — internal scanners, health checks, trusted gateways — from being flagged as an attack.

To add an entry, enter an IP address or CIDR range and save; remove entries from the list when they are no longer needed.

Note: for more on security attack detection and security events, see the security chapter.

---

Related documents

• Quick start - connecting an agent with an API key after installation
• Incidents - the incidents raised by inspection conditions
• Applications - using the category filter in the application list