Elasticsearch and OpenSearch logs

Connect Elasticsearch, Elastic Cloud or OpenSearch so the agent can search your logs, with a read-only API key or login.

In the appPlatformSettingsLive systemElasticsearch (or OpenSearch)

With your logs, the agent can see what happened when the requester hit the problem: the error, the request, the time. If your logs are in Elasticsearch (also on Elastic Cloud) or OpenSearch, it searches them there.

The agent only reads logs. It can search up to the last 7 days, and gets at most 1,000 entries per search. Values that look like secrets are removed from every entry before the agent sees it.

The log watcher uses the same connection.

One log source per platform

A platform reads its logs from one place: Loki, Elasticsearch, OpenSearch or Datadog. To switch, click Disconnect under the one that is connected, then choose the other. A client with logs in two places is two platforms.

Only this platform's namespaces

If the platform lists namespaces (Namespaces it may look at, on the Kubernetes card of the same page), every search is limited to them, using the Namespace field. So several platforms can share one Elasticsearch, and none sees another's logs.

Connect it#

1
Go to Settings › Live system. Click Connect a cluster or logs (or Connect another), and choose Elasticsearch or OpenSearch. Its Logs row shows under the card.
2
Enter its address.
3
Choose the Login, and enter the API key, or the username and password.
4
Check the Index pattern, and the field names if your logs use other ones.
5
Describe your logs in How your logs are labelled.
6
Click Save, under the fields.
7
Click Check connection.

Field	What it does
Address	The base address, for example `https://logs.example.com:9200`. On Elastic Cloud, your deployment's Elasticsearch endpoint.
Login	API key (sent as `Authorization: ApiKey …`), Username and password, or None.
API key or Password	Stored encrypted and never shown again. Leave it empty to keep the saved one.
Username	Shown for Username and password.
Index pattern	Which indexes hold the logs. `logs-*` unless you change it. Separate several with commas.
Field names	Under Your logs use other field names?: the fields that hold the Time (`@timestamp`), Level (`log.level`), Message (`message`) and Namespace (`kubernetes.namespace`). The defaults fit Filebeat, Elastic Agent, Logstash and Fluent Bit with the Elastic Common Schema.
How your logs are labelled	Notes that help the agent write good searches.

Check connection appears once the settings are saved. It says which product and version answered, how many log entries the index pattern holds, and any field name it couldn't find in your logs.

With your own runner

When the agent runs on your own runner, the API key or password is kept by you. Save the settings first. Then choose how it is kept: Scrambled for your runner or In your store. Your runner then checks, and reads for the log watcher, with it. It needs runner 0.12.0 or later. See Two ways to keep your keys.

Make a read-only API key#

Give the agent a key that can only read the log indexes.

1
In Kibana, open Stack Management › API keys and click Create API key.
2
Name it, for example cerberius-agents.
3
Turn on Control security privileges, and give it only read access to your log indexes:

{
  "logs-read": {
    "indices": [
      { "names": ["logs-*"], "privileges": ["read", "view_index_metadata"] }
    ]
  }
}

5
Create it, and copy the Encoded value. Paste that as the API key.

On OpenSearch, make a user with a role that has read and indices:admin/mappings/fields/get on your log indexes, and choose Username and password. Amazon OpenSearch Service with signed requests (IAM) comes with cloud accounts; a username and password from its fine-grained access control works now.

The address must be reachable

Cerberius Agents must be able to reach the address. If saving says it points to a private or internal network, use a public address, with the login above.

What it can and can't do#

The agent never writes the search itself, only the words to look for. Cerberius Agents builds a plain search from them, with the time range and the namespaces. It only ever calls:

GET / (which product and version),
GET /<index pattern>/_count and GET /<index pattern>/_field_caps (for the check, and the field names),
POST /<index pattern>/_search with a plain search.

A search never holds a script, an aggregation or a change. Whatever the key could do, nothing is written or deleted.

How the agent searches#

During an investigation, the agent:

lists the fields your logs have, to find the right one
searches with Lucene query syntax: bare words search the message, for example "payment failed" AND kubernetes.pod.name:api*
picks the time range, up to the last 7 days, and up to 1,000 entries at a time
can ask for more fields of each entry, such as user.email

Every search shows in the agent's steps on the reply page.

Change or remove it#

Change any field and click Save. A new address (another host, scheme or port) removes the saved API key or password when you save, so a key is never sent somewhere it wasn't made for. Enter it again with the new address. To stop using it, click Disconnect under it: its settings and keys are removed from this platform.