CloudWatch, Azure Monitor and Cloud Logging

If your logs are in your cloud's own log store, the agent can search them there:

• Amazon CloudWatch, with your Amazon Web Services account;
• Azure Monitor (Log Analytics, where Microsoft Sentinel keeps its logs too), with your Microsoft Azure account;
• Google Cloud Logging, with your Google Cloud account.

They need no keys of their own: they read with the cloud account, connected first under Settings › Accounts and cloud. For each search, the agent gets a short-lived pass that can only read, never the account's key.

The agent searches up to the last 7 days, and gets at most 1,000 entries per search. Values that look like secrets are removed from every entry before the agent sees it. The log watcher uses the same connection.

Connect it#

1. 1

   Connect the cloud account under Settings › Accounts and cloud, if it isn't yet.

2. 2

   Under that cloud's settings, turn on Use CloudWatch for logs (or Use Azure Monitor for logs, or Use Cloud Logging for logs). Or go to Settings › Live system, click Connect a cluster or logs, and choose it there.

3. 3

   Fill in where to look (below).

4. 4

   Describe your logs in How your logs are labelled.

5. 5

   Click Save, then Check connection.

If the platform lists namespaces (on the Kubernetes card of Live system), every search is limited to them:

• CloudWatch: your log groups are already this platform's own, so only a line that names another namespace is left out. Plain-text and Lambda lines stay.
• Azure Monitor and Cloud Logging: a workspace or project may hold other platforms' logs, so only entries from those namespaces are read. An entry that names no namespace is left out.

How the agent searches#

The agent writes plain words or "quoted phrases"; an entry must have all of them. It can't write the store's own query: we write it, and the words go in as text that can't change it. So a search never reaches past your log groups, workspace, projects or namespaces.

• CloudWatch matches the words exactly, case included. A word can't hold " or \.
• Azure Monitor matches them anywhere in the message, in any case.
• Cloud Logging matches them in any field, in any case.

Which entries are errors#

For the log watcher, an error is an entry whose level says so (error, fatal, critical, alert, emergency or panic). An entry with no level counts only when its message says error, exception, fatal, critical, traceback or panic. So INFO retrying after error is not an error. On Cloud Logging, Google's own audit logs are left out.

CloudWatch and Cloud Logging limit how often their logs can be searched, so the log watcher there doesn't read the lines around each error again. Azure Monitor does.