[Doc] Document Grafana 12.4 changes

documentation/docs/alert/alert_rules.md

@@ -5,15 +5,16 @@ Alert rules describe the circumstances under which you want to be alerted. The e
 
 An alert rule consists of one or more queries and expressions, a condition, the frequency of evaluation, and the duration over which the condition is met. For example, you might configure an alert to fire and trigger a notification when MongoDB is down.
 
-An alert rule can be in three possible states:
+During evaluation, an alert rule moves through the following states:
 
-- Normal: Everything is working correctly and the conditions specified in the rule has not been met. This is the default state for newly created rules.
-- Pending: The conditions specified in the alert rule has been met, but for a time that is less than the configured duration.
-- Firing: Both the conditions and the duration specified in the alert rule have both been met.
+- **Normal**: Everything is working correctly, and the conditions specified in the rule have not been met. This is the default state for newly created rules.
+- **Pending**: The conditions specified in the alert rule have been met, but for less time than the configured duration.
+- **Firing**: Both the alert condition and the configured duration have been met.
+- **Recovering**: The alert conditions are no longer met, but the alert remains active until the metric is stable below the threshold. This prevents alerts from repeatedly flipping between **Firing** and **Normal** when metrics briefly cross a threshold.
 
 It takes at least one evaluation cycle for an alert rule to transition from one state to another (e.g., from `Normal` to `Pending`).
 
-## Alert rules templates
+## Alert rule templates
 
 PMM provides a set of Alert Rule templates with common events and expressions for alerting. These templates can be used as a basis for creating Alert Rules. You can also create your own templates if you need custom expressions.
 
@@ -37,7 +38,7 @@ This section focuses on creating an alert rule based on PMM templates. For infor
 Before creating PMM alert rules, configure the required alert resources:
 {.power-number}
 
-1. Go to **PMM Configuration > Settings > Advanced Settings** and ensure that the **Percona Alerting** option is enabled. When this is disabled, the **Alerting** page displays only Grafana-managed alert rules. This means that you will not be able to create alerts based on PMM templates.
+1. Go to **Configuration > Settings > Advanced settings** and ensure that the **Percona Alerting** option is enabled. When this is disabled, the **Alerting** page displays only Grafana-managed alert rules. This means that you will not be able to create alerts based on PMM templates.
 2. Go to **Dashboards** and check the folders available for storing alert rules. If none of the available folders are relevant for your future alert rules, click **New > New Folder** and create a custom one.
 3. Go to **Alerting > Alert rule templates** and check the default PMM templates. If none of the templates include a relevant expression for the type of alerts that you want to create, click **Add template** to create a custom template instead.
 
@@ -109,7 +110,7 @@ If you want to create custom templates, you can test the MetricsQL expressions f
 To test expressions for custom templates:
 {.power-number}
 
-1. On the main menu in PMM, choose **Explore > Metrics**.
+1. On the main menu in PMM, choose **Explore > PromQL builder**.
 2. Enter your expression in the **Metrics** field and click **Run query**.
 
 For example, to check the CPU usage, Go to **Explore > PromQL builder** and run the query expression below:
@@ -136,3 +137,9 @@ If you want to learn about creating Grafana alerts instead, check out [Grafana's
     - **Label** must be an exact match. You can find a complete list of labels using the <i class="uil uil-compass"></i> **Explore** menu in PMM.
 
 5. Click **Save and Exit** to close the page and go to the **Alert Rules** tab where you can review, edit and silence your new alert.
+
+## Recording rules
+
+Record an alert rules to pre-compute a query on a schedule and store the result as a new metric, which you can then reference in alert rules. This is useful when the same query powers multiple alert rules, for example, average CPU load across all nodes or replication lag across a cluster.
+
+To create a recording rule, click **Alerting > Alert rules** + New recording rule**. For full instructions, see [Create recording rules](https://grafana.com/docs/grafana/latest/alerting/alerting-rules/create-recording-rules/) in the Grafana documentation.

documentation/docs/alert/silence_alerts.md

@@ -22,7 +22,7 @@ To create a new silence:
 4. Enter one or more matching labels by filling out the **Name** and **Value** fields. Matchers determine which rules the silence will apply to. Note that all labels specified here must be matched by an alert for it to be silenced.
 5. Enter any additional comments you would like about this silence - by default, the date the silence was created is placed here.
 6. Review the affected alert instances that will be silenced.
-7. Click **Save silece**.
+7. Click **Save silence**.
 
 For more information on working with silences, see [About alerting silences](https://grafana.com/docs/grafana/latest/alerting/manage-notifications/create-silence/) in the Grafana documentation.
 

documentation/docs/install-pmm/install-pmm-client/connect-database/azure.md

@@ -8,7 +8,7 @@
 The Microsoft Azure feature is turned off by default. To turn it on:
 {.power-number}
 
-1. Go to  **PMM Configuration > Settings > Advanced Settings**.
+1. Go to **Configuration > Settings > Advanced Settings**.
 
 2. Click the <i class="uil uil-toggle-off"></i> toggle in the **Technical preview features** section of the page.
 
@@ -105,8 +105,7 @@ To get the tenant ID, client ID and the client secret:
 ![!](../../../images/azure_app_secret_02.png)
 ![!](../../../images/azure_app_secret_03.png)
 
-5. Copy the **value** of the application secret. Once you leave this page you won't be able to see the secret again and you will have to generate a new
-   one.
+5. Copy the **value** of the application secret. Once you leave this page you won't be able to see the secret again and you will have to generate a new one.
 ![!](../../../images/azure_app_secret_04.png)
 
 
@@ -121,7 +120,7 @@ To get the tenant ID, client ID and the client secret:
 ![!](../../../images/azure_app_permissions_03.png)
 ![!](../../../images/azure_app_permissions_04.png)
 
-When you fill in all fields press the *Discover* button and you will see a list of available databases for monitoring.
+When you fill in all fields press the **Discover** button and you will see a list of available databases for monitoring.
 
 ![!](../../../images/PMM_Add_Instance_Azure_2.png)
 
@@ -140,13 +139,13 @@ You can find more details on how to create DB on Azure at:
 - <https://docs.microsoft.com/en-us/azure/mysql/>
 
 !!! hint alert alert-success "Tip"
-    You must set `pg_stat_statements.track = all` in your PostgreSQL Server settings to use PMM Query Analytics. [Read more](../connect-database/postgresql.md#configure-monitoring-extension.
+    You must set `pg_stat_statements.track = all` in your PostgreSQL Server settings to use PMM Query Analytics. [Read more](../connect-database/postgresql.md#configure-monitoring-extension).
 
 ![!](../../../images/PMM_Add_Instance_Azure_3.png)
 
-In the list of databases on the Discovery page click **Start Monitoring** to add the selected Azure Database to PMM.
+In the list of databases on the **Discovery** page click **Start Monitoring** to add the selected Azure Database to PMM.
 
-Fill in all required fields and click **Add service**.
+Fill in all required fields. To set how long PMM waits when connecting, enter a value in **Connection timeout**. Leave it empty to use the default of 2s. Click **Add service**.
 
 PMM can use 3 exporters to collect metrics:
 

documentation/docs/reference/faq.md

@@ -147,12 +147,16 @@ PMM protects an exporter's output from unauthorized access by adding an authoriz
 
 ## How to provision PMM Server with non-default admin password?
 
-Currently, there is no API available to change the `admin` password at deployment time. However, you can use the `GF_SECURITY_ADMIN_PASSWORD` environment variable to set the password for the default `admin` user.
+You can set the password for the default `admin` user on first startup with the `GF_SECURITY_ADMIN_PASSWORD` environment variable, and optionally set a custom admin username with `GF_SECURITY_ADMIN_USER`.
+
+These variables are applied only during initial PMM data initialization. They are ignored on subsequent starts and container restarts.
 
 ```sh
 docker run -d --name pmm-server \
+  -e GF_SECURITY_ADMIN_USER="your_username" \
   -e GF_SECURITY_ADMIN_PASSWORD="your_secure_password" \
   -p 443:8443 \
+  --volume pmm-data:/srv \
   percona/pmm-server:latest
 ```
 

documentation/docs/reference/ui/ui_components.md

@@ -24,7 +24,7 @@ You'll find these options in the left-side menu:
 | :material-monitor-dashboard:| Operating System  | Monitor server-level metrics including CPU, memory, disk, and network performance. |
 | :material-view-grid-outline:| All Dashboards | Create and organize dashboards, create [folders](../../use/dashboards-panels/manage-dashboards/create-folders.md), import dashboards, create playlists, and manage snapshots. |
 | :material-chart-timeline-variant: | Query Analytics (QAN) | Analyze database queries over time, identify slow queries, optimize performance, and troubleshoot issues. |
-| :material-compass-outline:  | Explore | Run ad-hoc queries with [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) to investigate specific metrics without creating dashboards. |
+| :material-compass-outline:  | Explore | Investigate metrics without creating dashboards. **PromQL builder** lets you write custom [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) queries. You can also enable **Explore metrics** to visually browse metrics without writing queries. **Explore metrics** requires [enabling the Grafana Metrics Drilldown plugin](#enable-explore-metrics). |
 | :material-bell-outline: |  | Alerts | Create and manage [alerts](../../alert/index.md) that notify you when metrics exceed thresholds. |
 |:material-earth: | Advisors | Run health assessment checks on your databases and view recommendations for improving performance.|
 | :material-flask-outline: | Inventory | View and manage all monitored nodes, services, and agents registered in PMM. Check database and agent status, organize services by clusters, and add or remove monitored instances. |
@@ -68,3 +68,16 @@ Filter your monitoring data using these contextual options:
 - **PMM Annotations**: Toggle visibility of important events on your timelines
 
 These selectors change based on the dashboard you're viewing, showing only relevant options.
+
+## Enable Explore metrics
+
+Starting with PMM 3.8.0, **Explore metrics** (part of the Grafana Metrics Drilldown app) is no longer enabled by default. To re-enable it:
+{.power-number}
+
+1. Go to **Users and Access > Plugins and data > Plugins**.
+2. Search for **Grafana Metrics Drilldown**.
+3. Select the plugin and click **Enable**.
+
+Once enabled, **Explore metrics** reappears under **Explore** in the left sidebar.
+
+![Explore metrics in the PMM sidebar](../../images/PMM_Explore_metrics.jpg)

documentation/docs/release-notes/3.8.0.md

@@ -15,28 +15,62 @@ TODO
 
 ## ✨ Release highlights
 
-TODO
+### Grafana 12.4 upgrade
+
+PMM 3.8.0 ships with [Grafana 12.4](https://grafana.com/docs/grafana/latest/whatsnew/whats-new-in-v12-0/), so you will notice several day-to-day monitoring improvements right away:
+
+**Less alert noise**: Alert rules now support a [**Recovering** state](https://grafana.com/docs/grafana/latest/whatsnew/whats-new-in-v12-0/#recovering-state-for-alert-rules). This helps avoid repeated fire/resolved notifications when a metric keeps moving above and below an alert threshold, so you get fewer noisy alerts for spiky metrics like replication lag, query response time, and connection counts.
+
+    ![Recovering alert state](../images/recovering-alert.jpg)
+
+**Faster monitoring dashboards**: Tables in PMM dashboards load, sort, and filter significantly faster. This affects dashboards across PMM including **Nodes Overview**, **MySQL User Details**, **PostgreSQL Instances Overview**, MongoDB, and HA/PXC **Cluster Summary**.
+
+**Explore metrics now opt-in**: Starting with Grafana 12, **Explore metrics** is no longer enabled by default. If you used it to visually browse metrics without writing queries, you need to re-enable it. Go to **Users and Access > Plugins and data > Plugins**, search for **Grafana Metrics Drilldown**, and click **Enable**. Once enabled, **Explore metrics** reappears under **Explore** in the left sidebar. See [Enable Explore metrics](../reference/ui/ui_components.md#enable-explore-metrics).
+
+
+
+**Recording rules**: Alert rules now support [**recording rules**](https://grafana.com/docs/grafana/latest/alerting/alerting-rules/create-recording-rules/). A recording rule runs a query on a schedule and saves the result as a new metric, so your alerts reference the pre-computed result instead of re-running the full query on every evaluation cycle. This is especially useful when the same query powers multiple alert rules, for example, average CPU load across all nodes or replication lag across a cluster.
+
+**More changes in Grafana 12**: Grafana 12 removes Angular support. If you have custom dashboards using legacy **Graph (old)** or **Table (old)** panels, they are automatically converted to **Time series** and **Table** panels the first time you open them after upgrading to PMM 3.8.0. For the full list of changes, see the [Grafana 12 release notes](https://grafana.com/docs/grafana/latest/whatsnew/whats-new-in-v12-0/).
 
 ## 📦 Components upgrade
 
-TODO
+- **Grafana**: Upgraded to 12.4.
 
 ## 🔒 Security updates
 
-TODO
+- [PMM-15005](https://perconadev.atlassian.net/browse/PMM-15005): Removed a self-signed TLS certificate that was previously included in the PMM Server image to suppress ClickHouse startup log noise. Because the same certificate was shared across all PMM Server installations, CVE scanners flagged it as a security risk. Since ClickHouse in PMM doesn't use TLS, the startup log noise is now suppressed without needing a certificate at all.
+
+## ⚠️ Deprecations
+
+- [PMM-14968](https://perconadev.atlassian.net/browse/PMM-14968): UI-based upgrades are now officially deprecated and will be removed in PMM 3.9.0 (July 2026). PMM now shows a warning on all pages where UI-based upgrades were available. Switch to [Docker](../pmm-upgrade/upgrade_docker.md), [Podman](../pmm-upgrade/upgrade_podman.md), or [Helm](../pmm-upgrade/upgrade_helm.md) before then. If you use Watchtower, remove it before switching to avoid conflicts.
 
 ## 📈 Improvements
 
 - [PMM-12392](https://perconadev.atlassian.net/browse/PMM-12392): Improved how PMM collects processlist data on MySQL 5.7.39+ and 8.0.22+ by switching from Information Schema to Performance Schema. On busy MySQL instances, this reduces the chance of query blocking and deadlocks, lowering the overall monitoring impact on your MySQL server workload.
 
-- [PMM-14937](https://perconadev.atlassian.net/browse/PMM-14937): When adding a MySQL, PostgreSQL, MongoDB, ProxySQL, Valkey, Amazon RDS, or Azure service, you can now set a **Connection timeout** to control how long PMM waits before giving up on a connection attempt. This is useful when monitoring remote or high-latency databases where the default 2s timeout is too short.
+- [PMM-14937](https://perconadev.atlassian.net/browse/PMM-14937): When adding a MySQL, PostgreSQL, MongoDB, ProxySQL, Valkey, Amazon RDS, or Azure service, you can now set a **Connection timeout** to control how long PMM waits before giving up on a connection attempt. This is useful when monitoring remote or high-latency databases where the default timeout is too short.
+
+- [PMM-14068](https://perconadev.atlassian.net/browse/PMM-14068): The [PostgreSQL Instance Summary](../reference/dashboards/dashboard-postgresql-instance-summary.md) dashboard is now easier to read: summary panels use a consistent color scheme to help you spot issues faster, the layout fits laptop screens without cutting off panels, and tables no longer require horizontal scrolling to see all data.
+
+- [PMM-14930](https://perconadev.atlassian.net/browse/PMM-14930): Building on the [native PMM navigation introduced in 3.6.0](3.6.0.md), the **Settings** page now also runs in native PMM UI. You get the same settings with **Metrics resolution**, **Advanced settings**, and **SSH key** tabs in a page that loads faster and looks consistent with the rest of the new interface. Access it at **Configuration > Settings** in the left sidebar.
+
+    ![PMM Settings in native UI](../images/PMM_Settings_native_ui.png)
+
+- [PMM-14995](https://perconadev.atlassian.net/browse/PMM-14995): PMM has been updated with Percona's refreshed brand. You'll notice the updated logo across the sidebar, login screen, and help center, along with new technology icons for MySQL, PostgreSQL, MongoDB, and Valkey, and a refreshed color palette throughout the interface.
 
 ## ✅ Fixed issues
 
 - [PMM-14748](https://perconadev.atlassian.net/browse/PMM-14748): Fixed dashboards resetting the time zone to the browser default when navigating between pages. All dashboards now respect the time zone set in your profile preferences. See [Set time zone](../reference/ui/timezone.md).
 
 - [PMM-14791](https://perconadev.atlassian.net/browse/PMM-14791): Resolved an issue in the **Disk Space** graphs on the **Disk Details** dashboard where values were hard to read, legend labels were missing, and charts were displayed incorrectly.
 
+- [PMM-14512](https://perconadev.atlassian.net/browse/PMM-14512): Fixed the **BP Data Dirty** panel on the **MySQL InnoDB Details** dashboard showing no data after upgrading from PMM 2. The panel now correctly displays the ratio of dirty to data pages in the InnoDB buffer pool.
+
+- [PMM-14851](https://perconadev.atlassian.net/browse/PMM-14851): Fixed a bug in the PMM Helm chart that could cause `ClusterRole` to fail when deploying PMM on Kubernetes. The chart now deploys without RBAC errors.
+
+- [PMM-14934](https://perconadev.atlassian.net/browse/PMM-14934): Fixed an issue where the `GF_SECURITY_ADMIN_USER` and `GF_SECURITY_ADMIN_PASSWORD` environment variables were ignored when starting PMM Server with a named Docker volume (`-v pmm-data:/srv`). You can now use these variables to set the admin username and password at first start, regardless of how the data volume is mounted. As part of this fix, **the first start after upgrading takes about 20–30 seconds longer** than usual while PMM initializes. Subsequent starts are unaffected. PMM HA deployments are unimpacted.
+
 ## 🚀 Ready to upgrade to PMM 3.8.0?
 
 - [New installation](../quickstart/quickstart.md)