Alerts and Notifications in Netdata
Netdata provides two ways to send alert notifications. You can use either one or both at the same time.
Alerts are based on each node's health status. You can change thresholds, add new alerts, or silence-specific ones using Netdata's alerting system.
How Alert Notifications Work
| Method | Where Alerts Are Sent From | Customization | Highlights |
|---|---|---|---|
| Netdata Cloud | Cloud UI | Medium | Centralized alerting using connected nodes' health status |
| Netdata Agent | Local Netdata Agent | High | Node-level alerting with wide integration support |
You can enable one or both methods depending on your needs.
Quick Start
Use this table to choose and set up your preferred alerting method:
| Option | Setup Location | Setup Effort | Best For |
|---|---|---|---|
| Netdata Cloud | In the Cloud UI | Low | Teams managing multiple nodes |
| Netdata Agent | On each Netdata node | Medium | Full control and flexibility |
Set Up Alerts via Netdata Cloud
- Connect your nodes to Netdata Cloud
- In the UI, go to:
Space → Notifications - Choose an integration (e.g. Slack, Amazon SNS, Splunk)
- Set alert severity filters as needed
See all supported Cloud integrations
Set Up Alerts via Netdata Agent
-
Open the notification config:
sudo ./edit-config health_alarm_notify.conf -
Enable your preferred method, for example, email:
SEND_EMAIL="YES"
DEFAULT_RECIPIENT_EMAIL="you@example.com" -
Ensure your system can send mail (via
sendmail, SMTP relay, etc.) -
Restart the agent:
sudo systemctl restart netdata
See all Agent-based integrations
About the Agent's Health Monitoring
The Netdata Agent continuously monitors system health and performance. It offers:
- Hundreds of pre-configured alerts covering system, app, and service metrics
- No setup is required to work out of the box
- Dynamic customization you can fully control how, when, and what triggers an alert
See which collectors support alerts
Alert Philosophy and Design
The Anatomy of Intelligent Alerts
| Component | Description |
|---|---|
| Metrics Data | Collected every second (and at "event frequency" for eBPF metrics), giving you the most precise foundation for alerting |
| Filtering | Alerts run against specific metrics series, with support for pattern matching, labels, hostnames, and operating systems |
| Frequency | Configurable check intervals based on how quickly you need to know about issues |
| Templates | Write once, apply everywhere with dimension templates and the ability to apply logic to multiple charts |
| Calculation | Convert raw metrics into human-readable formats, like percentages, for easier understanding |
| Thresholds | Pre-configured values that determine when metrics are outside acceptable ranges |
| Hysteresis | Prevention of "flapping" alerts for metrics that hover around thresholds |
| Severity Levels | CLEAR, WARNING, and CRITICAL states to indicate the urgency of action needed |
Preventing False Positives and Alert Fatigue
| Feature | Benefit |
|---|---|
| Intelligent Defaults | Thresholds are carefully selected based on real-world experience |
| Dynamic Thresholds | Some alerts adapt to your system's baseline performance |
| Hysteresis Protection | Prevents floods of notifications when metrics fluctuate around a threshold |
| Notification Delays | Configurable delays ensure transient issues don't trigger alerts |
| Severity Separation | Different notification channels for different alert severities |
| Role-Based Routing | Ensures alerts reach only the appropriate stakeholders |
Troubleshooting Alerts
Netdata provides several tools to help you understand and resolve alert notifications:
Netdata Assistant
The Netdata Assistant is an AI-powered feature that guides you through troubleshooting alerts. When an alert triggers, you can:
- Click on the alert in your Netdata Cloud dashboard
- Press the Assistant button to open a floating window
- Receive customized information and troubleshooting tips specific to that alert
The Assistant window follows you as you navigate through Netdata dashboards, making it easy to reference while investigating the issue.
Community Resources
For more complex alert troubleshooting, Netdata maintains a dedicated Alerts Troubleshooting space in our community forum.
You can also get help through our GitHub repository or Discord channel if you prefer those platforms. After resolving an issue, consider sharing your experience in the forum to help other users.
Customizing Alerts
You can tune alerts to match your environment by adjusting thresholds, writing custom alert conditions, silencing alerts temporarily or permanently, and using statistical functions for smarter alerting.
Related Documentation
Do you have any feedback for this page? If so, you can open a new issue on our netdata/learn repository.