Feat/tr/cluster dlq alarms #137
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nsearch cluster lives or not
theodorehreuter
commented
Apr 30, 2025
| resource "aws_cloudwatch_metric_alarm" "critical_stac_server_dlq_alarm" { | ||
| alarm_name = "CRITICAL: ${local.name_prefix}-stac-server-dead-letter SQS DLQ Critical Alarm" | ||
| alarm_description = "CRITICAL: 10 or more messages are persisting in the ${local.name_prefix}-stac-server SQS dead letter queue" | ||
| evaluation_periods = 5 |
There was a problem hiding this comment.
might be high. Thought is that if a bunch of messages are there for over 5 minutes its really properly stuck there, thus a CRITICAL alarm
…iption to accurately reflect threshold
cvangerpen
requested changes
May 8, 2025
| deploy_stac_server_outside_vpc = var.deploy_stac_server_outside_vpc | ||
| fd_web_acl_id = var.deploy_waf_rule ? module.base_infra.web_acl_id : var.ext_web_acl_id | ||
| warning_sns_topic_arn = module.base_infra.warning_sns_topic_arn | ||
| critical_sns_topic_arn = module.base_infra.critical_sns_topic_arn |
There was a problem hiding this comment.
We should allow the user to toggle whether these stac-server alarms are deployed or not via boolean flag; automatically deploying alarms can be handy but they may prefer to manage all tracked dimensions themselves, too.
Suggest using the same approach used in the cirrus module for consistency:
There was a problem hiding this comment.
added deploy_alarms trigger similar to cirrus module
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed Changes
This PR is filling in some gaps in alarm systems that were exposed by some infra failures.
stac-serverfor the dead letter queuestac-serverOpenSearch cluster healthOne outstanding question - optional deployment of alarms. I'm choosing to have them deployed automatically but open to suggestions for why they could be optionally deployed.
Checklist