Skip to content

feat: Added auto-scaling functionality to the consumer and redoer. #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 10, 2025
Merged

feat: Added auto-scaling functionality to the consumer and redoer. #37

merged 4 commits into from
Oct 10, 2025

Conversation

jamesiarmes
Copy link
Member

@jamesiarmes jamesiarmes commented Oct 10, 2025 •
edited
Loading

Also removed SENZING_DATASOURCES from the tools container since the consumer can create data sources at runtime now.

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 10, 2025 •
edited
Loading

Plan output for service config


Note: Objects have changed outside of OpenTofu

OpenTofu detected the following changes made outside of OpenTofu since the
last "tofu apply" which may have affected this plan:

  # module.system.module.exporter.docker_image.container has been deleted
  - resource "docker_image" "container" {
      - id       = "sha256:883af577db3bf561b4bd99c760d997d13a3cd75041077e29cf54cc138c667410207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:jarmes" -> null
      - name     = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:jarmes" -> null
        # (2 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.system.module.tools.docker_image.container has been deleted
  - resource "docker_image" "container" {
      - id       = "sha256:33d2f55231d7b4f86f4a9cecca12191063d0c88f69e965ce765f4e1f9bd5a64b207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-tools:jarmes" -> null
      - name     = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-tools:jarmes" -> null
        # (2 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.system.module.consumer.module.task.docker_image.container has been deleted
  - resource "docker_image" "container" {
      - id       = "sha256:df657de5ccf355761d04be3bd8bb885f236974ad5c6ac56eadd28b25b4963ed6207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-consumer:jarmes" -> null
      - name     = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-consumer:jarmes" -> null
        # (2 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.system.module.redoer.module.task.docker_image.container has been deleted
  - resource "docker_image" "container" {
      - id       = "sha256:992f5095958ee7fbc04dd2659e3fc8abc3f8d8c0a18ec54ccc4305f4742eeceb207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:jarmes" -> null
      - name     = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:jarmes" -> null
        # (2 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

OpenTofu used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
  - destroy
+/- create replacement and then destroy
 <= read (data resources)

OpenTofu will perform the following actions:

  # module.system.aws_cloudwatch_metric_alarm.queue_active will be created
  + resource "aws_cloudwatch_metric_alarm" "queue_active" {
      + actions_enabled                       = true
      + alarm_actions                         = (known after apply)
      + alarm_description                     = "Monitor for active messages in the ingestion queue to scale service containers."
      + alarm_name                            = "sqs-senzing-development-queue-active"
      + arn                                   = (known after apply)
      + comparison_operator                   = "GreaterThanOrEqualToThreshold"
      + dimensions                            = {
          + "QueueName" = "sqs-senzing-development-queue"
        }
      + evaluate_low_sample_count_percentiles = (known after apply)
      + evaluation_periods                    = 1
      + id                                    = (known after apply)
      + metric_name                           = "ApproximateNumberOfMessagesVisible"
      + namespace                             = "AWS/SQS"
      + period                                = 60
      + region                                = "us-west-1"
      + statistic                             = "Sum"
      + tags_all                              = {
          + "application" = "sqs-senzing-development"
          + "environment" = "development"
          + "program"     = "safety-net"
          + "project"     = "sqs-senzing"
        }
      + threshold                             = 1
      + treat_missing_data                    = "notBreaching"
    }

  # module.system.aws_cloudwatch_metric_alarm.queue_empty will be created
  + resource "aws_cloudwatch_metric_alarm" "queue_empty" {
      + actions_enabled                       = true
      + alarm_actions                         = (known after apply)
      + alarm_description                     = "Monitor for an empty ingestion queue to scale down service containers."
      + alarm_name                            = "sqs-senzing-development-queue-empty"
      + arn                                   = (known after apply)
      + comparison_operator                   = "LessThanOrEqualToThreshold"
      + dimensions                            = {
          + "QueueName" = "sqs-senzing-development-queue"
        }
      + evaluate_low_sample_count_percentiles = (known after apply)
      + evaluation_periods                    = 5
      + id                                    = (known after apply)
      + metric_name                           = "ApproximateNumberOfMessagesVisible"
      + namespace                             = "AWS/SQS"
      + period                                = 60
      + region                                = "us-west-1"
      + statistic                             = "Maximum"
      + tags_all                              = {
          + "application" = "sqs-senzing-development"
          + "environment" = "development"
          + "program"     = "safety-net"
          + "project"     = "sqs-senzing"
        }
      + threshold                             = 0
      + treat_missing_data                    = "breaching"
    }

  # module.system.aws_iam_policy.queue will be updated in-place
  ~ resource "aws_iam_policy" "queue" {
        id               = "arn:aws:iam::207495628382:policy/sqs-senzing-development-queue-access-20250925044339106000000001"
        name             = "sqs-senzing-development-queue-access-20250925044339106000000001"
      ~ policy           = jsonencode(
          ~ {
              ~ Statement = [
                    {
                        Action   = [
                            "kms:Decrypt",
                        ]
                        Effect   = "Allow"
                        Resource = [
                            "arn:aws:kms:us-west-1:207495628382:key/a7e04278-b830-4097-84f5-823173e80116",
                        ]
                        Sid      = "KeyAccess"
                    },
                  ~ {
                      ~ Action   = [
                          + "sqs:ChangeMessageVisibility",
                            "sqs:DeleteMessage",
                            # (2 unchanged elements hidden)
                        ]
                        # (2 unchanged attributes hidden)
                    },
                ]
                # (1 unchanged attribute hidden)
            }
        )
        tags             = {
            "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
        # (7 unchanged attributes hidden)
    }

  # module.system.module.consumer.aws_appautoscaling_policy.down["this"] will be created
  + resource "aws_appautoscaling_policy" "down" {
      + alarm_arns         = (known after apply)
      + arn                = (known after apply)
      + id                 = (known after apply)
      + name               = "sqs-senzing-development-consumer-down"
      + policy_type        = "StepScaling"
      + region             = "us-west-1"
      + resource_id        = "service/sqs-senzing-development/sqs-senzing-development-consumer"
      + scalable_dimension = "ecs:service:DesiredCount"
      + service_namespace  = "ecs"

      + step_scaling_policy_configuration {
          + adjustment_type = "ExactCapacity"
          + cooldown        = 60

          + step_adjustment {
              + metric_interval_upper_bound = "0"
              + scaling_adjustment          = 0
            }
        }
    }

  # module.system.module.consumer.aws_appautoscaling_policy.up["this"] will be created
  + resource "aws_appautoscaling_policy" "up" {
      + alarm_arns         = (known after apply)
      + arn                = (known after apply)
      + id                 = (known after apply)
      + name               = "sqs-senzing-development-consumer-up"
      + policy_type        = "StepScaling"
      + region             = "us-west-1"
      + resource_id        = "service/sqs-senzing-development/sqs-senzing-development-consumer"
      + scalable_dimension = "ecs:service:DesiredCount"
      + service_namespace  = "ecs"

      + step_scaling_policy_configuration {
          + adjustment_type = "ExactCapacity"
          + cooldown        = 60

          + step_adjustment {
              + metric_interval_lower_bound = "0"
              + metric_interval_upper_bound = "250001"
              + scaling_adjustment          = 1
            }
          + step_adjustment {
              + metric_interval_lower_bound = "1000001"
              + metric_interval_upper_bound = "1250001"
              + scaling_adjustment          = 5
            }
          + step_adjustment {
              + metric_interval_lower_bound = "1250001"
              + metric_interval_upper_bound = "1500001"
              + scaling_adjustment          = 6
            }
          + step_adjustment {
              + metric_interval_lower_bound = "1500001"
              + metric_interval_upper_bound = "1750001"
              + scaling_adjustment          = 7
            }
          + step_adjustment {
              + metric_interval_lower_bound = "1750001"
              + metric_interval_upper_bound = "2000001"
              + scaling_adjustment          = 8
            }
          + step_adjustment {
              + metric_interval_lower_bound = "2000001"
              + metric_interval_upper_bound = "2250001"
              + scaling_adjustment          = 9
            }
          + step_adjustment {
              + metric_interval_lower_bound = "2250001"
              + scaling_adjustment          = 10
            }
          + step_adjustment {
              + metric_interval_lower_bound = "250001"
              + metric_interval_upper_bound = "500001"
              + scaling_adjustment          = 2
            }
          + step_adjustment {
              + metric_interval_lower_bound = "500001"
              + metric_interval_upper_bound = "750001"
              + scaling_adjustment          = 3
            }
          + step_adjustment {
              + metric_interval_lower_bound = "750001"
              + metric_interval_upper_bound = "1000001"
              + scaling_adjustment          = 4
            }
        }
    }

  # module.system.module.exporter.docker_image.container will be created
  + resource "docker_image" "container" {
      + id          = (known after apply)
      + image_id    = (known after apply)
      + name        = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:f9b8421e7d1804db620530d33f83e09518a47867"
      + repo_digest = (known after apply)
      + triggers    = {
          + "image_tage" = "f9b8421e7d1804db620530d33f83e09518a47867"
        }

      + build {
          # At least one attribute in this block is (or was) sensitive,
          # so its contents will not be displayed.
        }
    }

  # module.system.module.exporter.docker_registry_image.container must be replaced
+/- resource "docker_registry_image" "container" {
      ~ id                   = "sha256:cd1a4ddda07a1855d3e80e1fbede03c6b6e30589385f14329622b1c196515149" -> (known after apply)
      ~ name                 = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:jarmes" -> "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:f9b8421e7d1804db620530d33f83e09518a47867" # forces replacement
      ~ sha256_digest        = "sha256:cd1a4ddda07a1855d3e80e1fbede03c6b6e30589385f14329622b1c196515149" -> (known after apply)
      ~ triggers             = { # forces replacement
          - "sha" = "sha256:883af577db3bf561b4bd99c760d997d13a3cd75041077e29cf54cc138c667410207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:jarmes"
        } -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)

      ~ auth_config {
          ~ password = (sensitive value)
            # (2 unchanged attributes hidden)
        }
    }

  # module.system.module.redoer.aws_appautoscaling_policy.down["this"] will be created
  + resource "aws_appautoscaling_policy" "down" {
      + alarm_arns         = (known after apply)
      + arn                = (known after apply)
      + id                 = (known after apply)
      + name               = "sqs-senzing-development-redoer-down"
      + policy_type        = "StepScaling"
      + region             = "us-west-1"
      + resource_id        = "service/sqs-senzing-development/sqs-senzing-development-redoer"
      + scalable_dimension = "ecs:service:DesiredCount"
      + service_namespace  = "ecs"

      + step_scaling_policy_configuration {
          + adjustment_type = "ExactCapacity"
          + cooldown        = 60

          + step_adjustment {
              + metric_interval_upper_bound = "0"
              + scaling_adjustment          = 0
            }
        }
    }

  # module.system.module.redoer.aws_appautoscaling_policy.up["this"] will be created
  + resource "aws_appautoscaling_policy" "up" {
      + alarm_arns         = (known after apply)
      + arn                = (known after apply)
      + id                 = (known after apply)
      + name               = "sqs-senzing-development-redoer-up"
      + policy_type        = "StepScaling"
      + region             = "us-west-1"
      + resource_id        = "service/sqs-senzing-development/sqs-senzing-development-redoer"
      + scalable_dimension = "ecs:service:DesiredCount"
      + service_namespace  = "ecs"

      + step_scaling_policy_configuration {
          + adjustment_type = "ExactCapacity"
          + cooldown        = 60

          + step_adjustment {
              + metric_interval_lower_bound = "0"
              + scaling_adjustment          = 1
            }
        }
    }

  # module.system.module.tools.data.aws_caller_identity.identity will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "identity" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.system.module.tools.data.aws_ecr_authorization_token.token will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_ecr_authorization_token" "token" {
      + authorization_token = (sensitive value)
      + expires_at          = (known after apply)
      + id                  = (known after apply)
      + password            = (sensitive value)
      + proxy_endpoint      = (known after apply)
      + region              = (known after apply)
      + registry_id         = "207495628382"
      + user_name           = (known after apply)
    }

  # module.system.module.tools.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.system.module.tools.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
      + region      = (known after apply)
    }

  # module.system.module.tools.aws_iam_policy.task will be updated in-place
  ~ resource "aws_iam_policy" "task" {
        id               = "arn:aws:iam::207495628382:policy/sqs-senzing-development-tools-task"
        name             = "sqs-senzing-development-tools-task"
      ~ policy           = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "ssm:GetParameters",
                          - "ssm:GetParameter",
                        ]
                      - Effect   = "Allow"
                      - Resource = "arn:aws:ssm:us-west-1:207495628382:parameter/sqs-senzing/development/*"
                      - Sid      = "SSMAccess"
                    },
                  - {
                      - Action   = [
                          - "ssmmessages:CreateControlChannel",
                          - "ssmmessages:CreateDataChannel",
                          - "ssmmessages:OpenControlChannel",
                          - "ssmmessages:OpenDataChannel",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "EcsExecAccess"
                    },
                  - {
                      - Action   = [
                          - "cloudwatch:PutMetricData",
                          - "ec2:DescribeTags",
                          - "ec2:DescribeVolumes",
                          - "logs:CreateLogGroup",
                          - "logs:CreateLogStream",
                          - "logs:DescribeLogGroups",
                          - "logs:DescribeLogStreams",
                          - "logs:PutLogEvents",
                          - "logs:PutRetentionPolicy",
                          - "xray:GetSamplingRules",
                          - "xray:GetSamplingStatisticSummaries",
                          - "xray:GetSamplingTargets",
                          - "xray:PutTelemetryRecords",
                          - "xray:PutTraceSegments",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "CloudWatchAccess"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags             = {
            "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
        # (6 unchanged attributes hidden)
    }

  # module.system.module.tools.docker_image.container will be created
  + resource "docker_image" "container" {
      + id          = (known after apply)
      + image_id    = (known after apply)
      + name        = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-tools:f9b8421e7d1804db620530d33f83e09518a47867"
      + repo_digest = (known after apply)
      + triggers    = {
          + "image_tage" = "f9b8421e7d1804db620530d33f83e09518a47867"
        }

      + build {
          # At least one attribute in this block is (or was) sensitive,
          # so its contents will not be displayed.
        }
    }

  # module.system.module.tools.docker_registry_image.container must be replaced
+/- resource "docker_registry_image" "container" {
      ~ id                   = "sha256:7238058a503df93c5b4cd825bc08a95f3527630d82e0a3026dd1f239ad1c9ae6" -> (known after apply)
      ~ name                 = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-tools:jarmes" -> "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-tools:f9b8421e7d1804db620530d33f83e09518a47867" # forces replacement
      ~ sha256_digest        = "sha256:7238058a503df93c5b4cd825bc08a95f3527630d82e0a3026dd1f239ad1c9ae6" -> (known after apply)
      ~ triggers             = { # forces replacement
          - "sha" = "sha256:33d2f55231d7b4f86f4a9cecca12191063d0c88f69e965ce765f4e1f9bd5a64b207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-tools:jarmes"
        } -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)

      ~ auth_config {
          ~ address  = "https://207495628382.dkr.ecr.us-west-1.amazonaws.com" -> (known after apply)
          ~ password = (sensitive value)
          ~ username = "AWS" -> (known after apply)
        }
    }

  # module.system.module.consumer.module.scaling_target.aws_appautoscaling_target.ecs_target[0] will be created
  + resource "aws_appautoscaling_target" "ecs_target" {
      + arn                = (known after apply)
      + id                 = (known after apply)
      + max_capacity       = 10
      + min_capacity       = 0
      + region             = "us-west-1"
      + resource_id        = "service/sqs-senzing-development/sqs-senzing-development-consumer"
      + role_arn           = (known after apply)
      + scalable_dimension = "ecs:service:DesiredCount"
      + service_namespace  = "ecs"
      + tags_all           = {
          + "application" = "sqs-senzing-development"
          + "environment" = "development"
          + "program"     = "safety-net"
          + "project"     = "sqs-senzing"
        }

      + suspended_state (known after apply)
    }

  # module.system.module.consumer.module.service.aws_ecs_service.main[0] will be destroyed
  # (because index [0] is out of range for count)
  - resource "aws_ecs_service" "main" {
      - arn                                = "arn:aws:ecs:us-west-1:207495628382:service/sqs-senzing-development/sqs-senzing-development-consumer" -> null
      - availability_zone_rebalancing      = "DISABLED" -> null
      - cluster                            = "arn:aws:ecs:us-west-1:207495628382:cluster/sqs-senzing-development" -> null
      - deployment_maximum_percent         = 200 -> null
      - deployment_minimum_healthy_percent = 100 -> null
      - desired_count                      = 0 -> null
      - enable_ecs_managed_tags            = false -> null
      - enable_execute_command             = false -> null
      - force_delete                       = false -> null
      - health_check_grace_period_seconds  = 0 -> null
      - iam_role                           = "/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS" -> null
      - id                                 = "arn:aws:ecs:us-west-1:207495628382:service/sqs-senzing-development/sqs-senzing-development-consumer" -> null
      - launch_type                        = "FARGATE" -> null
      - name                               = "sqs-senzing-development-consumer" -> null
      - platform_version                   = "LATEST" -> null
      - propagate_tags                     = "SERVICE" -> null
      - region                             = "us-west-1" -> null
      - scheduling_strategy                = "REPLICA" -> null
      - tags                               = {
          - "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        } -> null
      - tags_all                           = {
          - "application"    = "sqs-senzing-development"
          - "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
          - "environment"    = "development"
          - "program"        = "safety-net"
          - "project"        = "sqs-senzing"
        } -> null
      - task_definition                    = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-dev-consumer:36" -> null
      - triggers                           = {} -> null
      - wait_for_steady_state              = false -> null

      - deployment_circuit_breaker {
          - enable   = false -> null
          - rollback = false -> null
        }

      - deployment_configuration {
          - bake_time_in_minutes = "0" -> null
          - strategy             = "ROLLING" -> null
        }

      - deployment_controller {
          - type = "ECS" -> null
        }

      - network_configuration {
          - assign_public_ip = false -> null
          - security_groups  = [
              - "sg-0f7c26de2ae898193",
            ] -> null
          - subnets          = [
              - "subnet-03dfcfff330d289fb",
              - "subnet-0b64a14539d697a4e",
            ] -> null
        }
    }

  # module.system.module.consumer.module.service.aws_ecs_service.main_ignore_desired_count_changes[0] will be created
  + resource "aws_ecs_service" "main_ignore_desired_count_changes" {
      + arn                                = (known after apply)
      + availability_zone_rebalancing      = "DISABLED"
      + cluster                            = "sqs-senzing-development"
      + deployment_maximum_percent         = 200
      + deployment_minimum_healthy_percent = 100
      + enable_ecs_managed_tags            = false
      + enable_execute_command             = false
      + force_delete                       = false
      + iam_role                           = (known after apply)
      + id                                 = (known after apply)
      + launch_type                        = "FARGATE"
      + name                               = "sqs-senzing-development-consumer"
      + platform_version                   = (known after apply)
      + propagate_tags                     = "SERVICE"
      + region                             = "us-west-1"
      + scheduling_strategy                = "REPLICA"
      + tags                               = {
          + "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
      + tags_all                           = {
          + "application"    = "sqs-senzing-development"
          + "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
          + "environment"    = "development"
          + "program"        = "safety-net"
          + "project"        = "sqs-senzing"
        }
      + task_definition                    = (known after apply)
      + triggers                           = (known after apply)
      + wait_for_steady_state              = false

      + deployment_configuration (known after apply)

      + deployment_controller {
          + type = "ECS"
        }

      + network_configuration {
          + assign_public_ip = false
          + security_groups  = [
              + "sg-0f7c26de2ae898193",
            ]
          + subnets          = [
              + "subnet-03dfcfff330d289fb",
              + "subnet-0b64a14539d697a4e",
            ]
        }
    }

  # module.system.module.consumer.module.task.data.aws_caller_identity.identity will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "identity" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.system.module.consumer.module.task.data.aws_ecr_authorization_token.token will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_ecr_authorization_token" "token" {
      + authorization_token = (sensitive value)
      + expires_at          = (known after apply)
      + id                  = (known after apply)
      + password            = (sensitive value)
      + proxy_endpoint      = (known after apply)
      + region              = (known after apply)
      + registry_id         = "207495628382"
      + user_name           = (known after apply)
    }

  # module.system.module.consumer.module.task.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.system.module.consumer.module.task.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
      + region      = (known after apply)
    }

  # module.system.module.consumer.module.task.aws_iam_policy.task will be updated in-place
  ~ resource "aws_iam_policy" "task" {
        id               = "arn:aws:iam::207495628382:policy/sqs-senzing-dev-consumer-task"
        name             = "sqs-senzing-dev-consumer-task"
      ~ policy           = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "ssm:GetParameters",
                          - "ssm:GetParameter",
                        ]
                      - Effect   = "Allow"
                      - Resource = "arn:aws:ssm:us-west-1:207495628382:parameter/sqs-senzing/dev/*"
                      - Sid      = "SSMAccess"
                    },
                  - {
                      - Action   = [
                          - "ssmmessages:CreateControlChannel",
                          - "ssmmessages:CreateDataChannel",
                          - "ssmmessages:OpenControlChannel",
                          - "ssmmessages:OpenDataChannel",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "EcsExecAccess"
                    },
                  - {
                      - Action   = [
                          - "cloudwatch:PutMetricData",
                          - "ec2:DescribeTags",
                          - "ec2:DescribeVolumes",
                          - "logs:CreateLogGroup",
                          - "logs:CreateLogStream",
                          - "logs:DescribeLogGroups",
                          - "logs:DescribeLogStreams",
                          - "logs:PutLogEvents",
                          - "logs:PutRetentionPolicy",
                          - "xray:GetSamplingRules",
                          - "xray:GetSamplingStatisticSummaries",
                          - "xray:GetSamplingTargets",
                          - "xray:PutTelemetryRecords",
                          - "xray:PutTraceSegments",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "CloudWatchAccess"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags             = {
            "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
        # (6 unchanged attributes hidden)
    }

  # module.system.module.consumer.module.task.docker_image.container will be created
  + resource "docker_image" "container" {
      + id          = (known after apply)
      + image_id    = (known after apply)
      + name        = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-consumer:f9b8421e7d1804db620530d33f83e09518a47867"
      + repo_digest = (known after apply)
      + triggers    = {
          + "image_tage" = "f9b8421e7d1804db620530d33f83e09518a47867"
        }

      + build {
          # At least one attribute in this block is (or was) sensitive,
          # so its contents will not be displayed.
        }
    }

  # module.system.module.consumer.module.task.docker_registry_image.container must be replaced
+/- resource "docker_registry_image" "container" {
      ~ id                   = "sha256:d57b622184cc1d92448e7d96552d0ab001fc2c9f4accc453fbc9e98dd3716d5e" -> (known after apply)
      ~ name                 = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-consumer:jarmes" -> "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-consumer:f9b8421e7d1804db620530d33f83e09518a47867" # forces replacement
      ~ sha256_digest        = "sha256:d57b622184cc1d92448e7d96552d0ab001fc2c9f4accc453fbc9e98dd3716d5e" -> (known after apply)
      ~ triggers             = { # forces replacement
          - "sha" = "sha256:df657de5ccf355761d04be3bd8bb885f236974ad5c6ac56eadd28b25b4963ed6207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-consumer:jarmes"
        } -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)

      ~ auth_config {
          ~ address  = "https://207495628382.dkr.ecr.us-west-1.amazonaws.com" -> (known after apply)
          ~ password = (sensitive value)
          ~ username = "AWS" -> (known after apply)
        }
    }

  # module.system.module.exporter.module.ecs_task.aws_ecs_task_definition.main[0] must be replaced
+/- resource "aws_ecs_task_definition" "main" {
      ~ arn                      = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-development-exporter:13" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-development-exporter" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [
              ~ {
                  - mountPoints            = []
                    name                   = "otel-collector"
                  - portMappings           = []
                  - systemControls         = []
                  - volumesFrom            = []
                    # (7 unchanged attributes hidden)
                },
              ~ {
                  ~ environment            = [
                      ~ {
                            name  = "LOG_LEVEL"
                          ~ value = "DEBUG" -> "INFO"
                        },
                        {
                            name  = "Q_URL"
                            value = "https://sqs.us-west-1.amazonaws.com/207495628382/sqs-senzing-development-queue"
                        },
                        # (1 unchanged element hidden)
                    ]
                  ~ image                  = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:jarmes" -> "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-exporter:f9b8421e7d1804db620530d33f83e09518a47867"
                    name                   = "sqs-senzing-development-exporter"
                  ~ portMappings           = [
                      ~ {
                          - hostPort      = 80
                          - protocol      = "tcp"
                            # (1 unchanged attribute hidden)
                        },
                    ]
                  - systemControls         = []
                  - volumesFrom            = []
                    # (9 unchanged attributes hidden)
                },
            ] # forces replacement
        )
      ~ id                       = "sqs-senzing-development-exporter" -> (known after apply)
      ~ revision                 = 13 -> (known after apply)
        tags                     = {
            "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
        # (12 unchanged attributes hidden)

      - volume {
          - configure_at_launch = false -> null
          - name                = "logs" -> null
        }
      - volume {
          - configure_at_launch = false -> null
          - name                = "senzing-home" -> null
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "logs"
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "senzing-home"
        }
    }

  # module.system.module.redoer.module.scaling_target.aws_appautoscaling_target.ecs_target[0] will be created
  + resource "aws_appautoscaling_target" "ecs_target" {
      + arn                = (known after apply)
      + id                 = (known after apply)
      + max_capacity       = 1
      + min_capacity       = 1
      + region             = "us-west-1"
      + resource_id        = "service/sqs-senzing-development/sqs-senzing-development-redoer"
      + role_arn           = (known after apply)
      + scalable_dimension = "ecs:service:DesiredCount"
      + service_namespace  = "ecs"
      + tags_all           = {
          + "application" = "sqs-senzing-development"
          + "environment" = "development"
          + "program"     = "safety-net"
          + "project"     = "sqs-senzing"
        }

      + suspended_state (known after apply)
    }

  # module.system.module.redoer.module.service.aws_ecs_service.main[0] will be destroyed
  # (because index [0] is out of range for count)
  - resource "aws_ecs_service" "main" {
      - arn                                = "arn:aws:ecs:us-west-1:207495628382:service/sqs-senzing-development/sqs-senzing-development-redoer" -> null
      - availability_zone_rebalancing      = "DISABLED" -> null
      - cluster                            = "arn:aws:ecs:us-west-1:207495628382:cluster/sqs-senzing-development" -> null
      - deployment_maximum_percent         = 200 -> null
      - deployment_minimum_healthy_percent = 100 -> null
      - desired_count                      = 0 -> null
      - enable_ecs_managed_tags            = false -> null
      - enable_execute_command             = false -> null
      - force_delete                       = false -> null
      - health_check_grace_period_seconds  = 0 -> null
      - iam_role                           = "/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS" -> null
      - id                                 = "arn:aws:ecs:us-west-1:207495628382:service/sqs-senzing-development/sqs-senzing-development-redoer" -> null
      - launch_type                        = "FARGATE" -> null
      - name                               = "sqs-senzing-development-redoer" -> null
      - platform_version                   = "LATEST" -> null
      - propagate_tags                     = "SERVICE" -> null
      - region                             = "us-west-1" -> null
      - scheduling_strategy                = "REPLICA" -> null
      - tags                               = {
          - "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        } -> null
      - tags_all                           = {
          - "application"    = "sqs-senzing-development"
          - "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
          - "environment"    = "development"
          - "program"        = "safety-net"
          - "project"        = "sqs-senzing"
        } -> null
      - task_definition                    = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-dev-redoer:6" -> null
      - triggers                           = {} -> null
      - wait_for_steady_state              = false -> null

      - deployment_circuit_breaker {
          - enable   = false -> null
          - rollback = false -> null
        }

      - deployment_configuration {
          - bake_time_in_minutes = "0" -> null
          - strategy             = "ROLLING" -> null
        }

      - deployment_controller {
          - type = "ECS" -> null
        }

      - network_configuration {
          - assign_public_ip = false -> null
          - security_groups  = [
              - "sg-0f7c26de2ae898193",
            ] -> null
          - subnets          = [
              - "subnet-03dfcfff330d289fb",
              - "subnet-0b64a14539d697a4e",
            ] -> null
        }
    }

  # module.system.module.redoer.module.service.aws_ecs_service.main_ignore_desired_count_changes[0] will be created
  + resource "aws_ecs_service" "main_ignore_desired_count_changes" {
      + arn                                = (known after apply)
      + availability_zone_rebalancing      = "DISABLED"
      + cluster                            = "sqs-senzing-development"
      + deployment_maximum_percent         = 200
      + deployment_minimum_healthy_percent = 100
      + enable_ecs_managed_tags            = false
      + enable_execute_command             = false
      + force_delete                       = false
      + iam_role                           = (known after apply)
      + id                                 = (known after apply)
      + launch_type                        = "FARGATE"
      + name                               = "sqs-senzing-development-redoer"
      + platform_version                   = (known after apply)
      + propagate_tags                     = "SERVICE"
      + region                             = "us-west-1"
      + scheduling_strategy                = "REPLICA"
      + tags                               = {
          + "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
      + tags_all                           = {
          + "application"    = "sqs-senzing-development"
          + "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
          + "environment"    = "development"
          + "program"        = "safety-net"
          + "project"        = "sqs-senzing"
        }
      + task_definition                    = (known after apply)
      + triggers                           = (known after apply)
      + wait_for_steady_state              = false

      + deployment_configuration (known after apply)

      + deployment_controller {
          + type = "ECS"
        }

      + network_configuration {
          + assign_public_ip = false
          + security_groups  = [
              + "sg-0f7c26de2ae898193",
            ]
          + subnets          = [
              + "subnet-03dfcfff330d289fb",
              + "subnet-0b64a14539d697a4e",
            ]
        }
    }

  # module.system.module.redoer.module.task.docker_image.container will be created
  + resource "docker_image" "container" {
      + id          = (known after apply)
      + image_id    = (known after apply)
      + name        = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:f9b8421e7d1804db620530d33f83e09518a47867"
      + repo_digest = (known after apply)
      + triggers    = {
          + "image_tage" = "f9b8421e7d1804db620530d33f83e09518a47867"
        }

      + build {
          # At least one attribute in this block is (or was) sensitive,
          # so its contents will not be displayed.
        }
    }

  # module.system.module.redoer.module.task.docker_registry_image.container must be replaced
+/- resource "docker_registry_image" "container" {
      ~ id                   = "sha256:2e5200a7581fef2f050920c8b23314e4f7b1ed1b2e41f18b418adb92cb235210" -> (known after apply)
      ~ name                 = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:jarmes" -> "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:f9b8421e7d1804db620530d33f83e09518a47867" # forces replacement
      ~ sha256_digest        = "sha256:2e5200a7581fef2f050920c8b23314e4f7b1ed1b2e41f18b418adb92cb235210" -> (known after apply)
      ~ triggers             = { # forces replacement
          - "sha" = "sha256:992f5095958ee7fbc04dd2659e3fc8abc3f8d8c0a18ec54ccc4305f4742eeceb207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:jarmes"
        } -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)

      ~ auth_config {
          ~ password = (sensitive value)
            # (2 unchanged attributes hidden)
        }
    }

  # module.system.module.tools.module.ecr.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.system.module.tools.module.ecr.data.aws_iam_policy_document.repository[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "repository" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions = [
              + "ecr:BatchCheckLayerAvailability",
              + "ecr:BatchGetImage",
              + "ecr:DescribeImageScanFindings",
              + "ecr:DescribeImages",
              + "ecr:DescribeRepositories",
              + "ecr:GetAuthorizationToken",
              + "ecr:GetDownloadUrlForLayer",
              + "ecr:GetLifecyclePolicy",
              + "ecr:GetLifecyclePolicyPreview",
              + "ecr:GetRepositoryPolicy",
              + "ecr:ListImages",
              + "ecr:ListTagsForResource",
            ]
          + sid     = "PrivateReadOnly"

          + principals {
              + identifiers = [
                  + (known after apply),
                ]
              + type        = "AWS"
            }
        }
    }

  # module.system.module.tools.module.ecr.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.system.module.tools.module.ecr.aws_ecr_repository_policy.this[0] will be updated in-place
  ~ resource "aws_ecr_repository_policy" "this" {
        id          = "sqs-senzing-development-tools"
      ~ policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = [
                          - "ecr:ListTagsForResource",
                          - "ecr:ListImages",
                          - "ecr:GetRepositoryPolicy",
                          - "ecr:GetLifecyclePolicyPreview",
                          - "ecr:GetLifecyclePolicy",
                          - "ecr:GetDownloadUrlForLayer",
                          - "ecr:GetAuthorizationToken",
                          - "ecr:DescribeRepositories",
                          - "ecr:DescribeImages",
                          - "ecr:DescribeImageScanFindings",
                          - "ecr:BatchGetImage",
                          - "ecr:BatchCheckLayerAvailability",
                        ]
                      - Effect    = "Allow"
                      - Principal = {
                          - AWS = "arn:aws:iam::207495628382:root"
                        }
                      - Sid       = "PrivateReadOnly"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # module.system.module.tools.module.ecs_task.aws_ecs_task_definition.main[0] must be replaced
+/- resource "aws_ecs_task_definition" "main" {
      ~ arn                      = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-development-tools:62" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-development-tools" -> (known after apply)
      ~ container_definitions    = jsonencode(
            [
              - {
                  - command                = [
                      - "--config=/etc/ecs/container-insights/otel-task-metrics-config.yaml",
                    ]
                  - environment            = [
                      - {
                          - name  = "OTEL_LOG_LEVEL"
                          - value = "info"
                        },
                    ]
                  - essential              = false
                  - image                  = "public.ecr.aws/aws-observability/aws-otel-collector:latest"
                  - logConfiguration       = {
                      - logDriver = "awslogs"
                      - options   = {
                          - awslogs-group         = "/aws/ecs/sqs-senzing/development/tools"
                          - awslogs-region        = "us-west-1"
                          - awslogs-stream-prefix = "otel-collector"
                        }
                    }
                  - mountPoints            = []
                  - name                   = "otel-collector"
                  - portMappings           = []
                  - readonlyRootFilesystem = true
                  - secrets                = [
                      - {
                          - name      = "AOT_CONFIG_CONTENT"
                          - valueFrom = "arn:aws:ssm:us-west-1:207495628382:parameter/sqs-senzing/development/otel"
                        },
                    ]
                  - systemControls         = []
                  - volumesFrom            = []
                },
              - {
                  - cpu                    = 1024
                  - environment            = [
                      - {
                          - name  = "LOG_LEVEL"
                          - value = "DEBUG"
                        },
                      - {
                          - name  = "PGHOST"
                          - value = "sqs-senzing-development-senzing.cluster-c7qqmqeoy39j.us-west-1.rds.amazonaws.com"
                        },
                      - {
                          - name  = "PGSSLMODE"
                          - value = "require"
                        },
                      - {
                          - name  = "Q_URL"
                          - value = "https://sqs.us-west-1.amazonaws.com/207495628382/sqs-senzing-development-queue"
                        },
                      - {
                          - name  = "SENZING_DATASOURCES"
                          - value = "PEOPLE CUSTOMERS"
                        },
                    ]
                  - essential              = true
                  - image                  = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-development-tools:jarmes"
                  - linuxParameters        = {
                      - initProcessEnabled = true
                    }
                  - logConfiguration       = {
                      - logDriver = "awslogs"
                      - options   = {
                          - awslogs-group         = "/aws/ecs/sqs-senzing/development/tools"
                          - awslogs-region        = "us-west-1"
                          - awslogs-stream-prefix = "ecs"
                        }
                    }
                  - memory                 = 4096
                  - memoryReservation      = 4096
                  - mountPoints            = [
                      - {
                          - containerPath = "/var/lib/amazon"
                          - readOnly      = false
                          - sourceVolume  = "aws-lib"
                        },
                      - {
                          - containerPath = "/var/log"
                          - readOnly      = false
                          - sourceVolume  = "logs"
                        },
                      - {
                          - containerPath = "/home/senzing"
                          - readOnly      = false
                          - sourceVolume  = "senzing-home"
                        },
                    ]
                  - name                   = "sqs-senzing-development-tools"
                  - portMappings           = [
                      - {
                          - containerPort = 80
                          - hostPort      = 80
                          - protocol      = "tcp"
                        },
                    ]
                  - readonlyRootFilesystem = true
                  - secrets                = [
                      - {
                          - name      = "PGPASSWORD"
                          - valueFrom = "arn:aws:secretsmanager:us-west-1:207495628382:secret:rds!cluster-2e4a2e07-8cf4-45ac-aec7-db2686d406d5-HPW6AD:password::"
                        },
                      - {
                          - name      = "PGUSER"
                          - valueFrom = "arn:aws:secretsmanager:us-west-1:207495628382:secret:rds!cluster-2e4a2e07-8cf4-45ac-aec7-db2686d406d5-HPW6AD:username::"
                        },
                      - {
                          - name      = "SENZING_ENGINE_CONFIGURATION_JSON"
                          - valueFrom = "arn:aws:ssm:us-west-1:207495628382:parameter/sqs-senzing/development/senzing"
                        },
                    ]
                  - systemControls         = []
                  - volumesFrom            = []
                },
            ] # forces replacement
        ) -> (known after apply) # forces replacement
      ~ id                       = "sqs-senzing-development-tools" -> (known after apply)
      ~ revision                 = 62 -> (known after apply)
        tags                     = {
            "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
        # (12 unchanged attributes hidden)

      - volume {
          - configure_at_launch = false -> null
          - name                = "aws-lib" -> null
        }
      - volume {
          - configure_at_launch = false -> null
          - name                = "logs" -> null
        }
      - volume {
          - configure_at_launch = false -> null
          - name                = "senzing-home" -> null
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "aws-lib"
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "logs"
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "senzing-home"
        }
    }

  # module.system.module.consumer.module.task.module.ecr.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.system.module.consumer.module.task.module.ecr.data.aws_iam_policy_document.repository[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "repository" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions = [
              + "ecr:BatchCheckLayerAvailability",
              + "ecr:BatchGetImage",
              + "ecr:DescribeImageScanFindings",
              + "ecr:DescribeImages",
              + "ecr:DescribeRepositories",
              + "ecr:GetAuthorizationToken",
              + "ecr:GetDownloadUrlForLayer",
              + "ecr:GetLifecyclePolicy",
              + "ecr:GetLifecyclePolicyPreview",
              + "ecr:GetRepositoryPolicy",
              + "ecr:ListImages",
              + "ecr:ListTagsForResource",
            ]
          + sid     = "PrivateReadOnly"

          + principals {
              + identifiers = [
                  + (known after apply),
                ]
              + type        = "AWS"
            }
        }
    }

  # module.system.module.consumer.module.task.module.ecr.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.system.module.consumer.module.task.module.ecr.aws_ecr_repository_policy.this[0] will be updated in-place
  ~ resource "aws_ecr_repository_policy" "this" {
        id          = "sqs-senzing-dev-consumer"
      ~ policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = [
                          - "ecr:ListTagsForResource",
                          - "ecr:ListImages",
                          - "ecr:GetRepositoryPolicy",
                          - "ecr:GetLifecyclePolicyPreview",
                          - "ecr:GetLifecyclePolicy",
                          - "ecr:GetDownloadUrlForLayer",
                          - "ecr:GetAuthorizationToken",
                          - "ecr:DescribeRepositories",
                          - "ecr:DescribeImages",
                          - "ecr:DescribeImageScanFindings",
                          - "ecr:BatchGetImage",
                          - "ecr:BatchCheckLayerAvailability",
                        ]
                      - Effect    = "Allow"
                      - Principal = {
                          - AWS = "arn:aws:iam::207495628382:root"
                        }
                      - Sid       = "PrivateReadOnly"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # module.system.module.consumer.module.task.module.ecs_task.aws_ecs_task_definition.main[0] must be replaced
+/- resource "aws_ecs_task_definition" "main" {
      ~ arn                      = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-dev-consumer:36" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-dev-consumer" -> (known after apply)
      ~ container_definitions    = jsonencode(
            [
              - {
                  - command                = [
                      - "--config=/etc/ecs/container-insights/otel-task-metrics-config.yaml",
                    ]
                  - environment            = [
                      - {
                          - name  = "OTEL_LOG_LEVEL"
                          - value = "info"
                        },
                    ]
                  - essential              = false
                  - image                  = "public.ecr.aws/aws-observability/aws-otel-collector:latest"
                  - logConfiguration       = {
                      - logDriver = "awslogs"
                      - options   = {
                          - awslogs-group         = "/aws/ecs/sqs-senzing/dev/consumer"
                          - awslogs-region        = "us-west-1"
                          - awslogs-stream-prefix = "otel-collector"
                        }
                    }
                  - mountPoints            = []
                  - name                   = "otel-collector"
                  - portMappings           = []
                  - readonlyRootFilesystem = true
                  - secrets                = [
                      - {
                          - name      = "AOT_CONFIG_CONTENT"
                          - valueFrom = "arn:aws:ssm:us-west-1:207495628382:parameter/sqs-senzing/development/otel"
                        },
                    ]
                  - systemControls         = []
                  - volumesFrom            = []
                },
              - {
                  - cpu                    = 1024
                  - environment            = [
                      - {
                          - name  = "LOG_LEVEL"
                          - value = "DEBUG"
                        },
                      - {
                          - name  = "Q_URL"
                          - value = "https://sqs.us-west-1.amazonaws.com/207495628382/sqs-senzing-development-queue"
                        },
                    ]
                  - essential              = true
                  - image                  = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-consumer:jarmes"
                  - linuxParameters        = {
                      - initProcessEnabled = true
                    }
                  - logConfiguration       = {
                      - logDriver = "awslogs"
                      - options   = {
                          - awslogs-group         = "/aws/ecs/sqs-senzing/dev/consumer"
                          - awslogs-region        = "us-west-1"
                          - awslogs-stream-prefix = "ecs"
                        }
                    }
                  - memory                 = 4096
                  - memoryReservation      = 4096
                  - mountPoints            = [
                      - {
                          - containerPath = "/var/log"
                          - readOnly      = false
                          - sourceVolume  = "logs"
                        },
                      - {
                          - containerPath = "/home/senzing"
                          - readOnly      = false
                          - sourceVolume  = "senzing-home"
                        },
                    ]
                  - name                   = "sqs-senzing-dev-consumer"
                  - portMappings           = [
                      - {
                          - containerPort = 80
                          - hostPort      = 80
                          - protocol      = "tcp"
                        },
                    ]
                  - readonlyRootFilesystem = true
                  - secrets                = [
                      - {
                          - name      = "SENZING_ENGINE_CONFIGURATION_JSON"
                          - valueFrom = "arn:aws:ssm:us-west-1:207495628382:parameter/sqs-senzing/development/senzing"
                        },
                    ]
                  - systemControls         = []
                  - volumesFrom            = []
                },
            ] # forces replacement
        ) -> (known after apply) # forces replacement
      ~ id                       = "sqs-senzing-dev-consumer" -> (known after apply)
      ~ revision                 = 36 -> (known after apply)
        tags                     = {
            "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
        # (12 unchanged attributes hidden)

      - volume {
          - configure_at_launch = false -> null
          - name                = "logs" -> null
        }
      - volume {
          - configure_at_launch = false -> null
          - name                = "senzing-home" -> null
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "logs"
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "senzing-home"
        }
    }

  # module.system.module.redoer.module.task.module.ecs_task.aws_ecs_task_definition.main[0] must be replaced
+/- resource "aws_ecs_task_definition" "main" {
      ~ arn                      = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-dev-redoer:6" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:us-west-1:207495628382:task-definition/sqs-senzing-dev-redoer" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [
              ~ {
                  - mountPoints            = []
                    name                   = "otel-collector"
                  - portMappings           = []
                  - systemControls         = []
                  - volumesFrom            = []
                    # (7 unchanged attributes hidden)
                },
              ~ {
                  ~ environment            = [
                      ~ {
                            name  = "LOG_LEVEL"
                          ~ value = "DEBUG" -> "INFO"
                        },
                        {
                            name  = "Q_URL"
                            value = "https://sqs.us-west-1.amazonaws.com/207495628382/sqs-senzing-development-queue"
                        },
                    ]
                  ~ image                  = "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:jarmes" -> "207495628382.dkr.ecr.us-west-1.amazonaws.com/sqs-senzing-dev-redoer:f9b8421e7d1804db620530d33f83e09518a47867"
                  ~ memory                 = 4096 -> 2048
                  ~ memoryReservation      = 4096 -> 2048
                    name                   = "sqs-senzing-dev-redoer"
                  ~ portMappings           = [
                      ~ {
                          - hostPort      = 80
                          - protocol      = "tcp"
                            # (1 unchanged attribute hidden)
                        },
                    ]
                  - systemControls         = []
                  - volumesFrom            = []
                    # (7 unchanged attributes hidden)
                },
            ] # forces replacement
        )
      ~ id                       = "sqs-senzing-dev-redoer" -> (known after apply)
      ~ memory                   = "4096" -> "2048" # forces replacement
      ~ revision                 = 6 -> (known after apply)
        tags                     = {
            "awsApplication" = "arn:aws:resource-groups:us-west-1:207495628382:group/sqs-senzing-development/0949oli7hmptcuydpbudaxsl9k"
        }
        # (11 unchanged attributes hidden)

      - volume {
          - configure_at_launch = false -> null
          - name                = "logs" -> null
        }
      - volume {
          - configure_at_launch = false -> null
          - name                = "senzing-home" -> null
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "logs"
        }
      + volume {
          + configure_at_launch = (known after apply)
          + name                = "senzing-home"
        }
    }

Plan: 22 to add, 5 to change, 10 to destroy.

Changes to Outputs:
  ~ image_tag              = "jarmes" -> "f9b8421e7d1804db620530d33f83e09518a47867"

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: tfplan

To perform exactly these actions, run the following command to apply:
    tofu apply "tfplan"

@jamesiarmes jamesiarmes requested a review from Copilot October 10, 2025 21:14
Copilot
Copilot AI reviewed Oct 10, 2025
View reviewed changes
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds auto-scaling functionality to the consumer and redoer services by implementing AWS Application Auto Scaling with CloudWatch alarms based on SQS queue metrics. The scaling system monitors queue depth to scale up containers when messages are present and scale down when the queue remains empty for a configurable threshold.

  • Added auto-scaling configuration variables for maximum containers, message thresholds, and empty queue timing
  • Implemented AWS Application Auto Scaling policies with step scaling for both consumer and redoer services
  • Created CloudWatch alarms to trigger scaling actions based on SQS queue message count

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tofu/modules/system/variables.tf Added auto-scaling configuration variables
tofu/modules/system/ecs.tf Updated service modules to use cluster name and added scaling policies
tofu/modules/system/alarms.tf Created CloudWatch alarms for queue monitoring and scaling triggers
tofu/modules/persistent_service/ Added auto-scaling functionality with step scaling policies and scaling targets
tofu/config/service/ Propagated new variables to service configuration
.github/workflows/ Added new environment variables to CI/CD workflows

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@jamesiarmes jamesiarmes marked this pull request as ready for review October 10, 2025 21:23
@jamesiarmes jamesiarmes requested a review from a team as a code owner October 10, 2025 21:23
@jamesiarmes jamesiarmes merged commit 8b16625 into main Oct 10, 2025
10 checks passed
@jamesiarmes jamesiarmes deleted the auto-scaling-consumer branch October 10, 2025 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jamesiarmes