Create pre-production stage for CI pipeline

tests/ci/cdk/README.md

@@ -2,7 +2,26 @@
 
 AWS-LC CI uses AWS CDK to define and deploy AWS resources (e.g. AWS CodeBuild, ECR).
 
-## CI Setup
+## Table of Contents
+- [CDK Setup](#cdk-setup)
+  - [Before running CDK command](#before-running-cdk-command)
+  - [Minimal permissions](#minimal-permissions)
+  - [Pipeline Commands](#pipeline-commands)
+  - [CI Commands](#ci-commands)
+- [AWS-LC Benchmarking Framework](#aws-lc-benchmarking-framework)
+  - [Framework Setup](#framework-setup)
+  - [How to Use](#how-to-use)
+    - [Start from Pull Request](#start-from-pull-request)
+    - [Start Locally](#start-locally)
+    - [Examine Output](#examine-output)
+- [Files](#files)
+- [Development Reference](#development-reference)
+  - [Useful commands](#useful-commands)
+  - [Useful Docker image build commands](#useful-docker-image-build-commands)
+    - [Linux Docker image build](#linux-docker-image-build)
+    - [Windows Docker image build (DEPRECATED)](#windows-docker-image-build-deprecated)
+
+## CDK Setup
 
 ### Before running CDK command:
 
@@ -63,17 +82,50 @@ To setup or update the CI in your account you will need the following IAM permis
   * secretsmanager:DeleteSecret
   * secretsmanager:GetSecretValue
 
-### Commands
+### Pipeline Commands
+Use these commands to deploy the CI pipeline. Any changes to the CI or Docker images will be updated automatically after the pipeline is deployed. 
+
+These commands are run from `aws-lc/tests/ci/cdk`.
+
+[SKIP IF NO CROSS-ACCOUNT DEPLOYMENT] Give the pipeline account administrator access to the deployment account's CloudFormation. Repeat this step depending on how many deployment environment there are. You only need to run this step once when the pipeline is deploying to a new account for the first time.
+```
+cdk bootstrap aws://${DEPLOY_ACCOUNT_ID}/us-west-2 --trust ${PIPELINE_ACCOUNT_ID} --trust-for-lookup ${PIPELINE_ACCOUNT_ID} --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess
+```
+
+If not done previously, bootstrap cdk for the pipeline account before running the next commands.
+```
+cdk bootstrap aws://${PIPELINE_ACCOUNT_ID}/us-west-2
+```
+
+To deploy dev pipeline to the same account as your CI:
+```
+./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --github-source-version ${GITHUB_SOURCE_VERSION} --deploy-account ${DEPLOY_ACCOUNT_ID} --action deploy-dev-pipeline
+```
+
+To deploy dev pipeline but pipeline is hosted in a separate account:
+```
+./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --github-source-version ${GITHUB_SOURCE_VERSION} --pipeline-account ${PIPELINE_ACCOUNT_ID} --deploy-account ${DEPLOY_ACCOUNT_ID} --action deploy-dev-pipeline
+```
+
+To deploy production pipeline using default parameters:
+```
+./run-cdk.sh --action deploy-production-pipeline
+```
+
+### CI Commands
+Use these commands if you wish to deploy individual stacks instead of the entire pipeline.
+
+These commands are run from `aws-lc/tests/ci/cdk`.
+
+If not done previously, bootstrap cdk before running the commands below. Make sure that AWS_ACCOUNT_ID is the AWS account you wish to deploy the CI stacks to.
 
-These commands are run from `aws-lc/tests/ci/cdk`. \
-If not done previously, bootstrap cdk before running the commands below:
 ```shell
 cdk bootstrap aws://${AWS_ACCOUNT_ID}/us-west-2
 ```
 
 You may also need to request an increase to certain account quotas:
 ```shell
-open https://${CDK_DEPLOY_REGION}.console.aws.amazon.com/servicequotas/home/services/ec2/quotas
+open https://${DEPLOY_REGION}.console.aws.amazon.com/servicequotas/home/services/ec2/quotas
 ```
 * **EC2-VPC Elastic IPs** = 20
 
@@ -82,23 +134,23 @@ Note: `GITHUB_REPO_OWNER` specifies the GitHub repo targeted by this CI setup.
 
 To set up AWS-LC CI, run command:
 ```
-./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action deploy-ci --aws-account ${AWS_ACCOUNT_ID}
+./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action deploy-ci --deploy-account ${AWS_ACCOUNT_ID}
 ```
 
 To update AWS-LC CI, run command:
 ```
-./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action update-ci --aws-account ${AWS_ACCOUNT_ID}
+./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action update-ci --deploy-account ${AWS_ACCOUNT_ID}
 ```
 
 To create/update Linux Docker images, run command:
 ```
-./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action build-linux-img --aws-account ${AWS_ACCOUNT_ID}
+./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action build-linux-img --deploy-account ${AWS_ACCOUNT_ID}
 ```
 
 To destroy AWS-LC CI resources created above, run command:
 ```
 # NOTE: this command will destroy all resources (AWS CodeBuild and ECR).
-./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action destroy-ci --aws-account ${AWS_ACCOUNT_ID}
+./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action destroy-ci --deploy-account ${AWS_ACCOUNT_ID}
 ```
 
 For help, run command:
@@ -155,6 +207,10 @@ Below is CI file structure.
 │   ├── __init__.py
 │   ├── ecr_stack.py
 │   ├── ...
+├── pipeline
+│   ├── __init__.py
+│   ├── pipeline_stack.py
+│   ├── ...
 ├── cdk.json
 ├── requirements.txt
 ├── run-cdk.sh
@@ -167,7 +223,8 @@ Below is CI file structure.
 * `README.md` — The introductory README for this project.
 * `app.py` — The “main” for this sample application.
 * `cdk.json` — A configuration file for CDK that defines what executable CDK should run to generate the CDK construct tree.
-* `cdk` — A CDK module directory
+* `cdk` — A module directory that contains all CI-related stacks and utilities
+* `pipeline` - A module directory that defines a continuous deployment pipeline for the CI.
 * `requirements.txt` — This file is used by pip to install all of the dependencies for your application. In this case, it contains only -e . This tells pip to install the requirements specified in setup.py. It also tells pip to run python setup.py develop to install the code in the cdk module so that it can be edited in place.
 * `setup.py` — Defines how this Python package would be constructed and what the dependencies are.
 
@@ -244,7 +301,7 @@ aws codebuild start-build-batch --project-name aws-lc-docker-image-build-linux
 # Go to AWS console, you can check CodeBuild by clicking "Developer Tools > CodeBuild > Build projects".
 ```
 
-#### Windows Docker image build
+#### Windows Docker image build (DEPRECATED)
 Windows docker image build requires more resources (like EC2 host, S3, SSM and so on) set up because DIND (Docker in Docker) is not supported by Windows.
 Below are some commands specific to windows docker image build.
 

tests/ci/cdk/app.py

@@ -5,56 +5,47 @@
 
 from aws_cdk import Environment, App
 
-# from cdk.bm_framework_stack import BmFrameworkStack
-from cdk.aws_lc_analytics_stack import  AwsLcGitHubAnalyticsStack
-from cdk.aws_lc_android_ci_stack import AwsLcAndroidCIStack
-from cdk.aws_lc_github_ci_stack import AwsLcGitHubCIStack
-from cdk.aws_lc_github_fuzz_ci_stack import  AwsLcGitHubFuzzCIStack
-from cdk.aws_lc_ec2_test_framework_ci_stack import AwsLcEC2TestingCIStack
 from cdk.linux_docker_image_batch_build_stack import LinuxDockerImageBatchBuildStack
+from pipeline.ci_util import add_ci_stacks
+from pipeline.pipeline_stack import AwsLcCiPipeline
 from cdk.windows_docker_image_build_stack import WindowsDockerImageBuildStack
-from cdk.aws_lc_github_ci_x509_stack import AwsLcGitHubX509CIStack
 from cdk.ecr_stack import EcrStack
-from util.metadata import AWS_ACCOUNT, AWS_REGION, LINUX_X86_ECR_REPO, LINUX_AARCH_ECR_REPO, WINDOWS_X86_ECR_REPO
+from util.metadata import (
+    LINUX_X86_ECR_REPO,
+    LINUX_AARCH_ECR_REPO,
+    WINDOWS_X86_ECR_REPO,
+    PIPELINE_ACCOUNT,
+    PIPELINE_REGION,
+    DEPLOY_ACCOUNT,
+    DEPLOY_REGION,
+)
 
 # Initialize app.
 app = App()
 
-# Initialize env.
-env = Environment(account=AWS_ACCOUNT, region=AWS_REGION)
-
-# Define AWS ECR stacks.
-# ECR holds the docker images, which are pre-built to accelerate the code builds/tests of git pull requests.
-EcrStack(app, "aws-lc-ecr-linux-x86", LINUX_X86_ECR_REPO, env=env)
-EcrStack(app, "aws-lc-ecr-linux-aarch", LINUX_AARCH_ECR_REPO, env=env)
-EcrStack(app, "aws-lc-ecr-windows-x86", WINDOWS_X86_ECR_REPO, env=env)
-
-# Define CodeBuild Batch job for building Docker images.
-LinuxDockerImageBatchBuildStack(app, "aws-lc-docker-image-build-linux", env=env)
-
-# AWS CodeBuild cannot build Windows Docker images because DIND (Docker In Docker) is not supported on Windows.
-# Windows Docker images are created by running commands in Windows EC2 instance.
-WindowsDockerImageBuildStack(app, "aws-lc-docker-image-build-windows", env=env)
-
-# Define CodeBuild Batch job for testing code.
-x86_build_spec_file = "cdk/codebuild/github_ci_linux_x86_omnibus.yaml"
-AwsLcGitHubCIStack(app, "aws-lc-ci-linux-x86", x86_build_spec_file, env=env)
-arm_build_spec_file = "cdk/codebuild/github_ci_linux_arm_omnibus.yaml"
-AwsLcGitHubCIStack(app, "aws-lc-ci-linux-arm", arm_build_spec_file, env=env)
-integration_build_spec_file = "cdk/codebuild/github_ci_integration_omnibus.yaml"
-AwsLcGitHubCIStack(app, "aws-lc-ci-integration", integration_build_spec_file, env=env)
-win_x86_build_spec_file = "cdk/codebuild/github_ci_windows_x86_omnibus.yaml"
-AwsLcGitHubCIStack(app, "aws-lc-ci-windows-x86", win_x86_build_spec_file, env=env)
-fuzz_build_spec_file = "cdk/codebuild/github_ci_fuzzing_omnibus.yaml"
-AwsLcGitHubFuzzCIStack(app, "aws-lc-ci-fuzzing", fuzz_build_spec_file, env=env)
-analytics_build_spec_file = "cdk/codebuild/github_ci_analytics_omnibus.yaml"
-AwsLcGitHubAnalyticsStack(app, "aws-lc-ci-analytics", analytics_build_spec_file, env=env)
-# bm_framework_build_spec_file = "cdk/codebuild/bm_framework_omnibus.yaml"
-# BmFrameworkStack(app, "aws-lc-ci-bm-framework", bm_framework_build_spec_file, env=env)
-ec2_test_framework_build_spec_file = "cdk/codebuild/ec2_test_framework_omnibus.yaml"
-AwsLcEC2TestingCIStack(app, "aws-lc-ci-ec2-test-framework", ec2_test_framework_build_spec_file, env=env)
-android_build_spec_file = "cdk/codebuild/github_ci_android_omnibus.yaml"
-AwsLcAndroidCIStack(app, "aws-lc-ci-devicefarm-android", android_build_spec_file, env=env)
-AwsLcGitHubX509CIStack(app, "aws-lc-ci-x509")
+AwsLcCiPipeline(
+    app,
+    "AwsLcCiPipeline",
+    env=Environment(account=PIPELINE_ACCOUNT, region=PIPELINE_REGION),
+)
+
+if DEPLOY_ACCOUNT and DEPLOY_REGION:
+    # Initialize env.
+    env = Environment(account=DEPLOY_ACCOUNT, region=DEPLOY_REGION)
+
+    # Define AWS ECR stacks.
+    # ECR holds the docker images, which are pre-built to accelerate the code builds/tests of git pull requests.
+    EcrStack(app, "aws-lc-ecr-linux-x86", LINUX_X86_ECR_REPO, env=env)
+    EcrStack(app, "aws-lc-ecr-linux-aarch", LINUX_AARCH_ECR_REPO, env=env)
+    EcrStack(app, "aws-lc-ecr-windows-x86", WINDOWS_X86_ECR_REPO, env=env)
+
+    # Define CodeBuild Batch job for building Docker images.
+    LinuxDockerImageBatchBuildStack(app, "aws-lc-docker-image-build-linux", env=env)
+
+    # AWS CodeBuild cannot build Windows Docker images because DIND (Docker In Docker) is not supported on Windows.
+    # Windows Docker images are created by running commands in Windows EC2 instance.
+    WindowsDockerImageBuildStack(app, "aws-lc-docker-image-build-windows", env=env)
+
+    add_ci_stacks(app, env=env)
 
 app.synth()