diff --git a/README.md b/README.md index f61181c3..58671cd1 100644 --- a/README.md +++ b/README.md @@ -13,45 +13,64 @@ The SAP Testing Automation Framework is an open-source orchestration tool designed to validate SAP deployments on Microsoft Azure. It enables you to assess system configurations against SAP on Azure best practices and guidelines, and facilitates automation for various testing scenarios. -> **NOTE**: This repository is currently in public preview and is intended for testing and feedback purposes. As this is an early release, it is not yet production-ready, and breaking changes can be introduced at any time. - ![SAP Testing Automation Framework](./docs/images/sap-testing-automation-framework.png) -## πŸ“Š Key Features +## πŸ“Š Key Scenarios -- **High Availability Testing** - Thorough validation of the SAP HANA scale-up and SAP Central Services failover mechanism in a two node pacemaker cluster, ensuring the system operates correctly across various test cases. - - **Configuration Validation** - Ensures that SAP HANA scale-up and SAP Central Services configurations comply with SAP on Azure best practices and guidelines. - - **Functional Testing** - Executes test scenarios on the high availability setup to identify potential issues, whether during a new system deployment or before implementing cluster changes in a production environment. -- **Configuration Checks** - Validates OS parameters, database settings, Azure resources, and storage configurations against SAP and Azure best practices for supported databases. Performs comprehensive validation including kernel parameters, filesystem mounts, VM sizing, and network setup to ensure compliance with recommended guidelines. -- **Detailed Reporting** - Generates comprehensive reports, highlighting configuration mismatch or deviations from recommended best practices. Includes failover test outcomes, any failures encountered, and logs with insights to aid in troubleshooting identified issues. +SAP Testing Automation is designed as a scalable framework to orchestrate and validate a wide spectrum of SAP landscape scenarios through repeatable, policy-driven test modules. The framework currently offers following scenarios - -## πŸ† Purpose +> [!NOTE] +> +> The High Availability testing scenario in the SAP Testing Framework is **generally available (GA)**, while the Configuration Checks scenario is currently in **public preview**. -Testing is crucial for keeping SAP systems running smoothly, especially for critical business operations. This framework helps by addressing key challenges: +### High Availability Testing -- **Preventing Risks** - Identifies configuration issues and validates system behavior before problems affect production operations. It simulates system failures like node crashes, network issues, and storage failures to check if recovery mechanisms work properly, helping to catch potential issues early. -- **Meeting Compliance Requirements** - Provides detailed reports and logs that help with audits and ensure compliance with internal and regulatory standards. -- **Ensuring Quality** - The framework runs automated tests to verify whether the failover behavior of SAP components functions as expected on Azure across various test scenarios. It also ensures that the cluster and resource configurations are set up correctly, helping to maintain system reliability. -- **Automating Testing** - Automates validation processes from configuration checks to reporting, saving time and ensuring consistent results. +In the SAP Testing Automation Framework, thorough validation of high availability SAP HANA scale-up and SAP Central Services failover mechanism in a two node pacemaker cluster can be performed, ensuring the system operates correctly across different situations. -## 🚦 Get Started +- **High Availability Configuration Validation:** The framework helps to ensure that SAP HANA scale-up and SAP Central Services configurations and load balancer settings are compliant with SAP on Azure high availability configuration guidelines. +- **Functional Testing:** The framework executes series of real-world scenarios based on the SAP HANA and SAP Central Services high availability setup to identify potential issues, whether during a new system deployment or before implementing cluster changes in a production environment. The test cases are based on what is documented in how-to guides for SAP HANA and SAP Central Services configuration. +- **Offline configuration validation:** Offline validation is a mode of the framework that validates SAP HANA and SAP Central Services high availability cluster configurations without establishing a live SSH connection to the production cluster. Instead, it analyzes captured cluster information base (CIB) XML files exported from each cluster node. -There are two primary ways to get started with the SAP Testing Automated Framework. Choose the path that best fits your current environment and objectives: +### Configuration Checks (Preview) -### Option 1: Integration with SAP Deployment Automation Framework (SDAF) +The framework performs comprehensive configuration checks to ensure that the SAP system and its components are set up according to [SAP on Azure best practice](https://learn.microsoft.com/azure/sap/). This includes validating infrastructure settings, operating system parameter configurations, and network settings, in addition to the cluster configuration, to identify any deviations that could impact system performance or reliability. -If you already have [SDAF](https://github.com/Azure/sap-automation) environment set up, integrating the SAP Testing Automation Framework is a natural extension that allows you to leverage existing deployment pipelines and configurations. +- **Infrastructure Validation:** This includes validating the underlying infrastructure components, such as virtual machines, load balancer, and other resource configurations, to ensure they meet the requirements for running SAP workloads on Azure. +- **Storage Configuration Checks:** It validates settings of disks, storage accounts, Azure NetApp Files, including throughput, performance, and stripe size. +- **Operating System and SAP Parameter Validation:** The framework checks critical operating system parameters and SAP kernel settings to ensure they align with recommended configurations. +- **Cluster Configuration Validation:** This framework ensures that the high availability cluster resource settings adhere to best practices for high availability and failover scenarios. -### Option 2: Getting Started with High Availability Testing (Standalone) +The framework generates comprehensive reports, highlighting configuration mismatch or deviations from recommended best practices. For high availability scenarios, the report includes failover test outcomes, any failures encountered, and logs with insights to aid in troubleshooting identified issues. + +## πŸ† Purpose + +Testing is crucial for keeping SAP systems running smoothly, especially for critical business operations. This framework helps by addressing key challenges: -For users focused solely on validating SAP functionality and configurations, the standalone approach offers a streamlined process to test critical SAP components without the complexity of full deployment integration. - - For High Availability testing details, see the [High Availability documentation](./docs/HIGH_AVAILABILITY.md). - - For Configuration Checks and Testing details, see the [Configuration Checks documentation](./docs/CONFIGURATION_CHECKS.md). +- **Risk Prevention** - The high availability testing helps simulate system failures like node crashes, network issues, and storage failures to check if recovery mechanisms work properly, helping to catch problems before they affect real operations. Configuration validation detects misalignments with SAP on Azure best practices early. +- **Compliance Requirements** - Many businesses need to prove their SAP systems are reliable. This framework provides detailed reports and logs that help with audits and ensure compliance with internal and regulatory standards. +- **Quality Assurance** - The framework runs automated tests to verify whether the failover behavior of SAP components functions as expected on Azure across various test scenarios. It also ensures that the cluster and resource configurations are set up correctly, helping to maintain system reliability. +- **Test Automation** - Manually validating overall SAP systems' configurations and high availability (HA) setup is slow and error-prone. This framework automates the process, from setup to reporting, saving time and ensuring more accurate and consistent results. ## πŸ—οΈ Architecture and Components To learn how the framework works, refer to the [architecture and components](./docs/ARCHITECTURE.md) documentation. +## 🚦 Get Started + +There are two primary ways to get started with the SAP Testing Automation Framework. You can choose the path that best fits your current environment and objectives: + +### Option 1: Standalone Setup of SAP Testing Automation Framework + +For users focused solely on validating SAP functionality and configurations, the standalone approach offers a streamlined process to test critical SAP components without the complexity of full deployment integration. For more details on the setup, see following documents to get started - + +- Configure management server following the document [Setup Guide for SAP Testing Automation Framework](https://github.com/Azure/sap-automation-qa/blob/main/docs/SETUP.MD). + - For high availability testing scenarios, see [High Availability documentation](./docs/HIGH_AVAILABILITY.md). + - For Configuration Checks and Testing details, see the [Configuration Checks documentation](./docs/CONFIGURATION_CHECKS.md). + +### Option 2: Integration with SAP Deployment Automation Framework (SDAF) + +If you already have an [SAP Deployment Automation Framework](https://learn.microsoft.com/azure/sap/automation/deployment-framework) environment set up, integrating the SAP Testing Automation Framework is a natural extension that allows you to leverage existing deployment pipelines and configurations. For more details on the setup, see [Setup Guide for SAP Testing Automation Framework](./docs/SETUP.MD). + ## πŸ†˜Support For support and questions, please: diff --git a/docs/CONFIGURATION_CHECKS.md b/docs/CONFIGURATION_CHECKS.md index 1a65eab3..a229bd10 100644 --- a/docs/CONFIGURATION_CHECKS.md +++ b/docs/CONFIGURATION_CHECKS.md @@ -17,33 +17,32 @@ Configuration validation serves as a critical quality gate in the SAP deployment ## Configuration Check Categories -**Azure Compute** -- VM SKU appropriateness for SAP workloads -- Accelerated Networking enablement -- Availability Set/Zone configuration -- Proximity Placement Group setup - -**Storage Configuration** -- Premium SSD/Ultra Disk usage for critical paths -- Write Accelerator for log volumes -- Storage account redundancy settings -- Disk caching policies - -**SAP Database Configuration** -- SAP HANA: Memory allocation, system replication parameters -- IBM DB2: Hardware requirements, system language, OS tuning parameters - -**Pacemaker Cluster (HANA only)** -- Resource agent versions and parameters -- Fencing (STONITH) configuration -- Resource constraints and colocation rules -- Cluster communication settings - -**SAP HA Resources (HANA only)** -- Virtual hostname configuration -- File system mount options -- Service startup ordering -- Failover timeout values +The configuration checks are organized into logical groups that can be executed independently or all at once. The main categories are: + +1. **Infrastructure** + + - While not a separate execution category, infrastructure checks are performed as part of the other categories. + - **Azure Compute**: VM SKU, Accelerated Networking, Availability Set/Zone, Proximity Placement Group. + - **Storage**: Use of Premium SSD/Ultra Disk, Write Accelerator, disk caching policies, and redundancy settings. + +2. **Database** + + - Validates SAP HANA or IBM DB2 specific settings. + - **SAP HANA**: Checks memory allocation, system replication parameters, and Pacemaker cluster configurations (resource agents, fencing, constraints). + - **IBM DB2**: Verifies hardware requirements, system language, and OS tuning parameters. + +3. **Central Services** + + - Validates the configuration of ASCS (ABAP SAP Central Services) and ERS (Enqueue Replication Server) instances. + + - Checks for virtual hostname configuration, file system mount options, and service startup ordering. + + +3. **Application Servers** + - Validates the configuration of the application server instances. + + +> **Note**: High Availability (HA) configuration checks and functional tests are currently supported only for SAP HANA databases. For IBM DB2 databases, only non-HA configuration checks are available. ### 1. Setup Configuration @@ -58,18 +57,18 @@ Follow the steps (2.1 - 2.2) in [Setup Guide for SAP Testing Automation Framewor > **Note**: High Availability (HA) configuration checks and functional tests are currently supported only for SAP HANA databases. For IBM DB2 databases, only non-HA configuration checks are available. -### 3. Required Access and Permissions +### 3. Required Access and Permissions (required) + +Effective configuration validation requires that the management server's managed identity (system or user assigned) has read permissions on all target Azure resources. This allows the framework to inspect the settings of services including, but not limited to, Azure Load Balancers, storage solutions (Managed Disks, Azure Files, Azure NetApp Files), and network infrastructure. Lacking the necessary access will prevent the configuration checks from identifying potential misconfigurations in the environment. For more details on configuring system assigned managed identity vs user assigned managed identity, see [Setup Guide for SAP Testing Automation Framework](./SETUP.MD#configuring-access-using-managed-identity). -Ensure that the managed identity or service principal used by the controller virtual machine has the necessary permissions to access Azure resources and SAP systems for configuration validation. -1. "Reader" role to the user-assigned managed identity on the resource group containing the SAP VMs and the Azure Load Balancer. -1. "Reader" role to the user-assigned managed identity on the resource group containing the Azure NetApp Files account (if using Azure NetApp Files as shared storage). -1. "Reader" role to the user-assigned managed identity on the resource group containing the storage account (if using Azure File Share as shared storage). -1. "Reader" role to the user-assigned managed identity on the resource group containing the managed disks (if using Azure Managed Disks for SAP HANA data and log volumes). -1. "Reader" role to the user-assigned managed identity on the resource group containing the shared disks (if using Azure Shared Disks for SBD devices). +1. Depending on the type of managed identity method you want to use, configure managed identity on management server + - [Configuring access using system-assigned managed identity](./SETUP.MD#configuring-access-using-system-assigned-managed-identity). + - [Configuring access using user-assigned managed identity](SETUP.MD#configuring-access-using-user-assigned-managed-identity). +2. Grant the managed identity (system- or user-assigned) the built-in **Reader** role on every resource group that contains SAP system components (VMs, managed disks, load balancers, virtual network, shared storage such as Azure NetApp Files or Azure Files). If everything resides in a single resource group, one assignment there is sufficient; if components are split across multiple resource groups, add a **Reader** role assignment on each resource group to allow the configuration checks to read and validate all infrastructure settings. ### 4. Azure Login (required) -Ensure that you are logged into Azure CLI on the controller VM with the appropriate subscription context: +Ensure that you are logged into Azure CLI on the management server VM with the appropriate subscription context: ```bash # Login to Azure using System Assigned Managed Identity @@ -108,7 +107,7 @@ To execute the script, run following command: ### 6. Viewing Test Results -After the test execution completes, a detailed HTML report is generated that summarizes the PASS/FAIL status of each test case and includes detailed execution logs for every step of the automation run. +After the test execution completes, a detailed HTML report is generated. The report provide the summary of each test cases that got executed for each VM. **To locate and view your test report:** @@ -119,16 +118,17 @@ After the test execution completes, a detailed HTML report is generated that sum ```bash cd WORKSPACES/SYSTEM//quality_assurance/ ``` + 2. **Find your report file:** The report file is named using the following format: ``` - CONFIG_{SAP_SID}_{DATABASE_TYPE}_{INVOCATION_ID}.html + all_{DISTRO}_{INVOCATION_ID}.html ``` - - `SAP_SID`: The SAP system ID (e.g., HN1, NWP) - - `DATABASE_TYPE`: The database type (e.g., HANA) + - `DISTRO`: Linux distribution (SLES or RHEL) + - `INVOCATION_ID`: A unique identifier (Group invocation ID) for the test run which is logged at the end of test execution. Find example screenshot below: ![Test Execution Completion Screenshot](./images/execution_screenshot.png) diff --git a/docs/HA_OFFLINE_VALIDATION.md b/docs/HA_OFFLINE_VALIDATION.md index c5c701f3..9d5b8662 100644 --- a/docs/HA_OFFLINE_VALIDATION.md +++ b/docs/HA_OFFLINE_VALIDATION.md @@ -1,13 +1,12 @@ # SAP Automation QA - Offline Validation -## Overview - -The offline validation feature enables robust validation of SAP HANA and SAP Central Services High Availability cluster configurations without requiring live cluster access or without connecting to the SAP virtual machines. This capability allows you to analyze cluster configurations from previously collected CIB (Cluster Information Base) XML files, making it ideal for post-incident analysis, compliance auditing, and troubleshooting scenarios. -Offline validation provides a powerful capability for maintaining and auditing SAP HANA cluster configurations without impacting production systems. +The offline validation feature for SAP High Availability (HA) clusters within the SAP Testing Automation Framework allows for the robust validation of SAP HANA and SAP Central Services HA cluster configurations without requiring direct access to the live cluster environment. By using cluster configuration files (CIB XML), you can audit and validate your cluster setup, ensuring it meets the required standards and best practices. Offline validation provides the capability for maintaining configuration integrity, performing regular audits, and troubleshooting without connecting to the production systems. ## How Offline Validation Works -### Architecture Overview +The offline validation process relies on cluster configuration information extracted into CIB (Cluster Information Base) XML files. The validation engine processes these files to analyze the cluster's configuration against a set of predefined rules and best practices. + +### Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” @@ -18,26 +17,36 @@ Offline validation provides a powerful capability for maintaining and auditing S β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` +1. **CIB XML Files**: These files contain a snapshot of the Pacemaker cluster configuration from each node. +2. **Validation Engine**: The engine parses the CIB XML files and evaluates the configuration parameters, resource settings, and constraints. +3. **HTML Report**: The results are compiled into a detailed HTML report, which presents the validation checks in a clear, tabular format. ### Prerequisites -- SAP Testing Automation Framework (STAF) setup on a management server. Detailed setup instructions can be found in the [STAF Setup Guide](./HIGH_AVAILABILITY.md). -- Previously collected CIB XML files stored in the `WORKSPACES/SYSTEM//offline_validation/` directory. +Before performing offline validation, ensure the following requirements are met: + +- SAP Testing Automation Framework (STAF) setup on a management server. Detailed setup instructions can be found in the [STAF Setup Guide](./SETUP.md). +- You must collect CIB XML files from each node in the SAP HA cluster. These files must be stored in the appropriate directory structure `WORKSPACES/SYSTEM//offline_validation/` on the management server. ### Required Files Structure + +The offline validation process requires a specific directory structure on the management server. The CIB XML files must be placed within the workspace of the system being validated. + ```file WORKSPACES/SYSTEM// -β”œβ”€β”€ hosts.yaml # Ansible inventory -β”œβ”€β”€ sap-parameters.yaml # SAP system parameters -└── offline_validation/ # Output of commands for offline validation +β”œβ”€β”€ hosts.yaml # Ansible inventory file for the SAP system. +β”œβ”€β”€ sap-parameters.yaml # SAP system parameters file. +└── offline_validation/ # Directory for offline validation artifacts. β”œβ”€β”€ / - β”‚ └── cib # CIB XML file for node 1 + β”‚ └── cib # CIB XML file from the first node. └── / - └── cib # CIB XML file for node 2 + └── cib # CIB XML file from the second node. ``` ## How to Perform Offline Validation +Follow these steps to conduct an offline validation of your SAP HA cluster configuration. + ### Step 1: Initial Setup This setup is defined in the Getting Started section of the [High Availability Guide](./HIGH_AVAILABILITY.md). Ensure you have the following: @@ -46,43 +55,64 @@ This setup is defined in the Getting Started section of the [High Availability G - SAP system parameters file (`sap-parameters.yaml`). - Updated vars.yaml file with the necessary parameters. -### Step 2: Collect CIB XML Files and copy to management server +### Step 2: Collect CIB XML Files + +First, collect the cluster configuration from each node in your SAP system. + +1. Log in to each node of the HA cluster. + +2. Execute the following command to export the cluster configuration to a file named `cib`: + + ```bash + # Execute below command on both nodes + cibadmin --query | tee cib + ``` + + This command captures the complete cluster configuration in XML format. + +### Step 3: Transfer and Organize CIB Files + +Next, transfer the `cib` files to your management server and organize them into the required directory structure. + +1. For each node, create a corresponding directory on the management server: -#### 2.1 Collect CIB XML Files + ```bash + mkdir -p WORKSPACES/SYSTEM//offline_validation// + ``` - Before performing offline validation, you need to collect High Availability cluster configuration files (CIB XML files) from the SAP system nodes. This can be done by executing the following command on each node: + Replace `` with the name of your SAP system configuration and `` with the hostname of the node from which the `cib` file was collected. - ```bash - cibadmin --query | tee cib - ``` +2. Copy each `cib` file into its respective `` directory. - This command will create a file named `cib` in the current directory, which contains the cluster configuration in XML format. +### Step 4: Run Offline Validation -#### 2.2 Create the Required Directory Structure +Once the CIB files are in place, execute the offline validation script. - Copy these files to the management server under the `WORKSPACES/SYSTEM//offline_validation/` directory, maintaining the structure as shown above. Ensure the directory structure is created as follows: +1. Navigate to the scripts directory of the SAP Automation QA framework. +2. Run the `sap_automation_qa.sh` script with the `--offline` flag. You must also specify the target OS family using the `--extra-vars` parameter. - ```bash - mkdir -p WORKSPACES/SYSTEM//offline_validation// - ``` + For SUSE-based systems: + ```bash + ./scripts/sap_automation_qa.sh --offline --extra-vars='target_os_family=SUSE' + ``` - Place the `cib` file in the respective `/` directory. + For RHEL-based systems: + ```bash + ./scripts/sap_automation_qa.sh --offline --extra-vars='target_os_family=RHEL' + ``` -### Step 3: Run Offline Validation +> [!TIP] +> For troubleshooting or detailed logging, you can run the script in verbose mode by adding the `-vvv` flag: +> ```bash +> ./scripts/sap_automation_qa.sh --offline --extra-vars='target_os_family=' -vvv +> ``` - Execute the sap_automation_qa script for offline validation with the `--offline` flag. The target OS family is a requirement parameter (`target_os_family`) and must be specified using the `--extra-vars` option. +### Step 5: View Results - ```bash - ./scripts/sap_automation_qa.sh --offline --extra-vars='target_os_family=SUSE' - # or - ./scripts/sap_automation_qa.sh --offline --extra-vars='target_os_family=RHEL' - ``` +After the script completes, the validation results are stored in an HTML file. - Enable verbose logging for troubleshooting: - ```bash - ./scripts/sap_automation_qa.sh --extra-vars='target_os_family=' --offline -vvv - ``` +1. Navigate to the quality assurance directory for your system configuration: -### Step 4: View Results + `WORKSPACES/SYSTEM//quality_assurance/` - The validation results will be available in `WORKSPACES/SYSTEM//quality_assurance/` directory. Open the HTML file in a web browser to view the detailed parameter validation table with PASSED/INFO/FAILED statuses. \ No newline at end of file +2. Open the HTML file in a web browser to view the detailed validation report. The report includes tables that show each parameter checked, its expected value, and its actual configured value, with clear pass/fail indicators. \ No newline at end of file diff --git a/docs/HIGH_AVAILABILITY.md b/docs/HIGH_AVAILABILITY.md index 11a55c07..409cdcbc 100644 --- a/docs/HIGH_AVAILABILITY.md +++ b/docs/HIGH_AVAILABILITY.md @@ -1,57 +1,42 @@ # SAP High Availability Testing -A key component of the SAP Testing Automation framework is the SAP High Availability (HA) Testing. This helps in ensuring that an SAP system deployment complies to SAP on Azure best practices and guidelines. +The SAP on Azure Automation framework includes a High Availability (HA) testing component designed to validate that SAP deployments on Azure adhere to established best practices. This component executes a series of automated tests that simulate real-world failure scenarios to ensure the resilience and recovery capabilities of the SAP system. -The SAP High Availability Testing scenario executes a series of tests designed to simulate real-world failures, ensuring the system's recovery capabilities. Leveraging Ansible, the framework orchestrates various test cases, including node crashes, network disruptions, and storage failures, to validate the effectiveness of recovery mechanisms. Additionally, the framework captures comprehensive logs and generates detailed reports on the test outcomes. +Leveraging Ansible, the framework orchestrates these failure scenarios and validates the system's automated recovery processes. This document provides guidance on configuring and executing these HA tests. ## Supported Configurations -### Linux distribution +Azure offers various deployment options for SAP workloads on different operating system distributions. The SAP Testing Automation Framework executes its test scenarios on the SAP system configurations running on Linux distribution. You can find the support matrix on supported Linux distribution version, and high availability configuration pattern in [SAP Testing Automation Framework Supported Platforms and Features](https://learn.microsoft.com/azure/sap/automation/testing-framework-supportability#supported-sap-system-configurations). -Currently SAP Testing Automation Framework is supported for below Linux distros and version. +## Pre-requisites -| Distribution | Supported Release | -|--------------|-------------------| -| SUSE Linux Enterpise Server (SLES) | 15 SP4, 15 SP5, 15 SP6 | -| Red Hat Enterprise Linux (RHEL) | 8.8, 8.10, 9.2, 9.4 | +Before executing the HA tests, complete the following prerequisite steps. -### High Availability configuration pattern +### 1. Enable Cluster Services on Boot -| Component | Type | Cluster Type | Storage | -|-----------|------|--------------|---------| -| SAP Central Services | ENSA1 or ENSA2 | Azure Fencing Agent | Azure Files or ANF | -| SAP Central Services | ENSA1 or ENSA2 | ISCSI (SBD device) | Azure Files or ANF | -| SAP Central Services | ENSA1 or ENSA2 | Azure Shared Disks (SBD device) | Azure Files or ANF | -| SAP HANA | Scale-up | Azure Fencing Agent | Azure Managed Disk or ANF | -| SAP HANA | Scale-up | ISCSI (SBD device) | Azure Managed Disk or ANF | -| SAP HANA | Scale-up | Azure Shared Disks (SBD device) | Azure Managed Disk or ANF | - -For SAP Central Services on SLES, both the simple mount approach and the classic method are supported. - - -### Enabling Cluster Services on Boot - -Before executing the tests, ensure that the cluster services are configured to start automatically during system boot. Run the following command on one of the cluster nodes to enable this setting. The `--all` option ensures that the cluster services are enabled on all nodes within the cluster. +Ensure that cluster services are configured to start automatically on system boot. Execute the appropriate command for your Linux distribution on one of the cluster nodes: ```bash -crm cluster enable --all # for SUSE virtual machines -pcs cluster enable --all # for RedHat virtual machine -``` +# For SUSE Linux Enterprise Server (SLES) +crm cluster enable --all -### 1. Setup Configuration +# For Red Hat Enterprise Linux (RHEL) +pcs cluster enable --all +``` -Follow the steps (1.1 - 1.5) in [Setup Guide for SAP Testing Automation Framework](./SETUP.MD) to set up the framework on a management server. +### 2. Configure the Automation Framework -### 2. System Configuration +Follow the steps in the [Setup Guide for SAP Testing Automation Framework](./SETUP.MD) to prepare the framework on a designated management server. -Update the `TEST_TYPE` parameter in [`vars.yaml`](./../vars.yaml) file to `SAPFunctionalTests` to enable the High Availability test scenarios. +### 3. Configure the System for HA Testing -Follow the steps (2.1 - 2.2) in [Setup Guide for SAP Testing Automation Framework](./SETUP.MD#2-system-configuration) to configure your system details. +1. Update the `TEST_TYPE` parameter in the `vars.yaml` file to `SAPFunctionalTests` to enable the High Availability test scenarios. +2. Follow the steps in the [System Configuration section of the Setup Guide](./SETUP.MD#2-system-configuration) to provide the details of your SAP system. -### 3. Test Execution +## Test Execution -To execute the script, run following command: +Execute the tests using the `sap_automation_qa.sh` script from the `scripts` directory. You can run all tests or specify a subset of test cases. ```bash # Run all the tests with default parameters @@ -67,34 +52,30 @@ To execute the script, run following command: ./scripts/sap_automation_qa.sh --test_groups=HA_DB_HANA --test_cases=[primary-node-crash] -vvv ``` -### 4. Viewing Test Results - -After the test execution completes, a detailed HTML report is generated that summarizes the PASS/FAIL status of each test case and includes detailed execution logs for every step of the automation run. +## Viewing Test Results -**To locate and view your test report:** +Upon completion, the framework generates a detailed HTML report that summarizes the PASS/FAIL status of each test case and provides detailed execution logs. -1. **Navigate to your SAP system’s workspace directory:** +1. **Navigate to the workspace directory for your SAP system.** - Replace `` with the name of your SAP system configuration (for example, `DEV-WEEU-SAP01-X00`): + Replace `` with the name of your SAP system configuration (e.g., `DEV-WEEU-SAP01-X00`). - ```bash - cd WORKSPACES/SYSTEM//quality_assurance/ - ``` -2. **Find your report file:** + ```bash + cd WORKSPACES/SYSTEM//quality_assurance/ + ``` - The report file is named using the following format: +2. **Identify the report file.** - ``` - HA_{SAP_TIER}_{DATABASE_TYPE}_{OS_DISTRO_NAME}_{INVOCATION_ID}.html - ``` + The report file name follows this format: + `HA_{SAP_TIER}_{DATABASE_TYPE}_{OS_DISTRO_NAME}_{INVOCATION_ID}.html` - - `SAP_TIER`: The SAP tier tested (e.g., DB, SCS) - - `DATABASE_TYPE`: The database type (e.g., HANA) - - `OS_DISTRO_NAME`: The operating system distribution (e.g., SLES15SP4) - - `INVOCATION_ID`: A unique identifier (Group invocation ID) for the test run which is logged at the end of test execution. Find example screenshot below: + * `SAP_TIER`: The SAP tier tested (e.g., DB, SCS). + * `DATABASE_TYPE`: The database type (e.g., HANA). + * `OS_DISTRO_NAME`: The operating system (e.g., SLES15SP4). + * `INVOCATION_ID`: A unique identifier for the test run, which is logged at the end of the test execution. - ![Test Execution Completion Screenshot](./images/execution_screenshot.png) + ![Test Execution Completion Screenshot](./images/execution_screenshot.png) -3. **View the report** +3. **View the report.** - You can open the HTML report in any web browser to review the results and logs. + Open the HTML file in any web browser to review the test results and logs. diff --git a/docs/SETUP.MD b/docs/SETUP.MD index ac57a4d7..4869002e 100644 --- a/docs/SETUP.MD +++ b/docs/SETUP.MD @@ -1,58 +1,106 @@ # Setup Guide for SAP Testing Automation Framework -## Technical Requirements for running Automation Framework +The SAP Testing Automation Framework allows you to execute: -To run the SAP Testing Automation Framework, you must meet certain prerequisites and follow technical requirements. +1. High Availability (HA) functional test suites (HANA database, Central Services). +2. Configuration / compliance checks against an SAP on Azure deployment. -### SAP System Deployment on Microsoft Azure +Execution is driven by Ansible-based playbooks and Python helpers, using metadata defined in YAML files under a per‑system workspace. + +## Pre-requisites + +### 1. SAP System Deployment on Microsoft Azure - The SAP system must be hosted on Microsoft Azure Infrastructure-as-a-Service (IaaS). - The SAP system deployed should follow SAP on Azure best practices as outlined in: - [SAP HANA high availability on Azure Virtual Machine](https://learn.microsoft.com/azure/sap/workloads/sap-high-availability-guide-start). - [SAP Netweaver high availability on Azure Virtual Machine](https://learn.microsoft.com/azure/sap/workloads/sap-high-availability-guide-start) -### Management server +### 2. Management (Jump) server + +The SAP Testing Automation Framework requires a management server, which must run one of the supported operating system versions listed in [Supported distributions for the management server](https://learn.microsoft.com/en-us/azure/sap/automation/testing-framework-supportability#supported-distributions-for-management-server). + +### 3. Network Connectivity + +The management server must have network connectivity to the SAP system to perform tests and validations. If your management server is in a separate network from your SAP system, you must establish the connection by peering the networks as outlined in [manage a virtual network peering](https://learn.microsoft.com/azure/virtual-network/virtual-network-manage-peering?tabs=peering-portal). -The SAP Testing Automation Framework requires a jumpbox or management server with the following setup: +### 4. Identity and authorization -- **Operating System**: Supported (Ubuntu 22.04 LTS, SLES 15 SP4, 15 SP6, REDHAT 9.4). -- **Location**: Must be deployed on Azure. +The framework requires a managed identity to securely access Azure resources (like Load Balancers, Storage). You must configure **one** of the following two options. -### Azure RBAC +#### Option 1: User-Assigned Managed Identity -For the framework to access the properties of the Azure Load Balancer in a high availability SAP system on Azure, the management server must have a Reader role assigned to the Load Balancer. This can be done using either a system-assigned or user-assigned managed identity. +1. **Create a User-Assigned Managed Identity**: + + * Follow the guide to [manage user-assigned managed identities](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/how-manage-user-assigned-managed-identities). + * After creation, navigate to the identity's **Overview** page and copy the **Client ID**. You will need this for `sap-parameters.yaml`. + +2. **Assign the Identity to the Management VM**: + * Navigate to your management server VM in the Azure Portal. + * Under **Settings**, select **Identity**. + * Go to the **User assigned** tab and click **+ Add**. + * Select the user-assigned identity you created. + * For detailed steps, see [configure managed identities on Azure VMs](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/how-to-manage-ua-identity-vm). -#### Configuring access using system-assigned managed identity +3. **Assign Roles to the User-Assigned Identity**: + + This is a critical step to grant the identity permissions. We recommend to grant permission at the resource group level. + + **Method 1: Assign roles from the managed identity.** + + 1. Navigate to the user-assigned identity resource in the Azure Portal. + 2. Go to Azure role assignments to grant it the necessary permissions on other resources. The scope available here is limited to **Subscription**, **Resource group**, and a few other options. But if you want to assign roles per-resource like Azure load balancer, Azure NetApp Files etc., then follow method 2. + + **Method 2: Assign roles from the target resource.** + + 1. Navigate to the specific resource you want the identity to access (e.g., the Azure Load Balancer, Azure NetApp Files). + 2. Go to its **Access control (IAM)** blade. + 3. Click **Add > Add role assignment**. + 4. Select the required role (e.g., `Reader`\). + 5. For **Assign access to**, choose **Managed identity**. + 6. Click **+ Select members**, choose your subscription, and under **Managed identity**, select the user-assigned identity you created. + 7. Click **Review + assign**. + 8. Repeat this for every resource listed in the permissions table below. For detailed guidance, see [Assign Azure roles using the Azure portal](https://learn.microsoft.com/azure/role-based-access-control/role-assignments-portal). + +4. **Configure `sap-parameters.yaml`**: + + * Paste the **Client ID** you copied in step 1 into the `user_assigned_identity_client_id` field. -1. Enable system managed identity on the management server by following the steps in [Configure managed identities on Azure VMs](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/how-to-configure-managed-identities?pivots=qs-configure-portal-windows-vm#system-assigned-managed-identity). -1. Open the Azure Load Balancer used for the high availability deployment of your SAP system on Azure. -1. In the Azure Load Balancer panel, go to Access control (IAM). -1. Follow steps from [Use managed identity to access Azure Resource](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal) to complete the configuration. +#### Option 2: System-Assigned Managed Identity -#### Configuring access using user-assigned managed identity +A system-assigned identity is tied directly to the management server VM. It is created and deleted with the VM. -1. Create user-assigned managed identity as described in [manage user-assigned managed identities](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/how-manage-user-assigned-managed-identities?pivots=identity-mi-methods-azp#create-a-user-assigned-managed-identity) -1. Assign user-assigned managed identity to management server as described in [configure managed identities on Azure VMs](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/how-to-configure-managed-identities?pivots=qs-configure-portal-windows-vm#assign-a-user-assigned-managed-identity-to-an-existing-vm) +1. **Enable System-Assigned Identity on the Management VM**: + * Navigate to your management server VM in the Azure Portal. + * Under **Settings**, select **Identity**. + * Under the **System assigned** tab, switch the **Status** to **On** and click **Save**. + * For detailed steps, follow the guide: [Configure managed identities on Azure VMs](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/how-to-manage-ua-identity-vm). -**Permissions required for High Availability tests:** -1. Open the Azure Load Balancer used for the high availability deployment of your SAP system on Azure. -1. In the Azure Load Balancer panel, go to Access control (IAM). -1. Assign the required role to the user-assigned managed identity by following the steps in [assign roles using Azure portal](https://learn.microsoft.com/azure/role-based-access-control/role-assignments-portal). -1. Assign the "Reader" role to the user-assigned managed identity on the resource group containing the SAP VMs and the Azure Load Balancer. +2. **Assign Roles to the Management VM's Identity**: -**Permissions required for Configuration Checks:** -1. "Reader" role to the user-assigned managed identity on the resource group containing the SAP VMs and the Azure Load Balancer. -1. "Reader" role to the user-assigned managed identity on the resource group containing the Azure NetApp Files account (if using Azure NetApp Files as shared storage). -1. "Reader" role to the user-assigned managed identity on the resource group containing the storage account (if using Azure File Share as shared storage). -1. "Reader" role to the user-assigned managed identity on the resource group containing the managed disks (if using Azure Managed Disks for SAP HANA data and log volumes). -1. "Reader" role to the user-assigned managed identity on the resource group containing the shared disks (if using Azure Shared Disks for SBD devices). + This is a critical step to grant the management server VM permissions. We recommend to grant permission at the resource group level. -### Network Connectivity + **Method 1: Assign roles from management server VM identity.** -The management server must have network connectivity to the SAP system to perform tests and validations. You can establish this connection by peering the networks as outlined in [manage a virtual network peering](https://learn.microsoft.com/azure/virtual-network/virtual-network-manage-peering?tabs=peering-portal). + 1. Navigate to the management server VM, select **identity** in the Azure Portal. + 2. Go to Azure role assignments to grant it the necessary permissions on other resources. The scope available here is limited to **Subscription**, **Resource group**, and a few other options. But if you want to assign roles per-resource like Azure load balancer, Azure NetApp Files etc., then follow method 2. -### Analytics Integration (optional) + **Method 2: Assign roles from the target resource.** + + 1. Navigate to the specific resource you want the identity to access (e.g., the Azure Load Balancer, Azure NetApp Files). + 2. Go to its **Access control (IAM)** blade. + 3. Click **Add > Add role assignment**. + 4. Select the required role (e.g., `Reader`\). + 5. For **Assign access to**, choose **Managed identity**. + 6. Click **+ Select members**, choose your subscription, and under **Managed identity**, select the system-assigned identity of management server. + 7. Click **Review + assign**. + 8. Repeat this for every resource listed in the permissions table below. For detailed guidance, see [Assign Azure roles using the Azure portal](https://learn.microsoft.com/azure/role-based-access-control/role-assignments-portal). + +3. **Configure `sap-parameters.yaml`**: + * When using a system-assigned identity, the `user_assigned_identity_client_id` parameter in `sap-parameters.yaml` must be left blank or set to an empty string (`""`). + +### 5. Analytics Integration (optional) - **Analytics Integration** [Telemetry Setup Information](./TELEMETRY_SETUP.md) - Azure Log Analytics @@ -252,7 +300,7 @@ platform: "HANA" # The NFS provider used for shared storage. Supported values are: # - ANF (for Azure NetApp Files) # - AFS (for Azure File Share) -NFS_provider: "ANF" # or "AFS" +NFS_provider: "AFS" # or "ANF" # If you're using a user-assigned managed identity (as explained in "Azure RBAC" section above): # - Enter the client ID of that identity here @@ -283,28 +331,34 @@ ANF_account_name: "ANF-ACCOUNT-NAME" The required credential files depend on the authentication method used to connect to the SAP system: 1. **SSH Key Authentication**: If connecting via SSH key, place the private key inside `WORKSPACE/SYSTEM/` and name the file "ssh_key.ppk". + 1. **Password Authentication**: If connecting using a username and password, create a password file by running the following command. It takes the username from hosts.yaml file. - ```bash - echo "password" > WORKSPACES/SYSTEM//password - ``` + ```bash + echo "password" > WORKSPACES/SYSTEM//password + chmod 600 WORKSPACES/SYSTEM//password + ``` + +> [!NOTE] +> +> If you are using a credential file locally, then you need to comment `key_vault_id` and `secret_id` in `sap-parameters.yaml` 2.2.4. **Credential Files** (From Azure Key Vault) When using Azure Key Vault to store credentials, the framework retrieves authentication details directly from the key vault using the configured managed identity. - **Authentication Methods:** +**Authentication Methods:** - 1. **SSH Key Authentication**: Store the private SSH key content in Azure Key Vault as a secret. - 2. **Password Authentication**: Store the password in Azure Key Vault as a secret. The username is taken from the `hosts.yaml` file. + 1. **SSH Key Authentication**: Store the private SSH key content in Azure Key Vault as a secret. + 2. **Password Authentication**: Store the password in Azure Key Vault as a secret. The username is taken from the `hosts.yaml` file. - **Setup:** +**Setup:** 1. Ensure the managed identity has "Key Vault Secrets User" role on the key vault. 2. Configure `key_vault_id` and `secret_id` parameters in `sap-parameters.yaml` as shown in section 2.2.2. - **Important**: When using Key Vault authentication, do NOT create local credential files (`ssh_key.ppk` or `password` files). +**Important**: When using Key Vault authentication, do NOT create local credential files (`ssh_key.ppk` or `password` files). ## Update the framework