Architecture Compliance Suite

Important

This feature might not be applicable to all Platforms. Please check individual Platform pages, section Supported Features to confirm if this feature is listed as supported.

What is Arm ACS

Architecture Compliance Suite (ACS) is used to ensure architectural compliance across different implementations of the architecture. Arm Enterprise ACS includes a set of examples of the invariant behaviours that are provided by a set of specifications for enterprise systems (e.g. SBSA, SBBR, etc.), so that implementers can verify if these behaviours have been interpreted correctly.

Overview of ACS test

Reference design (RD) platform is targeted for enterprise, network and other infrastructure markets. This requires the platforms to be compliant to the various architectural specifications such as SBSA and SBBR. ACS compliance test helps to ensure that the RD platform and the software stack is compliant to these specifications. The ACS compilance test on RD platforms use a downloadable pre-built ACS disk image. On completion of the ACS test on RD platforms, the test results are stored on the ACS disk image and can be retrieved from it. The results can then be looked into to determine the conformance to the SBSA and SBBR standards.

Build the platform software

Note

This section assumes the user has completed the chapter Getting Started and has a functional working environment.

This section describes the procedure to build the RD software for ACS test. The components of the RD platform software stack that are built is limited to those that provide the EFI implementation and the EFI shell (i.e, SCP, TF-A and EDK2).

To build the software stack, the command to be used is

• If the target platform is SGI-575:

  ./build-scripts/sgi/build-test-acs.sh -p sgi575 <command>
  

• If the target platform is a Reference Design (RD) platform:

  ./build-scripts/rdinfra/build-test-acs.sh -p <platform name> <command>
  

Supported command line options are listed below

• <platform name>

  • Lookup for a platform name in Platform Names.

• <command>

  • Supported commands are

    • clean

    • build

    • package

    • all (all of the three above)

Examples of the build command are

• Command to clean, build and package the software stack needed for the ACS test on RD-N2 platform:

  ./build-scripts/rdinfra/build-test-acs.sh -p rdn2 all
  

• Command to perform an incremental build of the software components included in the software stack for the RD-N2 platform.

  ./build-scripts/rdinfra/build-test-acs.sh -p rdn2 build
  

Note

This command should be followed by the package command to complete the preparation of the FIP and the disk image.

• Command to package the previously built software stack and prepares the FIP and the disk image.

  ./build-scripts/rdinfra/build-test-acs.sh -p rdn2 package
  

Validate ACS conformance

For running the ACS tests, the ACS test suite disk image is required. The ACS test suite disk image can either by built from source by following the documentation at SystemReady SR ACS or the latest available prebuilt image (sr_acs_live_image.img.xz) can be downloaded from here and extract the sr_acs_live_image.img from sr_acs_live_image.img.xz as described in the README file. It is advised that the latest prebuilt image be used if there are no specfic reasons to build this disk image from ACS test suit source code.

Note

The fresh copy of the ACS test suite disk image should be used before starting the ACS tests. This is to ensure that the tests are not skipped due to presence of log files in the disk image from the previous execution of the test.

To boot and to start ACS test, the commands to be used are

• Set MODEL path before launching the model:

  export MODEL=<absolute path to the platform FVP binary>
  

• If the target platform is SGI-575:

  cd model-scripts/sgi
  

• If the target platform is a Reference Design (RD) platform

  cd model-scripts/rdinfra
  

• Launch the ACS test

  ./acs.sh -p <platform name> -v <path to sr_acs_live_image.img> -n [true|false] -a <additional_params>
  

Supported command line options are listed below

• -p <platform name>

  • Lookup for a platform name in Platform Names.

• -v <absolute path to the sr_acs_live prebuilt image>

  • The absolute path to the sr_acs_live_image.img has to be supplied as the parameter.

• -n [true|false] (optional)

  • Controls the use of network ports by the model. If network ports have to be enabled, use ‘true’ as the option. Default value is set to ‘false’.

• -a <additional_params> (optional)

  • Specify any additional model parameters to be passed. The model parameters and the data to be passed to those parameters can be found in the FVP documentation.

Example commands to perform the ACS tests are as listed below.

• Command to start the execution of the RD-N2 model and perform the ACS tests. The ACS test suite disk image named ‘sr_acs_live_image.img’ is picked up from the location /tmp/sr_acs_live_image.img.

  ./acs.sh -p rdn2 -v /tmp/sr_acs_live_image.img
  

Note

Follow the instructions below for the steps to be performed to complete the tests.

• Command to start the execution of the RD-N2 model with networking enabled and perform the ACS tests. The ACS test suite disk image named ‘sr_acs_live_image.img’ is picked up from the location /tmp/sr_acs_live_image.img. Additional parameters to the model are supplied using the -a command line parameter and networking support is enabled by using the -n parameter.

  ./acs.sh -p rdn2 -v /tmp/sr_acs_live_image.img -n true -a "-C board.flash0.diagnostics=1"
  

Note

Follow the instructions below for the steps to be performed to complete the tests.

The SBSA/SBBR tests are split into two phases - tests that execute from linux and the tests that execute from an EFI interface level.

Let the boot progress to the ‘Grub’ menu. To execute ACS test cases, choose the option ‘bbr/bsa’ from Grub menu, which will launch the EFI shell. Press ‘Enter’ key or wait till the timeout for the EFI startup script to run. The startup script will launch SBBR SCT test by default at first, skip the SCT test by pressing any key on the prompt ‘Press any key to stop the EFI SCT running’. This will start the SBSA UEFI test to start (if SCT is not skipped, the SBSA will run after completion of SCT).

On UEFI SBSA test completion, the startup script will reboot the platform, follow the same steps mentioned above till skipping SCT. This time SBSA UEFI test is not executed as the results are already captured in the sr_acs_live_image.img disk image and the validation OS starts boot. The Linux part of the test will be executed on validation OS boot complete.

On completion of the ACS SBSA and SBBR tests, the execution stops at the sr_acs_live_image filesystem command line prompt. The test can be stopped by terminating the FVP.

In case it is not required to run the complete ACS compliance, instead it is required to validate only the SBSA, the sr_acs_live_image.img has the provision. For this the SBSA test should be run from the EFI shell manually by executing the command listed below. This command skips the exerciser tests as well.

Shell> EFI\BOOT\bsa\sbsa\Sbsa.efi -skip 800

Running the test manually will not store the test result into sr_acs_live_image.img disk image, instead the test results will be available on console as the test proceeds to completion.

Select a SBSA compliance level (optional)

SBSA specification classifies hardware into different levels, level-3 through level-7. The ACS disk image is typically configured for a default level. For ACS disk image v3.0, the default SBSA level is 4. For running the ACS tests for any higher or lower level, press ESC from UEFI shell and run the SBSA efi binary manually to select the appropriate compliance level to be tested. An example of the command to be used to select the compliance level is listed below.

Shell> EFI\BOOT\bsa\sbsa\Sbsa.efi -l <y>

Here, y can be 3, 4, 5 or 6 for the SBSA compliance level.

Retrieve the test results

On completion of SBSA/SBBR tests, the test results can be retrieved by mounting the second partition of ACS test suite disk image that was used for the test.

mkdir /tmp/acs-disk
sudo mount -o loop,offset=537919488 sr_acs_live_image.img /tmp/acs-disk/

The test results can be found in the directories below:

• UEFI SBSA test report: /tmp/acs-disk/acs_results/uefi/

• Linux SBSA test report : /tmp/acs-disk/acs_results/linux/

• FWTS result : /tmp/acs-disk/acs_results/fwts/

Unmount the disk after analysing the logs using the following commands.

losetup
sudo umount /dev/loop_x     # _x can be 0, 1, 2... based on losetup output