.. _cmn_cyprus_multichip_config_label: CMN-Cyprus Multichip Configuration ================================== Introduction ------------ This document gives an overview of the CMN-Cyprus multichip configuration in the RD-Fremont-Cfg2 platform. Overview -------- The following diagram shows the cross-chip CCG connections in the the RD-Fremont-Cfg2 platform: :: +----------------------------------+ +-------------------------------+ | | | | | CCG6+-------------+CCG6 | | | | CCG4| |CCG4 CCG7+-------------+CCG7 | | CHIP 0 | | CHIP 2 | | CCG8+----+ +-----+CCG8 CCG5| |CCG5 | | | | | | CCG9+-+ | | +--+CCG9 | | | | | | | | | | CCG0 CCG1 CCG2 CCG3 | | | | | | CCG0 CCG1 CCG2 CCG3 | +----+-------+-------+-------+-----+ | | | | +-----+------+------+------+----+ | | | | | | | | | | | | | | | | | | | | | | | | +----+-------+-------+-------+-----+ | | | | +-----+------+------+------+----+ | CCG3 CCG2 CCG1 CCG0 | | | | | | CCG3 CCG2 CCG1 CCG0 | | | | | | | | | | CCG4+-|--|--+ | | | | | | | | | | | CCG5+-|--|-----+ | | | | | | | | |CCG8 | | +--------+CCG4 CCG8| | CHIP 1 | | | CHIP 3 | | | +-----------+CCG5 | |CCG9 | | CCG9| | CCG6+-------------+CCG6 | | | | | | CCG7+-------------+CCG7 | | | | | +----------------------------------+ +-------------------------------+ CMN-Cyprus Driver config data for multichip ------------------------------------------- Please refer the following file in the workspace to get the complete info on the configuration data passed to the CMN-Cyprus driver module in SCP firmware: ``/scp/product/neoverse-rd/rdfremont/scp_ramfw/config_cmn_cyprus.c`` The CML configuration structures can be found in the following file: ``/scp/module/cmn_cyprus/include/mod_cmn_cyprus.h`` The ``mod_cmn_cyprus_cml_config`` structure is used to describe the CCG blocks. Each chip in the RD-Fremont-Cfg2 platform is connected to the other chips via CCG blocks. This info is described using the following tables in the config data: - ``cml_config_table_chip_0``: This table describes the CCG blocks that connect the Chip 0 to Chip 1, Chip 2 and Chip 3 respectively. - ``cml_config_table_chip_1``: This table describes the CCG blocks that connect the Chip 1 to Chip 0, Chip 2 and Chip 3 respectively. - ``cml_config_table_chip_2``: This table describes the CCG blocks that connect the Chip 2 to Chip 0, Chip 1 and Chip 3 respectively. - ``cml_config_table_chip_3``: This table describes the CCG blocks that connect the Chip 3 to Chip 0, Chip 1 and Chip 2 respectively. The following section explains how the Chip 0 CCG blocks are described in the ``cml_config_table_chip_0`` table. **Please note that LCN is enabled along with CPAG and two CCG nodes per CPAG group is considered in this config data.** 1. Configure the logical IDs for the CCGs. The logical ID ranges from 0 to (total CCG block per chip - 1). - Refer ``rdfremontcfg2_cmn_cyprus_ccg_port`` in config_cmn_cyprus.c file for the list of CCG logical IDs. - For the CCG blocks that connect Chip 0 to Chip 1, ``CCG_PORT_0`` and ``CCG_PORT_1`` is used. 2. Configure HAIDs for the CCG blocks. This is a unique HAID across all the chips. - The Chip ID is used to calculate a unique HAIDs for the CCG blocks. Hence, ``(CCG_PER_CHIP * PLATFORM_CHIP_0) + CCG_PORT_0`` and ``(CCG_PER_CHIP * PLATFORM_CHIP_0) + CCG_PORT_1`` is used. 3. Configure the table of remote memory regions. As LCN is enabled, the remote memory regions must be specified as System Cache Group(SCG) regions. The remote DRAM regions which are non-contiguous are specified as secondary regions for the same SCG. Apart from these, the start and the end node positions for the remote SCG is also configured. - Since the CCG0, CCG1 blocks in Chip 0 is connected to Chip 1, the address range of the Chip 1 is specified in this table: i.e., [0x1000000000 - 0x1fffffffff] and [0x100000000000 - 0x1fffffffffff] - Specify the unique HAIDs of the CCG3 and CCG2 blocks in Chip 1 as the target HAIDs. 4. Configure the Remote Chip ID. This is used to setup the AgentID (RAID and HAID) to LinkID LUT. 5. Set ``enable_smp_mode`` to true as the CCG block must be programmed for multichip SMP communications. 6. Set ``enable_direct_connect_mode`` to true in order to enable the upper link layer to upper link layer connection between the CCLAs of two chips. This enables the connection of CXS interface from the CCLA on one CMN-Cyprus to the CXS interface of the other. 7. Set ``enable_cpa_mode`` to true in order to enable CCIX Port Aggregation Mode. 8. Set ``cpag_id`` to 0, as the CPA group IDs are assigned sequentially and ``ccg_count`` to 2 as we consider two CCG ports per CPA group. Steps 1 - 8 are repeated for configuring the CPAG that connect Chip 0 to Chip 2 and Chip 3 respectively. Similarly, the CPA groups in Chip 1, 2 and 3 are specified in the respective CCG config tables. Also, the ``enable_lcn`` flag is set in each chip's CMN config data to enable LCN programming. The CMN-Cyprus driver in each SCP, discovers and configures the local CMN mesh and then programs the CCGs to enable multichip SMP communication. For more info on the CMN-Cyprus driver programming, please refer :ref:`CMN-Cyprus Driver `. -------------- *Copyright (c) 2023-2024, Arm Limited. All rights reserved.*