PPU-SMI (v1.4)

Overview

PPU-SMI (PPU System Management Interface) is a command-line tool based on HGML (HanGuang Management Library) that manages and monitors PPU devices. With PPU-SMI, you can:

• Modify device configurations and feature switches.

• Query runtime parameters and feature enablement status for a specific PPU device.

• Collect runtime data and specific events, and export them to tables for analysis.

• Analyze device resource utilization for each application.

• Query topology information for multiple PPU devices.

2. Installation and usage

PPU-SMI is bundled with the PPU_SDK software package. After installing PPU_SDK, navigate to its directory and run the following script to set the required environment variables:

source envsetup.sh

After setting the environment variables, run ppu-smi -v to check the version information and confirm that PPU-SMI is available:

root@122d8d7a7e37:~# ppu-smi -v
T-Head ppu-smi version v1.0
Build date: Sep  7 2022, 10:43:22
Code node: 4a1989d

Run ppu-smi -h to view the help information:

root@122d8d7a7e37:~# ppu-smi -h
PPU System Management Interface -- v1.0

PPU-SMI provides monitoring information for T-head PPU devices.
The data is presented in either a plain text or an XML format, via stdout or a file.
PPU-SMI also provides several management operations for changing the device state.
Note that the functionality of PPU-SMI is exposed through the HGML C-based library.

ppu-smi [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -h,   --help
        Print usage information and exit.
    -v,   --version
        Print version information and exit.

  LIST OPTIONS:

    -L,   --list-ppus
        Display a list of PPUs connected to the system.
...

3. Device summary

3.1 List PPU devices

Run ppu-smi -L to list the PPU devices in the system.

root@0549cf16bb85:~# ppu-smi -L
PPU 0: PPU (UUID: GPU-019ea108-c110-0828-0000-0000c07e1a46)
PPU 1: PPU (UUID: GPU-019ea108-c120-040c-0000-0000c0267f1e)

• Device index: A zero-based number assigned to the device at power-on.

• Device name

• Device UUID

3.1.1 MIG device information

When MIG mode is enabled, ppu-smi -L also lists the MIG devices associated with each PPU device. For example:

root@0549cf16bb85:~# ppu-smi -L
PPU 0: PPU (UUID: GPU-019ea108-c110-0828-0000-0000c07e1a46)
PPU 1: PPU (UUID: GPU-019ea108-c120-040c-0000-0000c0267f1e)
    MIG g1.c1.i1, Device 0 (UUID: MIG-79c62632-04cc-574b-af7b-cb2e307121c85)
    MIG g1.c0.i0, Device 1 (UUID: MIG-79c62632-04cc-574b-af7b-cb2e307121c84)

3.2 Basic information summary

Run ppu-smi without options to view a summary of the PPU devices.

root@122d8d7a7e37:~# ppu-smi
Wed Sep  7 17:04:22 2022
+-------------------------------------------------------------------------------+
| PPU-SMI 1.2           Driver Version: 543.21.01     HGGC Version: 11.5        |
+---------------------------------+----------------------+----------------------+
| PPU  Name        Persistence M. | Bus-Id               | Volatile Uncorr. ECC |
| Fan  Temp  Perf   Pwr:Usage/Cap | Memory-Usage         | PPU-Util  Compute M. |
|                                 |                      |               MIG M. |
+=================================+======================+======================+
| 0  alixpu                N/A    | 00000000:5E:00.0     |                    0 |
| N/A  35C   P0       100W / 200W | 114MiB / 134473MiB   | 10%          Default |
|                                 |                      |             Disabled |
+---------------------------------+----------------------+----------------------+
| 1  alixpu                N/A    | 00000000:5E:00.1     |                    0 |
| N/A  34C   P0       100W / 200W | 4MiB / 134473MiB     | 10%          Default |
|                                 |                      |             Disabled |
+---------------------------------+----------------------+----------------------+

+-------------------------------------------------------------------------------+
| Processes:                                                                    |
| PPU    GI   CI   PID   Type   Process name                         PPU Memory |
|        ID   ID                                                     Usage      |
+===============================================================================+
| 0      N/A  N/A  25759 C      cuda_test1                               151MiB |
| 0      N/A  N/A  25677 C      cuda_test2                               151MiB |
+-------------------------------------------------------------------------------+

The output includes two tables: the PPU device list and the PPU process list. Unsupported information is displayed as N/A.

• PPU: The zero-based number assigned to the device at power-on.

• Name: The name of the device.

• Persistence M.: Indicates whether persistence mode is enabled.

• Fan: The target fan speed as a percentage of the maximum speed.

• Temp: The current device temperature.

• Perf: The device performance state. P0 is the highest state, while P1, P2, and so on indicate progressively lower performance states. N/A is displayed if not supported.

• Pwr:Usage/Cap: The current power draw and the device power limit, both in watts.

• Bus-Id: The device's PCI bus ID.

• Memory-Usage: The used and total device memory, in MiB.

• Volatile Uncorr. ECC: The total count of uncorrectable ECC errors since the driver was loaded.

• PPU-Util: The current PPU stream processor utilization.

• Compute M.: The current compute mode, such as Default, Prohibited, or Exclusive Process (displayed as E. Process).

• MIG M.: The current MIG mode.

• PPU: The zero-based number assigned to the device at power-on.

• CI ID: The ID of the compute instance used by the process.

• PID: The system process ID.

• Type: The compute type of this process, where C stands for Compute.

• Process name: Process name

• PPU Memory Usage: The PPU memory used by the process, in MiB.

3.2.1 MIG device information

If a PPU device has MIG mode enabled, running ppu-smi also displays the MIG devices table:

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+---------------------+----------+-------------------------+
| PPU  GI  CI  MIG |        Memory-Usage |      Vol |         Shared          |
|      ID  ID  DEV |                     |  CU  Unc | CpyEng ENC DEC OFA JPEG |
|                  |                     |      ECC |                         |
+==================+=====================+==========+=========================+
|  0   0   0    5  |   106MiB / 33618MiB |  64    0 |   4     2   2   0   0   |
+------------------+---------------------+----------+-------------------------+
|  1   1   2    3  |     6MiB / 16809MiB |  32    0 |   2     0   0   0   0   |
+------------------+---------------------+----------+-------------------------+
|  1   1   3    2  |     6MiB / 16809MiB |  32    0 |   2     0   0   0   0   |
+------------------+---------------------+----------+-------------------------+

• PPU: The zero-based number assigned to the device at power-on.

• GI ID: The GPU instance ID associated with the MIG device.

• CI ID: The compute instance ID associated with the MIG device.

• MIG DEV: The enumeration number of the MIG device.

• Memory-Usage: The used and total memory for this MIG device (in MiB).

• CU: The number of dedicated Compute Units (CUs) for this MIG device.

• Vol Unc ECC: The total count of uncorrectable ECC errors since the driver was loaded.

• Shared: Resources shared by this MIG device with other MIG devices on the same GPU instance:

  • CpyEng: The number of shared copy engines.

  • ENC: The number of shared encoders.

  • DEC: The number of shared decoders.

  • OFA: The number of shared OFA processing units.

  • JPEG: The number of shared JPEG processing units.

3.2.2 Other options

4. Query device information

root@122d8d7a7e37:~# ppu-smi -q

==============PPUSMI LOG==============

Timestamp                                   : Wed Sep  7 19:13:36 2022
Driver Version                              : 510.39.01
HGGC Version                                : 11.6
SDK Version                                 : 1.4.45-5188b0

Attached PPUs                               : 1
PPU 00000000:01:00.0
    Product Name                            : alixpu
    Persistence Mode                        : Disabled
    MIG Mode
        Current                             : N/A
        Pending                             : N/A
    Serial Number                           : N/A
    PPU UUID                                : PPU-7f53d39f-ce6e-dc78-c3d4-4c18653c19c0
    PCI
        Bus                                 : 0x01
        Device                              : 0x00
        Domain                              : 0x0000
        Device Id                           : 0x1E0410DE
...

root@122d8d7a7e37:~# ppu-smi -q -d MEMORY
...
PPU 00000000:01:00.0
    HBM Vendor                              : Samsung
    Memory Usage
        Total                               : 11264 MiB
        Used                                : 248 MiB
        Free                                : 11016 MiB

root@2b92dd1ad851:~# ppu-smi -q -d UTILIZATION
...
PPU 00000000:3E:00.0
    Utilization
        Ppu                                 : 38 %
        Core                                : 30 %
        Memory                              : 0 %
        Encoder                             : 0 %
        Decoder                             : 0 %
    PPU Utilization Samples
        Duration                            : 19.89 sec
        Number of Samples                   : 99
        Max                                 : 56 %
        Min                                 : 1 %
        Avg                                 : 31 %
    Memory Utilization Samples
        Duration                            : 19.89 sec
        Number of Samples                   : 99
        Max                                 : 0 %
        Min                                 : 0 %
        Avg                                 : 0 %
    ENC Utilization Samples
        Duration                            : 19.89 sec
        Number of Samples                   : 99
        Max                                 : 0 %
        Min                                 : 0 %
        Avg                                 : 0 %
    DEC Utilization Samples
        Duration                            : 19.89 sec
        Number of Samples                   : 99
        Max                                 : 0 %
        Min                                 : 0 %
        Avg                                 : 0 %

root@2b92dd1ad851:~# ppu-smi -q -d ECC
...
PPU 00000000:3E:00.0
    Ecc Mode
        Current                             : Enabled
        Pending                             : Enabled
    ECC Errors
        Volatile
            SRAM Correctable                : 0
            SRAM Uncorrectable              : 0
            DRAM Correctable                : 0
            DRAM Uncorrectable              : 0
        Aggregate
            SRAM Correctable                : 0
            SRAM Uncorrectable              : 0
            DRAM Correctable                : 0
            DRAM Uncorrectable              : 0

root@2b92dd1ad851:~# ppu-smi -q -d TEMPERATURE
...
PPU 00000000:3E:00.0
    Temperature
        PPU Current Temp                    : 42 C
        PPU Shutdown Temp                   : 95 C
        PPU Slowdown Temp                   : 92 C
        PPU Max Operating Temp              : 85 C
        PPU Target Temperature              : N/A
        Memory Current Temp                 : 42 C
        Memory Max Operating Temp           : 95 C

root@2b92dd1ad851:~# ppu-smi -q -d POWER
...
PPU 00000000:3E:00.0
    Power Readings
        Power Management                    : Supported
        Power Draw                          : 51.99 W
        Power Limit                         : 250.00 W
        Default Power Limit                 : 250.00 W
        Enforced Power Limit                : 250.00 W
        Min Power Limit                     : 150.00 W
        Max Power Limit                     : 250.00 W
    Power Samples
        Duration                            : 2.39 sec
        Number of Samples                   : 119
        Max                                 : 58.68 W
        Min                                 : 51.40 W
        Avg                                 : 55.23 W

root@2b92dd1ad851:~# ppu-smi -q -d CLOCK
...
PPU 00000000:08:00.0                        
    Clocks                                  
        CU                                  : 200 MHz
        Memory                              : 1800 MHz
        Video                               : 1000 MHz
    Applications Clocks                     
        CU                                  : 1700 MHz
        Memory                              : 1800 MHz
    Default Applications Clocks             
        CU                                  : 1700 MHz
        Memory                              : 1800 MHz
    Max Clocks                              
        CU                                  : 1700 MHz
        Memory                              : 1800 MHz
        Video                               : 1000 MHz

root@2b92dd1ad851:~# ppu-smi -q -d COMPUTE
...
PPU 00000000:3E:00.0
    Compute Mode                            : Default

root@2b92dd1ad851:~# ppu-smi -q -d PIDS
...
PPU 00000000:3E:00.0
    Processes
        Compute instance ID                 : N/A
        Process ID                          : 85637
            Type                            : Compute
            Name                            : ppu_test
            Used PPU Memory                 : 413 MiB
        Compute instance ID                 : N/A
        Process ID                          : 87771
            Type                            : Compute
            Name                            : ppu_test_threads
            Used PPU Memory                 : 413 MiB

root@122d8d7a7e37:~# ppu-smi -q -d SUPPORTED_CLOCKS
...
PPU 00000000:01:00.0
    Supported Clocks
        Memory                              : 7000 MHz
            CU                              : 2100 MHz
            CU                              : 2085 MHz
            CU                              : 2070 MHz
            CU                              : 2055 MHz
            CU                              : 2040 MHz
            ...
        Memory                              : 6800 MHz
            CU                              : 2100 MHz
            CU                              : 2085 MHz
            CU                              : 2070 MHz
            CU                              : 2055 MHz
            CU                              : 2040 MHz
            ...

root@122d8d7a7e37:~# ppu-smi -q -d PAGE_RETIREMENT
...
PPU 00000000:10:00.0
    Retired Pages
        Single Bit ECC                      : 0
        Double Bit ECC                      : 0
        Pending Page Blacklist              : No

root@122d8d7a7e37:~# ppu-smi -q -d ROW_REMAPPER
...
PPU 00000000:10:00.0
    Remapped Rows
        Correctable Error                   : 0
        Uncorrectable Error                 : 0
        Pending                             : No
        Remapping Failure Occurred          : No
        Bank Remap Availability Histogram
            Max                             : 3072 bank(s)
            High                            : 0 bank(s)
            Partial                         : 0 bank(s)
            Low                             : 0 bank(s)
            None                            : 0 bank(s)

root@2b92dd1ad851:~# ppu-smi --query-ppu=timestamp,index,name,compute_mode,memory.total,memory.used --format=csv
timestamp, index, name, compute_mode, memory.total [MiB], memory.used [MiB]
2022/09/08 09:24:11.132, 0, alixpu, Default, 40960 MiB, 606 MiB
2022/09/08 09:24:11.139, 1, alixpu, Default, 24576 MiB, 508 MiB

root@2b92dd1ad851:~# ppu-smi --help-query-ppu
List of valid properties to query for the switch "--query-ppu=":

"timestamp"
The timestamp of when the query was made in format "YYYY/MM/DD HH:MM:SS.msec".

"driver_version"
The version of the installed driver. This is an alphanumeric string.

"count"
The number of PPUs in the system.

"name" or "ppu_name"
The official product name of the PPU. This is an alphanumeric string. For all products.

"serial" or "ppu_serial"
This number matches the serial number physically printed on each board. It is a globally unique immutable alphanumeric value.

"uuid" or "ppu_uuid"
This value is the globally unique immutable alphanumeric identifier of the PPU. It does not correspond to any physical label on the board.

"pci.bus_id" or "ppu_bus_id"
PCI bus id as "domain:bus:device.function", in hex.
...

root@2b92dd1ad851:~# ppu-smi --query-ppu=timestamp,count,name,serial,index,uuid --format=csv
timestamp, count, name, serial, index, uuid
2022/09/08 09:42:34.913, 2, alixpu, 1320421013145, 0, PPU-0cdd7938-b576-2411-a408-3ad81dfc1a78
2022/09/08 09:42:34.920, 2, alixpu, 1323921045367, 1, PPU-16c4c41f-9214-5e29-3b86-7a26ab011d3e

root@2b92dd1ad851:~# ppu-smi --help-query-supported-clocks
List of valid properties to query for the switch "--query-supported-clocks=":

[Section about Supported Clocks properties]
List of possible memory and processor clocks combinations that the PPU can operate on (not taking into account HW brake reduced clocks).

"timestamp"
The timestamp of when the query was made in format "YYYY/MM/DD HH:MM:SS.msec".

"name" or "ppu_name"
The official product name of the PPU. This is an alphanumeric string. For all products.

...

"memory" or "mem"
Memory clock.

"processor" or "sm"
Streaming multiprocessor clock.

root@2b92dd1ad851:~# ppu-smi --query-supported-clocks=timestamp,ppu_bus_id,memory,processor --format=csv
timestamp, pci.bus_id, memory [MHz], processor [MHz]
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1410 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1395 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1380 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1365 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1350 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1335 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1320 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1305 MHz
2022/09/08 09:50:26.289, 00000000:3E:00.0, 1215 MHz, 1290 MHz
...

root@2b92dd1ad851:~# ppu-smi --help-query-compute-apps
List of valid properties to query for the switch "--query-compute-apps=":

[Section about Active Compute Processes properties]
List of processes having compute context on the device.

"timestamp"
The timestamp of when the query was made in format "YYYY/MM/DD HH:MM:SS.msec".

"name" or "ppu_name"
The official product name of the PPU. This is an alphanumeric string. For all products.

...

"pid"
Process ID of the compute application.

"process_name"
Process Name.

"used_ppu_memory" or "used_memory"
Amount memory used on the device by the context.

root@2b92dd1ad851:~# ppu-smi --query-compute-apps=timestamp,uuid,pid,process_name,used_ppu_memory --format=csv
timestamp, uuid, pid, process_name, used_ppu_memory [MiB]
2022/09/08 09:57:57.660, PPU-0cdd7938-b576-2411-a408-3ad81dfc1a78, 61785, ppu_test, 413 MiB
2022/09/08 09:57:57.660, PPU-0cdd7938-b576-2411-a408-3ad81dfc1a78, 62103, ppu_test_threads, 413 MiB

root@0549cf16bb85:~# ppu-smi --help-query-retired-pages
List of valid properties to query for the switch "--query-retired-pages=":

[Section about Retired Pages properties]
List of pages have been retired or pending to be retired.

"timestamp"
The timestamp of when the query was made in format "YYYY/MM/DD HH:MM:SS.msec".

"name" or "ppu_name"
The official product name of the PPU. This is an alphanumeric string. For all products.
...

"retired_pages.address"
Address of a retired page. Address might be different when ECC is Enabled or Disabled.

"retired_pages.timestamp"
Timestamp at which the page was retired.

"retired_pages.cause"
Reason that describes why the page was retired. Can take one of two values:
 - Double Bit ECC: Number of PPU device memory pages retired due to a double-bit ECC error.
 - Single Bit ECC: Number of PPU device memory pages retired due to multiple single-bit ECC errors.

root@2b92dd1ad851:~# ppu-smi --query-retired-pages=timestamp,uuid,retired_pages.address,retired_pages.timestamp,retired_pages.cause --format=csv
timestamp, uuid, retired_pages.address, retired_pages.timestamp, retired_pages.cause
12:32:15.329, PPU-3f53d39f-ce6e-dc78-c3d4-4c18653c19c0, 0x0000000073234722, 1663256354, Single Bit ECC

root@0549cf16bb85:~# ppu-smi --help-query-remapped-rows
List of valid properties to query for the switch "--query-remapped-rows=":

"timestamp"
The timestamp of when the query was made in format "YYYY/MM/DD HH:MM:SS.msec".

"name" or "ppu_name"
The official product name of the PPU. This is an alphanumeric string. For all products.
...

"remapped_rows.correctable"
The number of rows that have been remapped due to correctable ECC errors.

"remapped_rows.uncorrectable"
The number of rows that have been remapped due to uncorrectable ECC errors.

"remapped_rows.pending"
Whether or not there are pending row-remappings.

"remapped_rows.failure"
Whether or not a row remapping has failed in the past.

"remap_availability.bank_histogram.max"
The number of banks that have max remap availability(all reserved rows are available).

"remap_availability.bank_histogram.high"
The number of banks that have high remap availability.

"remap_availability.bank_histogram.partial"
The number of banks that have partial remap availability.
...

root@2b92dd1ad851:~# ppu-smi --query-remapped-rows=timestamp,uuid,remapped_rows.correctable,remapped_rows.uncorrectable,remap_availability.bank_histogram.max --format=csv
timestamp, uuid, remapped_rows.correctable, remapped_rows.uncorrectable, remap_availability.bank_histogram.max
2022/09/08 16:24:40.435, PPU-099ea108-0181-0230-0000-000060f19f20, 0, 0, 3072
2022/09/08 16:24:40.435, PPU-019ea108-01a1-0222-0000-000040dff62d, 0, 0, 3072
2022/09/08 16:24:40.435, PPU-019ea108-0121-0222-0000-0000605e6417, 0, 0, 3072

5. Modify device configuration

5.1 Device configuration options

PPU-SMI allows you to modify the device configuration. For example, to change the processor clock frequency, run ppu-smi -lpc 1410:

root@122d8d7a7e37:~# ppu-smi -lpc 1410
Set PPU clock to (min clock 1410MHz, max clock 1410MHz) for PPU 00000000:01:00.0.
All done.

The following options are supported. Each PPU-SMI command modifies only one type of device configuration at a time.

5.1.1 Reset a PPU device

Running ppu-smi -r resets all PPU devices. To reset a specific PPU device, use the -i option; for example, ppu-smi -r -i 0 resets PPU device 0. Resetting a PPU device requires the following prerequisites:

1. No HGGC-related applications (compute applications) are running on any PPU device.

2. No PPU-related monitoring or utility applications, such as ppu-smi, ppudbg, or PPU DCGM, are running on any PPU device.

If a PPU device reset fails because one of the listed applications is running, use the following commands to identify them:

# Find HGGC-related applications
ppu-smi pmon -c 1

# Find PPU-related monitoring and utility applications
lsof /dev/alixpu

5.2 Other options

Collect device statistics

PPU-SMI collects device sample and event information and outputs it in CSV format to help you analyze the device's operating status. This functionality is provided by the stats subcommand. To view its help information, run ppu-smi stats -h:

root@2b92dd1ad851:~# ppu-smi stats -h
Generates PPU statistics such as power samples,
utilization samples, xid events, clock change events
and performance capping events.

ppu-smi stats [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -i,   --id
        Enumeration index, Serial number, PCI bus ID or UUID.
        Provide comma separated values for more than one device
    -f,   --filename
        Log to a specified file, rather than to stdout
    -d,   --display
        Display specific metric:
            pwrDraw,temp,memUtil,ppuUtil,
            encUtil,decUtil,memClk,procClk,
            violPwr,violThm,xidEvent,sbEcc,
            dbEcc,pState,clkChg,pwrChg,migChg
        Metric can be combined with comma e.g. pwrDraw,temp
    -c,   --count
        Run for specified number of monitoring cycles and exit
    -h,   --help
        Display help information

Stats in following CSV format:
Device, Power Drawn (pwrDraw), Timestamp (us), Value (Watts)
Device, PPU Temperature (temp), Timestamp (us), Value (C)
Device, PPU Util (ppuUtil), Timestamp (us), Value (%)
...

Output format

The following is a sample of the CSV output from running ppu-smi stats:

root@122d8d7a7e37:~# ppu-smi stats
0, violPwr , 1662606252538553, 0
0, violThm , 1662606252537681, 0
0, temp    , 1662606252536736, 51
0, pState  , 1662606252559897, 0
0, clkChg  , 1662606252559964, 0
0, pState  , 1662606252568416, 0
0, clkChg  , 1662606252568499, 0
...
0, temp    , 1662606253541531, 52
0, pwrDraw , 1662606252543916, 45
0, pwrDraw , 1662606252567432, 54
0, pwrDraw , 1662606252584972, 66
0, pwrDraw , 1662606252605044, 66
...

The output is in a headerless CSV format, with columns defined as follows:

1. First column: The device index, a zero-based number assigned to the device when it powers on.

2. Second column: The abbreviated name of the reported metric, such as pwrDraw and temp.

3. Third column: The data collection timestamp in microseconds, measured from the start of the system_clock.

4. Fourth column: The sampled value. The units for each value are not displayed in the output; refer to the metric descriptions table for the meaning and unit of each metric.

Metric descriptions

You can specify the metrics to collect using the -d option. Separate multiple metrics with a comma (,), for example: ppu-smi stats -d pwrDraw,temp:

root@122d8d7a7e37:~# ppu-smi stats -d pwrDraw,temp
0, temp    , 1662606935107702, 51
0, pwrDraw , 1662606935059582, 38
0, pwrDraw , 1662606935079459, 39
0, pwrDraw , 1662606935099668, 39
...

If you do not specify metrics with the -d option, the tool collects all supported metrics by default. The following table describes the available metrics.

For event-based metrics, such as xidEvent, PPU-SMI generates output when an event occurs. For sampling-based metrics, such as pwrDraw, PPU-SMI aggregates and outputs the results once per second.

By default, PPU-SMI runs and collects data continuously until you stop it by pressing Ctrl + C.

Other options

7. Monitor device status

The dmon subcommand monitors device status, displaying metrics for each device in a scrolling list.

root@2b92dd1ad851:~# ppu-smi dmon
# ppu   pwr ptemp mtemp    cu  core   mem   enc   dec  mclk  pclk
# idx     W     C     C     %     %     %     %     %   MHz   MHz
    0   109    30    32     0     2     0     0     0  1800  1500
    1   119    33    34     0     0     0     0     0  1800  1500
    0   109    30    32     0     0     0     0     0  1800  1500
    1   121    33    34     0     0     0     0     0  1800  1500
...

Run ppu-smi dmon -h to view the help for this subcommand:

root@2b92dd1ad851:~# ppu-smi dmon -h
PPU statistics are displayed in scrolling format with one line
per sampling interval. Metrics to be monitored can be adjusted
based on the width of terminal window.

ppu-smi dmon [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -i,   --id=
        Enumeration index, Serial number, PCI bus ID or UUID.
        Provide comma separated values for more than one device
    -d,   --delay=
        Collection delay/interval in seconds [default=1sec]
    -c,   --count=
        Collect specified number of samples and exit
    -s,   --select=
        One or more metrics [default=puc]
        Can be any of the following:
            p - Power Usage and Temperature
            u - Utilization
            c - Proc and Mem Clocks
            v - Power and Thermal Violations
            m - SM Memory
            e - ECC Errors and PCIe Replay errors
            t - PCIe Rx and Tx Throughput
    -o,   --options=
        One or more from the following:
            D - Include Date (YYYYMMDD) in scrolling output
            T - Include Time (HH:MM:SS) in scrolling output
...

7.1 Monitoring metrics

You can use the -s option to specify the metrics you want to monitor. Each metric is identified by a single character. To monitor multiple metrics, concatenate their characters and pass the resulting string to the -s option. For example, ppu-smi dmon -s pc displays metrics related to power usage (p) and clocks (c).

root@2b92dd1ad851:~# ppu-smi dmon -s pc
# ppu   pwr ptemp mtemp  mclk  pclk
# idx     W     C     C   MHz   MHz
    0    41    45    45  1215   765
    1    30    36    36  1215   930
...

The following table describes the supported metrics:

The following list describes the meaning of each column in the scrolling output:

• Basic information

  • Date: The current date.

  • Time: The current time.

  • ppu: The zero-based enumeration index assigned to the device at power-on.

• Metric type p

  • pwr: Current power usage in watts (W).

  • ptemp: Current processor temperature in degrees Celsius (C).

  • mtemp: Current memory temperature in degrees Celsius (C).

• Metric type u

  • sm: Stream processor utilization in percent (%).

  • core: Core utilization in percent (%).

  • mem: Memory utilization in percent (%).

  • enc: Encoder utilization in percent (%).

  • dec: Decoder utilization in percent (%).

• Metric type c

  • pclk: Processor clock frequency in megahertz (MHz).

  • mclk: Memory clock frequency in megahertz (MHz).

• Metric type v

  • pviol: The percentage of time during the sampling interval that the clock was throttled due to the power limit (%).

  • tviol: The percentage of time during the sampling interval that the clock was throttled due to the thermal limit (%).

• Metric type m

• mem: The amount of memory used, in mebibytes (MiB).

• Metric type e

  • sbecc: The count of single-bit ECC errors since the driver loaded.

  • dbecc: The count of double-bit ECC errors since the driver loaded.

  • pci: The number of PCIe replay errors.

• Metric type t

  • rxpci: PCIe receive throughput in megabytes per second (MB/s).

  • txpci: PCIe transmit throughput in megabytes per second (MB/s).

7.2 Other options

8. Monitor process status

PPU-SMI supports monitoring processes via the pmon subcommand, which provides a scrolling output. Each line displays information for a single process on a single device, and each metric appears in a separate column. For example, run ppu-smi pmon:

root@122d8d7a7e37:~# ppu-smi pmon
# ppu     pid  type    sm   mem   enc   dec   command
# idx       #   C/G     %     %     %     %   name
    0    4563     C     0     0     0     0   ppu_test
    0    4991     C     0     0     0     0   ppu_test_thread
...

Run ppu-smi pmon -h to view the help for this subcommand:

root@122d8d7a7e37:~# ppu-smi pmon -h
Process statistics are displayed in scrolling format per sampling
interval. This tool lists the statistics for all the compute
processes running on each device. Metrics to be monitored
can be adjusted based on the width of terminal window.

ppu-smi pmon [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -i,   --id=
        Enumeration index, Serial number, PCI bus ID or UUID.
        Provide comma separated values for more than one device
    -d,   --delay=
        Collection delay/interval in seconds [default=1sec]
    -c,   --count=
        Collect specified number of samples and exit
    -s,   --select=
        One or more metrics [default=u]
        Can be any of the following:
            u - Utilization
            m - Memory usage
    -o,   --options=
        One or more from the following:
            D - Include Date (YYYYMMDD) in scrolling output
            T - Include Time (HH:MM:SS) in scrolling output
...

8.1 Metrics

Use the -s option to specify the metrics you want to monitor. Each metric corresponds to a single character. To monitor multiple metrics, combine their characters and pass them as a single string to the -s option. For example, ppu-smi pmon -s um displays metrics related to process utilization (u) and memory usage (m).

root@122d8d7a7e37:~# ppu-smi pmon -s um
# ppu     pid  type    sm   mem   enc   dec   mem   command
# idx       #   C/G     %     %     %     %    MB   name
    0    4563     C     0     0     0     0   151   ppu_test
    0    4991     C     0     0     0     0   151   ppu_test_thread

The following table describes the supported metrics:

The following list describes each column in the scrolling output:

• Basic Information

  • Date: The current date.

  • Time: The current time.

  • ppu: The device's zero-based enumeration index, assigned at power-on.

  • pid: The process's system PID.

  • type: The process type. C indicates a compute process.

• Metric type u

  • sm: streaming processor utilization, in percent (%).

  • mem: memory utilization, in percent (%).

  • enc: encoder utilization, in percent (%).

  • dec: decoder utilization, in percent (%).

• Metric type m

  • mem: The process's memory usage, in MiB.

8.2 Other options

9. Query ICN link information

Use the icn subcommand to query ICN link information. Results are displayed per device and per link. For example, run ppu-smi icn -s.

root@dfc623e46a90:~# ppu-smi icn -s
PPU 0: PPU (UUID: GPU-019ea108-c180-040a-0000-000000000000)
    Link 0: 50 GB/s
    Link 1: <inactive>
    Link 2: <inactive>
    Link 3: 50 GB/s
    Link 4: 50 GB/s
    Link 5: 50 GB/s
    Link 6: 50 GB/s
PPU 1: PPU (UUID: GPU-019ea108-c180-060c-0000-000000000000)
    Link 0: 50 GB/s
    Link 1: <inactive>
    Link 2: <inactive>
    Link 3: 50 GB/s
...

Run ppu-smi icn -h to view the help information for the ICN feature. You can use the -i and -l options to constrain the query to specific devices and links. If you do not specify these options, the command queries all devices and links. You can specify only one ICN query option (such as -s) at a time:

root@dfc623e46a90:~# ppu-smi icn -h
icn -- Display ICN link information.

ppu-smi icn [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -h,   --help
        Display help information
    -i,   --id=
        Enumeration index, Serial number, PCI bus ID or UUID.
        Provide comma separated values for more than one device.
    -l,   --link=
        Specify a target link ID (0-based link index),
        Without this parameter, all links information are displayed.

    [any one of]

    -s,   --status
        Display link state (active/inactive).
    -c,   --capabilities
        Display link capabilities.
    -p,   --pcibusid
        Display remote node PCI bus ID for a link.
    -r,   --remotelinkinfo
        Display remote device PCI bus ID and ICN link ID for a link.

9.1 Query ICN link state

Use ppu-smi icn -s to query the link state and bandwidth. You can use the -i option to constrain the query to a specific device. For example, run ppu-smi icn -s -i 0. The output fields are described below:

root@dfc623e46a90:~# ppu-smi icn -s -i 0
PPU 0: PPU (UUID: GPU-019ea108-c180-040a-0000-000000000000)
    Link 0: 50 GB/s
    Link 1: <inactive>
    Link 2: <inactive>
    Link 3: 50 GB/s
    Link 4: 50 GB/s
    Link 5: 50 GB/s
    Link 6: 50 GB/s

• Link 0: 50 GB/s: ICN link 0 on PPU 0 is connected to another device at 50 GB/s.

• Link 1: <inactive>: ICN link 1 on PPU 0 is not connected to any device.

9.2 Query ICN link capabilities

Use ppu-smi icn -c to query the capabilities of each ICN link. Only active ICN links are displayed. The following is a sample output:

root@dfc623e46a90:~# ppu-smi icn -c
PPU 0: PPU (UUID: GPU-019ea108-c180-040a-0000-000000000000)
    Link 0, P2P is supported: true
    Link 0, Access to system memory is supported: false
    Link 0, P2P atomics is supported: true
    Link 0, System memory atomics is supported: false
    Link 0, SLI is supported: false
    Link 0, Link is supported: true
    Link 3, P2P is supported: true
    Link 3, Access to system memory is supported: false
    Link 3, P2P atomics is supported: true
    Link 3, System memory atomics is supported: false
    Link 3, SLI is supported: false
    Link 3, Link is supported: true
...

9.3 Query remote device PCI bus ID

Use ppu-smi icn -p to query the PCI bus ID of the remote device for each ICN link. Only active ICN links are displayed. The following is a sample output:

root@dfc623e46a90:~# ppu-smi icn -p
PPU 0: PPU (UUID: GPU-019ea108-c180-040a-0000-000000000000)
    Link 0: 00000001:CE:00.0
    Link 3: 00000000:89:00.0
    Link 4: 00000000:89:00.0
    Link 5: 00000000:CC:00.0
    Link 6: 00000001:CE:00.0
PPU 1: PPU (UUID: GPU-019ea108-c180-060c-0000-000000000000)
    Link 0: 00000001:D1:00.0
    Link 3: 00000000:C9:00.0
    Link 4: 00000000:86:00.0
    Link 5: 00000000:86:00.0
    Link 6: 00000001:D1:00.0
...

9.4 Query remote device information

Use ppu-smi icn -r to query information about the remote device of an ICN link, including the remote device's PCI bus ID and its ICN link index. The following is a sample output:

root@dfc623e46a90:~# ppu-smi icn -r
PPU 0: PPU (UUID: GPU-019ea108-c180-040a-0000-000000000000)
    Link 0: Remote Device 00000001:CE:00.0: Link 6
    Link 3: Remote Device 00000000:89:00.0: Link 1
    Link 4: Remote Device 00000000:89:00.0: Link 1
    Link 5: Remote Device 00000000:CC:00.0: Link 3
    Link 6: Remote Device 00000001:CE:00.0: Link 6
PPU 1: PPU (UUID: GPU-019ea108-c180-060c-0000-000000000000)
    Link 0: Remote Device 00000001:D1:00.0: Link 7
    Link 3: Remote Device 00000000:C9:00.0: Link 2
    Link 4: Remote Device 00000000:86:00.0: Link 0
    Link 5: Remote Device 00000000:86:00.0: Link 0
    Link 6: Remote Device 00000001:D1:00.0: Link 7
...

9.5 Query cable status

Use ppu-smi icn -cs to query the presence status of each port of the ICN link. The following is a sample output:

root@dfc623e46a90:~# ppu-smi icn -cs
PPU 0: t-head ppu 0 (UUID: GPU-3f53d39f-ce6e-dc78-c3d4-4c18653c19c0)
    Link 0: Connected
    Link 1: Disconnected
    Link 2: Connected
...
PPU 1: t-head ppu 1 (UUID: GPU-3f53d39f-ce6e-dc78-c3d4-4c18653c19c1)
    Link 0: Connected
    Link 1: Connected
    Link 2: Connected
...

9.6 Query link width

Use ppu-smi icn -lw to query the ICN link width. Only active ICN links are displayed. The following is a sample output:

root@dfc623e46a90:~# ppu-smi icn -lw
PPU 0: PPU-ZW810 (UUID: GPU-019ea108-4111-0220-0000-0000006ef62f)
    Link 0: 16x
    Link 3: 16x
    Link 4: 16x
    Link 5: 16x
    Link 6: 8x
PPU 1: PPU-ZW810 (UUID: GPU-019ea108-8191-042a-0000-0000c0d6cc1d)
    Link 0: 16x
    Link 3: 16x
    Link 4: 16x
    Link 5: 16x
    Link 6: 8x
...

• 16x: The link width is 16-bit.

• 8x: The link width is 8-bit.

9.7 Query link-to-port mapping

Use ppu-smi icn -lm to query the mapping between ICN links and physical ports. The following is a sample output:

root@dfc623e46a90:~# ppu-smi icn -lm
PPU 0: PPU-ZW810 (UUID: GPU-019ea108-4111-0220-0000-0000006ef62f)
    Link 0: Physical Port 2
    Link 1: N/A
    Link 2: Physical Port 5
...
PPU 1: PPU-ZW810 (UUID: GPU-019ea108-8191-042a-0000-0000c0d6cc1d)
    Link 0: Physical Port 2
    Link 1: N/A
    Link 2: Physical Port 5
...

• Links without a physical port mapping are displayed as N/A.

9.8 Query link statistics and errors

Use ppu-smi icn -e to query ICN link statistics and error counts. Only active ICN links are displayed. The following is a sample output:

PPU 0: PPU-ZW810 (UUID: GPU-019ea108-4111-0220-0000-0000006ef62f)
    Link 0: 2 link up times
    Link 0: 1 link down times
    Link 0: 0 FEC correctable errors
    Link 0: 0 FEC uncorrectable errors
    Link 0: 0 TX packet errors
    Link 0: 0 RX packet errors
    Link 0: 13 total TX packets
    Link 0: 11 total RX packets

    Link 3: 2 link up times
    Link 3: 1 link down times
    Link 3: 0 FEC correctable errors
    Link 3: 0 FEC uncorrectable errors
    Link 3: 0 TX packet errors
    Link 3: 0 RX packet errors
    Link 3: 16 total TX packets
    Link 3: 8 total RX packets
...

PPU 1: PPU-ZW810 (UUID: GPU-019ea108-8191-042a-0000-0000c0d6cc1d)
    Link 0: 2 link up times
    Link 0: 1 link down times
    Link 0: 0 FEC correctable errors
    Link 0: 0 FEC uncorrectable errors
    Link 0: 0 TX packet errors
    Link 0: 0 RX packet errors
    Link 0: 9 total TX packets
    Link 0: 4 total RX packets
...

• link up times: The number of times the link has come up.

• link down times: The number of times the link has gone down.

• FEC correctable errors: The count of FEC correctable errors on the link.

• FEC uncorrectable errors: The count of FEC uncorrectable errors on the link.

• TX packet errors: The number of transmission packet errors on the link.

• RX packet errors: The number of reception packet errors on the link.

• total TX packets: The total number of packets transmitted on the link.

• total RX packets: The total number of packets received on the link.

9.9 Query link throughput

Use ppu-smi icn -gt r to query the current data throughput summary for ICN links. The following is a sample output:

root@0549cf16bb85:~# ppu-smi icn -gt r
PPU 0: PPU (UUID: GPU-019ea108-c110-0828-0000-000000000000)
    Link 0: Raw Tx: 1618498333 KiB
    Link 0: Raw Rx: 1657515730 KiB
    Link 3: Raw Tx: 1653010223 KiB
    Link 3: Raw Rx: 1693049419 KiB
    Link 4: Raw Tx: 1622645860 KiB
    Link 4: Raw Rx: 1662045442 KiB
...
PPU 1: PPU (UUID: GPU-019ea108-c120-040c-0000-000000000000)
    Link 0: Raw Tx: 1618498333 KiB
    Link 0: Raw Rx: 1657515730 KiB
    Link 3: Raw Tx: 1653010223 KiB
    Link 3: Raw Rx: 1693049419 KiB
    Link 4: Raw Tx: 1622645860 KiB
    Link 4: Raw Rx: 1662045442 KiB
...

10. Querying device topology

PPU-SMI queries topological information between devices. Use the topo subcommand to retrieve this information. For example, run ppu-smi topo -m:

root@dfc623e46a90:~# ppu-smi topo -m
         PPU0    PPU1    PPU2    PPU3    PPU4    PPU5    PPU6    PPU7    CPU Affinity    NUMA Affinity
 PPU0    X       ICN2    SYS     ICN1    SYS     SYS     ICN2    SYS     0-47,96-143     0
 PPU1    ICN2    X       ICN1    SYS     SYS     SYS     SYS     ICN2    0-47,96-143     0
 PPU2    SYS     ICN1    X       ICN2    ICN1    ICN1    SYS     SYS     0-47,96-143     0
 PPU3    ICN1    SYS     ICN2    X       ICN1    ICN1    SYS     SYS     0-47,96-143     0
 PPU4    SYS     SYS     ICN1    ICN1    X       ICN2    SYS     ICN1    48-95,144-191   1
 PPU5    SYS     SYS     ICN1    ICN1    ICN2    X       ICN1    SYS     48-95,144-191   1
 PPU6    ICN2    SYS     SYS     SYS     SYS     ICN1    X       ICN2    48-95,144-191   1
 PPU7    SYS     ICN2    SYS     SYS     ICN1    SYS     ICN2    X       48-95,144-191   1
...

Run ppu-smi topo -h to view help for topology queries. From the mutually exclusive options list, specify only one option at a time. When using the -n or -p options, you must specify the target devices using the -i option:

root@dfc623e46a90:~# ppu-smi topo -h
topo -- Display topological information about the system.

ppu-smi topo [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -h,   --help
        Display help information
    -i,   --id=
        Enumeration index, Serial number, PCI bus ID or UUID.
        Provide comma separated values for more than one device.
        Must be used in conjunction with -n or -p.
    -ri,  --rear-id=
        When used with the option to display matrix (-m or -mp), 
        Show PPU devices that match the specified rear id.
    -po,  --ppu-only
        When used with the option to display matrix (-m or -mp),
        Show PPU devices only.
    -rg,  --rear-group
        When used with the option to display matrix (-m or -mp), 
        Group PPU device by rear id.

    [any one of]

    -m,   --matrix
        Display the PPUDirect communication matrix for the system.
    -mp,  --matrix_pci
        Display the PPUDirect communication matrix for the system (PCI Only).
    -lni, --list-network-interface
        Display a list of RDMA network interface controller(NIC) connected to the system.
    -c,   --cpu=
        Specify a CPU number, Display all PPUs with an affinity.
    -n,   --nearest_ppus=
        Display the nearest PPUs for a given traversal path.
        Could be one of the following:
            0 = a single PCIe switch on a dual PPU board
            1 = a single PCIe switch
            2 = multiple PCIe switches
            3 = a PCIe host bridge
            4 = an on-CPU interconnect link between PCIe host bridges
            5 = an SMP interconnect link between NUMA nodes
        Used in conjunction with -i which must be a single device ID.
    -p,   --ppu_path
        Display the most direct path traversal for a pair of PPUs.
        Used in conjunction with -i which must be a pair of device IDs.
    -p2p, --p2pstatus=
        Displays the p2p status between the PPUs of a given p2p capability.
        Could be one of the following:
            r - p2p read capabiity
            w - p2p write capability
            n - p2p ICN link capability
            a - p2p atomics capability
            p - p2p prop capability

10.1 Querying the topology matrix

Run ppu-smi topo -m to query the connection status between pairs of devices, and between PPU devices and RDMA NICs. If an ICN link is active, its information is prioritized. The query results are described below:

root@dfc623e46a90:/# ppu-smi topo -m
         PPU0    PPU1    PPU2    PPU3    PPU4    PPU5    PPU6    PPU7    NIC0    NIC1    NIC2    NIC3    NIC4    NIC5    NIC6    NIC7    CPU Affinity    NUMA Affinity
 PPU0    X       ICN2    SYS     ICN1    SYS     SYS     ICN2    SYS     PXB     SYS     SYS     SYS     SYS     SYS     PXB     SYS     0-47,96-143     0
 PPU1    ICN2    X       ICN1    SYS     SYS     SYS     SYS     ICN2    PIX     SYS     SYS     SYS     SYS     SYS     PIX     SYS     0-47,96-143     0
 PPU2    SYS     ICN1    X       ICN2    ICN1    ICN1    SYS     SYS     SYS     PXB     SYS     SYS     SYS     SYS     SYS     PXB     0-47,96-143     0
 PPU3    ICN1    SYS     ICN2    X       ICN1    ICN1    SYS     SYS     SYS     PXB     SYS     SYS     SYS     SYS     SYS     PXB     0-47,96-143     0
 PPU4    SYS     SYS     ICN1    ICN1    X       ICN2    SYS     ICN1    SYS     SYS     SYS     PXB     SYS     SYS     SYS     SYS     48-95,144-191   1
 PPU5    SYS     SYS     ICN1    ICN1    ICN2    X       ICN1    SYS     SYS     SYS     SYS     PXB     SYS     SYS     SYS     SYS     48-95,144-191   1
 PPU6    ICN2    SYS     SYS     SYS     SYS     ICN1    X       ICN2    SYS     SYS     PXB     SYS     SYS     PXB     SYS     SYS     48-95,144-191   1
 PPU7    SYS     ICN2    SYS     SYS     ICN1    SYS     ICN2    X       SYS     SYS     PIX     SYS     SYS     PIX     SYS     SYS     48-95,144-191   1
 NIC0    PXB     PIX     SYS     SYS     SYS     SYS     SYS     SYS     X       SYS     SYS     SYS     SYS     SYS     PIX     SYS     0-47,96-143     0
 NIC1    SYS     SYS     PXB     PXB     SYS     SYS     SYS     SYS     SYS     X       SYS     SYS     SYS     SYS     SYS     PIX     0-47,96-143     0
 NIC3    SYS     SYS     SYS     SYS     SYS     SYS     PXB     PIX     SYS     SYS     X       SYS     SYS     PIX     SYS     SYS     48-95,144-191   1
 NIC4    SYS     SYS     SYS     SYS     PXB     PXB     SYS     SYS     SYS     SYS     SYS     X       SYS     SYS     SYS     SYS     48-95,144-191   1
 NIC5    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     X       SYS     SYS     SYS     0-47,96-143     0
 NIC6    SYS     SYS     SYS     SYS     SYS     SYS     PXB     PIX     SYS     SYS     PIX     SYS     SYS     X       SYS     SYS     48-95,144-191   1
 NIC7    PXB     PIX     SYS     SYS     SYS     SYS     SYS     SYS     PIX     SYS     SYS     SYS     SYS     SYS     X       SYS     0-47,96-143     0

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  ICN# = Connection traversing a bonded set of # ICN links

NIC Legend:

  NIC0: mlx5_bond_0
  NIC1: mlx5_bond_1
  NIC2: mlx5_bond_2
  NIC3: mlx5_bond_3
  NIC4: mlx5_bond_4
  NIC5: mlx5_bond_5
  NIC6: mlx5_bond_6
  NIC7: mlx5_bond_7

PPU Rear Group:

  Rear ID 0: PPU 0,1,2,3,4,5,6,7

• ICN2: Indicates that two active ICN links exist between two devices, for example, between PPU0 and PPU1.

• SYS: If no active ICN link exists between two devices (e.g., PPU0 and PPU2), the topology matrix shows the connection status over the PCIe bus. See the Legend for details.

• CPU Affinity 0-47,96-143: Indicates the CPU core affinity of the PPU device. For example, PPU0 has an affinity with CPU cores 0 through 47 and 96 through 143.

• NUMA Affinity 0: Indicates the NUMA node affinity of the PPU device. For example, PPU0 has an affinity with NUMA node 0.

• NIC0: Represents an RDMA NIC connected to the system. Find the corresponding network interface name in the NIC Legend.

• PPU Rear Group: A list of PPU devices grouped by their rear ID. In this section, each row lists the PPU devices for a specific rear.

Use the -po option to display only PPU devices in the topology matrix, excluding RDMA NICs. Use the -mp option to display only PCI bus connectivity, excluding the connection status of ICN links.

For example, run ppu-smi topo -mp -po to query the PCI bus connectivity between PPU devices. The following is an example of the output:

root@dfc623e46a90:/# ppu-smi topo -mp -po
         PPU0    PPU1    PPU2    PPU3    PPU4    PPU5    PPU6    PPU7    CPU Affinity    NUMA Affinity
 PPU0    X       PXB     SYS     SYS     SYS     SYS     SYS     SYS     0-47,96-143     0
 PPU1    PXB     X       SYS     SYS     SYS     SYS     SYS     SYS     0-47,96-143     0
 PPU2    SYS     SYS     X       PXB     SYS     SYS     SYS     SYS     0-47,96-143     0
 PPU3    SYS     SYS     PXB     X       SYS     SYS     SYS     SYS     0-47,96-143     0
 PPU4    SYS     SYS     SYS     SYS     X       PXB     SYS     SYS     48-95,144-191   1
 PPU5    SYS     SYS     SYS     SYS     PXB     X       SYS     SYS     48-95,144-191   1
 PPU6    SYS     SYS     SYS     SYS     SYS     SYS     X       PXB     48-95,144-191   1
 PPU7    SYS     SYS     SYS     SYS     SYS     SYS     PXB     X       48-95,144-191   1

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge

10.1.1 Filter and group by rear ID

PPU-SMI lets you filter the topology matrix by rear ID, showing topological information for only the PPU devices connected to the specified rear. Use the -ri option to specify the rear ID. Find a PPU device's corresponding rear ID by running commands like ppu-smi -q or ppu-smi --query-ppu, or by checking the "PPU Rear Group" information in the topology matrix output.

For example, run ppu-smi topo -m -po -ri 1 to query the topological information for PPU devices with a rear ID of 1. The sample output below shows that only PPU devices with rear ID 1 (PPU 1, 3, 5, 7, 9, 11, 13, and 15) are displayed, while those connected to other rears are not:

root@dfc623e46a90:/# ppu-smi topo -m -po -ri 1
         PPU1    PPU3    PPU5    PPU7    PPU9    PPU11   PPU13   PPU15  CPU Affinity    NUMA Affinity
 PPU1    X       ICN2    SYS     ICN1    SYS     SYS     ICN2    SYS    0-47,96-143     0
 PPU3    ICN2    X       ICN1    SYS     SYS     SYS     SYS     ICN2   0-47,96-143     0
 PPU5    SYS     ICN1    X       ICN2    ICN1    ICN1    SYS     SYS    0-47,96-143     0
 PPU7    ICN1    SYS     ICN2    X       ICN1    ICN1    SYS     SYS    0-47,96-143     0
 PPU9    SYS     SYS     ICN1    ICN1    X       ICN2    SYS     ICN1   48-95,144-191   1
 PPU11   SYS     SYS     ICN1    ICN1    ICN2    X       ICN1    SYS    48-95,144-191   1
 PPU13   ICN2    SYS     SYS     SYS     SYS     ICN1    X       ICN2   48-95,144-191   1
 PPU15   SYS     ICN2    SYS     SYS     ICN1    SYS     ICN2    X      48-95,144-191   1

...

PPU Rear Group:

  Rear ID 1: PPU 1,3,5,7,9,11,13,15

PPU-SMI also supports grouping the topology matrix by rear ID. In the output, PPU devices are sorted first by rear ID and then by PPU index. For example, PPU devices with rear ID 0 are grouped together and appear first.

Use the -rg option to enable grouping by rear ID. For example, run ppu-smi topo -mp -po -rg. The resulting output shows PPUs sorted by rear ID:

root@dfc623e46a90:/# ppu-smi topo -mp -po -rg
         PPU0    PPU2    PPU4    PPU6    PPU8    PPU10   PPU12   PPU14   PPU1    PPU3    PPU5    PPU7    PPU9    PPU11   PPU13   PPU15   CPU Affinity    NUMA Affinity
 PPU0    X       PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     0-1,64-65       0
 PPU2    PHB     X       PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     0-1,64-65       0             
 PPU4    SYS     PXB     X       PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     0-1,64-65       0  
 PPU6    PXB     NODE    PIX     X       SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     0-1,64-65       0             
 PPU8    NODE    PIX     PHB     SYS     X       NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    0-1,64-65       0             
 PPU10   PIX     PHB     SYS     PXB     NODE    X       PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     0-1,64-65       0             
 PPU12   PHB     SYS     PXB     NODE    PIX     PHB     X       PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     0-1,64-65       0             
 PPU14   SYS     PXB     NODE    PIX     PHB     SYS     PXB     X       PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     0-1,64-65       0             
 PPU1    PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     X       SYS     PXB     NODE    PIX     PHB     SYS     PXB     0-1,64-65       0             
 PPU3    NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     X       NODE    PIX     PHB     SYS     PXB     NODE    0-1,64-65       0             
 PPU5    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    X       PHB     SYS     PXB     NODE    PIX     0-1,64-65       0             
 PPU7    PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     X       PXB     NODE    PIX     PHB     0-1,64-65       0             
 PPU9    SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     X       PIX     PHB     SYS     0-1,64-65       0             
 PPU11   PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     X       SYS     PXB     0-1,64-65       0             
 PPU13   NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     X       NODE    0-1,64-65       0             
 PPU15   PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    PIX     PHB     SYS     PXB     NODE    X       0-1,64-65       0

 ...

 PPU Rear Group:

  Rear ID 0: PPU 0,2,4,6,8,10,12,14
  Rear ID 1: PPU 1,3,5,7,9,11,13,15

10.2 Querying RDMA NICs

PPU-SMI supports querying information about the RDMA NICs in the system using the -lni option. For example, running ppu-smi topo -lni displays a list of network interfaces, including their names and PCI bus IDs. The following is a sample output:

root@dfc623e46a90:/# ppu-smi topo -lni
NIC 0: mlx5_bond_0 (PCI Bus Id: 00000000:8A:00.1)
NIC 1: mlx5_bond_1 (PCI Bus Id: 00000000:CF:00.1)
NIC 2: mlx5_bond_2 (PCI Bus Id: 00000001:D2:00.0)
NIC 3: mlx5_bond_3 (PCI Bus Id: 00000001:87:00.0)
NIC 4: mlx5_bond_4 (PCI Bus Id: 00000000:2A:00.1)
NIC 5: mlx5_bond_5 (PCI Bus Id: 00000001:D2:00.1)
NIC 6: mlx5_bond_6 (PCI Bus Id: 00000000:8A:00.0)
NIC 7: mlx5_bond_7 (PCI Bus Id: 00000000:CF:00.0)

10.3 Querying devices by CPU affinity

PPU-SMI lets you query which devices have an affinity with a specific CPU. Use the -c option to specify the CPU index. For example, run ppu-smi topo -c 0 to query the list of devices with an affinity for CPU 0. The following is a sample output:

root@dfc623e46a90:/# ppu-smi topo -c 0
The PPUs that have an affinity with CPU 0 are: 0, 1, 2, 3

10.4 Querying nearest PPU devices

PPU-SMI lets you query the nearest devices to a specified PPU device. Use the -n option to specify the traversal path (defined in the help information) and the -i option to specify the target PPU device. For example, run ppu-smi topo -n 5 -i 0. The following is a sample output:

root@dfc623e46a90:/# ppu-smi topo -n 5 -i 0
Device 0 is connected by way of a SMP interconnect link between NUMA nodes to device(s): 1, 2, 3, 4, 5, 6, 7

10.5 Querying the shortest path between PPUs

PPU-SMI lets you query the shortest path between two PPU devices. Use the -i option to specify the two PPU devices. For example, run ppu-smi topo -p -i 0,1. The following is a sample output:

root@dfc623e46a90:/# ppu-smi topo -p -i 0,1
Device 0 is connected to device 1 by way of potentially multiple PCIe switches.

10.6 Querying P2P capability

Use the -p2p option to check the support status of a P2P capability between devices. The definitions for each P2P capability are provided in the help information. For example, to query the P2P read capability, run ppu-smi topo -p2p r. The following is a sample output:

root@dfc623e46a90:/# ppu-smi topo -p2p r
         PPU0    PPU1    PPU2    PPU3    PPU4    PPU5    PPU6    PPU7
 PPU0    X       OK      OK      OK      OK      OK      OK      OK
 PPU1    OK      X       OK      OK      OK      OK      OK      OK
 PPU2    OK      OK      X       OK      OK      OK      OK      OK
 PPU3    OK      OK      OK      X       OK      OK      OK      OK
 PPU4    OK      OK      OK      OK      X       OK      OK      OK
 PPU5    OK      OK      OK      OK      OK      X       OK      OK
 PPU6    OK      OK      OK      OK      OK      OK      X       OK
 PPU7    OK      OK      OK      OK      OK      OK      OK      X

Legend:

  X    = Self
  OK   = Status Ok
  CNS  = Chipset not supported
  PNS  = PPU not supported
  TNS  = Topology not supported
  NS   = Not supported
  U    = Unknown

11. Multi-Instance GPU (MIG) management

PPU-SMI lets you query, create, and destroy instances in Multi-Instance GPU (MIG) mode. The following are key concepts related to MIG:

• GPU instance: A partition of the PPU hardware resources. Each partition is a GPU instance, and all GPU instances run in isolation from each other.

• GPU instance profile: A supported configuration for partitioning PPU hardware resources. A profile defines the dedicated resources for an instance, such as the number of CUs and the amount of memory.

• compute instance: A partition of a GPU instance. A compute instance has dedicated CUs but shares other resources. Applications can be configured to run on a specific compute instance.

• compute instance profile: A supported configuration for partitioning a GPU instance. A profile defines the number of dedicated CUs and the shared resources for a compute instance.

Enable MIG mode on a PPU device with ppu-smi -mig 1 (see 5. Modify device configuration for more information). Use the mig subcommand to query MIG-related information. For example, run ppu-smi mig -lgip to list supported GPU instance profile information:

root@0549cf16bb85:~# ppu-smi mig -lgip
+---------------------------------------------------------------------------------+
| GPU instance profiles:                                                          |
| PPU  Name                 Profile  Instances   Memory  P2P  CU      DEC   ENC   |
|                             ID     Free/Total   GiB         CpyEng  JPEG  OFA   |
+=================================================================================+
| 1    MIG 8g48gb              3        0/1      48.00   No   64      4     4     |
|                                                             2       4     0     |
+---------------------------------------------------------------------------------+
| 1    MIG 4g24gb              2        1/2      24.00   No   32      2     2     |
|                                                             2       2     0     |
+---------------------------------------------------------------------------------+
| 1    MIG 2g12gb              1        2/4      12.00   No   16      1     1     |
|                                                             2       1     0     |
+---------------------------------------------------------------------------------+
| 1    MIG 1g6gb               0        4/8      6.00    No   8       0     0     |
|                                                             2       0     0     |
+---------------------------------------------------------------------------------+

Run ppu-smi mig -h to view MIG help information. You can specify one mig query subcommand at a time. You can restrict the scope of some subcommands with the -i, -gi, or -ci options. These options can be used individually or in combination. For example, -i 0 -gi 1 restricts the operation to GPU instance 1 on PPU0.

root@dfc623e46a90:/# ppu-smi mig -h
mig -- Multi Instance GPU management.

ppu-smi mig [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -h,   --help
        Display help information
    -i,   --id=
        Enumeration index, Serial number, PCI bus ID or UUID.
        Provide comma separated values for more than one device.
    -gi,  --gpu-instance-id=
        GPU instance ID.
        Provide comma separated values for more than one GPU instance.
    -ci,  --compute-instance-id=
        Compute instance ID.
        Provide comma separated values for more than one compute instance.
    -C,   --default-compute-instance
        When used with the option to create a GPU instance (-cgi),
        Create compute instance with the default profile.

    [any one of]

    -lgip,--list-gpu-instance-profiles
        List supported GPU instance profiles.
        Option -i can be used to restrict the command to run on a specific PPU.
    -lgipp,--list-gpu-instance-possible-placements
        List possible GPU instance placements in the following format:
          {Start,Start...}:Size
        Option -i can be used to restrict the command to run on a specific PPU.
    -cgi, --create-gpu-instance=
        Create GPU instances for the given profile tuples.
        A profile tuple consists of a profile name or ID and an optional placement specifier,
        which consists of a colon and a placement start index.
        Provide comma separated values for more than one profile tuple(e.g. 1:0,4:2).
        Option -i can be used to restrict the command to run on a specific PPU.
    -dgi, --destroy-gpu-instance
        Destroy GPU instances.
        Options -i and -gi can be used individually or combined
        to restrict the command to run on a specific PPU or GPU instance.
    -lgi, --list-gpu-instances
        List GPU instances.
        Option -i can be used to restrict the command to run on a specific PPU.
    -r,   --reset-gpu-instance
        Trigger reset of the GPU instance.
        Options -i and -gi can be used individually or combined
        to restrict the command to run on a specific PPU or GPU instance.
    -lcip,--list-compute-instance-profiles
        List supported compute instance profiles.
        Options -i and -gi can be used individually or combined
        to restrict the command to run on a specific PPU or GPU instance.
    -lcipp,--list-compute-instance-possible-placements
        List possible compute instance placements in the following format:
          {Start,Start...}:Size
        Options -i and -gi can be used individually or combined
        to restrict the command to run on a specific PPU or GPU instance.
    -cci, --create-compute-instance=
        Create compute instance for the given profile tuples.
        A profile tuple consists of a profile name or ID and an optional placement specifier,
        which consists of a colon and a placement start index.
        Provide comma separated values for more than one profile tuple(e.g. 1:0,4:2).
        If no profile name or ID is given, then the default*
        compute instance profile ID will be used.
        Options -i and -gi can be used individually or combined
        to restrict the command to run on a specific PPU or GPU instance.
    -dci, --destroy-compute-instance
        Destroy compute instances.
        Options -i, -gi and -ci can be used individually or combined
        to restrict the command to run on a specific PPU or GPU instance or compute instance.
    -lci, --list-compute-instances
        List compute instances.
        Options -i and -gi can be used individually or combined
        to restrict the command to run on a specific PPU or GPU instance.

11.1 Query GPU instance profiles

Use the -lgip option to query GPU instance profiles. You can then use the -cgi option with a specific profile to create a GPU instance. Use the -i option to restrict the query to a specific device. For example, run ppu-smi mig -i 1 -lgip. The output includes the following fields:

root@a475cc8d4c49:/# ppu-smi mig -i 1 -lgip
+---------------------------------------------------------------------------------+
| GPU instance profiles:                                                          |
| PPU  Name                 Profile  Instances   Memory  P2P  CU      DEC   ENC   |
|                             ID     Free/Total   GiB         CpyEng  JPEG  OFA   |
+=================================================================================+
| 1    MIG 8g96gb              3        1/1      96.00   No   64      4     4     |
|                                                             2       4     0     |
+---------------------------------------------------------------------------------+
| 1    MIG 4g48gb              2        2/2      48.00   No   32      2     2     |
|                                                             2       2     0     |
+---------------------------------------------------------------------------------+
| 1    MIG 2g24gb              1        4/4      24.00   No   16      1     1     |
|                                                             2       1     0     |
+---------------------------------------------------------------------------------+
| 1    MIG 1g12gb              0        8/8      12.00   No   8       0     0     |
|                                                             2       0     0     |
+---------------------------------------------------------------------------------+

• PPU: The zero-based enumeration index assigned to the device upon power-on.

• Name: The name of the GPU instance profile. You can use this name with the -cgi option to create a GPU instance.

• Profile ID: The ID of the GPU instance profile. You can use this ID with the -cgi option to create a GPU instance.

• Instances Free/Total: The number of available and total GPU instances for this profile.

• Memory GiB: Device memory allocated to this profile, in GiB.

• P2P: Indicates whether this profile supports peer-to-peer capabilities.

• CU: The number of dedicated CUs for this profile.

• DEC: The number of dedicated decoders for this profile.

• ENC: The number of dedicated encoders for this profile.

• CpyEng: The number of dedicated copy engines for this profile.

• JPEG: The number of dedicated JPEG processing units for this profile.

• OFA: The number of dedicated OFA processing units for this profile.

11.2 Query possible placements

Use the -lgipp option to list possible placements for GPU instance profiles. You can then use the -cgi option to create a GPU instance at a specific placement.

Possible placements are displayed in the format {start0, start1, start2}:size, where start... represents a possible start index and size is the size of the placement. All placements for a given profile have the same size. For example, the following output indicates that an instance of size 4 can be created at placement 0 or 4.

{0,4}:4

For example, run ppu-smi mig -i 1 -lgipp. The following is a sample output:

root@a475cc8d4c49:/# ppu-smi mig -i 1 -lgipp
PPU 1 profile ID 0 placements: {0,1,2,3,4,5,6,7}:1
PPU 1 profile ID 1 placements: {0,2,4,6}:2
PPU 1 profile ID 2 placements: {0,4}:4
PPU 1 profile ID 3 placements: {0,4}:4

11.3 Create GPU instances

Use the -cgi option to create one or more GPU instances. To specify multiple GPU instances, separate them with a comma (,). For each GPU instance you want to create:

1. Specify the GPU instance profile by its name or ID.

   1. For profile names, you can use the full name or a short name. For example, for a profile named MIG 1g12gb, you can specify either MIG 1g12gb or 1g12gb.

2. To specify a start placement for the GPU instance, append a colon (:) and the placement start index to the profile information.

   1. This step is optional.

Example: Create a GPU instance using GPU instance profile ID 3 without specifying a placement.

-cgi 3

Example: Create two GPU instances using GPU instance profile IDs 0 and 1 without specifying a placement.

-cgi 0,1

Example: Create a GPU instance using GPU instance profile ID 1, specifying the placement to start at 4.

-cgi 1:4

Example: Create a GPU instance using the short profile name 1g12gb without specifying a placement.

-cgi 1g12gb

Example: Create two GPU instances using the profile names MIG 1g12gb and MIG 2g24gb, and specify a placement for each.

-cgi "MIG 1g12gb:0,MIG 2g24gb:4"

You can use the -i option to specify the PPU device for the new GPU instance. If you omit this option, the command attempts to create the instance on every PPU device. For example, run ppu-smi mig -i 1 -cgi 1. The following is a sample output:

root@a475cc8d4c49:~# ppu-smi mig -i 1 -cgi 1
Successfully created GPU instance ID 0 on PPU 1 using profile MIG 2g24gb (Profile ID 1)

11.4 List GPU instances

Use the -lgi option to list existing GPU instances. Use the -i option to restrict the query to a specific device. For example, run ppu-smi mig -i 1 -lgi. The following is a description of the output:

root@0549cf16bb85:~# ppu-smi mig -i 1 -lgi
+---------------------------------------------------------+
| GPU instances:                                          |
| PPU  Name                 Profile  Instance  Placement  |
|                             ID        ID     Start:Size |
+=========================================================+
| 1    MIG 2g12gb              1        0         0:2     |
+---------------------------------------------------------+
| 1    MIG 2g12gb              1        2         2:2     |
+---------------------------------------------------------+

• PPU: The zero-based enumeration index assigned to the device upon power-on.

• Name: The name of the GPU instance.

• Profile ID: The ID of the GPU instance profile that was used to create this GPU instance.

• Instance ID: The ID of the GPU instance. You can use this ID with the -gi option to specify this GPU instance in other commands.

• Placement: The placement information for this GPU instance.

11.5 Destroy GPU instances

Use the -dgi option to destroy one or more GPU instances. You can restrict the operation to a specific device with the -i option or to a specific GPU instance with the -gi option. The -i and -gi options can be used individually or in combination.

For example, to destroy GPU instance 1 on PPU 1, run ppu-smi mig -dgi -i 1 -gi 1. The following is a sample output:

root@0549cf16bb85:~# ppu-smi mig -dgi -i 1 -gi 1
Successfully destroyed GPU instance ID 1 from PPU 1

11.6 Reset GPU instances

Use the -r option to reset one or more GPU instances. Resetting a GPU instance does not affect other GPU instances. You can restrict the operation to a specific device by using the -i option or to a specific GPU instance by using the -gi option. The -i and -gi options can be used individually or in combination.

For example, to reset GPU instance 0 on PPU 1, run ppu-smi mig -i 1 -gi 0 -r. The following is a sample output:

root@0549cf16bb85:~# ppu-smi mig -i 1 -gi 0 -r
Successfully trigger reset of GPU instance ID 0 from PPU 1.

11.7 Query compute instance profiles

Use the -lcip option to list the compute instance profiles available for partitioning a GPU instance. You can restrict the query to a specific device by using the -i option or to a specific GPU instance by using the -gi option. The -i and -gi options can be used individually or in combination. For example, to query the compute instance profiles for GPU instance 0 on PPU 1, run ppu-smi mig -lcip -i 1 -gi 0. The following is a description of the output:

root@0549cf16bb85:~# ppu-smi mig -lcip -i 1 -gi 0
+--------------------------------------------------------------------------------------+
| Compute instance profiles:                                                           |
| PPU    GPU     Name                 Profile  Instances   Exclusive       Shared      |
|      Instance                         ID     Free/Total     CU       DEC   ENC  OFA  |
|         ID                                                          CpyEng JPEG      |
+======================================================================================+
| 1       0      MIG 1u.2g12gb           0       16/16         1         0    0    0   |
|                                                                        2    0        |
+--------------------------------------------------------------------------------------+
| 1       0      MIG 2u.2g12gb           1        8/8          2         0    0    0   |
|                                                                        2    0        |
+--------------------------------------------------------------------------------------+
| 1       0      MIG 3u.2g12gb           2        4/4          3         0    0    0   |
|                                                                        2    0        |
+--------------------------------------------------------------------------------------+
| 1       0      MIG 4u.2g12gb           3        4/4          4         0    0    0   |
|                                                                        2    0        |
+--------------------------------------------------------------------------------------+
| 1       0      MIG 8u.2g12gb           4        2/2          8         0    0    0   |
|                                                                        2    0        |
+--------------------------------------------------------------------------------------+
| 1       0      MIG 12u.2g12gb          5        1/1         12         0    0    0   |
|                                                                        2    0        |
+--------------------------------------------------------------------------------------+
| 1       0      MIG 16u.2g12gb         6*        1/1         16         1    1    0   |
|                                                                        2    1        |
+--------------------------------------------------------------------------------------+

• PPU: The zero-based enumeration index assigned to the device upon power-on.

• GPU instance ID: The ID of the GPU instance.

• Name: The name of the compute instance profile. You can use this name with the -cci option to create a compute instance.

• Profile ID: The ID of the compute instance profile. You can use this ID with the -cci option to create a compute instance.

  • An ID marked with an asterisk (*) indicates the default compute instance profile. When you create a default compute instance by using the -C or -cci option, the system creates a compute instance with this profile.

• Instances Free/Total: The number of available and total compute instances for this profile.

• Exclusive CU: The number of dedicated CUs in a compute instance created with this profile.

• DEC: The number of shared decoders for this profile.

• ENC: The number of shared encoders for this profile.

• CpyEng: The number of shared copy engines for this profile.

• JPEG: The number of shared JPEG processing units for this profile.

• OFA: The number of shared OFA processing units for this profile.

11.8 Query possible placements

Use the -lcipp option to list possible placements for compute instance profiles. You can then use the -cci option to create a compute instance at a specific placement. You can restrict the operation to a specific device by using the -i option or to a specific GPU instance by using the -gi option. The -i and -gi options can be used individually or in combination. Possible placements are displayed in the format {start0, start1, start2}:size, where start... represents a possible start index and size is the size of the placement. All placements for a given profile have the same size. For example, the following output indicates that an instance of size 4 can be created at placement 0 or 4.

{0,4}:4

For example, run ppu-smi mig -i 3 -gi 0 -lcipp. The following is a sample output:

root@0549cf16bb85:~# ppu-smi mig -i 3 -gi 0 -lcipp
PPU 3 GPU instance 0 profile ID 0 placements: {0,1,2,3,4,5,6,7}:1
PPU 3 GPU instance 0 profile ID 1 placements: {0,2,4,6}:2
PPU 3 GPU instance 0 profile ID 2 placements: {0,4}:3
PPU 3 GPU instance 0 profile ID 3 placements: {0,4}:4
PPU 3 GPU instance 0 profile ID 4 placements: {0}:8

11.9 Create compute instances

Use the -cci option to create one or more compute instances. You can specify the name or ID of the corresponding compute instance profile. If you do not specify a profile, the command uses the default compute instance profile. To specify multiple compute instances, separate them with a comma (,). For each compute instance you want to create:

1. Specify the compute instance profile by its name or ID.

   1. For profile names, you can use the full name or a short name. For example, for a profile named MIG 1u.1g6gb, you can specify either MIG 1u.1g6gb or 1u.1g6gb.

2. To specify a start placement for the compute instance, append a colon (:) and the placement start index to the profile information.

   1. This step is optional.

You can restrict the operation to a specific device by using the -i option or to a specific GPU instance by using the -gi option. The -i and -gi options can be used individually or in combination. You can also create a default compute instance when you create a GPU instance. For example, run ppu-smi mig -i 1 -cgi 1 -C:

root@0549cf16bb85:~# ppu-smi mig -i 1 -cgi 1 -C
Successfully created GPU instance ID 4 on PPU 1 using profile MIG 2g12gb (Profile ID 1)
Successfully created compute instance ID 0 on PPU 1 GPU instance ID 4 using profile MIG 16u.2g12gb (Profile ID 6)

Example: Create a compute instance using compute instance profile ID 3.

-cci 3

Example: Create two compute instances using compute instance profile IDs 3,4.

-cci 3,4

Example: Create a compute instance using compute instance profile ID 1, specifying the placement to start at 4.

-cci 1:4

Example: Create a compute instance using the short profile name 1u.2g12gb.

-cci 1u.2g12gb

Example: Create two compute instances using the profile names MIG 1u.2g12gb and MIG 2u.2g12gb.

-cci "MIG 1u.2g12gb,MIG 2u.2g12gb"

Example: Create a compute instance using the default compute instance profile by not specifying a profile name or ID.

ppu-smi mig -cci -i 1 -gi 0

You can use the -i and -gi options to specify where to create the compute instance. If you do not specify these options, the command attempts to create the instance on every PPU device and GPU instance. For example, run ppu-smi mig -i 1 -gi 0 -cci 3. The following is a sample output:

root@0549cf16bb85:~# ppu-smi mig -i 1 -gi 0 -cci 3
Successfully created compute instance ID 0 on PPU 1 GPU instance ID 0 using profile MIG 4u.2g12gb (Profile ID 3)

11.10 List compute instances

Use the -lci option to list existing compute instances. You can restrict the query to a specific device by using the -i option or to a specific GPU instance by using the -gi option. The -i and -gi options can be used individually or in combination. For example, run ppu-smi mig -i 1 -gi 0 -lci. The following is a description of the output:

root@0549cf16bb85:~# ppu-smi mig -i 1 -gi 0 -lci
+-------------------------------------------------------------------+
| Compute instances:                                                |
| PPU    GPU     Name                 Profile  Instance  Placement  |
|      Instance                         ID        ID     Start:Size |
|         ID                                                        |
+===================================================================+
| 1       0      MIG 4u.2g12gb           3        0         0:0     |
+-------------------------------------------------------------------+
| 1       0      MIG 8u.2g12gb           4        1         0:1     |
+-------------------------------------------------------------------+

• PPU: The zero-based enumeration index assigned to the device upon power-on.

• GPU instance ID: The ID of the GPU instance.

• Name: The name of the compute instance.

• Profile ID: The ID of the compute instance profile that was used to create this compute instance.

• Instance ID: The ID of this compute instance. You can use this ID with the -ci option to specify this compute instance in other commands.

• Placement: The placement information for this compute instance.

11.11 Destroy compute instances

Use the -dci option to destroy one or more compute instances. You can restrict the operation to a specific device by using the -i option, to a specific GPU instance by using the -gi option, or to a specific compute instance by using the -ci option. The -i, -gi, and -ci options can be used individually or in combination.

For example, to destroy compute instance 1 of GPU instance 0 on PPU 1, run ppu-smi mig -dci -i 1 -gi 0 -ci 1. The following is a sample output:

root@0549cf16bb85:~# ppu-smi mig -dci -i 1 -gi 0 -ci 1
Successfully destroyed compute instance ID 1 from PPU 1 GPU instance 0

12. Virtual GPU (vGPU)

PPU-SMI supports querying information about device virtualization (vGPU) and allows you to create and delete vGPU instances. Using vGPU features requires administrator privileges.

The following are key concepts related to vGPU:

• vGPU type: A supported configuration for a vGPU on a PPU device. It defines how PPU resources are partitioned, such as the number of copy engines (CEs) and the amount of memory allocated to each vGPU instance.

• vGPU instance: A vGPU instance created from a specific vGPU type. You can use a vGPU instance to create a virtual machine (VM).

The following describes the states of a vGPU instance:

• active instance: A created vGPU instance that is associated with a VM.

• alive instance: Any created vGPU instance, including those that are not associated with a VM.

You can enable vGPU mode on a PPU device by running ppu-smi -vm VGPU (for more information, see 5. Modify device configuration). Use the vgpu subcommand to query vGPU information. For example, run ppu-smi vgpu -s to view the vGPU types supported by the device:

root@0549cf16bb85:~# ppu-smi vgpu -s
PPU 0: PPU (UUID: GPU-019ea108-c110-0420-0000-0000e0b7fe3c)
    vGPU Type 0: vGPU-0 (Class: MIG-Backed: 2 CE 5GB Memory 0 ICN 0 DVG 0 EVG)
    vGPU Type 1: vGPU-1 (Class: MIG-Backed: 2 CE 5GB Memory 0 ICN 1 DVG 0 EVG)
    vGPU Type 2: vGPU-2 (Class: MIG-Backed: 2 CE 5GB Memory 0 ICN 0 DVG 1 EVG)
    vGPU Type 3: vGPU-3 (Class: MIG-Backed: 2 CE 5GB Memory 1 ICN 0 DVG 0 EVG)
    vGPU Type 4: vGPU-4 (Class: MIG-Backed: 2 CE 5GB Memory 1 ICN 1 DVG 0 EVG)
    vGPU Type 5: vGPU-5 (Class: MIG-Backed: 2 CE 5GB Memory 1 ICN 0 DVG 1 EVG)
...

Run ppu-smi vgpu -h to display help information. You can use only one vgpu query or operation sub-option per command. Use the -i option to specify one or more PPU devices. All vgpu query subcommands support the -l option to periodically output results at an interval specified in seconds. For example, -l 5 outputs the results every 5 seconds until you press Ctrl+C to stop it.

root@0549cf16bb85:~# ppu-smi vgpu -h
vgpu -- Virtual GPU management.

ppu-smi vgpu [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -h,   --help
        Display help information
    -i,   --id=
        Enumeration index, Serial number, PCI bus ID or UUID.
        Provide comma separated values for more than one device.
    -v,   --verbose
        Display detailed information about supported vGPU types or vGPU types that can be created.
    -l,   --loop=
        Display information at the specified time interval in seconds until Ctrl-C is pressed.
    -f,   --force
        When used with the option to delete a vGPU instance (-di),
        Delete vGPU instance regardless of VM state.

    [any one of]

    -q,   --query
        Display information about currently running vGPU instances (VM active).
    -a,   --alive
        Display information about currently alive vGPU instances.
    -s,   --supported
        Display supported vGPU types.
    -c,   --creatable
        Display the vGPU types that can currently be created.
    -ci,  --create-instance=
        Create a vGPU instance for the given vGPU type ID.
    -di,  --delete-instance=
        Delete a vGPU instance for the given vGPU instance ID.
        Use option -f to delete vGPU instance regardless of VM state.

12.1 Query supported vGPU types

Use the -s option to query the vGPU types supported by a PPU device. Use the -i option to restrict the query to one or more specific devices. For example, run ppu-smi vgpu -i 0 -s:

root@0549cf16bb85:~# ppu-smi vgpu -i 0 -s
PPU 0: PPU (UUID: GPU-019ea108-c110-0420-0000-0000e0b7fe3c)
    vGPU Type 0: vGPU-0 (Class: MIG-Backed: 2 CE 5GB Memory 0 ICN 0 DVG 0 EVG)
    vGPU Type 1: vGPU-1 (Class: MIG-Backed: 2 CE 5GB Memory 0 ICN 1 DVG 0 EVG)
    vGPU Type 2: vGPU-2 (Class: MIG-Backed: 2 CE 5GB Memory 0 ICN 0 DVG 1 EVG)
    vGPU Type 3: vGPU-3 (Class: MIG-Backed: 2 CE 5GB Memory 1 ICN 0 DVG 0 EVG)
    vGPU Type 4: vGPU-4 (Class: MIG-Backed: 2 CE 5GB Memory 1 ICN 1 DVG 0 EVG)
    vGPU Type 5: vGPU-5 (Class: MIG-Backed: 2 CE 5GB Memory 1 ICN 0 DVG 1 EVG)
...

The list for each PPU device contains the following information:

• vGPU type ID

• vGPU type name

• vGPU type classification details

You can combine the -s and -v options to query more detailed vGPU type information. For example, run ppu-smi vgpu -i 0 -s -v. The output includes the following fields:

root@0549cf16bb85:~# ppu-smi vgpu -i 0 -s -v
+-------------------------------------------------------------------+
| Supported vGPU types:                                             |
| PPU  Type   Name                 Instances     GPU       Memory   |
|       ID                         Free/Total  Instance     GiB     |
|                                                 ID                |
+===================================================================+
| 0      0    vGPU-0                  4/8         N/A      5.00     |
+-------------------------------------------------------------------+
| 0      1    vGPU-1                  0/1         N/A      5.00     |
+-------------------------------------------------------------------+
| 0      2    vGPU-2                  2/4         N/A      11.00    |
+-------------------------------------------------------------------+

• PPU: The zero-based enumeration index assigned to the device upon power-on.

• Type ID: The ID of the vGPU type.

• Name: The name of the vGPU type.

• Instances Free/Total: The number of vGPU instances that can currently be created / The total number of supported vGPU instances.

• GPU Instance ID: The MIG GPU instance ID corresponding to the vGPU type. N/A indicates that there is no mapping.

• Memory GiB: The amount of memory, in GiB, allocated to each vGPU instance.

12.2 Query creatable vGPU types

Use the -c option to query the vGPU types that can currently be created on a PPU device. Use the -i option to restrict the query to specific devices. For example, run ppu-smi vgpu -i 0 -c. The output lists only the vGPU types that can be created with the currently available PPU resources:

root@0549cf16bb85:~# ppu-smi vgpu -i 0 -c
PPU 0: PPU (UUID: GPU-019ea108-c110-0420-0000-0000e0b7fe3c)
    vGPU Type 0: vGPU-0 (Class: MIG-Backed: 2 CE 5GB Memory 0 ICN 0 DVG 0 EVG)
    vGPU Type 3: vGPU-3 (Class: MIG-Backed: 2 CE 5GB Memory 1 ICN 0 DVG 0 EVG)
...

The list for each PPU device contains the following information:

• vGPU type ID

• vGPU type name

• vGPU type classification details

To view more details about creatable vGPU types, combine the -c and -v options. For example, run ppu-smi vgpu -i 0 -c -v. The output includes the following fields:

root@0549cf16bb85:~# ppu-smi vgpu -i 0 -c -v
+-------------------------------------------------------------------+
| Supported vGPU types:                                             |
| PPU  Type   Name                 Instances     GPU       Memory   |
|       ID                         Free/Total  Instance     GiB     |
|                                                 ID                |
+===================================================================+
| 0      0    vGPU-0                  4/8         N/A      5.00     |
+-------------------------------------------------------------------+
| 0      2    vGPU-2                  2/4         N/A      11.00    |
+-------------------------------------------------------------------+

• PPU: The zero-based enumeration index assigned to the device upon power-on.

• Type ID: The ID of the vGPU type.

• Name: The name of the vGPU type.

• Instances Free/Total: The number of vGPU instances that can currently be created / The total number of supported vGPU instances.

• GPU Instance ID: The MIG GPU instance ID corresponding to the vGPU type. N/A indicates that there is no mapping.

• Memory GiB: The amount of memory, in GiB, allocated to each vGPU instance.

12.3 Create a vGPU instance

Use the -ci option to create a vGPU instance from a specified vGPU type.

Use the -i option to restrict the operation to a specific device. For example, to create a vGPU instance from vGPU type ID 0 on PPU device 0, run ppu-smi vgpu -i 0 -ci 0. The following is a sample output:

root@0549cf16bb85:~# ppu-smi vgpu -ci 0 -i 0
Successfully created vGPU instance 0 (PCI Bus ID 00000000:5E:00.0) on PPU 0 using vGPU type 0.

12.4 Query alive vGPU instances

Use the -a option to query information about vGPU instances in the alive state. Use the -i option to restrict the query to a specific device. For example, run ppu-smi vgpu -i 0 -a. The output includes the following information:

root@0549cf16bb85:~# ppu-smi vgpu -i 0 -a
PPU 00000000:10:00.0
    Alive vGPUs                             : 2
    vGPU Instance 1
        vGPU Instance ID                    : 1
        vGPU Name                           : vGPU-10
        vGPU Type                           : 10
        vGPU UUID                           : VGPU-3f53d39f-ce6e-dc78-c3d4-4c18653c19c0
        vGPU PCI Bus ID                     : 00000000:10:01.0
        MDEV UUID                           : N/A
        GPU Instance ID                     : N/A
        ECC Mode                            : Enabled
        Memory Size                         : 28672 MiB
    vGPU Instance 5
        vGPU Instance ID                    : 5
        vGPU Name                           : vGPU-1
        vGPU Type                           : 1
        vGPU UUID                           : VGPU-3f53d39f-ce6e-dc78-c3d4-4c18653c19c1
        vGPU PCI Bus ID                     : 00000000:10:01.4
        MDEV UUID                           : N/A
        GPU Instance ID                     : N/A
        ECC Mode                            : Enabled
        Memory Size                         : 7168 MiB

For each PPU device:

• Alive vGPUs: The number of vGPU instances currently created on the PPU device.

For each vGPU instance:

• vGPU Instance ID: The ID of the vGPU instance. You can pass this ID to the -di option to delete the vGPU instance.

• vGPU Name: The name of this vGPU instance.

• vGPU Type: The ID of the corresponding vGPU type.

• vGPU UUID: The UUID of the vGPU instance.

• vGPU PCI Bus ID: The PCI bus ID of the vGPU instance, in Bus:Device.Function format.

• MDEV UUID: The UUID of the MIG device corresponding to the vGPU instance. N/A indicates that there is no mapping.

• GPU Instance ID: The MIG GPU instance ID corresponding to the vGPU instance. N/A indicates that there is no mapping.

• ECC Mode: The ECC mode of the vGPU instance.

• Memory Size: The total amount of memory allocated to the vGPU instance.

12.5 Query active vGPU instances

Use the -q option to query information about vGPU instances in the active state (associated with a VM). Use the -i option to restrict the query to a specific device. For example, run ppu-smi vgpu -i 0 -q. The output includes the following information:

root@0549cf16bb85:~# ppu-smi vgpu -i 0 -q
PPU 00000000:10:00.0
    Active vGPUs                            : 1
    vGPU Instance 1
        vGPU Instance ID                    : 1
        VM UUID                             : ee7b7a4b-388a-4357-a425-5318b2c65b30
        vGPU Name                           : vGPU-10
        vGPU Type                           : 10
        vGPU UUID                           : VGPU-3f53d39f-ce6e-dc78-c3d4-4c18653c19c0
        vGPU PCI Bus ID                     : 00000000:10:01.0
        MDEV UUID                           : N/A
        Guest Driver Version                : 0.8.0
        GPU Instance ID                     : N/A
        ECC Mode                            : Enabled
        Memory Usage
            Total                           : 28672 MiB
            Used                            : 3 MiB
            Free                            : 28669 MiB

For each PPU device:

• Active vGPUs: The number of vGPU instances on the PPU device that are associated with a VM.

For each vGPU instance:

• vGPU Instance ID: The ID of the vGPU instance. You can pass this ID to the -di option to delete the vGPU instance.

• VM UUID: The UUID of the virtual machine.

• VM Domain ID: The domain ID of the virtual machine.

• vGPU Name: The name of this vGPU instance.

• vGPU Type: The ID of the corresponding vGPU type.

• vGPU UUID: The UUID of the vGPU instance.

• vGPU PCI Bus ID: The PCI bus ID of the vGPU instance, in Bus:Device.Function format.

• MDEV UUID: The UUID of the MIG device corresponding to the vGPU instance. N/A indicates that there is no mapping.

• Guest Driver Version: The guest driver version within the VM.

• GPU Instance ID: The MIG GPU instance ID corresponding to the vGPU instance. N/A indicates that there is no mapping.

• ECC Mode: The ECC mode of the vGPU instance.

• Memory Usage: The memory usage of the vGPU instance.

  • Total: The total amount of memory allocated to the vGPU instance.

  • Used: The amount of memory used by the vGPU instance.

  • Free: The amount of available memory.

12.6 Delete a vGPU instance

Use the -di option to delete a vGPU instance by specifying its ID. Use the -i option to restrict the operation to a specific device. By default, you cannot delete a vGPU instance that is in use by a VM. Use the -f option to force deletion regardless of the VM state. For example, to delete the vGPU instance with ID 1 on PPU device 0, run ppu-smi vgpu -i 0 -di 1. The following is a sample output:

root@0549cf16bb85:~# ppu-smi vgpu -i 0 -di 1
Successfully deleted vGPU instance 1 from PPU 0.

13. Drain a PPU device

PPU-SMI allows you to set and query the drain state of a PPU device, allowing you to drain a faulty PPU device from the system. When a PPU device is drained, it behaves as follows:

• Existing compute tasks continue to run without disruption.

• The PPU device does not accept new compute tasks.

• The PPU device becomes invisible to tools like PPU-SMI and PPUDBG.

For example, to set the drain state of the PPU device with the PCI bus ID 0001:AA:00.0 to draining, run ppu-smi drain -p 0001:AA:00.0 -m 1:

root@dfc623e46a90:~# ppu-smi drain -p 0001:AA:00.0 -m 1
Successfully set PPU 0001:AA:00.0 drain state to: draining.

To view the help information for the drain feature, run ppu-smi drain -h. You must specify the target PPU device by using the -p option with its PCI bus ID:

root@dfc623e46a90:~# ppu-smi drain -h
drain -- Displays/modifies PPU drain states for power idling.

ppu-smi drain [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -h,   --help
        Display help information
    -p,   --pciid=
        PPU PCI ID in the format XXXX:YY:Z.a
            XXXX = domain
            YY   = bus
            Z    = device
            a    = function
    
    [any one of]

    -m,   --modify=
        Modify the drain state of a PPU specified by -p.
            0 = not draining
            1 = draining
    -q,   --query
        Query the drain state of a PPU specified by -p.

13.1 Set the drain state

Use the -m option to set or clear the drain state of a PPU device. You must specify the target PPU device by using the -p option with its PCI bus ID. For example, to drain a PPU device, run ppu-smi drain -p 0001:AA:00.0 -m 1:

root@dfc623e46a90:~# ppu-smi drain -p 0001:AA:00.0 -m 1
Successfully set PPU 0001:AA:00.0 drain state to: draining.

13.2 Query the drain state

Use the -q option to query the drain state of a PPU device. You must specify the target PPU device by using the -p option with its PCI bus ID. For example, to query the drain state, run ppu-smi drain -p 0001:AA:00.0 -q:

root@dfc623e46a90:~# ppu-smi drain -p 0001:AA:00.0 -q
The current drain state of PPU 0001:AA:00.0 is: draining.

14. GPM management

PPU-SMI lets you set and query the status of GPU performance monitoring (GPM). When enabled, GPM uses the PPU device performance counter to collect data. This prevents other applications that rely on the performance counter from running. You can use PPU-SMI to query the GPM status, pause GPM to allow other applications to collect performance counter data, and then resume GPM when they finish.

For example, run ppu-smi gpm --get-stream-state to query the GPU performance monitoring (GPM) stream state on PPU devices:

root@dfc623e46a90:~# ppu-smi gpm --get-stream-state
PPU 0 GPM stream state: Enabled.
PPU 1 GPM stream state: Disabled.

Run ppu-smi gpm -h to view help for the GPM feature. You can use the -i option to specify one or more PPU devices. Separate multiple device IDs with a comma (,).

root@dfc623e46a90:~# ppu-smi gpm -h
gpm -- GPU Performance Monitoring management.

ppu-smi gpm [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
    -h,   --help
        Display help information
    -i,   --id=
        Enumeration index, Serial number, PCI bus ID or UUID. 
        Provide comma separated values for more than one device.

    [any one of]

    -s,   --set-stream-state=
        Set GPU Performance Monitoring Stream State:
        0/DISABLED, 1/ENABLED
    -g,   --get-stream-state
        Get GPU Performance Monitoring Stream State
    --get-sample-state
        Get GPU Performance Monitoring Sample State

14.1 Set the GPM stream state

Use the -s option to enable or disable the GPM stream. When the GPM stream is disabled:

• Applications that subscribe to GPM data will receive N/A for performance results.

• GPU performance monitoring (GPM) no longer uses the PPU device performance counter.

For example, run ppu-smi gpm -s DISABLED -i 0 to disable the GPM stream on PPU 0:

root@dfc623e46a90:~# ppu-smi gpm -s DISABLED -i 0
Set GPM stream state to DISABLED for PPU 00000000:5E:00.0.

14.2 Get the GPM stream state

Use the -g option to query the GPM stream state. You can use the -i option to restrict the query to specific devices. For example, run ppu-smi gpm -g -i 0 to query the GPM stream state for PPU 0:

root@dfc623e46a90:~# ppu-smi gpm -g -i 0
PPU 0 GPM stream state: Disabled.

14.3 Get the GPM sample state

Use the --get-sample-state option to query the GPM sample state. This indicates whether an application is using GPM to collect performance metrics. For example, run ppu-smi gpm --get-sample-state to query the GPM sample state for all PPU devices:

root@dfc623e46a90:~# ppu-smi gpm --get-sample-state
PPU 0 GPM sample state: Enabled.
PPU 1 GPM sample state: Disabled.

Known issues

• SDK v1.00: The reported application clock frequency may be lower than the actual value.

• SDK v1.00: In the mig subcommand, queries without an instance ID may fail.

• The ppu-smi -q -d UTILIZATION command does not support sampling information.

• The stats subcommand does not support sampling information.

• The ppu-smi icn -gt d command does not support querying ICN link data throughput.

• The ppu-smi icn -gt r command underreports the ICN link data volume.

• The throttling percentage caused by temperature or power limits is not reported.

• Querying the PCIe replay counter and error counters is not supported.