Configure DoMINO-Automotive-Aero NIM at Runtime#

Use this documentation for details about how to configure the DoMINO-Automotive-Aero NIM at runtime.

GPU Selection#

The container should be run on specific GPUs. Multi-GPU support is currently unavailable. Expose specific GPUs inside the container by using either:

  • The --gpus flag, for example --gpus='"device=1"'.

  • The environment variable NVIDIA_VISIBLE_DEVICES, for example -e NVIDIA_VISIBLE_DEVICES=1.

The device IDs to use as inputs are the output of nvidia-smi -L.

GPU 0: Tesla H100 (UUID: GPU-b404a1a1-d532-5b5c-20bc-b34e37f3ac46)
GPU 1: NVIDIA GeForce RTX 3080 (UUID: GPU-b404a1a1-d532-5b5c-20bc-b34e37f3ac46)

See the NVIDIA Container Toolkit documentation for more instructions.

Shared Memory Flag#

DoMINO-Automotive-Aero NIM uses Triton’s Python backend capabilities that scales with the number of CPU cores available. You may need to increase the available shared memory given to the microservice container.

Example providing 1g of shared memory:

docker run ... --shm-size=1g ...

Model Profiles#

The DoMINO-Automotive-Aero NIM has the following model profiles that can be used:

DoMINO#

NIM_MODEL_PROFILE: b8ea86b5ad2bd507ca67a9226a816817ecd8f658c75987ea0acfd4c3081cff9a

Environment Variables#

The DoMINO-Automotive-Aero NIM allows a few customizations that are referenced on the start up of the container. The below variables can be used to change the NIM behavior.

Variable

Default

Description

NGC_API_KEY

Your NGC API key with read access to the model registry for the model profile you are using.

NIM_MODEL_PROFILE

“b8ea8….cff9a”

The model package to load into NIM on launch. This is downloaded from NGC assuming that you have the correct permissions.

NIM_HTTP_API_PORT

8000

Publish the NIM service to the specified port inside the container. Make sure to adjust the port passed to the -p/--publish flag of docker run to reflect that.

NIM_DISABLE_MODEL_DOWNLOAD

Disable model download on container startup.
On this page