Nsys profile --pytorch fails under torch compile

I am getting this error: torch._dynamo.exc.InternalTorchDynamoError: ModuleNotFoundError: No module named ‘nsys_nvtx_annotations’

I am simply trying to prifle a neural network training script which has torch.compile in it. Please help!

@guysz00 can you comment

@rishabhranjan0207 can you tell me what version of Nsight Systems you were using and the OS?

NVIDIA Nsight Systems version 2025.2.1.130-252135690618v0

OS: Ubuntu 22.04.4 LTS

@Guy_Sz were there pytorch trace improvements in 2025.3?

@rishabhranjan0207 thank you for reporting this!

Currently, the option --pytorch=functions-trace is not compatible with torch.compile. We will consider adding support for that.

You can workaround this by removing the unsupported annotations from <nsys_installation_dir>/target-linux-x64/PythonFunctionsTrace/pytorch.json, which are torch.nn.modules.*.forward and torch.nn.functional.* (of course, those annotations will be missing).

Note that --pytorch=autograd-nvtx and --pytorch=autograd-shapes-nvtx is expected to work fine.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.