Compatibility issue: vLLM ≥0.9.0 requires PyTorch 2.7, but Gaudi stack supports only PyTorch 2.6.0

[cygwin]

Hello Intel Gaudi Team,

I am currently working with Intel Gaudi accelerators and attempting to integrate them with vLLM, a popular open-source LLM inference engine.
During my integration, I have identified a compatibility gap between the Gaudi supported PyTorch version and the requirements of vLLM:

Environment

• Gaudi SW stack version: 25.0.1

• PyTorch version (Gaudi official support): 2.6.0 (per Support Matrix)

• vLLM versions tested: 0.8.x (works), 0.9.0 and above (fails)

Issue

• Starting from vLLM 0.9.0, the project requires PyTorch 2.7.x.

• This was introduced after PR #18056, which removed legacy typing.List usage in favor of Python 3.9+ native list[...] annotations.

• In PyTorch 2.6.0, some functions (e.g., infer_schema) still expect List[Tensor].

• With vLLM ≥0.9.0 (built against torch 2.7.x), only list[Tensor] signatures exist.

• This mismatch causes schema inference errors and prevents vLLM from running on Gaudi with PyTorch 2.6.0.

Impact

• Users on Gaudi cannot run the latest vLLM (0.9.x and newer).

• Only older versions (≤0.8.x) or Habana’s internal fork are usable, which prevents access to important features and fixes from the mainline vLLM project.

Request

1. Could you please share if there is a roadmap or plan for Gaudi to support PyTorch 2.7.x?

2. If not yet planned, would it be possible to provide a temporary patch/workaround for PyTorch 2.6.0 compatibility with vLLM ≥0.9.0?

3. Any guidance for the community on the recommended path forward would be very helpful.

Thank you for your support.

[MyLinhG]

Hello cygwin,

Thank you for your patience. PyTorch 2.7 is supported (per Support Matrix) on Intel Gaudi SW version 1.22.0. 

Please confirm that you are seeing it as well.

thank you,