SYCL for llama.cpp.python

Pre2802 — Thu, 04 Apr 2024 05:52:29 GMT

I'm trying to use SYCL as my hardware acclerator for using my GPU in Windows 10
My GPU is

I have installed intel OneAPI toolkit.
Im not able to use my GPU despite doing the following commands in command prompt
1. I ran my setvars.bat file in C:\Program Files (x86)\Intel\oneAPI directory
2. set CMAKE_ARGS="-DLLAMA_SYCL=on -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx"
3. set FORCE_CMAKE=1
4. pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose

The llama-cpp-python gets installed properly but I'm getting error when am loading the import statement

Python 3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from llama_cpp import Llama Traceback (most recent call last): File "C:\Python\Python311\site-packages\llama_cpp\llama_cpp.py", line 70, in _load_shared_library return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\ctypes\__init__.py", line 376, in __init__ self._handle = _dlopen(self._name, mode) ^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: [WinError 193] %1 is not a valid Win32 application During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python\Python311\site-packages\llama_cpp\__init__.py", line 1, in <module> from .llama_cpp import * File "C:\Python\Python311\site-packages\llama_cpp\llama_cpp.py", line 83, in <module> _lib = _load_shared_library(_lib_base_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Python\Python311\site-packages\llama_cpp\llama_cpp.py", line 72, in _load_shared_library raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}") RuntimeError: Failed to load shared library 'C:\Python\Python311\site-packages\llama_cpp\llama.dll': [WinError 193] %1 is not a valid Win32 application

Re: SYCL for llama.cpp.python

Alex_Y_Intel — Tue, 09 Apr 2024 00:03:37 GMT

This doesn't look like a compiler issue. This forum is for questions related to Intel DPC++/C++ compiler.

topic Re: SYCL for llama.cpp.python in Intel® oneAPI DPC++/C++ Compiler

SYCL for llama.cpp.python

Re: SYCL for llama.cpp.python