How to monitor NPU ram usage and utilization.

[Arya_Pulkit]

How to track the NPU RAM usage and utilization in a  windows system when inference through an LLM model?

[DeancR_Intel]

Hi Arya_Pulkit,

Thank you for posting in the Community!

To better assist you with tracking NPU RAM usage and utilization during inference with a large language model (LLM), could you please provide some details about your system? Here are a few questions to get started:

• System Configuration:
  • What is the make and model of your NPU?
  • What is the make and model of your CPU and GPU?
  • How much RAM does your system have?
• Operating System:
  • Which version of Windows are you using?
• Any steps you have tried to resolve this?
• Current application you're using to monitor the usage.

Best regards,

Dean R.

Intel Customer Support Technician

[Arya_Pulkit]

• System Configuration:
  • What is the make and model of your NPU? - Intel(R) AI Boost
  • What is the make and model of your CPU and GPU? CPU: Intel® Core™ Ultra 7 - 155H /GPU: Intel(R)Arc(TM)Graphics
  • How much RAM does your system have? -16 GB
• Operating System:
  • Which version of Windows are you using? - Microsoft Windows 11 Pro

• Any steps you have tried to resolve this?
  • Still trying to figure out how can I monitor realtime ram usage and utilization.
• Current application you're using to monitor the usage.
  • I am trying to run Lama and  monitor resource consumption.

[DeancR_Intel]

Hi Arya_Pulkit,

This is an acknowledgement message for your case, do please wait as this concern will be investigated internally to be able to provide a resolution to your request. I will get back to your once I have the necessary information.

Best regards,

Dean R.

Intel Customer Support Technician

[DeancR_Intel]

Hi Arya_Pulkit,

For more detailed instructions, you can refer to this article: How to Check If My Intel® Processor Has an Integrated Neural...

Could you please let me know why you want to monitor the NPU RAM usage and utilization? Are you experiencing any specific issues with your computer?

Additionally, could you provide the make and model of your computer? If possible, please also share the SSU (System Support Utility) logs. This information will help me assist you more effectively.

You can find the download link here:

• Intel® System Support Utility for Windows*
• Here's a quick guide on using the Intel SSU:

1. Download and launch SSU.exe.
2. Check the box for "Everything."

• When the scanning is complete, click "Next."
• Click "Save."
• Please send the saved file to us.

Best regards,

Dean R.

Intel Customer Support Technician

[DeancR_Intel]

Hi Arya_Pulkit,

I wanted to follow up on the previous message regarding your inquiry. Have you had a chance to review it? If you have any questions or need further information, please let me know.

Best regards,

Dean R.

Intel Customer Support Technician

[DeancR_Intel]

Hi Arya_Pulkit,

I have not heard back from you so I will close this inquiry now. If you need further assistance, please submit a new question as this thread will no longer be monitored.

Best regards,

Dean R.

Intel Customer Support Technician