Hardware limitations on USB endpoints (XHCI)

[DCox4]

Hi all.

I would like to know how to find information about the number of endpoints that an intel device can support.

We currently use the C1107U and the J1900I-C

We prefer mini pc type setups for portability of our products.

[edit] I have also tested this on my Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz and finding the same issue

We have noticed that windows 8 is able to support at least 18 connected devices, plus several hubs. We have noticed that Debian systems, however, have issues at about 12 devices and we see that the XHCI driver is reaching its endpoint limitation. We guess that this limitation is 64 endpoints (since hubs can consume 4 or 5 endpoints each)

I have spoken to the Linux kernal developers and they state that "Intel Ivy Bridge system xhci host, the 64 endpoint is a hardware limitation."

I have been able to disable the XHCI driver and force a rollback to the EHCI driver. This allows me to attach many more devices, but I am concerned that this is not a "safe" trick.

This also makes us concerned that the windows driver is may be cheating as well.

Our devices are all USB1.1

We run them on USB3 external powered industrial hubs.

Our devices are custom, so we can change the USB descriptors as necessary.

Currently we ask for (per device), 2 end points, less than 10mA and less than 12mb/sec.

so my Questions:

1. Is it "ok" to run EHCI, given the limitations observed in XHCI? Does EHCI have more endpoints? and why?

2. What options do we have to get more devices connected? We plan to further scale up our operations. For example "i5, i7" etc.

3. What do we need to look at in the chip-specifications in order to be able to know what the limitations actually are?

4. A lot of quick response to my questions regarding this matter have been "USB supports 127 connected devices" and "perhaps you are hitting a current limitation". Is this really true, given the above information?

[Adolfo_S_Intel]

Hello Spirail,

Thank you for your inquiry. We are investigating your request, and we'll reply again soon with additional information.

Regards,

Adolfo

[Adolfo_S_Intel]

Hello Spirail

I hope this information address your concerns:

1. Is it "ok" to run EHCI, given the limitations observed in XHCI? Does EHCI have more endpoints? and why?

• Yes, you can run EHCI without problems. There are potentially more endpoints with EHCI; for every one XHCI connected device you could instead have two EHCI devices. The XHCI limitation is based on system memory, it is not a limitation of the USB specification.

2. What options do we have to get more devices connected? We plan to further scale up our operations. For example "i5, i7" etc.

• Newer platforms allow for two root hubs; that would potentially provide you with 127 connected devices per root hub.

3. What do we need to look at in the chip-specifications in order to be able to know what the limitations actually are?

• The limitation will be the amount of system memory, along with the USB specification for 127 connected devices per root hub.

4. A lot of quick response to my questions regarding this matter have been "USB supports 127 connected devices" and "perhaps you are hitting a current limitation". Is this really true, given the above information?

• Yes, that information is correct.

Regards,

Adolfo

[DCox4]

Please remember that we are not talking about "theoretical" maximums supported by the USB standard,

but REAL limitations based on the chip or chip-set.

Sating 127 "is the supported amount" is like stating that a ticketing system can support a 1000 tickets per show, but the venu still only fits 100 people.

My specific responses are here:

1. Is it "ok" to run EHCI, given the limitations observed in XHCI? Does EHCI have more endpoints? and why?

• Yes, you can run EHCI without problems. There are potentially more endpoints with EHCI; for every one XHCI connected device you could instead have two EHCI devices. The XHCI limitation is based on system memory, it is not a limitation of the USB specification.

So. Since we have observed that XHCI is limiting at 64 endpoint. I can expect the EHCI to limit at about 128 endpoints?

Since devices are typically at least 2 endpoints and hubs are about 4 or 5, i can expect a DEVICES limitation of around 50 devices. (no where near 127 devices)

2. What options do we have to get more devices connected? We plan to further scale up our operations. For example "i5, i7" etc.

• Newer platforms allow for two root hubs; that would potentially provide you with 127 connected devices per root hub.

Does this increase the amount of system memory? Based on 1. I assume I would not get any more endpoint, hence this setup does NOT really allow and more real devices.

3. What do we need to look at in the chip-specifications in order to be able to know what the limitations actually are?

• The limitation will be the amount of system memory, along with the USB specification for 127 connected devices per root hub.

Since the limitation is the system memory. (and has nothing to do with the number 127)

Where can I look to see what the system memory is? Where does it show exactly how much is allocated to USB memory? Is it shared with other resources?

I want to be able to make a calculation on how many endpoints can be supported.

Summary.

What I really need now is to confirm that EHCI in a stable way supports twice as many endpoints as XHCI. (where does it say this in the documentation / standards?)

How to get an actual number of supported endpoints for an intel chip.

Would OHCI and UHCI allow for more endpoints? Is it possible to do this?

Finally. the C1107U does not support disabling XHCI in the bios. Is there another way to force a rollback to EHCI in Linux?

[Gabriel_T_Intel]

Hello Spirail.

Thank you very much for your patience, we are investigating your request, and we'll reply again soon with additional information.

Best Regards.

G. Thomas

[DCox4]

Thanks good to hear!

[Adolfo_S_Intel]

Hello Spirail

Sorry for the delay in the response. Here is the feedback that we've got from one of our experts.

First Limitation

Fundamentally, the customer is correct. You will never be able to achieve 127 actual USB devices attached to a Host Controller (i.e. Intel system). As the customer pointed out, each USB device (USB key, keyboard, mouse, etc.) is typically counted as two endpoints (two logical USB units), and each USB hub, multiplier, or repeater is counted as another 4+ endpoints. So, it comes to about 50+ devices per each Host Controller.

It's because of this reason that Intel added a second USB Host Controller in the majority of its Core platform chipsets. If you look at the Features page of the 8 Series Chipset ( when partnered with the 4th Gen Intel Core processor), page 39, http://www.intel.com/content/www/us/en/chipsets/8-series-chipset-pch-datasheet.html http://www.intel.com/content/www/us/en/chipsets/8-series-chipset-pch-datasheet.html, it states having "Two EHCI Host Controllers, supporting up to fourteen external USB 2.0 ports". So, with two Host Controllers, each at 50+ device, you have a potential of connecting up to 100+ USB devices per 4th Gen Core platform system.

Note that this is not the same for the Intel Atom processor Family, including the Celeron J1900 processors. This only has a single USB Host Controller, so you are limited back down to 50+ devices.

Second Limitation

So, why 127 devices? This number is based on the address size used by USB devices. The USB Specification states that a USB packet will have 7 address bits. Calculating 2^7 will give all the address location, subtract 1 for the Host Controller. So, 2^7 – 1 = 127 devices possible on each USB Host Controller.

During system BIOS initialization and configuration, before Windows, Linux, or any Operating System ever launches, 4KB of system memory will be allocated for each potentially attached USB endpoint (4KB x 128 = 512KB). Note that this is per endpoint, not per device. This memory allocation assures that system memory will be available for all USB endpoint / devices inserted during system runtime. However, the 4KB is based on USB 2.0 endpoints (EHCI). All USB 3.0 endpoints (XHCI) require 8KB of system memory allocation. Plus, each USB 3.0 device is allocated XHCI endpoints and EHCI endpoints, for backwards compatibility. Eventually, you will reach a limitation where insufficient memory has been allocated for your USB devices.

As mentioned before, this 512KB allocation is performed by system BIOS. To change this allocation (higher or lower) you would need to work with your Independent BIOS Vendor (IBV). This is not an end-user modification, or overcome by changing BIOS settings. The IBV will need to make changes to their BIOS code, recompile the code, and install on your system.

Could you provide us with more information regarding the C1107U, we haven't been able to find any Intel part with that number.

If it is a third party motherboard, can you give us more detail about what Intel hardware is using this motherboard?

Have you consulted with the manufacturer of the board about how to disable XHCI?

I hope this is useful for you

Best Regards,

Adolfo.

[DCox4]

Hi AdolfoS

Its been a long time, I hope that is ok. Our company has been focusing on other areas that do not hit these limitations, however, recently this area has come up again for us. So I have a few more questions in order to help me to ask the right things from motherboard manufacturers.

1. This "bios initialization of memory of usb end points". Is there a specific name for that? Its really tricky to get more information about it. Where does the memory come from? is it a RAM allocation (the common stuff we all know about) or some sort of "other" system memory. ie: onchip sram, registers, etc.

What I'm getting at is "is there some sort of hardware specific limitation" or can you really just customize a bios to use more of the 2Gb (or whatever) ram and allocate it to get more xhci drivers.

2. When there are "two ehci hosts" does that mean that twice as much memory will be allocated in the bios? I know that this comes down to the motherboard manufacturer, however I'm "guessing" that its unlikely that they do anything about allocating the extra memory since the requirement for this many devices is uncommon.

3. It seems a little strange that the bios allocates memory for usb2.0 endpoints, yet supports usb3 devices and hence the allocation gets halved. Do you think that newer machines will allocate more memory?

Also, we have some systems here were we can disable xhci in the bios forcing a rollback to ehci. It helps, but it difficult to buy the right machine at the right time. We have even found that newer computers are less likely to have this option.

An interesting side note is that raspberry-pi allows you to force even UHCI drivers, however there are still memory limitations.

Best regards

David

[DCox4]

A further note on all this.

I talked to some bios developers about this.

As well as the above questions, it would be nice to see if you agree on the following points.

1. there is no limitation in bios or OS that allocates how much memory you have. Its 100% a hardware problem.

2. bios "does" some memory allocation routines, but its only for "itself" to use when its running. It has nothing to do with the os.

3. there are some chip-sets that even have 2 (and even 3) ehci controllers, but I need to actually make sure I am distributing my devices across several controllers to take advantage of this.

4. taking advantage of extra ehci controllers means knowing the hardware architecture. Ie: controllers are physically allocated to different sets of usb ports. I may also need to buy a machine and at a PCI based host controller to get access to this.

[Adolfo_S_Intel]

Hello David

We are working on your issue and will reply back with more information as soon as possible.

Best Regards,

Adolfo

[Natalie_Z_Intel]

Hi, Spirail! The information you need requires a non-disclosure agreement to exist. Intel's policy does not allow a CNDA to be issued to a person, only a company (legal entity). You may also consider working with an http://www.intel.com/content/www/us/en/intelligent-systems/intelligent-systems-distributors.html Intel distributor who may be able to assist you with your project. They will still require a CNDA though. LynnZ.

[DCox4]

ok. my email is mailto:david.cox@justtheengineer.com david.cox@justtheengineer.com

send me an email and ill give you my company details.

/David

[Natalie_Z_Intel]

EDC Support (mailto:edc.support@intel.com edc.support@intel.com) will help you to apply for a Privileged account. Be on the lookout for an email for that team. When applying use your company email address. The online sales team will then contact you to discuss your project, etc, to determine the eligibility in establishing a CNDA between your company and Intel. If approved, then you will be able to access content that is classified as Intel Confidential. LynnZ

[RGrim2]

David and LynnZ - We have been struggling with the same issues that David is having. The limited number of USB 3.0 devices is not normally an issue if the computer has adequate USB 2.0 ports. We are seeing commercial computers, however, that don't have any 3.0 support at all. This is particularly common with the new small format computers and some touch screen computers. Some PC BIOS allow the user to disable the USB 3.0 ports but this disables the entire port, 2.0 and 3.0 support. Desktop based installations could take advantage of a PCI based USB 2.0 card but this is not an option on this group of computers that lack a user accessible PCB bus.

I have applied for a Privileged account and sent an email to EDC Support. Any additional input you would provide would be appreciated.

Rich

[CarlosAM_INTEL]

Hello Rich,

Thank you for contacting the Intel Embedded Community.

In order to better understand your consultation, we would like to address the following questions:

Could you please clarify if the platforms related to this situation are your project or a third party design?

In case that the affected design is yours, could you please tell us what is the platform (processor and chipsets) related to this condition?

If the affected is a third party board, could you please let us know the name of the manufacturer, the model, and all the documentation related to it?

Please give us all the requested information to help you in the best way possible.

Thanks in advance for your cooperation.

Best Regards,

Carlos A.

[idata]

Hi Rich,

We understand that you have been contacted by your account manager, Roger, to further discuss your project and assist with establishing a CNDA, if needed. I noticed that Carlos asked you a few questions but you and Roger may have already discussed this.

-Scott

[Natalie_Z_Intel]

Hey, Rich! I understand that your account manager, Roger, was able to have a conference call and get you started with the things you need. Thanks so much for using the Embedded Community to find technical assistance. Good luck with your project!! LynnZ.

[JLube]

Our customer has run into this same limitation of USB 3.0 ports having limited number of endpoints. We hope to recommend to him that he can solve the issue by forcing the USB 3.0 ports to use the EHCI driver instead of the xHCI driver. How would we force Win 7 USB 3.0 ports to load / use only the EHCI driver. (Same question for Win 8.1 and Win 10.)

Thanks,

[CarlosAM_INTEL]

Hello jlubeck,

Thank you for contacting the Intel Embedded Community.

The driver suggestion is related to Windows 7 because the eHCI driver is used by default. Due to this fact, please uninstall the xHCI driver.

On the other hand, the Windows 8.1 or 10 drivers by default is the xHCI. We suggest you to contact the Operating Systems developer to further help you with this situation, or you can contact your BIOS manufacturer to verify if there is a BIOS setting that can help you to solve this inconvenience.

Please let us know if this information is useful to you.

Best Regards,

Carlos A.

[JLube]

HI Carlos,

On my desktop system, I have a Renesas USB 3.0 host controller running Win 7 - 64 bit. We uninstalled the xHCI driver and then did a "update driver software" and selected "Search automatically" but the system did not load an EHCI driver. We've seen some blogs / forums that it was done in Linux, but we have not confirmed that ourselves.

Thanks,

John

[CarlosAM_INTEL]

Hello jlubeck,

Thanks for your update.

Could you please let us know all the information of the desktop that you have mentioned? Please include the name of the manufacturer, model of the motherboard, and other info that you consider important.

Thanks in advance for your collaboration.

Best Regards,

Carlos A.