Intel® Collaboration Suite for WebRTC
Community support and discussions on the Intel® Collaboration Suite for WebRTC (Intel® CS for WebRTC).
Announcements
Welcome to the Intel Community. If you get an answer you like, please mark it as an Accepted Solution to help others. Thank you!
For the latest information on Intel’s response to the Log4j/Log4Shell vulnerability, please see Intel-SA-00646
1136 Discussions

Intel webrtc version 4 stream subscription failed above 128 subscribers

Nithin_P_
New Contributor I
470 Views

Hi

                  Today i have done a load test on intel mcu version 4. two presenter published and viewers subscribing mixed stream.when the viewers count reach above 128 then no new users can subscribe the stream. when new users joined then showing an error in the console. i have attached the error.but this error not affecting existing subscribers but unable to subscribe for new users.I think this is a very serious issue for large scale applications

I have attached the console error. please review 

invermcu version :  4.0

server : centos7

used cluster with multiple webrtc agents

browser used : google chrome latest version

Thank you,

0 Kudos
20 Replies
Kiran_Raj_RA
Beginner
466 Views

i am also facing the same issue when using cluster

Uncaught (in promise) Error: roomController.subscribe failed, reason: Failed in adding output to audio-engine
    at new t (ics.js:1497)
    at t.value (ics.js:1176)
    at t.value (ics.js:834)
    at O (ics.js:1311)
    at ics.js:1384
    at ics.js:148
    at Array.map (<anonymous>)
    at t.dispatchEvent (ics.js:147)
    at r.<anonymous> (ics.js:1719)
    at r.emit (bundle.js?t=3543453:8916)
    at r.onevent (bundle.js?t=3543453:8916)
    at r.onpacket (bundle.js?t=3543453:8916)
    at r.<anonymous> (bundle.js?t=3543453:8916)
    at r.emit (bundle.js?t=3543453:8916)
    at r.ondecoded (bundle.js?t=3543453:8916)
    at s.<anonymous> (bundle.js?t=3543453:8916)

Nithin_P_
New Contributor I
466 Views

Hi,

            I have tested again , about 175 to 180 users this issue happening. during this range the cpu utlization of the server which running audio agent become very high.I think this issue is not present in previous versions. may be this is due to the new update - audio mixing feature 

Thank you,

Nithin_P_
New Contributor I
466 Views

Hi,

This issue happening freequently. when we restart audio agent then the issue resolved and new users can subscribe 

Kiran_Raj_RA
Beginner
466 Views

2018-05-04 13:09:08,569  - WARN: mcu.media.AcmmFrameMixer - No free Id, max participants reached(128)!
2018-05-04 13:09:08,569  - WARN: mcu.media.AcmmFrameMixer - Can not add output participant

Nithin_P_
New Contributor I
466 Views

Hi,

            Again tested 

On  128 th subscriber following errors logged in intel 4.0 server

Error logged In Audio agent

WARN: mcu.media.AcmmFrameMixer - No free Id, max participants reached(128)!
WARN: mcu.media.AcmmFrameMixer - Can not add output participant

Error logged In Conference agent

2018-05-04 15:08:58.546  - INFO: Conference - roomController.subscribe failed, reason: Failed in adding output to audio-engine
2018-05-04 15:08:58.546  - INFO: Conference - Exception: roomController.subscribe failed, reason: Failed in adding output to audio-engine
2018-05-04 15:09:00.144  - ERROR: RoomController - subscribe failed, reason: Failed in adding output to audio-engine
2018-05-04 15:09:00.144  - INFO: Conference - roomController.subscribe failed, reason: Failed in adding output to audio-engine
2018-05-04 15:09:00.144  - INFO: Conference - Exception: roomController.subscribe failed, reason: Failed in adding output to audio-engine
2018-05-04 15:09:00.790  - ERROR: RoomController - subscribe failed, reason: Failed in adding output to audio-engine
2018-05-04 15:09:00.790  - INFO: Conference - roomController.subscribe failed, reason: Failed in adding output to audio-engine
2018-05-04 15:09:00.791  - INFO: Conference - Exception: roomController.subscribe failed, reason: Failed in adding output to audio-engine

Error logged  browser console

Uncaught (in promise) Error: roomController.subscribe failed, reason: Failed in adding output to audio-engine
    at new t (ics.js:1497)
    at t.value (ics.js:1176)
    at t.value (ics.js:834)
    at O (ics.js:1311)
    at ics.js:1384
    at ics.js:148
    at Array.map (<anonymous>)
    at t.dispatchEvent (ics.js:147)
    at r.<anonymous> (ics.js:1719)
    at r.emit (bundle.js?t=3543453:8916)
    at r.onevent (bundle.js?t=3543453:8916)
    at r.onpacket (bundle.js?t=3543453:8916)
    at r.<anonymous> (bundle.js?t=3543453:8916)
    at r.emit (bundle.js?t=3543453:8916)
    at r.ondecoded (bundle.js?t=3543453:8916)
    at s.<anonymous> (bundle.js?t=3543453:8916)

Final result is no subscribers after 128 th user can subscribe stream from the room.existing subscribers have no issues

intel webrtc version : 4.0

server : centos 7.2

browser : chrome latest version

Thank you,

Qiujiao_W_Intel
Employee
466 Views

Thanks Nithin and Kiran for pointing this issue, we can reproduce it locally and we are working on this issue, will get back to you when it's fixed.

Kiran_Raj_RA
Beginner
466 Views

Hi Qiujiao W,

Thanks for the reply, We deployed conference server cluster with the below cluster strategy

we have 2 webrtc,audio,video agents  with cluster strategy least-used 

1. In webrtc agent the load is eventually distributing and least-used working perfect.
2. In audio and video agent least-used not working and the load is coming to a single server only.

 

Qiujiao_W_Intel
Employee
466 Views

Hi, Kiran, we cannot reproduce least-used policy not working for audio/video agent issue, could you provide more detail info on your scenario and testing environment? and is there any error log in MCU logs?

Nithin_P_
New Contributor I
466 Views

Hi Qiujiao,

                 When we can expect the fix. we are absolutely stuck. All our events have min 1000 to 2000 subscribers so that we cant upgrade the version. 128 user limit is a very small figure for a webinar application.

Thank you,

Qiujiao_W_Intel
Employee
466 Views
Nithin, we are working on this issue now and will get back to you ASAP. BTW, did you successfully subscribe 1000 to 2000 mix stream with previous version?
Nithin_P_
New Contributor I
466 Views

HI,

No, We have checked maximum up to 500 subscribers and 3 publishers  in a single room ( mix = true ) on version 3.4.1, 

Limitation we faced in previous versions 

1. due to ports not releasing issue in webrtc agent(webrtc stability issue) 
sometimes the users cant enter into the room so we used 5 to 10 webrtc agents and round-robin 
cluster stratergy for distributing webrtc hits so they can enter in the room(sometimes it fail) in version 3.4.1

2. The major challenge is when some one start a screen share the cpu hits a huge value in webrtc agent because screenshare video is not mixed so each user will subscribe it.

3. audio video sync in mixed stream as well as recorded video is another issue in 

Currently we are in beta version. when move to live there will be 1000 to 2000 participant subscribe mixed stream ( based on the current flash version running on live ). but publisher count will be three or four.

Nithin_P_
New Contributor I
466 Views

Hi Qiujiao,

Did you fix the issue. Do we need to wait for next version ?

Qiujiao_W_Intel
Employee
466 Views

Yes, we will have a minor version release which will fix this issue soon.

Kiran_Raj_RA
Beginner
466 Views

Hi, Qiujiao W,

When can we get the next release that fix 128 user issue, or is it possible to provide any other solution to add more users.

Kiran_Raj_RA
Beginner
466 Views

Hi, Qiujiao W,

We have done alive event with 5 publishers (audio only) mixed mode and 150 subscribers (mcu v3.4.1). the audio feels like slow motion and missing, so what will be if we use 4.0 is the same slow we feel or is it work better. 

Nithin_P_
New Contributor I
466 Views

Hi Qiujiao,

Any fix on 128 maximum user limit issue ?

Qiujiao_W_Intel
Employee
466 Views

Hi, Nithin, we will have a 4.0.1 release early next week which will fix 128 limit issue, stay tuned

Qiujiao_W_Intel
Employee
466 Views

HI, Kiran, audio missing issue in your scenario will also be fixed in 4.0.1.

Qiujiao_W_Intel
Employee
466 Views

Hi, Kiran and Nithin, our 4.0.1 package is available now, please have a try.

Qiujiao_W_Intel
Employee
271 Views

Please use 'viewer' role for subscribe-only user so that they will not trigger any publish related pipeline

Reply