Intel® Collaboration Suite for WebRTC
Community support and discussions on the Intel® Collaboration Suite for WebRTC (Intel® CS for WebRTC).
Announcements
Welcome to the Intel Community. If you get an answer you like, please mark it as an Accepted Solution to help others. Thank you!
For the latest information on Intel’s response to the Log4j/Log4Shell vulnerability, please see Intel-SA-00646

mcu crashing

Stephen_-_
New Contributor I
178 Views

I have been leaving my webcam on overnight connected to my server to test the stability of release 3.0 update 2 on a ubuntu 14.04.1 VPS

It seems to eventually freeze, and need a reboot, when I do "bin/stop-all" it states "no mcu to stop" which makes me think the problem may be in the mcu.

I'm not really sure if I'm reading the logs right but I hope they could provide some answers. I wish at the least there was a way of detecting when a part of the application has failed and maybe automatically reboot rather than just stay crashed. I haven't modified the server at all yet and it's just using the basic example.

Some parts in the logs:

There's a really long 11MB erizo log file and towards the end it says:

2016-03-01 10:53:55.825  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:54:00.828  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:54:05.830  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:54:14.744  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:54:15.831  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:54:20.832  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:54:25.835  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:54:30.836  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:55:20,929  - WARN: WebRtcConnection - Bad source SSRC in RTCP feedback packet: 1165194345
2016-03-01 10:56:35,404  - WARN: WebRtcConnection - Bad source SSRC in RTCP feedback packet: 2228027394
2016-03-01 10:56:35.407  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:56:35.408  - DEBUG: ErizoJS - KeepAlive from ErizoController
2016-03-01 10:56:35.410  - INFO: ErizoJSController - Removing muxer /#EuASka3hdxdipU7CAAAH
2016-03-01 10:56:35,424  - DEBUG: mcu.media.SoftVideoCompositor - Configuring layout
2016-03-01 10:56:35,424  - DEBUG: mcu.media.SoftVideoCompositor - configChanged is true
2016-03-01 10:56:35,424  - DEBUG: mcu.Mixer - notifyAsyncEvent - event: UpdateStream
2016-03-01 10:56:35,424  - DEBUG: mcu.WebRTCGateway - closeAll
2016-03-01 10:56:35,424  - INFO: WebRtcConnection - WebRtcConnection Destructor
2016-03-01 10:56:35,425  - INFO: NiceConnection - video - NICE State Changing from 2 to 3 0x54c9ea0
2016-03-01 10:56:35,451  - DEBUG: mcu.media.SoftVideoCompositor - commit customlayout
2016-03-01 10:56:35,451  - DEBUG: mcu.media.SoftVideoCompositor - configChanged sets to false after commitLayout!
2016-03-01 10:56:35.656  - INFO: ErizoJSController - Removing subscribers /#EuASka3hdxdipU7CAAAH
2016-03-01 10:56:35.657  - INFO: ErizoJSController - Removing publisher /#EuASka3hdxdipU7CAAAH
2016-03-01 10:56:35.658  - INFO: ErizoJSController - Publishers: 1
2016-03-01 10:56:35.660  - INFO: ErizoJSController - Removing muxer 56cf4791da728cad1bc2cd28
2016-03-01 10:56:35.661  - INFO: ErizoJSController - Iterating and closing  /#EuASka3hdxdipU7CAAAH { '/#EuASka3hdxdipU7CAAAH': { wrtc: WebRtcConnection {}, resolution: undefined } } { wrtc: WebRtcConnection {}, resolution: undefined }
2016-03-01 10:56:35,678  - DEBUG: mcu.Mixer - closeAll
2016-03-01 10:56:35,679  - INFO: WebRtcConnection - WebRtcConnection Destructor
2016-03-01 10:56:35,679  - INFO: NiceConnection - video - NICE State Changing from 2 to 3 0x49c8740
2016-03-01 10:56:35,857  - DEBUG: mcu.media.VideoMixer - closeAll
2016-03-01 10:56:36,192  - DEBUG: mcu.media.VideoMixer - Closed all media in this Mixer
2016-03-01 10:56:36,212  - DEBUG: mcu.media.BufferManager - BufferManager destroyed
2016-03-01 10:56:36.212  - INFO: ErizoJSController - Removing subscribers 56cf4791da728cad1bc2cd28
2016-03-01 10:56:36.213  - INFO: ErizoJSController - Removing publisher 56cf4791da728cad1bc2cd28
2016-03-01 10:56:36.213  - INFO: ErizoJSController - Publishers: 0
2016-03-01 10:56:36.213  - INFO: ErizoJSController - Removed all publishers. Killing process.

I don't know if that's relevant or not.

In woogeen-mcu.stdout it says towards the end:

2016-03-01 10:55:10.296  - INFO: RoomController - Adding mixer id  56cf4791da728cad1bc2cd28
2016-03-01 10:55:10.397  - INFO: RoomController - Adding mixer id  56cf4791da728cad1bc2cd28
2016-03-01 10:55:10.499  - INFO: RoomController - Adding mixer id  56cf4791da728cad1bc2cd28
2016-03-01 10:55:10.601  - INFO: RoomController - Adding mixer id  56cf4791da728cad1bc2cd28
2016-03-01 10:55:10.705  - INFO: RoomController - Adding mixer id  56cf4791da728cad1bc2cd28
2016-03-01 10:55:10.807  - INFO: RoomController - Adding mixer id  56cf4791da728cad1bc2cd28
2016-03-01 10:55:10.908  - INFO: RoomController - Adding mixer id  56cf4791da728cad1bc2cd28
2016-03-01 10:55:10.909  - DEBUG: RoomController - Answer timeout
2016-03-01 10:55:10.910  - INFO: RoomController - Erizo created
/root/intel/Release-v3.0.1/mcu/erizoController/roomController.js:1
(function (exports, require, module, __filename, __dirname) { "use strict";var logger=require("./../common/logger").logger,erizoController=require("./erizoController"),log=logger.getLogger("RoomController");exports.RoomController=function(e){var i={},o={},r={},n={},t={},l=e.amqper,a=e.agent_id,u=e.id,c=5e3,s=2,d=[],f=function(e){return function(i){if(n)if(i!==!0){if(n.ka_count++,n.ka_count>s){for(var o=n.publishers,r=o.length-1;r>=0;--r)g("unpublish",o);l.callRpc("ErizoAgent_"+a,"deleteErizoJS",,{callback:function(){}}),delete n}}else n.ka_count=0}},v=function(){for(var e in n)l.callRpc("ErizoJS_"+e,"keepAlive",[],{callback:f(e)})};setInterval(v,c);var p=function(e,i){return GLOBAL.config.erizoController.outOfProcessMixer||void 0===e||void 0===r?void l.callRpc("ErizoAgent_"+a,"createErizoJS",[],{callback:function(e){log.debug("Answer",e),"timeout"===e||n||(n={publishers:[],ka_count:0}),i(e),l.callRpc("ErizoJS

TypeError: Cannot read property 'publishers' of undefined
    at /root/intel/Release-v3.0.1/mcu/erizoController/roomController.js:1:1383
    at Object.GLOBAL.config.erizoController.outOfProcessMixer.r.(anonymous function).l.callRpc.callback (/root/intel/Release-v3.0.1/mcu/erizoController/roomController.js:1:933)
    at callbackError (/root/intel/Release-v3.0.1/mcu/common/amqper.js:1:2752)
    at ontimeout [as _onTimeout] (timers.js:195:34)
    at Timer.listOnTimeout (timers.js:92:15)

in woogeen-nuve.stdout it says:

2016-03-01 07:42:32.773  - INFO: Room - apply custom layout templates
2016-03-01 10:54:56.114  - INFO: CloudHandler - ErizoAgent 2  in  104.233.83.134 does not respond. Deleting it.
2016-03-01 10:54:56.118  - INFO: CloudHandler - [CLOUD HANDLER]: Warning! No erizoAgent is available.
2016-03-01 10:55:21.151  - INFO: CloudHandler - ErizoController 1  in  104.233.83.134 does not respond. Deleting it.
2016-03-01 10:55:21.152  - INFO: CloudHandler - [CLOUD HANDLER]: Warning! No erizoController is available.
2016-03-01 10:55:21.152  - INFO: CloudHandler - [CLOUD HANDLER]: Warning! No erizoAgent is available.
2016-03-01 10:56:35.458  - INFO: CloudHandler - I received a keepAlive mess from a removed erizoController
2016-03-01 10:56:36.029  - INFO: CloudHandler - [CLOUD HANDLER]: ErizoController in host  104.233.83.134 does not respond.

Any idea what might be the issue? Is there an easy way of being notified when this happens so I can at least reboot it? I'm worried this could happen when I'm not there and then it will be unusable.

0 Kudos
5 Replies
Sergey_P_1
Beginner
178 Views

We have the same problem with crashing MCU server and reboot it every day.

Chunbo_H_Intel1
Employee
178 Views

Hi guys,

Thanks for the feedback. Our engineering team is now working on the solution. Thanks!

Bryan_A_
Beginner
178 Views

Hello !
Let these settings, and as the number of rooms and the number of clients:
erizoAgent.maxProcesses
erizoAgent.prerunProcesses

2 weeks ago, I had the same problem.
And now, I seem to have found a solution:
I increased the value erizoAgent.maxProcesses (rooms * 2)
and erizoAgent.prerunProcesses (rooms + 2)

Chunbo_H_Intel1
Employee
178 Views

Thanks for the sharing!

We've identified the issue of "TypeError: Cannot read property 'publishers' of undefined
    at /root/intel/Release-v3.0.1/mcu/erizoController/roomController.js:1:1383
    at Object.GLOBAL.config.erizoController.outOfProcessMixer.r.(anonymous function).l.callRpc.callback (/root/intel/Release-v3.0.1/mcu/erizoController/roomController.js:1:933)
    at callbackError (/root/intel/Release-v3.0.1/mcu/common/amqper.js:1:2752)
    at ontimeout [as _onTimeout] (timers.js:195:34)
    at Timer.listOnTimeout (timers.js:92:15)".

And we had some fix against it in the coming release!

Stay tuned!

software__revamp
Beginner
178 Views

If help needed, please contact me softwarevamp@gmail.com.

Reply