Intel® Collaboration Suite for WebRTC
Community support and discussions on the Intel® Collaboration Suite for WebRTC (Intel® CS for WebRTC).
Announcements
Welcome to the Intel Community. If you get an answer you like, please mark it as an Accepted Solution to help others. Thank you!
For the latest information on Intel’s response to the Log4j/Log4Shell vulnerability, please see Intel-SA-00646

Cluster video has a black screen

Lawrence1
Beginner
656 Views

Hi Team,

Test scenario as below

Server1 : Is running: rabbitmq mongodb management-api cluster-manager portal  webrtc-agent conference-agent audio-agent streaming-agent recording-agent sip-agent management-console sip-portal app

(Comment out video agent start script in start-all.sh)

Server2: Is running: video-agent

(Comment out all other components start scripts except video agent in start-all.sh)

the video agent is configured with the correct rabbit-mq server and cluster ip_address.

I can start all the components correctly on all the nodes. The firewall is turned off and all ports are open

black screen:The problem is that there is no streaming video when running the video agent on Server2.

Logs follow:

Server1:logs/webrtc-xxx

2019-08-07 17:03:38.838  - INFO: WorkingNode - pid: 8548
2019-08-07 17:03:38.841  - INFO: WorkingNode - Connecting to rabbitMQ server...
2019-08-07 17:03:38.855  - INFO: AmqpClient - Connecting to rabbitMQ server OK, options: { host: '172.31.28.159', port: 5672 }
2019-08-07 17:03:38.882  - INFO: InternalConnectionFactory - QUIC is not enabled for internal IO
2019-08-07 17:03:38.892  - INFO: WorkingNode - webrtc-b2b8837850a94ff8ec5c@172.31.28.159_0 as rpc server ready
2019-08-07 17:03:38.895  - INFO: WorkingNode - webrtc-b2b8837850a94ff8ec5c@172.31.28.159_0 as monitor ready
2019-08-07 17:04:16,709  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:04:16,723  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:05:54,351  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:05:54,362  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:21,728  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:21,738  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:22,060  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:22,074  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:48,179  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:48,181  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:48,214  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:48,230  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:48,235  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:48,249  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:52,183  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:52,207  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:52,929  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:52,941  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:54,898  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:54,949  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:56,159  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:56,171  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:56,314  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:06:56,352  - ERROR: owt.RawTransport - Error establishing the TCP connection: Connection refused
2019-08-07 17:07:18.650  - WARN: WorkingNode - Exiting on SIGTERM

Server2: logs/video-xxx

2019-08-07 17:03:47.668  - INFO: WorkingNode - pid: 11337
2019-08-07 17:03:47.671  - INFO: WorkingNode - Connecting to rabbitMQ server...
2019-08-07 17:03:47.682  - INFO: AmqpClient - Connecting to rabbitMQ server OK, options: { host: '172.31.28.159', port: 5672 }
2019-08-07 17:03:47.694  - INFO: InternalConnectionFactory - QUIC is not enabled for internal IO
2019-08-07 17:03:47.704  - INFO: WorkingNode - video-cbd446b5e5aed9c3dbf9@172.31.28.162_0 as rpc server ready
2019-08-07 17:03:47.708  - INFO: WorkingNode - video-cbd446b5e5aed9c3dbf9@172.31.28.162_0 as monitor ready
2019-08-07 17:04:10,629  - INFO: mcu.media.VideoMixer - Init maxInput(16), rootSize(640, 480), bgColor(16, 128, 128)
2019-08-07 17:07:48.494  - WARN: WorkingNode - Exiting on SIGTERM

Thanks

 

0 Kudos
6 Replies
Li_C_Intel
Employee
656 Views
Hi, Can Server1 reach Server2 in the network?
Lawrence1
Beginner
656 Views

Hi,

I turned off the firewall.

Server1 can reach Server2 in the network.

Ping IP:

Ping 172.31.28.162
PING 172.31.28.162 (172.31.28.162) 56 (84) bytes of data.
64 bytes from 172.31.28.162: icmp_seq = 1 ttl = 64 times = 3.08 ms
64 bytes from 172.31.28.162: icmp_seq = 2 ttl = 64 times = 4.16 ms
64 bytes from 172.31.28.162: icmp_seq = 3 ttl = 64 time = 4.71 Ms.

Ping 172.31.28.159
PING 172.31.28.159 (172.31.28.159) 56 (84) bytes of data.
64 bytes from 172.31.28.159: icmp_seq = 1 ttl = 64 times = 0.191 ms
64 bytes from 172.31.28.159: icmp_seq = 2 ttl = 64 times = 0.257 ms
64 bytes from 172.31.28.159: icmp_seq = 3 ttl = 64 time = 0.175
Ms 64 bytes from 172.31.28.159: icmp_seq = 4 ttl = 64 times = 0.198 ms

The cluster configuration is as follows

Lawrence1
Beginner
656 Views

The cluster configuration is as follows.

Is there an error in the configuration?

 

Server1:

/management_api/management_api.toml

[server]
keystorePath = "./cert/certificate.pfx"
superserviceID = "5d41639bdf3e3812bae053ba"
ssl = true #if use HTTPS
port = 3000 #default: 3000, port of server
numberOfProcess = 4 #default: 1

[cluster]
name = "owt-cluster"

[rabbit]
host = "172.31.28.159" #default: "localhost"
port = 5672 #default: 5672

[mongo]
dataBaseURL = "172.31.28.159/owtdb" #default: "localhost/owtdb"

 

/cluster_manager/cluster_manager.toml

[rabbit]
host = "172.31.28.159" #default: "localhost"
port = 5672 #default: 5672

[manager]
name = "owt-cluster"

#The time for cluster manager getting ready to handle the incoming 'schedule' requirements.
initial_time = 6000 #ms

#The interval of checking the attendences of all cluster workers.
check_alive_interval = 1000 #ms

#The threshold count of consecutive absences after which the corresponding cluster workers will be deleted.
check_alive_count = 3

#The default reservation time if not specified in the scheduling requirement.
schedule_reserve_time = 60000 #ms

[strategy]
general = "last-used"
portal = "last-used"
conference = "last-used"
webrtc = "last-used"
sip = "round-robin"
streaming = "round-robin"
recording = "randomly-pick"
audio = "most-used"
video = "least-used"
analytics = "least-used"
 

/portal/portal.toml

[portal]
keystorePath = "./cert/certificate.pfx"
hostname = "" #default: ""
ip_address = "47.52.141.92" #default: ""
# Port that the socket.io server listens at.
port = 8080 #default: 8080
ssl = true #default: true
force_tls_v12 = false #default: false, tls1 and tls1.1 will not be allowed if set to true
#The network inferface the socket.io will be bound to in case portal.ip_address (the 'ip_address' item above) is not specified or equal to "".
# networkInterface = "eth1"
# How many seconds before Socket.IO client sending a new ping packet. This value might be overwrote by client side.
# ping_interval = 25 #default: undefined
# How many seconds without a pong packet to consider the Socket.IO connection closed. This value might be overwrote by client side.
# ping_timeout = 60 #default: undefined
# Life time for reconnection ticket. Unit: second.
reconnection_ticket_lifetime = 600 #default: 600
# Client will leave conference if it cannot reconnect to Socket.IO server after |reconnection_timeout| seconds.
reconnection_timeout = 60 #default: 60


[cluster]
name = "owt-cluster"

#The number of times to retry joining if the first try fails.
join_retry = 60 #default: 60

#The interval of reporting the work load
report_load_interval = 1000 #default: 1000, unit: millisecond

#The max CPU load under which this worker can taks new tasks.
max_load = 0.85 #default: 0.85


[capacity]
#The ISP list this portal is able to handle
#If the ISP list is set to be non-empty, only the creating token requests with preference.ips being matched with one element of this list will be scheduled to this portal.
isps = [] #default: [], which means all ISPs.
#The region list this portal prefers to handle
#If the region list is set to be non-empty, the creating token requests with preference.region being matched with one element of this list will be priorly scheduled to this portal.
regions = [] #default: [], which means all regions.


[rabbit]
host = "172.31.28.159" #default: "localhost"
port = 5672 #default: 5672

[mongo]
dataBaseURL = "172.31.28.159/owtdb" #default: "localhost/owtdb"
 

/webrtc_agent/agent.toml

[agent]
#Max processes that agent can run
maxProcesses = 13 #default: 13

#Number of precesses that agent runs when it starts. 1 <= prerunProcesses <= maxProcesses.
prerunProcesses = 2 #default: 2

[cluster]
name = "owt-cluster"

#The number of times to retry joining if the first try fails.
join_retry = 60 #default: 60

#The interval of reporting the work load
report_load_interval = 1000 #default: 1000, unit: millisecond

#The max network load under which this worker can take new tasks.
max_load = 0.85 #default: 0.85

#The bandwidth of network-interface used for WebRTC peerconnections. 
network_max_scale = 1000 #unit: Mbps
[capacity]
#The ISP list this agent is able to handle.
#If the ISP list is set to be non-empty, only the creating token requests with preference.ips being matched with one element of this list will be scheduled to this agent.
isps = [] #default: [], which means all ISPs.
#The region list this agent prefers to handle
#If the region list is set to be non-empty, the creating token requests with preference.region being matched with one element of this list will be priorly scheduled to this agent.
regions = [] #default: [], which means all regions.
[rabbit]
host = "172.31.28.159" #default: "localhost"
port = 5672 #default: 5672

[internal]
#The IP address used for internal-cluster media spreading. Will use the IP got from the 'network_interface' item if 'ip_address' is not specified or equal to "".
ip_address = "172.31.28.159" #default: ""

#The network interface used for internal-cluster media spreading. The first enumerated network interface in the system will be adopted if this item is not specified.
network_interface = "eth0" # default: undefined

# The internal listening port range, only works for TCP now
maxport = 20000 #default: 0
minport = 10000 #default: 0

#########################################################################################
[webrtc]
#The network inferface all peer-connections will be established through. All network interfaces in the system will be adopted if this item is not specified or specified with an empty array.
##Valid value is an array of objects that has two properties: name and replaced_ip_address. name is the name of network interface that will be used, replaced_ip_address is the IP address used for replacing the internal IP address in locally generated SDP's and ICE candidates during establishing the peer-connection(useful when behind NATs). replaced_ip_address is optional, if it is not specified, IP address from corresponding network interface will not be replaced.
#Example of valid value: [{name = "eth1"}, {name = "eth2", replaced_ip_address = "192.0.2.2"}].
network_interfaces = [{name = "eth0", replaced_ip_address = "47.52.141.92"}]  # default: []

keystorePath = "./cert/certificate.pfx"
#note, this won't work with all versions of libnice. With 0 all the available ports are used

# The webrtc port range
maxport = 9000 #default: 0
minport = 2000 #default: 0

#STUN server IP address and port to be used by the server.
#if "" is used, the address is discovered locally
stunport = 0 #default: 0
stunserver = "" #default: ""

#ThreadPool worker numbers for peer connection
num_workers = 24 #default: 24

 

/conference_agent/agent.toml

[agent]
#Max processes that agent can run
maxProcesses = -1 #default: -1, unlimited

#Number of precesses that agent runs when it starts. 1 <= prerunProcesses <= maxProcesses.
prerunProcesses = 2 #default: 2

#Roles defined in services have been moved to database

[cluster]
name = "owt-cluster"

#The number of times to retry joining if the first try fails.
join_retry = 60 #default: 60

#The interval of reporting the work load
report_load_interval = 1000 #default: 1000, unit: millisecond

#The max CPU load under which this worker can take new tasks.
max_load = 0.85 #default: 0.85
[rabbit]
host = "172.31.28.159" #default: "localhost"
port = 5672 #default: 5672

[mongo]
dataBaseURL = "172.31.28.159/owtdb" #default: "localhost/owtdb"

[internal]
# tcp/sctp available, tcp is default
protocol = "tcp"

 

/audio_agent/agent.toml

[agent]
#Max processes that agent can run
maxProcesses = -1 #default: -1, unlimited

#Number of precesses that agent runs when it starts. 1 <= prerunProcesses <= maxProcesses.
prerunProcesses = 2 #default: 2

[cluster]
name = "owt-cluster"

#The number of times to retry joining if the first try fails.
join_retry = 60 #default: 60

#The interval of reporting the work load
report_load_interval = 1000 #default: 1000, unit: millisecond

#The max CPU load under which this worker can take new tasks.
max_load = 0.85 #default: 0.85

[rabbit]
host = "172.31.28.159" #default: "localhost"
port = 5672 #default: 5672

[internal]
#The IP address used for internal-cluster media spreading. Will use the IP got from the 'network_interface' item if 'ip_address' is not specified or equal to "".
ip_address = "172.31.28.159" #default: ""

#The network interface used for internal-cluster media spreading. The first enumerated network interface in the system will be adopted if this item is not specified.
network_interface = "eth0" # default: undefined

# The internal listening port range, only works for TCP now
maxport = 20000 #default: 0
minport = 10000 #default: 0
 

Server2

/video_agent/agent.toml

[agent]
#Max processes that agent can run
maxProcesses = -1 #default: -1, unlimited

#Number of precesses that agent runs when it starts. 1 <= prerunProcesses <= maxProcesses.
prerunProcesses = 2 #default: 2


[cluster]
name = "owt-cluster"

#The number of times to retry joining if the first try fails.
join_retry = 60 #default: 60

#The interval of reporting the work load
report_load_interval = 1000 #default: 1000, unit: millisecond

#The max CPU/GPU load under which this worker can take new tasks.
max_load = 0.85 #default: 0.85


[rabbit]
host = "172.31.28.159" #default: "localhost"
port = 5672 #default: 5672


[internal]
#The IP address used for internal-cluster media spreading. Will use the IP got from the 'network_interface' item if 'ip_address' is not specified or equal to "".
ip_address = "172.31.28.159" #default: ""

#The network interface used for internal-cluster media spreading. The first enumerated network interface in the system will be adopted if this item is not specified.
network_interface = "eth0" # default: undefined

# The internal listening port range, only works for TCP now
maxport = 20000 #default: 0
minport = 10000 #default: 0

#########################################################################################
[video]
#If true and the machine has the capability, the mixer will be accelerated by hardware graphic chips
hardwareAccelerated = false

#If true and hardwareAccleration is enabled, setting this to true allows GACC HEVC encoder plugin to be used
#for better video quality.
#Warning: enabling this will introduce extra CPU resource consumption.
enableBetterHEVCQuality = false

#Multi Frame Encoding
#timeout[0, 100] in millisecond, setting to "0" disables this feature
MFE_timeout = 0 #default: 0

[avatar]
#widthxheight between the two dot ("180x180" between the "avatar." and ".yuv" in the default) in the location indicates the image size
location = "avatars/avatar_blue.180x180.yuv"
 

 

Lawrence1
Beginner
656 Views

Hello Team,

Any update would be appreciated.

Thank you.
 

YanbinZ_Intel
Employee
656 Views

We can't reproduce this issue .  Which version do you used ?  And could your please provide more detail information e.g test devices, test steps ?

 

Thanks

Yanbin

Naresh_R_1
New Contributor I
656 Views

Hi Team,

We have same issue in cluster servers. Mix stream subcribe stream is black. but forward stream always work.Any resolution ?

 

 

Reply