Nuve fails after it cannot connect to MongoDB

[Mark_d_10]

Hi,

On our servers the Nuve service crashes sometimes on a error where it can shortly not connect to the MongoDB cluster with Primary running on the localhost

This is what the nuve.stdout log is showing:

2017-10-18 08:44:24.937  - INFO: RoomResource - Room  59e714667da69514e86c20cf  does not exist
2017-10-18 08:44:25.306  - INFO: RoomResource - Representing room  59e714677da69514e86c20d0 of service  58b68bdc2a186407e0c9ad12
2017-10-18 08:44:30.998  - INFO: RoomsResource - Room created: ebe9a255-9311-466e-a47e-01b2a01ce250 for service kpnzorgService
2017-10-18 08:44:31.381  - INFO: RoomResource - Representing room  59e7146e7da69514e86c20d1 of service  58b68bdc2a186407e0c9ad12
2017-10-18 09:08:01.524  - INFO: RoomsResource - Room created: 1d249386-f679-4fa5-9e91-00ce1dc1d647 for service kpnzorgService
2017-10-18 09:08:03.546  - INFO: RoomResource - Representing room  59e719f07da69514e86c20d2 of service  58b68bdc2a186407e0c9ad12
2017-10-18 09:08:03.547  - INFO: RoomsResource - Room created: 94f5f6ea-eaa5-4e94-8cbc-07c19dd2f76e for service kpnzorgService
2017-10-18 09:08:03.935  - INFO: RoomResource - Representing room  59e719f17da69514e86c20d3 of service  58b68bdc2a186407e0c9ad12
2017-10-18 09:08:05.101  - INFO: RoomsResource - Room created: 6216bfb7-0a11-47ff-9e13-2c242ec5798b for service kpnzorgService
2017-10-18 09:08:05.487  - INFO: RoomResource - Representing room  59e719f47da69514e86c20d4 of service  58b68bdc2a186407e0c9ad12
2017-10-18 09:08:20.493  - INFO: RoomsResource - Room created: 6483ac93-c10b-48b2-b78a-a345cc7d8eca for service kpnzorgService
2017-10-18 09:08:20.870  - INFO: RoomResource - Representing room  59e71a037da69514e86c20d5 of service  58b68bdc2a186407e0c9ad12
2017-10-18 09:08:40.310  - INFO: RoomsResource - Room created: 107bf4bc-31ec-48cb-b128-a00c014e6be2 for service kpnzorgService
2017-10-18 09:08:41.127  - INFO: RoomResource - Representing room  59e71a057da69514e86c20d6 of service  58b68bdc2a186407e0c9ad12
2017-10-18 09:08:43.362  - INFO: ServiceRegistry - MongoDB: Error updating service:  { [MongoError: connection 1 to localhost:27017 closed]
  name: 'MongoError',
  message: 'connection 1 to rtc-db-1:27017 closed' }
MongoError: connection 1 to rtc-db-1:27017 closed
    at Function.MongoError.create (/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/node_modules/mongodb-core/lib/error.js:29:11)
    at Socket.<anonymous> (/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/node_modules/mongodb-core/lib/connection/connection.js:200:22)
    at Socket.g (events.js:260:16)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at TCP._onclose (net.js:487:12)
2017-10-18 09:08:43.456  - WARN: RoomRegistry - MongoDB: Error adding room:  { [MongoError: connection 1 to localhost:27017 closed]
  name: 'MongoError',
  message: 'connection 1 to rtc-db-1:27017 closed' }
MongoError: connection 1 to rtc-db-1:27017 closed
    at Function.MongoError.create (/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/node_modules/mongodb-core/lib/error.js:29:11)
    at Socket.<anonymous> (/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/node_modules/mongodb-core/lib/connection/connection.js:200:22)
    at Socket.g (events.js:260:16)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at TCP._onclose (net.js:487:12)
/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/lib/utils.js:98
    process.nextTick(function() { throw err; });
                                  ^

TypeError: Cannot read property 'sipInfo' of undefined
    at /usr/share/intelrtc/nuve/resource/roomsResource.js:1:1848
    at /usr/share/intelrtc/nuve/mdb/roomRegistry.js:1:526
    at /usr/share/intelrtc/nuve/node_modules/mongojs/lib/collection.js:87:23
    at /usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/lib/collection.js:524:20
    at /usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/lib/collection.js:655:26
    at handleCallback (/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/lib/utils.js:95:56)
    at Object.resultHandler [as cb] (/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/lib/bulk/ordered.js:407:11)
    at null.<anonymous> (/usr/share/intelrtc/nuve/node_modules/mongojs/node_modules/mongodb/node_modules/mongodb-core/lib/connection/pool.js:245:32)
    at g (events.js:260:16)
    at emitTwo (events.js:87:13)
    at emit (events.js:172:7)

Can there be made some retry?

In the hostsfile rtc-db-1 is pointed to 127.0.0.1 (so it is localhost actually)
The Mongo DB cluster only had a small hickup in connecting to it's secondary node:
 

2017-10-18T09:08:53.959+0000 [rsBackgroundSync] replSet sync source problem: 10278 dbclient error communicating with server: rtc-db-2:27017

So the MongoDB did not go into some Error state.
After restarting the Nuve everything worked fine again.

[Mark_d_10]

Maybe a better solution is to also allow connections MongoDB in ReplSet? So it can failover and still connect if there is issue with one server?

If I enter 2 database nodes in the dataBaseURL in the nuve.toml is also not working.

[Mark_d_10]

Any updates on this?

Will support for MongoDB replica sets be added in a future release?

 

[Lei_Z_Intel1]

For v3.4, Replica Set has already supported. For v3.5, both Replica Set and DB URL re-connection are supported.

========================================================================================

Configure dataBaseURL as follow for mongodb replica connections

<hostname>:port1,<hostname>:port2,<hostname>:port3/nuvedb? replicaSet=replicaname

Example:
dataBaseURL = “localhost:27017,localhost:27018,localhost:27019/nuvedb?replicaSet=rs0”

[Mark_d_10]

Thanks Lei, I will try it.
Maybe this configuration option can be put in the installation manual? Or as comment in the configuration file(s)?