Messages Struck in Queue

Messages Struck in Queue SearchSearch
Author Message
Acette Support
Frequent Contributor
Username: Mathewm

Post Number: 176
Registered: 04-2011
Posted on Thursday, October 24, 2024 - 10:13 am:   

Hi,

We had a case with one our installations where around 350,000 SMS were pushed through NowSMS on a day. What we noticed is that a large number of messages that reached NowSMS at 10:40 AM remained in the queue till 11:30 PM after which they were delivered. This was noticed only with messages that were submitted through HTTP with a particular userid and route. However other messages that were received through HTTP with same userid and route later were being delivered while the older messages still remained in the queue.
We are at loss as to which logs we should be looking to investigate why these messages were held up in the queue until the whole queue was cleared.
We also noticed that there were no errors (ESME_RTHROTTLED) in the SMSOUT logs. If it helps, can we share the SMSGW.ini file by email?

Looking forward to your guidance as we are in the process of preparing a Root Cause Analysis of the incident.
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 8545
Registered: 10-2002
Posted on Thursday, October 24, 2024 - 10:13 pm:   

Hi,

What version of NowSMS? I see from other posts that you are experimenting with the user setting regarding “send with priority’…that setting is likely the cause, as we have confirmed problems including stuck messages associated with using this setting.

In current versions, I’m not aware of any issues with this setting, other than it has does not have significant performance impact and we are re-engineering for a future release.
Acette Support
Frequent Contributor
Username: Mathewm

Post Number: 177
Registered: 04-2011
Posted on Friday, October 25, 2024 - 07:45 am:   

Hello Bryce,

Thanks for the response.
We are using version 2020.01.15 of NowSMS. Regarding the "Send messages with high priority" setting, this has been enabled only for one user and left unchecked for all other users. Do you suggest an upgrade to a later version which has relevant fixes?
We noticed this problem with a user queue for which this setting is disabled.
Further, Multi-part messages received earlier remain in this queue for a long time before they are sent out. Single-part messages received later in this queue are delivered immediately.

Thanks & Regards
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 8546
Registered: 10-2002
Posted on Friday, October 25, 2024 - 08:50 pm:   

My understanding that queue processing would get confused trying to process messages from a priority user that had already been processed. The connection to multiparty messages is that once a first part is sent, internally, the remaining parts are treated with some priority.

Yes, I would recommend updating.
Acette Support
Frequent Contributor
Username: Mathewm

Post Number: 178
Registered: 04-2011
Posted on Wednesday, October 30, 2024 - 02:20 pm:   

Hello Bryce,

Thanks for the details.

We found the issue that caused the delay in clearing the queue. Many multi-part messages sent to different destination numbers received SMPP error code "ESME_RMSGQFUL" from the SMS Provider. The retry logic would have caused an overhang as these messages were in large numbers.
We made use of the "SMPPRejectErrorCodes" configuration to add Error code 14 and reject these messages with permanent error. This helped in clearing the queues.
Thanks once again for sharing your thoughts.
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 8547
Registered: 10-2002
Posted on Wednesday, October 30, 2024 - 05:59 pm:   

Hi Mathew,

That is a very interesting observation. Often times when people report a queue build-up, I fail to think of common operational situations like the provider returning ESME_RMSGQFUL.

If this error is received, NowSMS will hold other messages for the same recipient in queue and only retry the oldest message for that recipient. Once that oldest message is processed, the other messages will be released for processing. Because only the first message is being retried, it can be difficult to determine that this error is leading to queue buildup.

Your use of the "SMPPRejectErrorCodes=14" configuration to reject these messages with permanent error seems an appropriate solution. Thank you for sharing.

Add Your Message Here, or click here to start a new topic.
Post:
Bold text Italics Underline Create a hyperlink Insert a clipart image
Options: Automatically activate URLs in message
Action: