Throttling error - 0х058 timeout

Throttling error - 0х058 timeout SearchSearch
Author Message
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 63
Registered: 01-2008
Posted on Monday, May 24, 2010 - 02:14 pm:   

Hello Des!

Can i write custome value in timeout resending message, when SMSC return Throttling error - 0х058 ?

Or can you explain what logic of resending messages when NowSMS get this error (Throttling error - 0х058) ?

thanks.
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 64
Registered: 01-2008
Posted on Monday, May 24, 2010 - 02:30 pm:   

And more little question..about logic resending when nowsms get error "Message queue full" from SMSC.
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 7913
Registered: 10-2002
Posted on Monday, May 24, 2010 - 07:26 pm:   

Hi Alexandre,

By default, if a throttling error is received, NowSMS pauses for 5 seconds on that connection before attempting to send another message.

This can be disabled with the following setting in the [SMSGW] section of SMSGW.INI (it applies to all SMPP connections):

SMPPThrottleErrorDelay=##

It specifies a time wait in seconds. 0 will disable any wait.

When this error happens, NowSMS will not immediately try to process the current message, it will move to the next message in the queue and come back to this message later.

The error will count as a retry, and a retry delay may be applied before the message that received this error is retried.

However, this retry cannot trigger a maximum number of retry attempts where the message is discarded. By default, the message that triggers a throttle error will be eligible for retries for up to 24 hours.

Similar logic is used for ESME_RMSGQFUL, except that there is no SMPPThrottleErrorDelay in this case.

-bn
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 65
Registered: 01-2008
Posted on Monday, May 24, 2010 - 08:20 pm:   

Thanks. But what time wait parameter in SMSGW for "Message queue full" error ?
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 7914
Registered: 10-2002
Posted on Monday, May 24, 2010 - 09:34 pm:   

There is none. We just move on to the next message and schedule the current message for a retry. (The retry scheduling logic is described in more detail here: http://blog.nowsms.com/2007/06/smpp-error-code-handling-in-nowsms.html).

-bn
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 66
Registered: 01-2008
Posted on Monday, May 24, 2010 - 10:17 pm:   

Thanks!. These are good options, but many SMSC have the criteria time wait the period for errors. It would be good if it would be possible to adjust the given conditions for separate SMPP (HTTP etc.) connections
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 67
Registered: 01-2008
Posted on Wednesday, May 26, 2010 - 09:14 am:   

One more question on-occasion errors Throttling error. SMSC demands from me that NowSMS processed Throttling error definitely.
I have made options in [SMSGW] such character:
SMPPThrottleErrorDelay=60
RetryDelay=60
RetryDelayMultiplier=2
RetryMaxAttempts=5

But SMSC has answered me that at reception of error Throttling error it is necessary to send after timeout too the message, that is on what there was error Throttling error. And NowSMS puts in the turn end such message.
Whether probably to make it by means of NowSMS? Or at NowSMS other logic of work with the given error?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2165
Registered: 08-2008
Posted on Wednesday, May 26, 2010 - 03:04 pm:   

Hi Alexandre,

I'm sorry, but our not logic is not able to do that.

It may be possible for us to add time wait periods for other error conditions.

However, retrying the current message immediately after the wait period is more complex, especially if NowSMS is operating in SMPP async mode where one thread is filling the queue and another is processing the acknowledgments. I will discuss this further with our engineering team, however the initial discussion suggested that this change was not practical.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 68
Registered: 01-2008
Posted on Thursday, May 27, 2010 - 08:50 am:   

SMSC confirms that after reception of error Throttling error is necessary перепосылать the same message on which there was this error as it can cause delays in message delivery. Still SMSC platform SDP speaks that, can is for example incorrect process the message if receives instead of the first segment of the message the second segment of the message.
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 69
Registered: 01-2008
Posted on Thursday, May 27, 2010 - 08:51 am:   

SMSC confirms that after reception of error Throttling error is necessary to send the same message on which there was this error as it can cause delays in message delivery. Still SMSC platform SDP speaks that, can is for example incorrect process the message if receives instead of the first segment of the message the second segment of the message.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2171
Registered: 08-2008
Posted on Thursday, May 27, 2010 - 11:38 am:   

Hi Alexandre,

I can see that the SDP may have a problem if the second segment of a message is received before the first. (However, in the real world, this can always happen.)

I will ask our engineering team if there is a way that they can address this issue.

However, there are logic problems involved in sending the exact same message again, immediately after a throttling error is received. Keep in mind that in async SMPP, we might have a large window size. Let's say the window size is 20. We may have submitted 20 messages before a throttling error is returned. This might result in 20 throttling errors, or maybe some of the messages get accepted.

Restarting at the exact point of the first message that generated a throttling error is extremely difficult, because for performance reasons we have separate logic threads filling the queue vs. processing the acknowledgments.

And I fail to see what the importance is of restarting at the exact same point.

I can, however, see the importance that this event should avoid sending the second part of a message before the first. And I will ask our engineering team to investigate this further to see if they can come up with a solution.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 70
Registered: 01-2008
Posted on Friday, May 28, 2010 - 11:26 am:   

Ok, i`ll wait answer from your engineering team :-)
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 71
Registered: 01-2008
Posted on Wednesday, June 02, 2010 - 04:36 pm:   

No any news for update or something else ? :-)
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2195
Registered: 08-2008
Posted on Wednesday, June 02, 2010 - 10:59 pm:   

Alexandre,

It may sound easy. However, given our current logic, it is not. This would require major changes.

The only change that we might be able to accommodate shortly would be an option to restart at the start of the queue after a throttling error.

This would not have the exact effect that you are asking for. However, perhaps it would be enough.

That said ... I am concentrating on SMPP async mode here.

If SMPP async mode is not enabled, then the changes are not so complex.

So perhaps I should ask ... are you using SMPP async mode for this connection?

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 72
Registered: 01-2008
Posted on Thursday, June 03, 2010 - 08:25 am:   

No, for this connection i use sync mode, this is limitation of SMSC for me now, but in future it may be async mode.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2196
Registered: 08-2008
Posted on Thursday, June 03, 2010 - 04:00 pm:   

For sync mode, it would not be difficult.

Does it sound a plan to do this:

1.) For sync mode, after a throttling error and delay, re-attempt sending the same message. (There should be a limit on the number of retries for the same message, I would suggest 3.)

2.) For async mode, after a throttling error, restart the queue processing so that if the throttling error was returned on part 1 of a message, we do not try to send part 2 next.

If you agree ... unfortunately, I don't think these changes can be made for approximately 2 weeks. (Maybe if we only do #1 that can be a little quicker.)

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 76
Registered: 01-2008
Posted on Thursday, June 03, 2010 - 08:23 pm:   

Yeah that be cool. But this options will work for all connection to SMSC or it only for one ?
As i said in my preview post:
"It would be good if it would be possible to adjust the given conditions for separate SMPP (HTTP etc.) connections"
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 78
Registered: 01-2008
Posted on Thursday, June 03, 2010 - 08:45 pm:   

Addition to my answer.
Yes for this purpose SMSC we will use sync mode at present, but in спсике connection as there will be also others SMSC, it would be desirable that it is new adjustment did not extend on other connections.
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 79
Registered: 01-2008
Posted on Thursday, June 03, 2010 - 08:47 pm:   

Addition to my answer. Yes for this purpose SMSC we will use sync mode at present, but in the list connection as there will be also others SMSC, it would be desirable that it is new adjustment did not extend on other connections.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2207
Registered: 08-2008
Posted on Friday, June 04, 2010 - 04:02 am:   

I believe it is time for us to allow the throttle delay to be specified on a per connection basis as SMSCs that have strict policies about how it should be handled (like your 60 second delay) are not always good policy choices for others. We will look toward allowing this. And possibly also a setting to treat the message queue full error with the throttle logic.

The other settings will be optional, I guess they could be settable per SMSC connection. At least that is how I will write up the change request.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 80
Registered: 01-2008
Posted on Friday, June 04, 2010 - 08:20 am:   

Ok. Thanks. I will wait for this update :-)
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2245
Registered: 08-2008
Posted on Tuesday, June 15, 2010 - 09:38 pm:   

Hi Alexandre,

This update is still being worked on. We will hopefully see it toward the end of this week.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 81
Registered: 01-2008
Posted on Wednesday, June 16, 2010 - 10:16 am:   

Ok. Thanks
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2249
Registered: 08-2008
Posted on Thursday, June 17, 2010 - 05:18 pm:   

Hi Alexandre,

We've uploaded an update to http://www.nowsms.com/download/nowsms20100617.zip.

The following details this:

* SMS Gateway/SMPP Client: When operating in sync mode, if ESME_RTHROTTLED is returned, after waiting for the throttle delay, the same message will be re-attempted up to 5 times.

* SMS Gateway/SMPP Client: The SMPP Throttle error delay is now configurable on a per connection basis. The default throttle error delay is 5 seconds. To set a global throttle error delay that applies to all connections, add SMPPThrottleErrorDelay=## (where ## is a number of seconds) under the [SMSGW] header of SMSGW.INI. To specify a delay that should be used only for a specific connection, add SMPPThrottleErrorDelay=## to the [SMPP - server:port] section for the connection.

* SMS Gateway/SMPP Client: Add configuration option to treat the ESME_RMSGQFUL (receive message queue full) error the same way as a throttling error, subjecting it to the SMPPThrottleErrorDelay pause. This option can be activated for inidividual SMPP connections if desired by adding ThrottleForQFull=Yes under the [SMPP - server:port] section header for a connection.

Also, when an SMPP connection is operating in sync mode, additional checks have been implemented to prevent sending segmented messages out of order.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 82
Registered: 01-2008
Posted on Friday, June 18, 2010 - 11:16 am:   

Big Thanks! I`ll test it and send to u report :-)
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 84
Registered: 01-2008
Posted on Wednesday, July 21, 2010 - 11:26 am:   

All work, but i want ask,about params:
RetryDelay=##
RetryDelayMultiplier=##
RetryMaxAttempts=##
That params must be under the [SMSGW] header of SMSGW.INI only ? Or they can used for a specific connection too?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2344
Registered: 08-2008
Posted on Wednesday, July 21, 2010 - 07:24 pm:   

Hi Alexandre,

The retry related parameters (RetryDelay, RetryDelayMultiplier, RetryDelayAfterAttempts, RetryDelayMax, RetryMaxAttempts & RetryMaxAge) are all now supported both as connection specific settings, and global settings.

If a setting is present in an SMSC specific section header (e.g., [SMPP - server:port]), this will override the retry settings for that connection. If no setting is present for a connection, the value in the [SMSGW] section will be used.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 85
Registered: 01-2008
Posted on Thursday, July 22, 2010 - 10:12 am:   

That all supported in last version 2010.06.17 ?
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 86
Registered: 01-2008
Posted on Thursday, July 22, 2010 - 10:20 am:   

I have forgotten to ask about parametres InitRetryDelay and InitRetryDelayMax. They can used for a specific connection too?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2347
Registered: 08-2008
Posted on Thursday, July 22, 2010 - 06:09 pm:   

Yes. In 2010.06.17.

It appears that this was not recorded in the changes file, however I can confirm that the change to support per-connection SMPPThrottleErrorDelay values also applied to all other retry related parameters.

It is possible that we may have missed some, but I can confirm that in addition to those previously listed, also included are InitRetryDelay, InitRetryDelayMax, InitRetryDelayMultiplier, InitRetryDelayAfterAttempts.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 88
Registered: 01-2008
Posted on Tuesday, July 27, 2010 - 12:13 pm:   

We have tested processing ESME_RMSGQFUL and have faced that that NowSMS tries to send anew the same message, and because of it other messages hang in queue. Whether probably to change function that after reception of error ESME_RMSGQFUL, the message moved to the queue end, i.e. it was exposed are lower a priority in comparison with other messages?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2368
Registered: 08-2008
Posted on Tuesday, July 27, 2010 - 04:07 pm:   

Don't use the ThrottleForQFull=Yes option.

The default setting only retries the current message for ESME_RTHROTTLED (and applies a delay before retry).

If ThrottleForQFull=Yes is set, ESME_RMSGQFUL is treated the same way as ESME_RTHROTTLED.
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 89
Registered: 01-2008
Posted on Tuesday, July 27, 2010 - 04:31 pm:   

Retry params is the same for ESME_RMSGQFUL ?

RetryDelay=##
RetryDelayMultiplier=##
RetryMaxAttempts=##

Or some another ?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2369
Registered: 08-2008
Posted on Tuesday, July 27, 2010 - 07:50 pm:   

The retry parameters apply the same to all error conditions, except that RetryMaxAttempts is not used for ESME_RMSGQFUL & ESME_RTHROTTLED errors. RetryMaxAge applies to ESME_RMSGQFUL & ESME_RTHROTTLED errors, with a default value of 1440 minutes (24 hours).

Special handling for a throttling error is the application of SMPPThrottleErrorDelay, and in recent updates, the message that triggers the error is retried up to 5 times.

For ESME_RMSGQFUL, the default behaviour is not to delay, and to attempt the next message in the queue.
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 90
Registered: 01-2008
Posted on Wednesday, July 28, 2010 - 09:09 am:   

Thanks for answers DES!
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 94
Registered: 01-2008
Posted on Thursday, July 29, 2010 - 09:33 am:   

We have spent a series of tests together with SMSC and have seen a strange situation at crossing Message queue full and is possible Throttling error, in the attached file a picture from Wireshark.

It can is connected with not correctly adjusted parametres of repeated sending, at occurrence of these errors?
Here my options to it SMSC:

[SMPP - SMSC:2775]
SMPPVersion=v3.4
UserName=login
Password=password
SenderAddressOverride=Yes
ReceiveSingleConnection=Yes
RetryDelay=60
RetryDelayMultiplier=2
RetryMaxAttempts=5
SMPPThrottleErrorDelay=60
InitRetryDelay=90
InitRetryDelayMax=1350
Receive=Yes
ReceiveMMS=No
UseSSL=No
DestTON=1
LongSMSAlt=Yes
SourceTON=0
SourceNPI=1
DestNPI=1
SenderAddress=TEST
RoutePrefOnly=Yes
Route1=+???????*
AllowedUserOnly=Yes
KeepAlive=45
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 2382
Registered: 08-2008
Posted on Thursday, July 29, 2010 - 06:42 pm:   

TCP Out-Of-Order is a TCP/IP stack level issue ... outside of our control.

It does not mean that there is an error occurring at the SMPP layer.

I would not expect it to be an actual problem, nor related to the "System error" responses.

It is not unusual for there to be occasional lost packets, retransmitted packets, out-of-order packets, or duplicate acknowledgments occurring during the course of a TCP/IP connection. The TCP protocol corrects these errors so that they are not seen by the higher layer protocols that are implemented on top of TCP.

If you're seeing a lot of these errors, then you might want to google for more information.

--
Des
NowSMS Support
Alexandre
Frequent Contributor
Username: Alexd

Post Number: 96
Registered: 01-2008
Posted on Friday, July 30, 2010 - 08:48 am:   

Ok thanks.