Missing UDH (sometimes)

Missing UDH (sometimes) SearchSearch
Author Message
Lars Nielsen, MobileNation ApS - Denmark
Frequent Contributor
Username: Larsmservice

Post Number: 251
Registered: 10-2003
Posted on Friday, December 16, 2005 - 11:48 am:   

Hi Bryce.

When sending a WAPPUSH through nowsms to one of my HTTP gateways, i sometimes notice that nowsms forgets to insert the UDH data into the GET/POST string.

It is quite critical on one of my gateways since they cannot receive the data when the UDH is missing. Then they return an error which nowsms do not undestand, and then i cannot get the Retry-stuff to work in my smsgw.ini. So what happends is that nowsms keeps posting the same message thousands of times. Actually untill i physically delete the message in the Q-folder.

Whoops that was 2 things ;-)
Generally i think its weird that nowsms sometimes forgets to replace the UDH data in the @@UDH@@.
Is @@UDH@@ case-sensitive ?

Please notice that i write "sonetimes". Mostly it will do it, but once or twice every 24 hours it "forgets" it.

Is it a bug or am i doing something wrong ?

Here are some logdata from various logfiles. Please notice that the UDH data is there...

4394FFC3.req,xxx.xxx.xxx.xxx,4512121212,OK -- HTTP - 1234.smsdomain.dk:80,Sender=1234;Binary=1;DCS=F5;UDH=0605040B8423F0;Data=6A0601AE02056A0045C60C03636473322E7177712E646B2F7469636B65742F313531323230 303532313230333736313862633536302F64656661756C742E617370000103446F776E6C6F616420 696E64686F6C6420667261205456322056616C68616C6C61000101

xxx.xxx.xxx.xxx,,4394FFC3.req,WAPPush

ThreadProcessModem: GET /smspush?to=4512121212&type=extendedraw&text=0056006900200066006F007200730074006 F006400200069006B006B0065002000640069006E0020006200650073006B00650064002E0020004 4007500200073006B007200650076003A0020004A0075006C002000686B6D00650072&user=uname &password=pword&udh=&smsc=operator.tdc&appnr=1234&mediacode=jul&price=00



Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 5338
Registered: 10-2002
Posted on Friday, December 16, 2005 - 04:43 pm:   

Hi Lars,

The message shown in the HTTP request is not a WAP Push message. It looks like it is a message in Unicode format. So there is no UDH. (And maybe your provider doesn't like a parameter being present when it is not used?)

What is not clear is why the message is Unicode ...

Are you submitting the message to NowSMS in Unicode format? The problem character seems to be toward the end.

Here's a Unicode decode:

0056 V
0069 i
0020
0066 f
006F o
0072 r
0073 s
0074 t
006F o
0064 d
0020
0069 i
006B k
006B k
0065 e
0020
0064 d
0069 i
006E j
0020
0062 b
0065 e
0073 s
006B k
0065 e
0064 d
002E .
0020
0044 D
0075 u
0020
0073 s
006B k
0072 r
0065 e
0076 v
003A :
0020
004A J
0075 u
006C l
0020
0068 h
6B6D 歭 (probably should be 006B 006D = k m)
0065 e
0072 r

Check to see how you are submitting this particular message.

In the meantime, we took a closer look at the retry logic for HTTP SMSC submissions. There are a few error codes for which the retry behaviour is too agressive. In particular, if the web server returns a 500 "Internal Server Error" response, the retries are very aggressive. There is a temporary update to address this issue at http://www.nowsms.com/download/lars.zip.

-bn
Lars Nielsen, MobileNation ApS - Denmark
Frequent Contributor
Username: Larsmservice

Post Number: 252
Registered: 10-2003
Posted on Sunday, December 18, 2005 - 10:24 pm:   

THE LINK IS INVALID: http://www.nowsms.com/download/lars.zip

Could you please upload it again Bryce. I need it ;-)
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 5346
Registered: 10-2002
Posted on Monday, December 19, 2005 - 02:03 pm:   

Weird. I should have tried it after uploading the file. It is working now.
Lars Nielsen, MobileNation ApS - Denmark
Frequent Contributor
Username: Larsmservice

Post Number: 253
Registered: 10-2003
Posted on Monday, December 19, 2005 - 02:52 pm:   

It works fine now...

Thanks...
devel
New member
Username: Devel

Post Number: 1
Registered: 01-2006
Posted on Tuesday, January 24, 2006 - 02:45 pm:   

Hi Lars/Bryce, now I face the same problems!! Thousands and thousands of SMS keep sending to the same sender ...

I have appliied this patch (5.51m) but it still keep sending. This does not work, please help!

RetryDelay=30
RetryDelayMultiplier=0
RetryDelayAfterAttempts=1
RetryMaxAttempts=1 <--- not working

Any advice?
Lars Nielsen, MobileNation ApS - Denmark
Frequent Contributor
Username: Larsmservice

Post Number: 256
Registered: 10-2003
Posted on Tuesday, January 24, 2006 - 02:57 pm:   

Try look into your smsdebug.log to see what answer you get from the sms-gateway that your nowsms is GET/POST(ing) to.

If the answer is not a HTTP 500 error or the like nowsms will not accept it as an error message.

I am not a nowsms employee so i am not entitled to help you, but i can try ;-) Bryce rules ;-)


;)Lars


devel
New member
Username: Devel

Post Number: 3
Registered: 01-2006
Posted on Tuesday, January 24, 2006 - 03:16 pm:   

No, it is not HTTP 500 error.. if its not, any idea how to make the RetryMaxAttempts=1 works?

Lars, is this what you have to solve this issue?
RetryDelay=30
RetryDelayMultiplier=0
RetryDelayAfterAttempts=1
RetryMaxAttempts=1

Its scary that every second it just push out the SMS...
devel
New member
Username: Devel

Post Number: 4
Registered: 01-2006
Posted on Tuesday, January 24, 2006 - 03:19 pm:   

For your info which i am sure you see this before, the SMS-IN directory ALWAYS has this .in file, and the Q directory .req keeps coming non-stop ...
devel
New member
Username: Devel

Post Number: 5
Registered: 01-2006
Posted on Tuesday, January 24, 2006 - 03:25 pm:   

its showing this:
Content-Length: 323
Content-Type: text/html
Set-Cookie: ASPSESSIONIDQABQDBQT=CHBCIGBDOCNIJFJDBOCEBJPO; path=/
Cache-control: private
devel
New member
Username: Devel

Post Number: 6
Registered: 01-2006
Posted on Tuesday, January 24, 2006 - 03:27 pm:   

oh its showing this before the one above:

23:22:26:828 [14] ThreadProcessConnection: Request processing complete
23:22:26:828 [5] HttpResponseWait: Ok
23:22:26:828 [5] RetrieveURL: got unknown response
23:22:26:828 [5] RetrieveURL: HTTP/1.1 500 Internal Server Error
devel
New member
Username: Devel

Post Number: 7
Registered: 01-2006
Posted on Tuesday, January 24, 2006 - 04:48 pm:   

Hi Bryce, please advice how to get the RetryMaxAttempts=1 to work regardless the error even its HTTP/1.1 500 Internal Server Error or others.

Its not a good practice to have a system with such "unlimited" processing of request (regardless of what it does ..) ... i think just matter of time some other guys will face the same issue (hope not like others who need to pay thousands of dollars of bill ...), and not to mention that telco might stop letting them connect to their smsc directly ...
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 5474
Registered: 10-2002
Posted on Friday, January 27, 2006 - 10:19 pm:   

Devel,

Did you try the lars.zip patch above?

This sounds like the same problem.

Oh wait ... you're talking about SMS-IN ... not out.

Or are you? I'm confused.

Lars' issue was a problem message in the outbound Q.

But I think you're talking about a problem message in the SMS-IN directory ... where your 2-way script is processing the message (and maybe submitting a new message in response?), but for some reason the script is returning an HTTP 500 error.

That is a problem. Basically, our logic is that we expect that the 2-way command will return an OK response (or a redirect). If there is a failure, we assume it is a temporary error, and keep retrying.

I do agree that retrying infinitely is not a good solution, as we can't assume that just because the HTTP response is bad that the script hasn't taken some action to process the message. We should protect from unanticipated script errors.

Recently, a similar problem was posted here:

http://support.nowsms.com/discus/messages/1/12418.html

In that case, ASP.Net doesn't like "<" or ">" to appear in a URL parameter, so I was able to explain how to disable the check in ASP.Net which chokes on these parameters.

But longer term, we should provide some protection from problem scripts.

I'm going to see if we can look at this with greater urgency.

Have you investigated why the script is returning a 500 error for this message?

The only temporary solution that I can offer you right now is that if you include @@MESSAGEID@@ in your 2-way command, NowSMS will insert the name of the ".IN" file here, so that you could check to make sure that you're not processing the same message repeatedly.

-bn
devel
New member
Username: Devel

Post Number: 14
Registered: 01-2006
Posted on Tuesday, March 14, 2006 - 05:18 am:   

Bryce, is the new 2006 nowsms version solved this unlimited loop issue? We still facing RetryMaxAttempts=1 not working once a while. Its a serious bug!
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 5647
Registered: 10-2002
Posted on Wednesday, March 15, 2006 - 07:19 pm:   

Yes, we did make some changes here.

But I'm confused about what exactly we are talking about here, because there seemed to be 2 different issues that got mixed together in this thread.

Lars original issue dealt with an HTTP SMSC connection, and retries.

If the HTTP SMSC request returned a 500 series error ("Internal Server Error"), then NowSMS would retry indefinitely.

That was not a good idea, and we changed that behaviour.

The secondary issue, which I believe you are concerned about deals with 2-way SMS ... and similar handling of HTTP response codes for 2-way SMS.

Our logic with 2-way SMS was that if the 2-way script returned any type of error, we assumed that the problem was temporary, and should be retried.

That might make sense in a perfect world. But we've gotten a lot of feedback from people where unexpected issues such as ASP.Net not liking "<" or ">" characters in a URL request, etc., lead to problems with the infinite and aggressive retries of the 2-way commands.

I don't know if there is a v5.51 patch that changes this behaviour. But I do know that this behaviour was changed in NowSMS 2006. The current logic is that if a 2-way command returns a non-OK HTTP response code, we rename the message file in the SMS-IN directory to have a ".BAD" extension, and do not perform any retry. This way, it is possible to investigate script problems (rename the ".BAD" to ".IN" to have it reprocessed).

-bn

devel
New member
Username: Devel

Post Number: 15
Registered: 01-2006
Posted on Saturday, March 18, 2006 - 03:21 am:   

Yes Bryce, in fact both of the cases above both will run into infinite loop. Good to see both has been fixed in new version.

Is there any special parameter we need to set on the .ini files for the above to work or we just use the maxretries parameter will do.

In another question not related to nowsms, will there be a NOWIMP 2006 coming soon?

Good job Bryce.
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 5678
Registered: 10-2002
Posted on Tuesday, March 21, 2006 - 04:12 pm:   

No special parameters needed, it's a change in behaviour. The max retries setting still only applies to outbound message delivery.

We've got some more updates planned on Now IMP, but there probably won't be anything too significant until late summer.