| SMSCRoute being ignored when using SMPP |  Search
|
|
NowSMS Support Forums ⬆ NowSMS Support - SMS Issues ⬆ Archive through February 09, 2012 ⬆ |
◄ ► |
| Author | Message | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 7 Registered: 06-2011 |
We have a problem with NowSMS v2011.07.05 not using the SMSCRoute we specified. In this particular scenario the SMSCRoute did not exist and instead of erroring the submission it simply picked an alternative route (seemingly randomly)which has lead to errors submitting and even worse some successful submissions on routes we haven't targetted. We need a way of configuring NowSMS to error any SMPP submissions that have a bad SMSCRoute specified and not try and compensate. This issue is serious as it causes billing and delivery issues. | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 9 Registered: 06-2011 |
After looking into this more we have found 4505 .err files that are caused by rejected by operator errors due to the random re-routing, but even though these were receipted to the SMPP user they were not sent to the 2-way callback as usual. Please can you look at patching these two issues ASAP as they are serious issues as our main system cannot currently predict NowSMS's behaviour and is therefore falling out of sync. The following two examples are from our logs and neither was recieved via the 2-way callback; I have obfuscated the message and usernames: 2011-08-03 16:57:26,SAR-blu+447969******-523abd60-2-1.req,84.45.99.73,+447969******,ERROR: Invalid Source Address - ESME_RINVSRCADR (0x0A) -- SMPP - smpp.ipx.com#3:6700,SubmitUser=*****;SMSCRouteName=SMPP-IPX-PC;Sender=83838;UDH= 050003290201;Text="********" 2011-08-03 16:57:26,blu523ABD7B.req,,83838,OK -- LocalUser:*******,SubmitUser=SMPP - smpp.ipx.com#3:6700;Sender=+447969******;Text="id:SAR-blu+447969******-523abd60- 2-1 sub:001 dlvrd:000 submit date:1108031657 done date:0000000000 stat:REJECTD err:00A text:B2EFBA1CD4143A83A0A" 2011-08-04 10:18:07,SAR-blu+447788******-523cdec6-2-1.req,84.45.99.73,+447788******,ERROR: Invalid Source Address - ESME_RINVSRCADR (0x0A) -- SMPP - smpp.ipx.com#2:6700,SubmitUser=*******;SMSCRouteName=SMPP-IPX-PC;Sender=83838;UD H=050003410201;Text="***************************************************" 2011-08-04 10:18:07,blu523CDED4.req,,83838,OK -- LocalUser:******,SubmitUser=SMPP - smpp.ipx.com#2:6700;Sender=+447788******;Text="id:SAR-blu+447788******-523cdec6- 2-1 sub:001 dlvrd:000 submit date:1108041018 done date:0000000000 stat:REJECTD err:00A text:B2EFBA1C644DCBCF693" | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3383 Registered: 08-2008 |
Hi Dave, I'm sorry, but I don't understand the problem that you are trying to describe. Or at a minimum, I need clarification. Who is specifying the invalid route? Is it being specified via an SMPP TLV parameters, or is it being specified via an accounting callback? -- Des NowSMS Support | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 10 Registered: 06-2011 |
Des, The customer connect using SMPP interface to NowSMS and then the account callbacks mechanism is used to route which bind the message is sent down using the SMSCRoute parameter. In this case it was set to a route that did not exist in the NowSMS configuration due to a mistake in our config data. We expected NowSMS to simply set the messages to .bad as the SMSCRoute specified did not exist, however instead it seems to have simply ignored the SMSCRoute and chosen random binds to send the message through! In addition to this issue we also noticed that many of the messages that received the reponse ESME_RINVSRCADR when submitted did not forward the subsequent received recipt to the 2-way interface but only to the SMPP user which has left our system out of sync. I hope this helps clarify the issues. Dave | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3387 Registered: 08-2008 |
Hi Dave, OK ... now I understand. If you are returning the invalid SMSCRoute for a PreAuth accounting callback, we could make a change to cause an invalid route as a trigger to reject the message. If the invalid SMSCRoute is returned in the normal callback (not the PreAuth), it is too late to reject the message. I agree that it is a good idea that we should reject the message if the PreAuth accounting response returns an invalid route name. We will pursue making this change. -- Des NowSMS Support | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 11 Registered: 06-2011 |
Des, I wasn't aware you could change the SMSCRoute in the PreAuth as it isn't mentioned here: http://www.nowsms.com/doc/advanced-configuration-settings/sms-accounting-callbac ks Our code currently does this at the SMSSend accounting call back but I guess we could move it to the PreAuth. What about the second issue regarding some receipts not being forwarded to the 2-way interface as they should; it looks like the when the message is rejected with a ESME_RINVSRCADR it isn't being forwarded to the 2-way just the SMPP? See my examples in the posts above. | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3388 Registered: 08-2008 |
Hi Dave, The original implementation did not allow the SMSCRoute to be set in the PreAuth callback. The issue was that for HTTP submissions to multiple recipients, the PreAuth callback only occurred once. There is an additional configuration setting that forces one PreAuth callback for each recipient, to allow the SMSC route to be set: SMSGW.INI / [SMSGW] / SMSAccountingPreAuthPerRecip=Yes This setting exists in all 2010 and later releases. If this setting is not present, the SMSCRoute= setting is still supported in the PreAuth callback for SMPP submissions, but not for HTTP submissions (even if to a single recipient). Regarding the second issue, it appears that the DuplicateUserReceiptsFor2Way=Yes setting was not working for non-delivery receipts generated directly by NowSMS. (The receipts that are generated if an upstream connection rejects a message submission with an SMPP error code.) We will get a fix applied for this in the next update. -- Des NowSMS Support | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 13 Registered: 06-2011 |
Des thanks for the update can you give me an ETA for the fix for DuplicateUserReceiptsFor2Way=Yes setting as we are losing accuracy for our reporting everyday. | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3393 Registered: 08-2008 |
Hi Dave, What version are you running right now? Our current schedule is to have the latest build completed testing by Thursday. -- Des NowSMS Support | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 14 Registered: 06-2011 |
Des, We are running version v2011.07.05 | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3409 Registered: 08-2008 |
Dave, I posted version 2011.08.10 at http://www.nowsms.com/download/nowsms20110810.zip It addresses the following issues: 1.) If a PreAuth callback returns an invalid route (requires SMSGW.INI / [SMSGW] / SMSAccountingPreAuthPerRecip=Yes setting to support HTTP message submissions), this version will refuse to accept the message. 2.) DuplicateUserReceiptsFor2Way=Yes fixed to also support the non-delivery receipts that occur when a message is rejected by an upstream SMSC connection. 3.) It makes an attempt at addressing the issue discussed in http://support.nowsms.com/discus/messages/1/70406.html, by adding a setting that copies any messages being routed to a local SMPP client account so that they are also processed via 2-way SMS commands. I say attempt, because although I've spent the better part of today testing this capability, it is possible that myself and our engineers missed something in the implementation. It is working in all of the scenarios that I can think of, but it may take additional time to discover scenarios that were not considered. The new setting is DuplicateUserMessagesFor2Way=Yes. If this setting is present, DuplicateUserReceiptsFor2Way=Yes is also enabled, so it is not necessary to set both. -- Des NowSMS Support | ||||
| Chris Frequent Contributor Username: Chrisc Post Number: 57 Registered: 12-2008 |
Hi Des, I've just finished my initial testing of the new version where I followed the number of changes you told us of in your post above and I would like to raise a few concerns. 1. - I started testing the PreAuth callback by returning an invalid route (namely 'blah'), but NowSMS still picked a random route and sent the message. To explain, I sent a message from an SMPP client to our testing NowSMS SMPP Server which fires off the PreAuth callback. Using the logic contained in our platform, the route that should've been picked was the 'blah' route. Obviously, this route is not contained in the SMSGW.ini file and the message should have been refused. Unfortunately, a random message was still picked and the message was sent. The SMSGW.ini was preconfigured with the SMSAccountingPreAuthPerRecip=Yes setting in the SMSGW section. Is there something obvious I'm missing?? 2. - Following the first test and leaving the preconfigured SMSAccountingPreAuthPerRecip parameter in and with the 'blah' route changed to its correct name, I continued with the 2nd test. I wanted to generate some traffic that would cause an obvious rejection at the carrier end so I sent a message to valid MSISDN with an Originator of 'test567890123' and 'noreplyblahblahb'. For some reason, the message still went to a different route altogether, although the moment I changed the Originator back to 'test' or 'noreply', the message went through the correct route and also got delivered OK. I'm not sure if the first test's configuration setting could've caused this perhaps, but it is still very strange behavior. 3. - I am happy to report though that the DuplicateUserMessagesFor2Way=Yes parameter does work and we can now see both SMPP inbound traffic as well as on our platform. Hopefully the information I've given above does provide you with enough information to look at these issues again. If however I've done something completely wrong or I've missed something obvious, please let me know. Regards Chris | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3414 Registered: 08-2008 |
Hi Chris, #1 and #2 both suggest to me that NowSMS is not seeing the route in the accounting response in either case. I'd suggest enabling the SMSDEBUG.LOG, which will show the HTTP request being issued, and the response that is being seen. Let's take a closer look at this sequence ... both the PreAuth callback and its response in the SMSDEBUG.LOG, as well as the SMSSend callback that follows. (The SMSSend callback that follows should have the SMSCRoute= parameter populated from the value in the PreAuth response.) -- Des NowSMS Support | ||||
| Chris Frequent Contributor Username: Chrisc Post Number: 58 Registered: 12-2008 |
Hi Des We've taken a look at the SMSDebug files and have found the SMSCRoute being specified as 'blah'. NowSMS still picked a random route and sent the message, while the SMSAccountingPreAuthPerRecip=Yes was in place. Further to this, by pure happenstance, when I sent the message the callback happened to fail, which returned a 202 Accepted response to NowSMS (I know, not exactly the type of response I'd use), but NowSMS continued processing the message. Do we have to follow a more strict policy of server responses, such as when an error occurs we should rather return a 500 response? Regards Chris | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3415 Registered: 08-2008 |
Hi Chris,
We'd like to see this log...at least as far in the debug as the PreAuth and SmsSend request/response, and the out log showing the message going out (or failing) on the wrong route. I can only figure that there is something about the format of the response that is causing the route not to be seen.
A 200 series response is considered ok. Keep in mind that this callback does not require a route to be set...and maybe that is the underlying problem here. There should be a configuration option that makes it a requirement for there to be a route. At least you would better see/know when the callback was failing or if there was a format problem with the response. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3416 Registered: 08-2008 |
Follow-up: Please feel free to email logs to nowsms@nowsms.com with Attention: Des in the subject line. Also for clarification, a 500 error would be more appropriate. | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3417 Registered: 08-2008 |
Hi Chris, The problem is that the PreAuth callback did not return a route. (If SMSCRoute=blah had been returned there, the message would have been rejected.) By the time it gets to the SmsSend callback, it is too late to reject the message, so any invalid route is simply ignored. My thinking is that the best way for us to avoid confusion on this issue is to implement a setting that requires the SMSCRoute to be set in the PreAuth callback. I've tasked our engineers to add such a setting. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3420 Registered: 08-2008 |
Hi Chris, OK ... I think this will be the solution. I posted version 2011.08.18 at http://www.nowsms.com/download/nowsms20110818.zip It adds a new setting ... SMSAccountingMustSetRoute=Yes (Use this setting instead of SMSAccountingPreAuthPerRecip=Yes) This setting indicates that the PreAuth accounting callback must set an SMSCRoute. If it does not (or if it is invalid), the message will not be accepted. (And because it is not accepted, it will not be subject to so-called random routing.) -- Des NowSMS Support | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 17 Registered: 06-2011 |
Des, For us to be able to move our routing decision making to the PreAuth callback we will need an additonal data item namely the MessageID so that we can correlate callbacks. Hopefully allocating an id to the message before PreAuth should be easy and the only impact would be non contigous MessageID's if you rejected which shouldn't be a n issue. Without this we will not be able to correlate and therefore still cannot utilise the new features. Thanks Dave | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3425 Registered: 08-2008 |
Hi Dave, I can't go into all of the logic, but at this time, providing/allocating a message ID prior to the PreAuth callback is not an option. Before we get too much further into this discussion, I want to backup and look at the original problem/issue. The issue is that an accounting callback is not required to supply routing information. If an accounting callback returns invalid routing information (or doesn't provide any routing information), NowSMS uses whatever routing logic has been configured. While this is not necessarily a bug, we agree that there are many configurations where routing should only be performed through an accounting callback. In installations that want to work in this way, if there is an error in the accounting callback, instead of routing the message randomly (because no routing information has been provided), we need an option to reject these messages that cannot be routed. My initial thought of how to address this was to add a configuration option to force routing information to be provided to the PreAuth callback. This option works well because in this scenario, the message can be rejected before it even makes it as far as NowSMS. You indicate that this will not work for you because you cannot provide routing information until a message ID has been assigned. Unfortunately, we cannot do this. If you need correlating data, it is possible to return an additional response in the PreAuth callback: UserData=xxxxxxxxx (best to be sure to terminate with a line break to signal end of data) If this field is present in the response, NowSMS will return it in subsequent SMSSend and SMSOut accounting callbacks ... which can be useful. However, that adds to the complexity considerably. Perhaps that would work for your needs, so I mention it. In the meantime, we are going to modify the SMSAccountingMustSetRoute=Yes logic to work when routes are set via the SMSSend callback instead of the PreAuth. When an invalid (or missing route) occurs in an SMSSend callback, NowSMS will have already accepted the message, but it will reject the message at that point (and return a non-delivery notification if one was requested). This will take some time ... I do not have a time estimate yet. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3436 Registered: 08-2008 |
Hi Chris/Dave, We made a change that I think will resolve the issue. There are now two separate configuration settings for scenarios where the route must be set by accounting callbacks. The setting SMSAccountingMustSetRoute=Yes that was introduced in the previous update now applies to the Type=SMSSend callback (not the PreAuth). Keep in mind that the SMSSend callback occurs after NowSMS has already accepted the message. When this setting is present, if an SMSSend callback does not return a route, the message will be queued for generating a non-delivery notification (if requested) and then rejected. (Regardless of this setting, if an SMSSend callback returns an invalid route, this will also occur.) A new setting, SMSAccountingMustSetRoutePreAuth=Yes works in a similar fashion, but requires that the PreAuth accounting response include a route. If it does not, and this configuration setting is specified, the message will be rejected, and the submitting client will receive an error response. (Regardless of this setting, if an SMSSend callback returns an invalid route, this will also occur.) I hope this addresses your issue. The update is at http://www.nowsms.com/download/nowsms20110826.zip. If you have any follow-up issues, please accept my apologies in advance, as I will be on holiday next week. A colleague will be monitoring in my absence. -- Des NowSMS Support | ||||
| Rik Clews New member Username: Rikclews Post Number: 28 Registered: 08-2006 |
Hi, I downloaded this version and set up using SMSAccountingMustSetRoute=Yes and when we return a valid route all seems to be in order. But if returning an invalid route (specifically a route name that isn't present in NowSMS) we get nothing other than an SMSOUT callback along the lines of Type=SMSOut&From=xxx&To=%2b447961xxxxxx&MessageID=xxx.req&SubmitIP=xxxx&Sender=T est&Validity=5M&Text=test+from+xxx&ReceiptRequested=Yes&SMSCRouteName=xxx&SMSCNa me=SMPP+-+127.0.0.1%2335%3a2775&Status=Error%3a++No+SMSC+defined+to+route+messag e However, we expected a rejected receipt forwarded to the client and the 2 way handler, I can't see where the SMSOUT call ends up, but nothing is forwarded to the client and nothing to the 2 way. Having said that from a couple of dozen test submissions we did receive an expected receipt once, although two copies arrived simultaneously one witha a done date of 0000000000 the other with a correct looking date. For this one we had the SMSOUT callback but also an SMSIN plus the 2 way. Just to check I restarted the server and retested and the first bad route gave the SMSIN and the double 2 way responses, but subsequent calls only had the SMSOUT. None of these receipts seemed to make it to the client either. I'm more than a bit confused by this! | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3449 Registered: 08-2008 |
Hi Rik, It depends on whether or not the submitting client requested a delivery receipt. If they did not, NowSMS does not return a non-delivery report. There is an issue that the "donedate" is currently not being generated correctly. If you are using the "DuplicateUserMessagesFor2Way=Yes" or "DuplicateUserReceiptsFor2Way=Yes" setting, the message routed to the user account will get an invalid "donedate", while the copy that goes to the 2-way command has the correct "donedate". We are preparing a fix for this "donedate" issue. (Note: Related to this, the delivery receipt generated also has an invalid validity period and service type. This is also being fixed.) If you want non-delivery notifications to always be generated if there is an error, there is a configuration setting to force this for SMPP submissions, but not for HTTP submissions. To enable this, under the [SMPP] section header of SMSGW.INI, add DefaultNonDelReceipt=Yes -- Des NowSMS Support | ||||
| Rik Clews New member Username: Rikclews Post Number: 29 Registered: 08-2006 |
Hi Des, I've done some retesting and still have issues, the rejected receipts are not being generated. Here's how I'm testing. Locally I have installed an evaluation copy of NowSMS v2010.11.04. On our staging servers I have NowSMS v2011.08.26. I set up two SMS users on staging with Inbound SMS Routing set, one has a fixed route that doesn't exist in NowSMS the other a route that is present. The INI has the following settings: DefaultNonDelReceipt=Yes DuplicateUserMessagesFor2Way=Yes SMSAccountingMustSetRoute=Yes DuplicateUserMessagesFor2Way=Yes AccountingExtended=No I have set up a SMSC connection to each account from my local NowSMS. I can receive inbounds locally via SMPP and get them on staging via 2-way as expected. The Good route user sends messages (receipt requested) and I get local SMPP receipts plus a copy via 2-way as expected. The bad route user sends messages (receipt requested), I get it on staging via the PreAuth SMSSEND callback and route it via the SMSSEND callback, but no receipt or error is received by the client or the 2-way. If I set AccountingExtended=Yes, I can see SMSOUT, SMSIN and 2-way calls for the good route, but only SMSOUT for the bad route - although that has the text "Status=Error: No SMSC defined to route message" appended to it, but although we do nothing just respond with OK, there is no SMSIN and nothing is passed to SMPP client or 2-way. We expect a rejected receipt here. Can you test that and see whats happening? Cheers, Rik | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3450 Registered: 08-2008 |
Hi Rik, I was about to ask for logs ... but let's try this quick ... The fact that you see that SMSOUT callback indicating there is no SMSC route is a very good sign. Make sure that DefaultNonDelReceipt=Yes is under the [SMPP] section in SMSGW.INI. The other settings are in the [SMSGW] section. I do have a patch that fixes the "donedate" issue ready to be uploaded, but I'd like to wait to make sure that there isn't some other issue we've missed. -- Des NowSMS Support | ||||
| Rik Clews New member Username: Rikclews Post Number: 30 Registered: 08-2006 |
Aha, I did miss the DefaultNonDelReceipt=Yes being under the [SMPP] section, but after I moved it from [SMSGW] I found no difference. I restarted NowSMS and the first bad route worked (double 2-way bad done date), then as previously, stopped working on subsequent bad route calls. I cleared the debug logs and rebooted the server, and the first call worked again, but this time the second call worked too! Sadly all subsequent attempts were unreceipted as before. In anticipation I've attached the debug logs that cover the last test (post reboot). Cheers, Rik
| ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3451 Registered: 08-2008 |
Thanks Rik, I see it now. I found a related configuration where a non-delivery receipt would fail to be generated 100% of the time. The problem should be fixed by tomorrow. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3452 Registered: 08-2008 |
Hi Rik, OK ... I think this should do it. In the first attempt at having the accounting callback return an invalid route at the SMSSend stage, there were multiple paths for generating the non-delivery receipt, and one of these paths failed to generate it. This version resolves that issue: http://www.nowsms.com/download/nowsms20110907.zip -- Des NowSMS Support | ||||
| Rik Clews New member Username: Rikclews Post Number: 31 Registered: 08-2006 |
Sorry to say this version is worse. I've downloaded it installed it (causing a reboot so all loaded as should be) and ran the same test as last time. The bad routes are now receipted to SMPP client and 2 way, but the good routes aren't - although they were yesterday! Now no receipts at all for good routed SMPP traffic. I saw a receipt which had the gateway Id, sender and recipient of someone else's http submission with the receipt text field clearly showing the text of one of my smpp tests, one of the good route unreceipted ones. So who's receipt was it? Logs attached - can you check it out? Cheers, Rik
| ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 18 Registered: 06-2011 |
Des, Although we appreciate fast response times to our issues can you ensure you are running sufficent unit and integration tests before releasing to us as serious issues such as no recipts for valid submissions should not be being raised by us. We can assist with testing the more difficult customer scenarios but the core functionality regression tests need to be done by the NowSMS team prior to release. Thanks Dave | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3454 Registered: 08-2008 |
Hi Rik, Can you give me a starting point to look at? The receipt that you know is wrong would be good. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3455 Registered: 08-2008 |
Hi Rik, OK ... I was too quick to ask for a starting point. I'm investigating more ... but the root of the issue seems to be that the SMSC is returning simplistic re-used message IDs such as 103, 104 and 105. Of course, this is common when stopping and restarting simulators. However, the most recent assignment of the message ID should take precedence over earlier assignments, so I will investigate this. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3456 Registered: 08-2008 |
Hi Rik, Unfortunately this is not a new issue. If an SMSC reuses the same message ID on the same day, the message tracking logic will assume that any receipts pertain to the first assignment of that message ID. If the SMSC uses the message ID one day, and then reuses it the next day, the tracking logic will assume that any receipts pertain to the most recent assignment of that message ID. There's no perfect fix for this, but it is basically a simulator issue where this problem is encountered, so it is not critical. (We will change the logic so that the tracking logic will assume that any receipts pertain to the most recent assignment of that message ID.) If I understand the problem that you are describing, this is why it occurs. Your TestGood "test 4" message was assigned an upstream message ID of 106. Earlier today, when running other tests, that same SMSC had already assigned message ID 106. When the delivery receipt was received, NowSMS processed it as if it was a delivery receipt for the earlier instance of that same message ID ... that is why it was not processed as expected. I don't think anything has gotten worse with this update. If you want to purge the message ID tracking databases before testing, delete the SMPPDATA subdirectory. Obviously, I don't advise that on a production system ... but this problem should not happen on a production system. -- Des NowSMS Support | ||||
| Bryce Norwood - NowSMS Support Board Administrator Username: Bryce Post Number: 7996 Registered: 10-2002 |
Hi Dave, I just want to jump into this thread for a minute. First, I want to assure you that we share your concerns. I've reviewed all of the interaction and updates involved here, and I do not believe that any point did any of these updates break any functionality provided by earlier releases. I would indeed be very concerned if important functionality like delivery receipt processing was broken. However, I believe this problem had nothing to do with the NowSMS update, and instead was caused by the problem Des described. If you stop and restart an SMPP simulator, it will immediately reuse previously issued message ID assignments. This results in an ambiguity for how the receipts should be processed. I concur with Des that if a message ID is reused, any receipt should be assumed to be associated with the most recent instance of the message ID. This is normally the case with NowSMS, but it is not the case if the upstream SMSC reuses the same message ID on the same day, as the first use for that day is the only one that is saved. We will fix this, but this should only be a problem for testing environment scenarios. This is not a new issue, and should not be construed as a fault in the 2011.09.07 update. Where we have fallen down in testing is with the implementation of two features that were requested by your organisation. Specifically: 1.) DuplicateUserReceiptsFor2Way=Yes originally did not do what it was supposed to do if the receipt was generated by NowSMS (triggered by an upstream SMSC rejection). 2.) Non-delivery receipts triggered by an SMSSend accounting callback returning an invalid route name originally was not consistent about actually generating non-delivery receipts. (This was a configuration dependent issue.) There was also a "donedate" problem with these non-delivery receipts, but that is minor. We've also had other recent issues not covered in this thread. 3.) NowSMS previously assumed that if an accounting callback was returning routing information, it must be correct. If it was incorrect, it ignored it. When this issue was raised, we agreed that this should be changed. However, our implementation did not match the way that you use accounting callbacks, so we've had several iterations to adjust it to your usage. 4.) NowSMS did not track various non-delivery status reports from upstream SMPP providers, treating them as only deliverable and undeliverable. You raised the issue that you also need the expired status to be maintained. We provided an update to address this ... and then were told that the issue wasn't just EXPIRED, it was all statues that were required to be supported. In the case of #1 and #2, we could have done a better job testing all possible scenarios when engineering the new functionality. We will endeavour to try harder to determine all possible scenarios and combinations in which a new setting or feature will be used. In the case of #3 and #4, unfortunately, we need to ask more questions before racing off to implement worthwhile ideas. In fact, the same could probably be said of #1 and #2. I apologise for any inconvenience. I do want to stand up for Des and his testing team, however (and the developers too). It does not appear that any existing functionality stopped working or was broken through any of this process. Indeed, these days, it does seem normal across this industry, that customers have to be weary of each and every update, not knowing what unexpected problems and bugs await them. I do not believe that was the case here. We will strive to do better when evaluating and implementing feature requests. But I don't want this to lead us to being more inclined to denying feature requests. All of the features that you asked for are good additions to the NowSMS product, and make it a more solid product. Unfortunately some new features require a few iterations to get them just right. -bn | ||||
| Rik Clews New member Username: Rikclews Post Number: 32 Registered: 08-2006 |
Ah Des, apologies, you are quite right. A re-test this morning has everything coming through as expected, plus I stopped and started our simulator a couple of times and knowing what to look for I found the SMSC Ids starting at 0 after each restart, I doubt O2 will be doing that anytime soon, but it may save future confusion if the latest Id was assumed as a match for the receipts. One small thing is that we receive 2 receipts for the bad routes, but they are the same now, one has a message Id of xx.in and the other xx.req, not a particular issue as we handle both and don't expect it to happen very often. More testing to do, but I think we can run with this one. Cheers, Rik | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3467 Registered: 08-2008 |
Hi Rik, We will have a fix for addressing the message ID reuse next week. As I understand, it is a simple matter of changing an SQL statement from "INSERT" to "INSERT or REPLACE". We also noticed that when one of these non-delivery notifications is generated for an invalid route, the log reports the status correctly, but also makes reference to a random SMSC. We are going to change the random SMSC reference to simply report "SYSTEM". Can you please explain the 2 receipts issue that you are seeing? I think that would be related to the DuplicateUserMessagesFor2Way=Yes setting ... one receipt going to the submitting user, and the other going to the 2-way command processor. -- Des NowSMS Support | ||||
| Rik Clews New member Username: Rikclews Post Number: 33 Registered: 08-2006 |
I log the 2 way inbound querystrings and for each SMPP bad route call I get something like the below (within 1 second of each other) at the 2 way handler: 2 way receive parameters: sender=%2b447700900999&smsprefix=id%3ared4D4CFE34&fullsms=id%3ared4D4CFE34+sub%3 a001+dlvrd%3a000+submit+date%3a1109091104+done+date%3a1109091104+stat%3aREJECTD+ err%3a00B+text%3atest+6+from+stgR222&recip=TestBad&messageid=red4D4CFE36.REC&rec eiptmessageid=red4D4CFE34.req&servicetype=&smscroute=SMPP-Test&msgdate=20110909& msgtime=110458&binary=0&udh=&pid=00&dcs=00 2 way receive parameters: sender=%2b447700900999&smsprefix=id%3ared4D4CFE34&fullsms=id%3ared4D4CFE34+sub%3 a001+dlvrd%3a000+submit+date%3a1109091104+done+date%3a1109091104+stat%3aREJECTD+ err%3a00B+text%3atest+6+from+stgR222&recip=TestBad&messageid=red4D4CFE38.in&rece iptmessageid=red4D4CFE34.req&servicetype=&smscroute=SMPP-Test&msgdate=20110909&m sgtime=110458&binary=0&udh=&pid=00&dcs=00 Cheers, Rik | ||||
| Dave Baddeley New member Username: Davebaddeley Post Number: 19 Registered: 06-2011 |
Bryce, Thanks for looking into and explaining the situation; it looks like the simulator id recycling was causing some issues after the reboots. I think at times it would be benefical if there could be a more direct communication, such as a Skype call to help flesh out requirements before a change; especially if complex. Dave | ||||
| Rik Clews New member Username: Rikclews Post Number: 34 Registered: 08-2006 |
Hi Des, I've come across one more anomaly, in that it takes considerably longer to stop the application than it did previously. We found it as we have had a scheduled task for the last 18 months that simply stops and starts the NowSMS service (doing a simple net stop/start), which was put in place due to server slow down (looked to be memory leaking) and this cured it. Since we released the new version that has caused this script to stop NowSMS OK but it no longer restarts, seems it takes too long to stop logs it hasn't responded then attempts the start but the command is ignored as its currently in a stopping state. My main concern is that this means the service is taking over 30 seconds to stop - seems a hellishly long shutdown? We do have 44 binds and 6 SMPP users on our live system, but that hasn't changed and in the previous version stopping was around 10 seconds. Any ideas why this has changed? Cheers, Rik | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3472 Registered: 08-2008 |
Hi Rik, Ok. I think I see how to recreate this problem. For some reason the shutdown delay is related to the SMPP keep-alive interval. If you can lower these values, shutdown is more responsive. I am investigating why this is, and what changed. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3476 Registered: 08-2008 |
Hi Rik, We've done a thorough review of the shutdown timing. The problem I mentioned about the keep-alive interval may not be what you were experiencing. As it turns out, this has been a long standing issue that delayed shutdown for any SMPP connections that did not have async windowing enabled. We have fixed this problem. However, as it does seem to have existed for quite sometime, it is unlikely that you would just start experiencing it, unless you had a configuration change. In the process, we found a bug where transceiver connections would take an extra 15 seconds to respond to a shutdown request. This is a relatively new bug related to changes to support multiple concurrent sessions for a single logical connection, and might be what you're seeing. We hadn't considered this a problem, as our shutdown period target was 20 seconds. But we can see where this unintended 15 second delay would be an increase over previous versions. A fix for this should be available for download in the next 24 to 48 hours. It also addresses the two issues that I mentioned in my post on Sept 9, and the duplicate SMSIN callback for locally generated non-delivery receipts. The duplicate SMSIN callback is a problem with the DuplicateUserMessagesFor2Way=Yes setting generating a duplicate receipt in a situation where it is not necessary. Normally, the duplicate receipt should only be generated if the original receipt is getting routed to a local user account. If the local user account is not configured to accept messages (i.e., HTTP submissions), the setting was duplicating these receipts even though it was not necessary. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3484 Registered: 08-2008 |
Hi Rik, Ok ... hopefully we can finally put these issues to rest. My description of the duplicate message problem was incorrect in the previous message. I was referring to a duplicate inbound receipt being routed to a 2-way command, not a duplicate SMSIN callback. That problem is fixed. The extra 15 seconds in the shutdown has also been fixed, along with additional speedup of the service shutdown. The simulator message ID reuse issue is fixed. Log and accounting callbacks now reference an SMSC of SYSTEM for non-delivery notifications triggered by an invalid route (instead of referencing a random SMSC connection). The update download is at http://www.nowsms.com/download/nowsms20110920.zip. -- Des NowSMS Support | ||||
| Rik Clews New member Username: Rikclews Post Number: 35 Registered: 08-2006 |
So far so good all that seems to be in order. Unfortunately there is another issue, if we reject (or fail to route) a message on the SMSSEND it still picks a random route and sends. Seems counter intuitive if we have a setting "SMSAccountingMustSetRoute" but if we don't set the route it still gets sent randomly. This is how we're getting it - we have the ability to restrict our users to UK only sends, in which case if the recipient is a non UK MSISDN when it comes to routing them we detect they can't send this and return the string "SMPPErrorCode=0x66" with a http status of 401 (unauthorized). Instead of the usual "SMSCRoute=xxx" and 200 (OK). Although there could be a multitude of reasons why we error to give the same result. What I'd expect to see here is the same processing as an invalid route so we don't send the message, just reject & receipt as rejected. Can you fiddle that in? I could simply return a bad route name, but that seems dangerous and I could see if we have an issue at our end we could be sending messages without realising or knowing which route they are going through. Cheers, Rik | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3486 Registered: 08-2008 |
Hi Rik, OK ... that is happening because of the "401" response. That is not expected and was not factored into our logic. If you change that to a "200", it would work. Here's more explanation of how this is handled. If a PreAuth callback returns a non-OK response (such as 401), the message will be rejected. Note, however that the SMPPErrorCode=0x66 will not be returned to the submitter, as this is only parsed if the HTTP response is 200 OK. (The user will receive the default SMPP error code, which is ESME_RTHROTTLED.) Historically, the HTTP response to the SMSSend callback has been ignored. If you want to block a message from being submitted, it needs to be done via the PreAuth. Then we added support for allowing a route name to be specified here, but the "SMSCRoute=" response is only read if the response status is 200 OK. You raise a good point, however, if SMSAccountingMustSetRoute=Yes is set, and the accounting callback fails or returns a non-OK response, then logic should dictate that the message routing has failed and the message should be discarded. We'll give this another try. -- Des NowSMS Support | ||||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3487 Registered: 08-2008 |
Follow-up ... Update at http://www.nowsms.com/download/nowsms20110922.zip The only change is that if SMSAccountingMustSetRoute=Yes is set, if the SMSSend callback fails or returns a non-OK response, the message will be rejected as having no routing information. -- Des NowSMS Support | ||||
| Rik Clews New member Username: Rikclews Post Number: 36 Registered: 08-2006 |
Hey Des, Downloaded, tested and released to our live platform. I do believe our work here is done! Thanks for the efforts. Cheers, Rik | ||||
Debug Logs
