SMSCReceiptMsgId flipping between Hex and Decimal

SMSCReceiptMsgId flipping between Hex and Decimal SearchSearch
Author Message
Volodymyr Mandrychenko
New member
Username: Vmandrychenko

Post Number: 2
Registered: 09-2007
Posted on Wednesday, November 05, 2008 - 09:58 pm:   

Hello,

I need help figuring out why SMSCReceiptMsgId is flipping between Hex and Decimal.

When the message is sent the SMSCReceiptMsgId I get from now sms is in Hex format. I am storing that ID for later, to map delivery receipt with original message. In about 10% to 25% now sms will return decimal ids with delivery receipts which are simply lost and making our reporting data inacurate.

Here is a snippet of SMS-IN log that shows both.

2008-11-05 00:00:02,15555555555,Text,id:4C5FEFB8 sub:001 dlvrd:001 submit date:0811050459 done date:0811050459 stat:ACKED err:003 text: ,25425,SMSCReceiptMsgId=4C5FEFB8
2008-11-05 00:00:03,15555555555,Text,id:0005627601 sub:001 dlvrd:001 submit date:0811042356 done date:0811050456 stat:DELIVRD err:004 text: ,25425,SMSCReceiptMsgId=5627601

Decimal numbers seem to be sequential.

This problem was happening in 1% of all cases. But starting Nov 1, I am seeing lot more (10% - 25%) of these invalid ids.

Upgrading to Now Sms 2008 from Now Sms 2007 had no effect.

Please help me resolve this issue.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 210
Registered: 08-2008
Posted on Thursday, November 06, 2008 - 12:38 am:   

Hi,

Those are untranslated message ids coming in from your provider.

Basically, when that message id comes in, NowSMS can't find a match in its database.

There was a problem with a specific SMSC (that also returned 10 digit decimal ids) where something about the receipt format was causing NowSMS to be unable to translate it ... which was fixed starting with the 2008.01.30 release.

I'm not aware of any other issues/problems in this area. The only other problem that we've seen recently involved garbage text in receipt messages (and that had to do with an SMSC that returned the wrong DCS value on receipts for Unicode messages).

So here's how we'll troubleshoot this.

Let's look at the example from your log above. Message id 0005627601.

Look through your SMSOUT-yyyymmdd.LOG prior to that message.

You want to see if you can find a log entry with one of the following:

SMSCMsgId=0005627601
SMSCMsgId=5627601
SMSCMsgId=55DED1
SMSCMsgId=0055DED1

Do you find one of these id's that would allow us to manually match up the message with its receipt?

If yes ... then we need to do some investigation to determine why NowSMS is not seeing the match up.

Are you perhaps using multiple connections to the same SMSC? Normally this is not a problem, if they appear in the SMSC list with the "#2", "#3", etc indicators. The issue there is that if the receipt comes back from a different connection, that could be a problem. NowSMS handles this automatically if it knows there are multiple connections to the same SMSC. But it might be fooled if you've used a HOSTS file trick on the multiple connections to the same SMSC look like different servers.

Another issue is that NowSMS only tracks the message id if the message is submitted with a delivery receipt requested. If a receipt is not requested, for performance reasons, NowSMS doesn't track the id. If the SMSC returns a receipt, even if one was not requested, that would cause it not to be translated properly.

Are you submitting messages via HTTP only? If so, you can put ReceiptRequested=Yes under the [SMSGW] header of SMSGW.INI to set this flag for all HTTP submissions.

(There is also a way to force this flag at the SMPP level, but I don't have my notes with me that explain how this is done. I'd have to look that up tomorrow if you need it.)

What if you don't find the corresponding message id in the SMSOUT log? Then we need to do some more investigation. Can you manually match up some SMSCMsgId values from SMSOUT with SMSCReceiptMsgId values from SMSIN? We can look closer at these to try to determine a pattern.

It's possible that there might be a problem that started on your provider's system on Nov 1.

But let's look at more examples to try to determine where the problem is.

--
Des
NowSMS Support
Volodymyr Mandrychenko
New member
Username: Vmandrychenko

Post Number: 3
Registered: 09-2007
Posted on Thursday, November 06, 2008 - 07:06 pm:   

I cannot find that message id in the logs. I have attached 2 OUT and 2 IN log files for Nov 4 and 5. I hope this helps you to determine the problem.

application/zipSMS-LOG
SMS-LOG.zip (8661.5 k)
Volodymyr Mandrychenko
New member
Username: Vmandrychenko

Post Number: 5
Registered: 09-2007
Posted on Thursday, November 06, 2008 - 07:58 pm:   

One thing to note. Our license expired on Nov 1, 2008, it seems that now sms went into some kind of trial mode with low throughput. We upgraded license and Now Sms version from CDs you sent us on Nov 3, 2008.

Here is number of invalid ID occurrences in chronological order.

1482 - 2008-10-31 - Now SMS 2007
10344 - 2008-11-01 - License Expired
9232 - 2008-11-02
21993 - 2008-11-03 License upgraded + Now Sms 2008 installed
24216 - 2008-11-04
24981 - 2008-11-05

Is there a possibility that this issue related to expired license and upgrade.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 216
Registered: 08-2008
Posted on Thursday, November 06, 2008 - 11:11 pm:   

Hi,

I'm not sure what you mean by license expired. I'm assuming that you just mean the updates & maintenance license. That expiration only prevents newer versions from being installed, it has no effect on operation.

I did some looking at the logs, and my preliminary analysis is that something is going wrong at the provider.

I'd like to get some more logs to prove this conclusively, and to give you the information that you need to talk to your provider about getting the problem resolved.

But before I do that, let me explain what I see.

In the SMSOUT log, let's take 20081104 as the example. You'll see two different id's in the log. After the time, you've got the NowSMS assigned message id, which is an incrementing 8 hex digit id (with the ".req" following it). In this particular log, all of these ids have a format of 4C5xxxxx.

You'll also see SMSCMsgId= values reported, and these are the ids assigned by your service provider. They seem to be using a 9 digit decimal number. Every id from the provider in this particular log starts with 170, so they have a format of 170xxxxxx.

If we look at the SMSIN log, we have two different formats for the receipt message ids. One is the 8 hex digit format, which is where NowSMS was able to translate the provider assigned message id back to the original NowSMS id.

In the other instances, this id is a 10 digit decimal number starting with 000. The first id of this format that we see in the log is 0004100772. The last is 0005627821. Unfortunately, I don't see any way that these numbers relate to the message id values that were originally returned when the message was submitted (170xxxxxx).

As an example, a message was sent to 17088215754 at 00:01:06. NowSMS assigned this an id of "4C5D2FA8". The provider indicated that it was assigned an id of "170200512".

At 00:01:49, we get a receipt back indicating that a message was delivered to this number, but referring to a message id of "0004105323".

Later that same second, we get a second receipt, but this one must have had the correct operator assigned id, because this time it gets translated back to the original NowSMS ID of "4C5D2FA8". (It must have come in from the provider with an id of "170200512", but NowSMS translates it before logging it.)

The "submit date" fields are different in these two receipts, so I think the first one is actually referring to an earlier message. But the message id "0004105323" is not close to the other "170xxxxxx" ids that the SMSC is assigning.

This suggests to me that the provider is having a message id translation problem of their own in their system.

Basically, you are getting delivery receipts back in to your system where the receipted_message_id in the receipt does not match any message_id returned in a submit message response (submit_sm_resp). The id sequences and formats are completely different.

If you'd like me to prove this, then I'd ask you to enable the SMSDEBUG.LOG. Beware, this will generate some pretty big files. Let NowSMS run in debug mode for about an hour ... then disable the SMSDEBUG.LOG and restart the service.

E-mail the SMSDEBUG.LOG, SMPPDEBUG.LOG, SMSIN-yyyymmdd.LOG and SMSOUT-yyyymmdd.LOG to nowsms@nowsms.com, putting "Attention: Des" in the subject line of your message.

I suspect the debug logs will confirm what I'm already seeing. But basically it looks like there is a delivery receipt message id problem at your provider. The delivery receipts they are returning do not consistently have correct message ids in them ... they don't match up with the ids that were returned by the SMSC when the messages were originally submitted.

--
Des
NowSMS Support
Volodymyr Mandrychenko
New member
Username: Vmandrychenko

Post Number: 6
Registered: 09-2007
Posted on Friday, November 07, 2008 - 07:54 pm:   

Thank you for very detailed explanation, it is very helpful. I am going to talk to our provider, to try to get to the bottom of this.