Receiving err 008

Receiving err 008 SearchSearch
Author Message
Muhammad Adnan farooq
New member
Username: Adnan

Post Number: 44
Registered: 10-2009
Posted on Wednesday, February 24, 2010 - 04:56 pm:   

One of our customer is having trouble while sending SMS from a device in the field to a short code. We can see error message "rejected err 008".

2010-02-23 23:09:25,celluland46,SMSReceive,720046,+15146619588,8252A981.req,Text,Text=id:82 52A260 sub:001 dlvrd:000 submit date:1002232003 done date:1002232304 stat:REJECTD err:008 text:|00005618,254,18,+1

Can you please guide us what does it mean by 008 as I am unable to find this error code?

Thanks
Adnan
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1845
Registered: 08-2008
Posted on Wednesday, February 24, 2010 - 05:52 pm:   

Hi Adnan,

In SMPP environments, this is a generic error that simply means "System Error" (ESME_RSYSERR).

I would look in your SMSOUT logs for messages destined to this short code to determine what SMSC connection actually rejected the message.

Verify that is the correct SMSC connection to which the message should be routed. If it is not correct, then you need to check the routing definitions to determine why the message is being routed to the wrong connection.

If the SMSC connection is correct, but the message is being rejected, try additional test messages to narrow down the problem.

I might be able to give more suggestions if I understood the logic path that is used on your system. When a device in the field sends an SMS to a short code, what is the path that this message should take if everything was working as you expect it to?

--
Des
NowSMS Support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1846
Registered: 08-2008
Posted on Wednesday, February 24, 2010 - 06:01 pm:   

I went back and looked at some of your recent threads, and it is likely that this is just a slight variation of previous problems.

The error code itself is likely not significant. The problem is likely that the message is not taking the expected path.

As I understand it, a message being sent to a short code is received by your system from an upstream SMS provider. It should never be sent out to your upstream SMS provider. Is that correct?

It would help if you could explain your configuration some more. You seem to have more than one NowSMS server. But I don't understand the logic flow of how they are supposed to work together.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1847
Registered: 08-2008
Posted on Wednesday, February 24, 2010 - 06:14 pm:   

I've been thinking about this some more.

Seeing the SMSOUT log entry would help a lot.

What you're seeing here is that 720046 tried to send a message to +15146619588. It was this message that failed, and an error is being sent back to 720046 indicating that the message could not be sent.

Based upon the configuration files you sent me previously, messages to +1514* should be routed to "SMPP - 10.191.2.30:2780".

I expect that we'll find in the SMSOUT log that 720046 sent a message to +15146619588, and the upstream SMS provider is rejecting the message for some reason with SMPP error code 8 -- "System Error" (ESME_RSYSERR).

If this is what you see, then you'd have to ask the upstream provider why they are blocking this send attempt from 720046 to +15146619588.

Note that I am making an educated guess about what is happening. Look at the SMSOUT log to verify that my assumption about what you will see is true.

--
Des
NowSMS Support
Muhammad Adnan farooq
New member
Username: Adnan

Post Number: 45
Registered: 10-2009
Posted on Wednesday, February 24, 2010 - 08:16 pm:   

Thanks for your analysis.

I will get back to you shortly.
Muhammad Adnan farooq
New member
Username: Adnan

Post Number: 46
Registered: 10-2009
Posted on Wednesday, February 24, 2010 - 10:40 pm:   

If a Short code sends SMS to the device in the field:
It reaches our Production NowSMS server first through SMPP bind,
then goes to ESME having NowSMS server,
then goes to our SMS provider through SMPP bind with it & then to the device in the field.

The above logs are from Production NowSMS server.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1849
Registered: 08-2008
Posted on Wednesday, February 24, 2010 - 10:50 pm:   

Take a look at the SMSOUT on the NowSMS server that connects to your SMS provider. I think that is where you will find references to the error when NowSMS is trying to submit the message to the upstream provider.

I believe you will also see this error 8 referenced in the SMSOUT log ... in which case the upstream provider is rejecting the message with the SMPP error code ESME_RSYSERR (8).
Muhammad Adnan farooq
New member
Username: Adnan

Post Number: 47
Registered: 10-2009
Posted on Wednesday, February 24, 2010 - 10:56 pm:   

I will see in detail in SMSOUT for ESME.

Can you please tell us why SMS provider will be rejecting these SMS with err 008?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1850
Registered: 08-2008
Posted on Wednesday, February 24, 2010 - 11:20 pm:   

No I can't. This is outside of our control.

Assuming other messages are going out fine, only your provider can tell you what is wrong with this sender/recipient combination.

I would check to see if this shortcode is having trouble sending to other numbers to see if the problem relates to their short code.

Also try having another short code send to this number as maybe the problem is with that number or SIM.

This is a very odd error for a provider to return....usually it means a configuration error on the system generating this error code (which is your provider).
Muhammad Adnan farooq
New member
Username: Adnan

Post Number: 48
Registered: 10-2009
Posted on Thursday, February 25, 2010 - 03:08 pm:   

I am getting rejected errors for different numbers, not specific to a single number.

2010-02-24 16:35:41,celluland46,SMSReceive,720046,+12048877419,825313D3.req,Text,Text=id:82 52FF88 sub:001 dlvrd:000 submit date:1002241330 done date:1002241630 stat:REJECTD err:008 text:|00005254,133,7F\r
2010-02-24 16:46:31,celluland46,SMSReceive,720046,+15146619588,825314FB.req,Text,Text=id:82 5300D6 sub:001 dlvrd:000 submit date:1002241340 done date:1002241641 stat:REJECTD err:008 text:|00005618,133,83\r
2010-02-24 16:52:52,celluland46,SMSReceive,720046,+15146619588,825315A3.req,Text,Text=id:82 53015A sub:001 dlvrd:000 submit date:1002241347 done date:1002241647 stat:REJECTD err:008 text:|00005618,133,83\r
2010-02-24 18:57:56,celluland46,SMSReceive,720046,+12048877419,82531F70.req,Text,Text=id:82 530EF0 sub:001 dlvrd:000 submit date:1002241552 done date:1002241853 stat:REJECTD err:008 text:|00005254,139,1,0,3
2010-02-24 19:03:16,celluland46,SMSReceive,720046,+12508135224,82531FB8.req,Text,Text=id:82 530FF8 sub:001 dlvrd:000 submit date:1002241557 done date:1002241858 stat:REJECTD err:008 text:|00002534,143,206,0
2010-02-24 19:03:17,celluland46,SMSReceive,720046,+15146619680,82531FBB.req,Text,Text=id:82 530FFB sub:001 dlvrd:000 submit date:1002241557 done date:1002241858 stat:REJECTD err:008 text:|00005254,254,18,+1
2010-02-24 19:03:17,celluland46,SMSReceive,720046,+15146619588,82531FBC.req,Text,Text=id:82 530FFC sub:001 dlvrd:000 submit date:1002241557 done date:1002241858 stat:REJECTD err:008 text:|00005618,254,18,+1
Muhammad Adnan farooq
New member
Username: Adnan

Post Number: 49
Registered: 10-2009
Posted on Thursday, February 25, 2010 - 03:37 pm:   

Below are the logs from SMSOUT that connects the SMS provider:

2010-02-24 13:52:02,5056D730.req,10.191.2.31,+12048877419,ERROR: System Error - ESME_RSYSERR -- SMPP - 207.245.224.211:3829,SubmitUser=production;Sender=720046;Text="|00005254,133,7F
"
2010-02-24 13:52:02,5056EB18.req,,720046,OK -- LocalUser:production,SubmitUser=SMPP - 207.245.224.211:3829;Sender=+12048877419;Text="id:5056D730 sub:001 dlvrd:000 submit date:1002241051 done date:1002241352 stat:REJECTD err:008 text:|00005254,133,7F
"
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1852
Registered: 08-2008
Posted on Thursday, February 25, 2010 - 04:14 pm:   

Hi Adnan,

I think you definitely need to talk to your upstream provider (the one associated with the "SMPP - 207.245.224.211:3829" connection), and ask them why they are returning the ESME_RSYSERR when this short code user is sending messages.


--
Des
NowSMS Support
Muhammad Adnan farooq
New member
Username: Adnan

Post Number: 50
Registered: 10-2009
Posted on Thursday, February 25, 2010 - 04:23 pm:   

Thanks DES. I have contacted my upstream provider.
I will contact you again if I need some further assistance.

Do you have a list of all error messages like 008?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1855
Registered: 08-2008
Posted on Thursday, February 25, 2010 - 04:46 pm:   

Here is a list of the SMPP error codes documented in the SMPP specification.

(Note that the "err:" number that appears in a non-delivery report will not always match a number in this list. If NowSMS generates the non-delivery report it will. But it is also possible for an upstream provider to generate a non-delivery report, and it is possible that they might use different error codes. This is why I wanted you to verify the error code in the SMSOUT-yyyymmdd log file. We can then see that the non-delivery message was generated by NowSMS when the upstream SMPP provider rejected the message, and in this case we can be certain that the error code is from this list.)

ESME_RINVMSGLEN0x00000001Message Length is invalid
ESME_RINVCMDLEN0x00000002Command Length is invalid
ESME_RINVCMDID0x00000003Invalid Command ID
ESME_RINVBNDSTS0x00000004Incorrect BIND Status for given command
ESME_RALYBND0x00000005ESME Already in Bound State
ESME_RINVPRTFLG0x00000006Invalid Priority Flag
ESME_RINVREGDLVFLG0x00000007Invalid Registered Delivery Flag
ESME_RSYSERR0x00000008System Error
Reserved0x00000009Reserved
ESME_RINVSRCADR0x0000000AInvalid Source Address
ESME_RINVDSTADR0x0000000BInvalid Dest Addr
ESME_RINVMSGID0x0000000CMessage ID is invalid
ESME_RBINDFAIL0x0000000DBind Failed
ESME_RINVPASWD0x0000000EInvalid Password
ESME_RINVSYSID0x0000000FInvalid System ID
Reserved0x00000010Reserved
ESME_RCANCELFAIL0x00000011Cancel SM Failed
Reserved0x00000012Reserved
ESME_RREPLACEFAIL0x00000013Replace SM Failed
ESME_RMSGQFUL0x00000014Message Queue Full
ESME_RINVSERTYP0x00000015Invalid Service Type
Reserved0x00000016-0x00000032Reserved
ESME_RINVNUMDESTS0x00000033Invalid number of destinations
ESME_RINVDLNAME0x00000034Invalid Distribution List name
Reserved0x00000035-0x0000003FReserved
ESME_RINVDESTFLAG0x00000040Destination flag is invalid (submit_multi)
Reserved0x00000041Reserved
ESME_RINVSUBREP0x00000042Invalid ‘submit with replace’ request (i.e. submit_sm with replace_if_present_flag set)
ESME_RINVESMCLASS0x00000043Invalid esm_class field data
ESME_RCNTSUBDL0x00000044Cannot Submit to Distribution List
ESME_RSUBMITFAIL0x00000045submit_sm or submit_multi failed
Reserved0x00000046-0x00000047Reserved
ESME_RINVSRCTON0x00000048Invalid Source address TON
ESME_RINVSRCNPI0x00000049Invalid Source address NPI
ESME_RINVDSTTON0x00000050Invalid Destination address TON
ESME_RINVDSTNPI0x00000051Invalid Destination address NPI
Reserved0x00000052Reserved
ESME_RINVSYSTYP0x00000053Invalid system_type field
ESME_RINVREPFLAG0x00000054Invalid replace_if_present flag
ESME_RINVNUMMSGS0x00000055Invalid number of messages
Reserved0x00000056-0x00000057Reserved
ESME_RTHROTTLED0x00000058Throttling error (ESME has exceeded allowed message limits)
Reserved0x00000059-0x00000060Reserved
ESME_RINVSCHED0x00000061Invalid Scheduled Delivery Time
ESME_RINVEXPIRY0x00000062Invalid message validity period (Expiry time)
ESME_RINVDFTMSGID0x00000063Predefined Message Invalid or Not Found
ESME_RX_T_APPN0x00000064ESME Receiver Temporary App Error Code
ESME_RX_P_APPN0x00000065ESME Receiver Permanent App Error Code
ESME_RX_R_APPN0x00000066ESME Receiver Reject Message Error Code
ESME_RQUERYFAIL0x00000067query_sm request failed
Reserved0x00000068-0x000000BFReserved
ESME_RINVOPTPARSTREAM0x000000C0Error in the optional part of the PDU Body.
ESME_ROPTPARNOTALLWD0x000000C1Optional Parameter not allowed
ESME_RINVPARLEN0x000000C2Invalid Parameter Length.
ESME_RMISSINGOPTPARAM0x000000C3Expected Optional Parameter missing
ESME_RINVOPTPARAMVAL0x000000C4Invalid Optional Parameter Value
Reserved0x000000C5-0x000000FDReserved
ESME_RDELIVERYFAILURE0x000000FEDelivery Failure (used for data_sm_resp)
ESME_RUNKNOWNERR0x000000FFUnknown Error
Reserved for SMPP extension0x00000100-0x000003FFReserved for SMPP extension
Reserved for SMSC vendor specific errors0x00000400-0x000004FFReserved for SMSC vendor specific errors
Reserved0x00000500-0xFFFFFFFFReserved
Muhammad Adnan farooq
Frequent Contributor
Username: Adnan

Post Number: 51
Registered: 10-2009
Posted on Thursday, February 25, 2010 - 05:21 pm:   

Thanks DES.
Muhammad Adnan farooq
Frequent Contributor
Username: Adnan

Post Number: 52
Registered: 10-2009
Posted on Friday, February 26, 2010 - 03:27 pm:   

DES

Our SMS provider has come back to us saying that there is some optional parameters that our SMPP application is adding to, that is the reason they are rejecting those mesages.

Message length: 18
Message
Optional parameters
Message state: DELIVERED (2)
MATE smpp_pdu:1->smpp_session:1
smpp_pdu: 1

Do you know where we can modify to fix this issue?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1867
Registered: 08-2008
Posted on Friday, February 26, 2010 - 03:51 pm:   

Hi Adnan,

Now that's strange. Very strange.

This optional parameter would be set for a delivery receipt.

It should not be set for a normal message.

My first thought is that your customer is doing something strange when they submit their reply. If they are set these parameters when they submit their message via SMPP, NowSMS would forward the parameters on.

Either that, or some delivery notifications are getting routed to the provider because of some sort of circular routing. However, based upon your log excerpts above, I do not believe this to be the case.

I think your customer is doing something funny when they submit the message to you.

They would have to be setting "esm_class" to indicate that the message is a receipt and/or including an optional parameter to indicate message delivered.

Before you contact your customer, look in the "Q" directory structure on the NowSMS server for a ".ERR" file.

It would be great if you could find 5056D730.ERR, which would correspond to the message from Wednesday's logs that is referenced above.

If you can't find that file, there will likely be another.

I am expecting to see ReceiptDelivered=Yes set in that file.

This would indicate that the customer is setting esm_class to indicate that the message is a receipt and/or including the optional parameter "message_state" when they are submitting a message to you.

Let's look at the .ERR file ... if ReceiptDelivered=Yes is present in that file, then you'll need to tell your customer to fix their SMPP client to stop setting these parameters, because it is causing their messages to be rejected.

--
Des
NowSMS Support
Muhammad Adnan farooq
Frequent Contributor
Username: Adnan

Post Number: 53
Registered: 10-2009
Posted on Friday, February 26, 2010 - 04:28 pm:   

I am looking into Q directory but unable to find any .ERR file which contains info related to the above customer mentioned in the logs.
Moreover, this problem is now not specific to only one customer as we can see number of customers SMS being rejected with err:008 by our SMS provider.

There is one Q directory which is global for all & also there is one inside every customers folder. I can see entries in the global one but couldnt see any thing in the individual folder.

Please suggest.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1869
Registered: 08-2008
Posted on Friday, February 26, 2010 - 04:46 pm:   

Are there any *.ERR files? (And are you looking on the right server ... this would be the server that connects to your upstream provider.)

In the SMSGW.INI file for that server, do you have ErrorQRetainDays=0? If so, remove this line. The setting change should take effect immediately without requiring a server restart. Then you should be able to catch a .ERR file for next time. (This setting stops the .ERR files from being generated.)

Aside from looking at these .ERR files to validate whether or not these parameters are present ... there's not much else I can do to diagnose the problem.

The only other thing you could do is run a Wireshark trace between your NowSMS Server and the SMSC, so that we can verify that this flag is actually being set.

To prove my theory, I'd also need to see a trace between the customer and the NowSMS server.

Alternatively, instead of a Wireshark trace, enabling the SMSDEBUG.LOG on your servers would allow us to see the raw SMPP. (Enabling the SMSDEBUG.LOG enables the SMPPDEBUG.LOG ... it is the SMPPDEBUG.LOG that we are interested in.)

Enabling the SMSDEBUG.LOG requires a service restart, which I know your operational people are reluctant to do.

But somehow, we need to see the raw SMPP ... or at least one of these .ERR files to start.

If what your provider is saying is true ... my theory is that the originating SMPP client is setting the esm_class to indicate that the message is a receipt and/or setting the optional message_state parameter.

However, before you go to the customer with this, I'd prefer to see evidence (.ERR file or SMPP protocol trace). The provider might have been looking at the wrong message when they told you the problem that they are seeing on their end, in which case we would be pursuing the wrong path.

Getting an .ERR file should be the easiest task ... so start with that.
Muhammad Adnan farooq
Frequent Contributor
Username: Adnan

Post Number: 54
Registered: 10-2009
Posted on Friday, February 26, 2010 - 04:46 pm:   

Under Q dir, all the files contains NonDelReceiptRequested=Yes.

Please also note one thing that we are not getting rejection message for every SMS. Sometimes SMS goes through & sometimes gets rejected with err 008.

EG:
2010-02-25 11:50:08,825374B4.req,,720046,OK -- LocalUser:celluland46,SubmitUser=SMPP - 10.191.2.30:2780;Sender=+15146619680;Text="!00005254,D,149,01/01/1980 12:00:26,3.14,1.10,B1
"
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1870
Registered: 08-2008
Posted on Friday, February 26, 2010 - 04:49 pm:   

Re-reading your message, I just realised that you might be looking at the wrong Q directory structure.

There are two sets of Q folders/directories on each server.

NowSMS\Q is the outbound message queue.

NowSMS\USERS\username\Q is the received message queue for individual user accounts.

You want to look for .ERR files in the outbound message queue.

And ... as you have two NowSMS servers, you want to be looking for .ERR files in the outbound message queue directory on your server that connects directly to your upstream provider.

As I understand it, your customer user accounts are on a separate server. So if you see a USERS\username\Q directory for all of your customer accounts, then my understanding is that this is the wrong server.

You want to look at NowSMS\Q (and subdirectories of this directory), on the server that connects to your upstream service provider, for the .ERR files that I am referring to.

--
Des
NowSMS Support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1871
Registered: 08-2008
Posted on Friday, February 26, 2010 - 04:57 pm:   

NonDelReceiptRequested=Yes is ok. (It means send me a delivery receipt if there is a failure, but don't bother sending me a delivery receipt if everything is ok.)

Although suspiciously, this setting does set the registered_delivery parameter in the SMPP message to a value of 2.

I wonder if the SMS provider is confused about their diagnosis.

They say they are seeing message_state = 2. But maybe they mean registered_delivery = 2.

Have you found any *.ERR files for messages rejected with this error yet? Can you post a sample?
Muhammad Adnan farooq
Frequent Contributor
Username: Adnan

Post Number: 55
Registered: 10-2009
Posted on Friday, February 26, 2010 - 08:11 pm:   

Below is the .ERR file for yesterday:

[SMS]
SubmittedBy=10.191.2.31
SubmitUser=cingular
Sender=41080179
PhoneNumber=+15005007851
Data=SETPARAMS 18=proxy
pid=00
dcs=00
Binary=0
NonDelReceiptRequested=Yes


I can see similar files like this but none of them has the above customers info.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 1879
Registered: 08-2008
Posted on Friday, February 26, 2010 - 08:30 pm:   

Hmm...the provider's explanation does not make sense. Optional parameter message_state=2 would not be set for this message.

registered_delivery=2 would be set because of NonDelReceiptRequested=Yes

I'd suggest making a copy of this .ERR file. Remove that line. Put back as a .REQ file with a new name (TEST1.REQ would be fine). Does it go out ok without that parameter? If so, this parm is added because of a special config setting that you have in SMSGW.INI and that could be removed. But Before we jump to that conclusion, let's verify.

You also may need to ask your provider to look again because at least for these messages the parameter they referenced is not set.