SMS Coding

SMS Coding SearchSearch
Author Message
Rui Da Costa e Silva
New member
Username: Rui_dacosta

Post Number: 1
Registered: 07-2018
Posted on Monday, July 16, 2018 - 05:24 pm:   

Good Morning.

So we are using a SMPP conection and we are having issues with the coding of the character.

We want to send the sms with unicode activated but when we send using DCS =8 and But we usually use ISO-8859-1 because we need the Latin characters with the accents.

At the moment when we send SMS with that coding if the message has only one SMS it all ok no problem.

But if it exceeds 1 SMS then we get the message like this :

- Estimado Rui. Informamos que o per?odo do seu seguro est? Prestes a terminar. Queira por favor regulariz?-lo evitando assim a suspens?o das garantias do seguro. Para mais infrmaÇ?es queria contactar-nos pelos terminais.

The message should be :

- - Estimado Rui. Informamos que o período do seu seguro está Prestes a terminar. Queira por favor regulariza-lo evitando assim a suspensão das garantias do seguro. Para mais informações queria contactar-nos pelos terminais.

So what can we do about the codification?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5938
Registered: 08-2008
Posted on Monday, July 16, 2018 - 09:44 pm:   

Hi,

In order to send those characters out over the air, the actual SMS must be encoded as Unicode (DCS=8).

Some SMPP providers support iso-8859-1 via the SMPP protocol, but they need to re-encode the text for the actual SMS. The actual SMS must use the GSM 7-bit character set or Unicode.

Resolving your problem will probably require changes in how you encode these messages.

To advise, I need to understand your configuration better.

You are connecting to NowSMS via SMPP, correct?

What is the outbound SMSC connection? Also SMPP?

--
Des
NowSMS Support
Rui Da Costa e Silva
New member
Username: Rui_dacosta

Post Number: 2
Registered: 07-2018
Posted on Tuesday, July 17, 2018 - 08:44 am:   

Hello

The SMS are enconded with "DCS=8".

Now SMS is conected via http and then it conects to the SMSC ia SMPP.

What kind of coding is used when we leave now sms in default??
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5940
Registered: 08-2008
Posted on Tuesday, July 17, 2018 - 09:40 pm:   

Hi,

I'm sorry that I am not understanding.

Is the following a correct interpretation?

You submit the following message to NowSMS via HTTP:

Estimado Rui. Informamos que o período do seu seguro está Prestes a terminar. Queira por favor regulariza-lo evitando assim a suspensão das garantias do seguro. Para mais informações queria contactar-nos pelos terminais.

Some characters are replaced with "?" ... but this only occurs on longer messages. (Normally this would be over 70 characters.)

NowSMS is sending out via SMPP (and the logs show DCS=8).

--

If my understanding above is correct, this is not a problem that I have seen before.

I'd like to look at your logs, so that I can better understand. In particular, the SMSWEB-yyyymmdd.LOG and SMSOUT-yyyymmdd.LOG. To expedite troubleshooting, I'm also going to ask for SMSDEBUG.LOG and SMPPDEBUG.LOG (there is a checkbox to enable SMSDEBUG.LOG on the "Serial #" page). Right before you collect the logs, send an example of a short message that works and a long message that does not.

E-mail the logs to nowsms@nowsms.com and include Attention: Des on the subject line, Also, please post a reply here to tell me that you have sent the logs.

The only thought that I have...in the SMPP SMSC configuration in NowSMS, under Advanced Settings is the WDP Adaptation setting enabled? That is the only setting that I can think of which might present this type of issue.

--
Des
NowSMS Support
Rui Da Costa e Silva
New member
Username: Rui_dacosta

Post Number: 3
Registered: 07-2018
Posted on Wednesday, July 18, 2018 - 08:58 am:   

Hi

We did not had the SMSDEBUG.LOG activated so i am not sure if the log is correct.

I have send the logs as you requested.

Thank You for the support.
Rui Da Costa e Silva
New member
Username: Rui_dacosta

Post Number: 4
Registered: 07-2018
Posted on Thursday, July 19, 2018 - 08:18 am:   

Hello

The email that was send yesterday was rejected by your side.

How can we send you the logs??

Regards
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5942
Registered: 08-2008
Posted on Thursday, July 19, 2018 - 12:55 pm:   

Hi,

Sorry for the delay. I did get the email.

I didn't get SMPPDEBUG.LOG, but I did receive the others.

From what I can see, NowSMS is recognizing the long Unicode message ( > 70 characters), and breaking the message in 67 character blocks. It is setting the DCS=8 for Unicode, and including the segment headers to allow the client to reassemble.

I'd like to see SMPPDEBUG.LOG to make sure I am not missing something unusual, but from what I can see NowSMS is encoding the message correctly.

I can only assume that your provider is reassembling the message and then re-encoding it wrong. Unfortunately, you would need to talk to them about it...in which case a Wireshark trace would probably help them....that trace should show 2 separate message submissions, one for each segment, in Unicode format with the correct character codes. (Our SMPPDEBUG.LOG would show this same information, which is why I would like to check it also.)

--
Des
NowSMS Support
Rui Da Costa e Silva
New member
Username: Rui_dacosta

Post Number: 5
Registered: 07-2018
Posted on Thursday, July 19, 2018 - 01:00 pm:   

Hello

That is also what i was thinking because on our logs it seems that everything is fine but then the SMS all come badly coded.

I must talk the ou provider then.

I will send you the SMPPDEBUG.LOG.

Thank you for the support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5943
Registered: 08-2008
Posted on Monday, July 23, 2018 - 02:50 am:   

Hi,

Sorry for another delay.

The debug logs rollover every day (and on service restarts), so the SMPPDEBUG.LOG you sent did not have the earlier message data.

So here is an example of me sending one of your examples, to show what you should see. When Unicodee format is used, the character display in SMPPDEBUG.LOG on the right side will appear with extra spaces. The left side is the actual data being sent...the right side is just to aid in locating info.

I've highlighted in red the Unicode encoding for a few characters. I suspect you will see the correct codes in your SMPPDEBUG.LOG. I think your provider is reassembling the message and then re-encoding it wrong, and you will need to talk to them about it.


21:25:31:959 (00000494) 45.40.135.245 ->: 00 00 00 D0 00 00 00 04 00 00 00 00 00 00 0A 2F /
21:25:31:959 (00000494) 45.40.135.245 ->: 00 00 00 00 01 01 39 39 39 39 39 39 39 39 39 39 9999999999
21:25:31:959 (00000494) 45.40.135.245 ->: 39 00 40 00 00 00 00 00 00 08 00 8C 05 00 03 A3 9 @
21:25:31:959 (00000494) 45.40.135.245 ->: 04 01 00 45 00 73 00 74 00 69 00 6D 00 61 00 64 E s t i m a d
21:25:31:959 (00000494) 45.40.135.245 ->: 00 6F 00 20 00 52 00 75 00 69 00 2E 00 20 00 49 o R u i . I
21:25:31:959 (00000494) 45.40.135.245 ->: 00 6E 00 66 00 6F 00 72 00 6D 00 61 00 6D 00 6F n f o r m a m o
21:25:31:959 (00000494) 45.40.135.245 ->: 00 73 00 20 00 71 00 75 00 65 00 20 00 6F 00 20 s q u e o
21:25:31:959 (00000494) 45.40.135.245 ->: 00 70 00 65 00 72 00 ED 00 6F 00 64 00 6F 00 20 p e r o d o
21:25:31:959 (00000494) 45.40.135.245 ->: 00 64 00 6F 00 20 00 73 00 65 00 75 00 20 00 73 d o s e u s
21:25:31:959 (00000494) 45.40.135.245 ->: 00 65 00 67 00 75 00 72 00 6F 00 20 00 65 00 73 e g u r o e s
21:25:31:959 (00000494) 45.40.135.245 ->: 00 74 00 E1 00 20 00 50 00 72 00 65 00 73 00 74 t P r e s t
21:25:31:959 (00000494) 45.40.135.245 ->: 00 65 00 73 00 20 00 61 14 00 00 04 00 00 00 00 e s a
21:25:31:959 (00000494) 45.40.135.245 ->: 14 01 00 04 00 00 00 00 14 02 00 04 00 00 03 B2
21:25:32:060 CheckSmppSubmitAsyncQ: Mark submit as complete
21:25:32:060 CheckSmppSubmitAsyncQ: SAR-+99999999999-5355a382-4-1
21:25:32:060 (00000494) 45.40.135.245 --: WaitForResponseSMPP: SMPP Debug: received submit or data response
21:25:32:060 (00000494) 45.40.135.245 <-: 46 byte packet
21:25:32:060 (00000494) 45.40.135.245 <-: 00 00 00 2E 80 00 00 04 00 00 00 00 00 00 0A 2F . /
21:25:32:060 (00000494) 45.40.135.245 <-: 53 41 52 2D 2B 39 39 39 39 39 39 39 39 39 39 39 SAR-+99999999999
21:25:32:060 (00000494) 45.40.135.245 <-: 2D 35 39 36 38 36 37 38 65 2D 34 2D 31 00 -5968678e-4-1
21:25:41:959 (00000494) 45.40.135.245 ->: 208 byte packet
21:25:41:959 (00000494) 45.40.135.245 ->: 00 00 00 D0 00 00 00 04 00 00 00 00 00 00 0A 30 0
21:25:41:959 (00000494) 45.40.135.245 ->: 00 00 00 00 01 01 39 39 39 39 39 39 39 39 39 39 9999999999
21:25:41:959 (00000494) 45.40.135.245 ->: 39 00 40 00 00 00 00 00 00 08 00 8C 05 00 03 A3 9 @
21:25:41:959 (00000494) 45.40.135.245 ->: 04 02 00 20 00 74 00 65 00 72 00 6D 00 69 00 6E t e r m i n
21:25:41:959 (00000494) 45.40.135.245 ->: 00 61 00 72 00 2E 00 20 00 51 00 75 00 65 00 69 a r . Q u e i
21:25:41:959 (00000494) 45.40.135.245 ->: 00 72 00 61 00 20 00 70 00 6F 00 72 00 20 00 66 r a p o r f
21:25:41:959 (00000494) 45.40.135.245 ->: 00 61 00 76 00 6F 00 72 00 20 00 72 00 65 00 67 a v o r r e g
21:25:41:959 (00000494) 45.40.135.245 ->: 00 75 00 6C 00 61 00 72 00 69 00 7A 00 61 00 2D u l a r i z a -
21:25:41:959 (00000494) 45.40.135.245 ->: 00 6C 00 6F 00 20 00 65 00 76 00 69 00 74 00 61 l o e v i t a
21:25:41:959 (00000494) 45.40.135.245 ->: 00 6E 00 64 00 6F 00 20 00 61 00 73 00 73 00 69 n d o a s s i
21:25:41:959 (00000494) 45.40.135.245 ->: 00 6D 00 20 00 61 00 20 00 73 00 75 00 73 00 70 m a s u s p
21:25:41:959 (00000494) 45.40.135.245 ->: 00 65 00 6E 00 73 00 E3 14 00 00 04 00 00 00 00 e n s
21:25:41:959 (00000494) 45.40.135.245 ->: 14 01 00 04 00 00 00 00 14 02 00 04 00 00 03 B2
21:25:42:049 CheckSmppSubmitAsyncQ: Mark submit as complete
21:25:42:049 CheckSmppSubmitAsyncQ: SAR-+99999999999-5355a382-4-2
21:25:42:049 (00000494) 45.40.135.245 --: WaitForResponseSMPP: SMPP Debug: received submit or data response
21:25:42:049 (00000494) 45.40.135.245 <-: 46 byte packet
21:25:42:049 (00000494) 45.40.135.245 <-: 00 00 00 2E 80 00 00 04 00 00 00 00 00 00 0A 30 . 0
21:25:42:049 (00000494) 45.40.135.245 <-: 53 41 52 2D 2B 39 39 39 39 39 39 39 39 39 39 39 SAR-+99999999999
21:25:42:049 (00000494) 45.40.135.245 <-: 2D 35 39 36 38 36 37 38 65 2D 34 2D 32 00 -5968678e-4-2


--
Des
NowSMS Support