Problem with special characters

Problem with special characters SearchSearch
Author Message
Cristian Regazzo
New member
Username: Regazzo

Post Number: 1
Registered: 07-2014
Posted on Friday, July 25, 2014 - 05:41 pm:   

Good afternoon,

I'm sending SMS messages to both the HTTP API or via the web interface, which contains the underscore character, and in my SmartPhone, the message arrives with the character ¿.

example:

Usually the Good_Morning message on my SmartPhone get: Good¿Morning.

How do I get the message arrives correctly with underscore?

Thank you.

Cristian Regazzo
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 4958
Registered: 08-2008
Posted on Friday, July 25, 2014 - 06:17 pm:   

Hi Cristian,

Are you using an SMPP SMSC connection?

Different providers use different character sets.

The following will solve 90% of SMPP character set issues:

In NowSMS, go to the "SMSC" page, highlight the connection and press the "Properties" button. Go into the “Advanced Settings” for the SMPP connection and try changing the “SMSC Character Set” to “iso-8859-1 as Default”. Note that in order to save and activate that setting change it may be necessary to press “OK” twice then “Apply” when you are returned to the “SMSC” list. (Different versions of NowSMS behave differently and the "Apply" button might not be enabled, indicating the settings change is already saved.) If NowSMS asks if you wish to test the SMSC connection, it is ok to answer “No”. Wait about 60 seconds for the server to activate the settings change and try another test message.

--
Des
NowSMS Support
Cristian Regazzo
New member
Username: Regazzo

Post Number: 2
Registered: 07-2014
Posted on Friday, July 25, 2014 - 06:32 pm:   

Good afternoon,

Sorry, forgot to mention that I am using the Now SMS Lite.

I'm sending SMS via HTTP, SMPP and not.

Sorry, did not find the configuration option mentioned by you in Now SMS Lite

Thank you

Att,
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 4959
Registered: 08-2008
Posted on Friday, July 25, 2014 - 06:40 pm:   

These issues would not apply to NowSMS Lite.

Are you experiencing problems with any other characters?

What model is the GSM modem connected to NowSMS Lite?

Is the recipient on the same mobile network as the GSM modem?
Cristian Regazzo
New member
Username: Regazzo

Post Number: 3
Registered: 07-2014
Posted on Friday, July 25, 2014 - 07:08 pm:   

Good afternoon,

The model GSM Modem is: D-Link DWM-166

This problem also happening with accented characters like á, é, í, ó, etc.

But I observed the following, if I send a message with any special characters along with ç (cedilla), all other special characters such as underline and acute accents are received correctly, Smartphone character.

Now I think the NowSMS Lite has some treatment when it detects ç (cedilla), fixing all other accented characters.

Standing by.

Thank you.

Att,
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 4960
Registered: 08-2008
Posted on Friday, July 25, 2014 - 07:59 pm:   

Hmm...

Unfortunately, I have never seen an issue like this before.

When you submit from the web interface, the web form is configured to use the UTF-8 character set, which is the character set NowSMS expects to see via HTTP.

When NowSMS sends a message out via a GSM modem, it uses the raw over the air (PDU) format, so there is no character set ambiguity like with other SMS protocols.

There must be something odd about this modem. But I can't imagine what it would be.


quote:

But I observed the following, if I send a message with any special characters along with ç (cedilla), all other special characters such as underline and acute accents are received correctly, Smartphone character.




Whenever a message contains any characters outside the GSM character set, NowSMS automatically uses Unicode encoding for the entire message. This preserves all characters, but has aside effect of limiting a single message to 70 characters instead of the normal 160 character limit. (Longer messages are segmented and sent using multiple physical messages with headers to tell the receiver how to reassemble as a single long message.) So, yes, NowSMS is doing something different here.

If a message contains only characters within the GSM character set, NowSMS is going to encode the message using the GSM character set, and use the 7-bit packing to further encode the characters. That is the raw format that goes out over the air.

If the characters are getting modified, this suggests the modem is unpacking the message from the 7-bit packing, then a character set translation is occurring, then the message is repacked in 7-bit format for sending. I just can't see how or why that would be happening.

Unfortunately, I don't think there is any easy fix for this, because it just doesn't make sense.

But here is what I would suggest:

1. Make sure you're using the latest firmware for that modem. Here is the product page with the latest firmware download: http://www.dlink.com/uk/en/support/product/dwm-156-3-75-hsupa-usb-adapter

2. If you have access to a different modem, try it instead.

3. Edit SMSGW.INI, and under the [Modem - your modem here] header, add:

CommandPostInit1=AT+CSCS="GSM"

Restart the NowSMS service. This command should have no effect because NowSMS uses PDU mode. However, since the modem appears to be confusing character sets, perhaps this setting to explicitly set the character set will help.

4. Enable the SMSDEBUG.LOG (option on the Serial # page). Send a message with at least one problem character from the modem to itself. I would like to look at the SMSDEBUG.LOG and compare the sent and received PDUs to confirm the problem.

--
Des
NowSMS Support
Cristian Regazzo
New member
Username: Regazzo

Post Number: 4
Registered: 07-2014
Posted on Friday, July 25, 2014 - 08:16 pm:   

Good afternoon,

The problem is just that, the only NowSMS Lite correctly sends the message with special characters when you change the default to Unicode.

The NowSMS Lite not recognizing this as a special underline caracterer thus does not change the shipping to Unicod

As I have set that all messages are always sent in the Unicode standard, even though the limitation of 70 caractes?

Standing by.

Att,

Cristian Regazzo
Cristian Regazzo
New member
Username: Regazzo

Post Number: 5
Registered: 07-2014
Posted on Friday, July 25, 2014 - 08:46 pm:   

Good afternoon,

I managed to solve my problem.

I configured with the DCS parameter value 12, thereby remedying the problem of underline.

Thank you for your attention.

Att,

Cristian Regazzo