How to send National Language SMS in 160 chars without Unicode

How to send National Language SMS in 160 chars without Unicode SearchSearch
Author Message
Ravichandra Reddy
New member
Username: Ravichandrak

Post Number: 1
Registered: 01-2015
Posted on Tuesday, January 20, 2015 - 10:01 am:   

Dear Support,

How to send National Language SMS in 160 characters without UniCode. If possible share the bytes for the same.

Kindly advise on the same.

Thanks,
Ravichandra.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5171
Registered: 08-2008
Posted on Tuesday, January 20, 2015 - 03:47 pm:   

Hi,

Unicode (UTF-16) coding is the most reliable encoding in SMS for characters outside the GSM character set. (Reference: http://www.nowsms.com/long-sms-text-messages-and-the-160-character-limit)

The fact that messages that contain characters outside of the GSM character set require Unicode encoding and are limited to 70 characters per message instead of the expected 160 character limit, is a frustrating limitation for many languages.

Messages longer than 70 characters can, of course, be sent, but they are sent as multipart (segmented) messages, and reassembled by the receiving client. If a message that contains Unicode characters is longer than 70 characters, it is broken into segments of 67 characters for sending. To send a 160 character message requires 3 SMS messages if the message contains characters that are not part of the GSM character set.

The original GSM protocol was developed in Western Europe, so it includes most Western European characters, plus capital letters in the Greek alphabet in order to facilitate Greek SMS support.

Shift tables are a way to support national language characters in SMS messages without requiring Unicode characters.

However, shift table support is necessary in the handset in order for the client to be able to see these characters when shift table encoding is used. Handsets without shift table support will not display the correct characters if shift table encoding is used.

NowSMS has implemented shift table support (locking shift table and single shift table) for the following languages: Turkish, Spanish, Portuguese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Oriya, Punjabi, Tamil, Telugu, and Urdu.

For the Turkish language, shift tables add support for the following national language characters: Ğ, ğ, Ş, ş, İ, ı, ç

For the Spanish language, shift tables add support for the following national language characters: ç, Á, Í, Ó, Ú, á, í, ó, ú.

For the Portuguese languages, shift tables add support for the following national language characters: Á À Â Ã ª á à â ã É Ê é ê Í í Ó Ô Õ º ó ô õ Ú Ü ú ü ` ç ∞

The 10 Indian languages that are supported contain too many characters to mention here ... and are so newly defined (3GPP Release 9) that they are unlikely to have any handset support for shift tables yet. However, we wanted our implementation to be flexible enough to support all currently defined shift tables.

References:
http://support.nowsms.com/discus/messages/1/70000.html
http://www.nowsms.com/shift-tables-national-language-sms-in-160-characters-witho ut-unicode

I do not have any binary examples, but those can be extracted from the SMSDEBUG.LOG by inspecting what NowSMS generates when shift table support is enabled.


--
Des
NowSMS Support
Ravichandra Reddy
New member
Username: Ravichandrak

Post Number: 2
Registered: 01-2015
Posted on Tuesday, February 03, 2015 - 04:38 am:   

Dear Des,

We could not able to send 160 character UNICode in single SMS from from portal. After 70 character its taking second SMS and attached screenshot for the same.

Please suggest how to send 160 UNICode characters in single SMS.

Thanks,
Ravichandra.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5189
Registered: 08-2008
Posted on Tuesday, February 03, 2015 - 04:23 pm:   

Hi,

That is how it works.

If a message contains characters outside the GSM character set, the message must be encoded as UTF-16 (UCS2), and only 70 characters fit in an SMS. Longer messages are sent with segmentation headers, with room for 67 Unicode characters per segment.

Please refer to the links in my previous reply, especially the first one: http://www.nowsms.com/long-sms-text-messages-and-the-160-character-limit

And if you are sending emoticons/emoji, some of those characters count as 2 Unicode characters (see http://www.nowsms.com/emoticons).

--
Des
NowSMS Support
Ravichandra Reddy
New member
Username: Ravichandrak

Post Number: 3
Registered: 01-2015
Posted on Tuesday, February 03, 2015 - 04:50 pm:   

Dear Des,

Thanks for the update.

I too agree with you, regarding stranded UNICode message.

How to send 160 characters Indian Regional Language(Hindi, Tamil, Telugu, etc...) in single SMS from NowSMS UI(http://127.0.0.1:8800)!!!

As per the following link we can send it,

http://www.nowsms.com/shift-tables-national-language-sms-in-160-characters-witho ut-unicode

Please advise on the same.

Thanks,
Ravichandra.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5192
Registered: 08-2008
Posted on Tuesday, February 03, 2015 - 05:01 pm:   

Hi Ravichandra,

Note that the receiving handsets must support shift tables for it to work.

Support for the Indian language via shift tables is still relatively new, and there may be few handsets that support this technique.

See http://support.nowsms.com/discus/messages/1/70000.html, which explains how to enable this support. Specifically note that the ShiftTables parameter must be added to SMSGW.INI.

Also, note that the web interface will still report 2 or more messages, as it is unaware of how the server performs this encoding.

The shift table headers add a few bytes, so you will get close to 160 characters, but you will get longer than 70.

Also note that only 1 language shift table can be used per message. So if you mix languages, Unicode encoding will be used.

--
Des
NowSMS Support
Ravichandra Reddy
New member
Username: Ravichandrak

Post Number: 4
Registered: 01-2015
Posted on Friday, February 13, 2015 - 09:14 am:   

Dear Des,


As you suggested, we have modified SMSGW.INI and sent couple of Hindi messages(One of Indian Regional Language) but I didn't display Hindi text('अ') on any handsets. We have tried with different handsets but nothing is displayed on any of handsets.

Please check the following test log and kindly guide/advise on the same,

Test Logs:
----------
2015-02-13 10:48:11,54CFAE52,127.0.0.1,919949000000,OK -- SMPP - 182.18.132.51:6000,SubmitUser=test;SMSCRouteName=test;Sender=test123;SMSCMsgId=1 232043105106010003;UDH=03250106;Text="अ"

Thanks,
Ravichandra.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5203
Registered: 08-2008
Posted on Friday, February 13, 2015 - 06:57 pm:   

Hi,

I'm not surprised.

As I stated in my first reply on this thread:

The 10 Indian languages that are supported contain too many characters to mention here ... and are so newly defined (3GPP Release 9) that they are unlikely to have any handset support for shift tables yet. However, we wanted our implementation to be flexible enough to support all currently defined shift tables.

That said, I would expect the handset to display the ¥ character if it does not support the shift table, because that character takes the place of ¥ when the Hindi shift table is activated.

You may want to add some English text before the अ and see if the handset displays the English text directly. If it does not, there may be an SMPP encoding issue as different SMPP servers have different expectations when UDH is present in a message. If the English text is not displayed, try setting "Encode long text messages with 7-bit packed encoding" under the advanced settings for the SMPP connection.

--
Des
NowSMS Support