SMS Shift Table Support

SMS Shift Table Support SearchSearch
Author Message
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 7949
Registered: 10-2002
Posted on Tuesday, March 01, 2011 - 05:07 pm:   

There was some discussion several months ago about SMS shift table support.

http://www.nowsms.com/shift-tables-national-language-sms-in-160-characters-witho ut-unicode
http://support.nowsms.com/discus/messages/1/69650.html

We have implemented shift table support (locking shift table and single shift table) to NowSMS for the following languages: Turkish, Spanish, Portuguese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Oriya, Punjabi, Tamil, Telugu, and Urdu.

Shift tables are a way to support national language characters in SMS messages without requiring Unicode characters. (If a message contains any characters that require Unicode encoding, the entire message becomes subject to a 70 character limitation per message instead of the normal 160 character limit. Shift tables are a way to overcome this limitation.)

Shift table support is necessary in the handset in order for the client to be able to see these characters when shift table encoding is used.

For the Turkish language, shift tables add support for the following national language characters: Ğ, ğ, Ş, ş, İ, ı, ç

For the Spanish language, shift tables add support for the following national language characters: ç, Á, Í, Ó, Ú, á, í, ó, ú.

For the Portuguese languages, shift tables add support for the following national language characters: Á À Â Ã ª á à â ã É Ê é ê Í í Ó Ô Õ º ó ô õ Ú Ü ú ü ` ç ∞

The 10 Indian languages that are supported contain too many characters to mention here ... and are so newly defined (3GPP Release 9) that they are unlikely to have any handset support for shift tables yet. However, we wanted our implementation to be flexible enough to support all currently defined shift tables.

Support for these shift table capabilities are still preliminary at this time. Additional information on how to enable NowSMS to support these shift tables is only being provided on request at this time. Please feel free to reply to this thread if you need more information.

-bn
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 7956
Registered: 10-2002
Posted on Thursday, May 05, 2011 - 04:21 pm:   

Follow-up.

SMS Shift Table support is available in NowSMS version 2011.03.21 and later.

http://www.nowsms.com/category/updates

2-way SMS automatically decodes messages that are encoded with any of the currently defined 13 shift tables (Turkish, Spanish, Portuguese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Oriya, Punjabi, Tamil , Telugu, and Urdu).

Outbound message encoding using SMS shift tables is only performed when explicitly configured.

To enable support for one or more SMS shift tables, it is necessary to edit the SMSGW.INI file, and under the [SMSGW] section header, add:

ShiftTables=##,##,##

The values specified are the 3GPP assigned decimal numeric values of shift tables that should be enabled for encoding outbound messages.

Supported values include:

1 - Turkish
2 - Spanish
3 - Portuguese
4 - Bengali
5 - Gujarati
6 - Hindi
7 - Kannada
8 - Malayalam
9 - Oriya
10 - Punjabi
11 - Tamil
12 - Telugu
13 - Urdu


It is very important to note that if a message encoded with shift tables is received by a device that does not support shift tables, that device will not display the message correctly. In the case of Turkish, Spanish and Portuguese, the character will appear as a somewhat similar character on a device that does not support shift tables. However, for the Indian languages, the result will be unreadable if shift table encoding is received on a device that does not support it.

-bn
Ankit Vyas
New member
Username: Ankit_eng85

Post Number: 1
Registered: 09-2011
Posted on Wednesday, September 21, 2011 - 08:21 am:   

Hi!I want to send SMS in any Indian Languages(Gujarati,Kannada,Tamil) in PDU format.if i send sms using these languages in any cell phone even if the phone is not supporting any language but i am using pdu mode so is it possible to receive that particular sms in readable format? My Requirement is like this if i send sms in any language it should be received in readable format.is there any idea about it?Please guide me or suggest me solution.I am new in this subject so please give your valuable guidelines.Thanks in advance.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3481
Registered: 08-2008
Posted on Wednesday, September 21, 2011 - 07:00 pm:   

SMS shift tables are relatively new for India, so I do not think they are widely enough supported for content provider use at this time.

You would send the message using Unicode character encoding.

However, there is no way to specify an alternate readable format if the receiving device does not support the Unicode characters you are sending. Please keep in mind that each individual SMS messages can support only 140 bytes of data (70 16-bit Unicode characters), so there is not room for things like alternate encodings.

The only practical solution is to include the alternate text in the actual text of the message to give users an instruction on how to reply with a keyword if the message is unreadable and they need English.

A single SMS message can be sent using multiple physical messages over the air, so you can send longer messages, but you generally want to keep the messages as short as practical.

One suggestion would be as part of a welcome message based upon the user's first interaction with the service. The welcome message could include English text that says something like "Reply ENG to switch to English language".

Exactly how this would work would depend on the logic flow of your application and content. Unfortunately, there are no magic solutions for handling this ... you need to work within limitations of the SMS medium.

--
Des
NowSMS Support
Ankit Vyas
New member
Username: Ankit_eng85

Post Number: 2
Registered: 09-2011
Posted on Thursday, September 22, 2011 - 07:24 am:   

Hi I think My Requirement is same as you are saying.I want to support Gujarati,Hindi,Kannada,Malayalam,Tamil,Telugu these languages in any of Cell phone(Black and White and color) Please let me know how can i use shift table for receiving SMS using these languages or give some idea what should i do for achieving this task.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3485
Registered: 08-2008
Posted on Thursday, September 22, 2011 - 03:19 pm:   

You do not seem to understand what I am saying.

SMS shift tables are relatively new for India, so I do not think they are widely enough supported for content provider use at this time.

The software in the receiving phone needs to support them. With Indian language shift tables having only just been defined in 3GPP Release 9, I find it unlikely that they will meet your requirements. You should use Unicode.
Andreas Feider
New member
Username: De_afeider

Post Number: 1
Registered: 11-2011
Posted on Thursday, November 10, 2011 - 01:55 pm:   

Hi Des, Bryce and all other who might be able to help.

I am trying to use the Web UI in order to send text messages using the National language table locking/single shift mechanism.

I use the form to send a "Binary SMS Message", choose "Other", DCS and PID is set to "0". I add the information to User Data Header field (e.g. "03250101" for National Language Table Locking Shift mechanism in Turkish language.)

Now I thought I can add the User Data in PDU format to the Binary Field. But the handset does not the characters I expected and in the logs I can see the field Text instead of the binary data. If I use another tool like PDUspy I can send the message and it is correctly shown on the receiving device. So the device and the provider are able to transfer and display text messages with National language locking shift.

Here is an example User Data Header and User Data I want to send with NowSMS:

PID: 0 DCS:0
UDH: 03240101
User Data: DA943B0B0618

So my question: Can you tell me how to use NowSMS in order to sent messages with values like above? Do you have an idea what I am doing wrong?
I am using is a licensed NowSMS version of 2011.07.05.
Is there maybe still a SMSGW.INI file where I have to change some configuration?

Best regards,
Andy.
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 8002
Registered: 10-2002
Posted on Thursday, November 10, 2011 - 11:49 pm:   

Hi Andy,

Are the NowSMS logs showing the correct text (it would be UTF-8 encoding in the logs).

If so, then you are encoding things correctly. When logging the messages, NowSMS always decodes shift table messages, and logs the decoded text instead of binary data.

It is only necessary to add a configuration parameter if you want NowSMS to take text messages from the web interface and perform shift table encoding. In that case, you would edit the SMSGW.INI file, and under the [SMSGW] section header, add:

ShiftTables=1

What is the SMSC connection? Since you mention, PDUSpy, I'm guessing it's a modem?

I just enabled the SMSDEBUG.LOG on my system and sent your message as a test message out via a modem connection. The PDU in the SMSDEBUG.LOG shows this: 0B03240101DA943B0B0618, which matches your data. Is this what you see too?

Try sending a message in to the modem, and let's compare working with non-working to see what bit is getting flipped where.

-bn
Andy28
New member
Username: De_afeider

Post Number: 2
Registered: 11-2011
Posted on Friday, November 11, 2011 - 08:05 am:   

Hi Bryce,

thanks for your help and sorry for the incomplete information. The connection is indeed a modem, but I could observe the same behaviour when sending the message via a UCP connection.
Nevertheless - thanks to your help - I have found the mistake.

I have sent the message created by PDUspy to the modem and SMSdebug.log showed the correct PDU value. When sending this, everything worked fine. Probably I made some kind of typo when filling in the binary data before.

Well, now all kind of national language shift table do work correctly and can be tested on the devices. Thanks your quick support.

Kind regards,
Andy