Incorrect GSM encoding for Alphanumeric sender IDs | Search |
NowSMS Support Forums ⬆ NowSMS Support - SMS Issues ⬆ Archive through May 20, 2013 ⬆ |
◄ ► |
Author | Message | |||
Hareeharan Kandavel New member Username: Haree Post Number: 5 Registered: 12-2012 |
Hi Support, We have been doing some testing with GSM characters in Alphanumeric sender IDs. We realised that Nowsms did not encode a few characters as per the GSM character set. However, we did not have any issues with encoding those characters in the message content. The incorrect encoding of GSM characters in Alphanumeric sender IDs include Hex Character 0x00 @ 0x01 £ 0x02 $ 0x03 ¥ 0x04 è 0x05 é 0x06 ù 0x07 ì 0x08 ò 0x09 Ç 0x0A 0x0B Ø 0x0C ø 0x0D 0x0E Å 0x0F å 0x10 Δ 0x11 _ 0x12 Φ 0x13 Γ 0x14 Λ 0x15 Ω 0x16 Π 0x17 Ψ 0x18 Σ 0x19 Θ 0x1A Ξ 0x1C Æ 0x1D æ 0x1E ß 0x1F É 0x3A : 0x40 ¡ Could you please help me with this so that all the above symbols could be used in Alphanumeric sender IDs? Thanks Haree | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4298 Registered: 08-2008 |
Hi Haree, I don't know what type of SMSC connection you are using. I am guessing SMPP? SMPP does not explicitly state what character set should be used for alphanumeric sender, so it may vary based upon provider. I suppose we should use whatever character set NowSMS is configured to use for the message text (we default to GSM for the message text). However, we use whatever the URL character set is. I will escalate this to engineering for further analysis, but I do have a concern that since the SMPP spec defines the field as a null terminated string, it will not be possible to support the @ character when the GSM character set is used. -- Des NowSMS Support | |||
Hareeharan Kandavel New member Username: Haree Post Number: 6 Registered: 12-2012 |
Hi Desosms, Can I have an update on this please as one of our customers would like to use special characters in AlphaNumeric sender ID's? Thanks, Haree | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4300 Registered: 08-2008 |
Hi Haree, You have not answered my previous question. Let me restate and add additional questions. I still don't know what type of SMSC connection you are using. I am guessing SMPP? What protocol is the customer using to connect to NowSMS? What version of NowSMS are you running? I may be able to release an updated SMPP DLL that gives the option to use the GSM character set to encode alphanumeric senders. However, even with that, I cannot guarantee how your service provider will encode the resulting message. I am only guessing that they are expecting the GSM character set. -- Des NowSMS Support | |||
Hareeharan Kandavel New member Username: Haree Post Number: 7 Registered: 12-2012 |
Hi Des, Apologies. Since the issue was escalated, I assumed that you would get back to me. I could confirm that I am using SMPP connection. We are using HTTP protocol to submit the messages to NowSMS. Currently, we are running version 2012.02.16. An updated SMPP DLL release would be highly appreciated. Once this is done, I will take this to the service providers to encode appropriately. Thanks, Haree | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4302 Registered: 08-2008 |
Hi Haree, I've posted an update to only the smssmpp.dll file at http://www.nowsms.com/download/smppharee.zip To install this DLL, you need to update to NowSMS 2012.06.28 or later. 2012.06.28, the official download is at http://www.nowsms.com/download-free-trial And 2012.11.30, a stable interim update is at http://www.nowsms.com/download/nowsms20121130.zip After running the install to update to either of those two versions, manually replace smssmpp.dll using the file in the zip. This update will encode alphanumeric sender addresses using the GSM character set if the SMPP character set is default or GSM/IA5. Note that it will not be possible to send the @ character in an alpha sender in this scenario. The reason is an SMPP limitation. The sender address parameter is defined as a null terminated ASCII string. In the GSM character set, @ is encoded as a null character. As null is being used as a terminator, it cannot be included in the string itself. This is not an issue for message text because the message text field in SMPP has an explicit length parameter and does not use null termination. -- Des NowSMS Support | |||
Hareeharan Kandavel New member Username: Haree Post Number: 8 Registered: 12-2012 |
Hi Des, Update to the DLL is very much appreciated. I have followed your instructions and did some testing which sorted out the issues with a few special characters. However, I am afraid NowSMS still don't encode quite a few characters according to GSM encoding. Please find the list of unsupported characters below: LATIN SMALL LETTER I WITH GRAVE ì LATIN SMALL LETTER O WITH GRAVE ò LATIN CAPITAL LETTER C WITH CEDILLA Ç LATIN CAPITAL LETTER O WITH STROKE Ø LATIN SMALL LETTER O WITH STROKE ø LATIN CAPITAL LETTER A WITH RING ABOVE Å LATIN SMALL LETTER A WITH RING ABOVE å LATIN CAPITAL LETTER AE Æ LATIN SMALL LETTER AE æ LATIN SMALL LETTER SHARP S (German) ß LATIN CAPITAL LETTER E WITH ACUTE É CURRENCY SIGN ¤ COLON : LATIN CAPITAL LETTER A WITH DIAERESIS Ä LATIN CAPITAL LETTER O WITH DIAERESIS Ö LATIN CAPITAL LETTER N WITH TILDE Ñ LATIN CAPITAL LETTER U WITH DIAERESIS Ü LATIN SMALL LETTER A WITH DIAERESIS ä LATIN SMALL LETTER O WITH DIAERESIS ö LATIN SMALL LETTER N WITH TILDE ñ LATIN SMALL LETTER U WITH DIAERESIS ü LATIN SMALL LETTER A WITH GRAVE à Could you please investigate this and suggest me a solution? Looking forward to your response. Thank you. Kind Regards, Haree | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4305 Registered: 08-2008 |
Hi Haree, As I mentioned previously ... I cannot guarantee how your service provider will encode the resulting message. The change that we made was to encode the alphanumeric sender using the character set configured for the SMPP connection (default being GSM). I tested with a few of the characters in your list (starting with the first one ... ì ... GSM character #7), and the resulting SMPP message showed the expected GSM character value. I would strongly suggest that you use Wireshark for testing so that you can inspect the SMPP encoding for yourself in order to better determine where the "problem" exists. That said, I do see two potential areas of confusion. 1.) The colon ":" character is used as a delimiter for port addressing by NowSMS. If it is present in a sender address, NowSMS will truncate the sender address at the ":" and remove all numeric characters. To disable this behaviour add AllowAlphaRecip=Yes to the [SMSGW] section of SMSGW.INI. (We will add automatic detection of this scenario in the future and only apply port address parsing when the address is otherwise numeric.) 2.) NowSMS expects HTTP submissions to use the UTF-8 character set for encoding. The "charset=" parameter can be used to specify a different input character set, however, currently this character set is only applied to the text of the message, not the sender address. The sender address must be encoded using UTF-8 when submitting via HTTP. -- Des NowSMS Support | |||
Hareeharan Kandavel New member Username: Haree Post Number: 9 Registered: 12-2012 |
Hi Des, Thank you for that information. I would like to confirm that all the characters in the alphanumeric sender ID's are now being encoded to the GSM character set. However, despite following your suggestions to encode "colon", I still don't see them being encoded correctly. I am now facing a similar issue with Roman8 and this also needs to have the sender address change applied that you have done earlier for GSM/IA5. Would you be able to apply a change to NowSMS so that a flag could be set that changes the entire SMPP packet with all its contents to the character set that has been applied in the SMSC configuration? Looking forward to your response. Thanks, Haree | |||
Bryce Norwood - NowSMS Support Board Administrator Username: Bryce Post Number: 8052 Registered: 10-2002 |
Hi Haree, OK ... that makes sense. Those characters that you have listed have different encodings in the Roman8 character set. We are only performing the alpha sender character set conversions for ASCII/iso-8859-1 or GSM/IA5. An updated SMSSMPP.DLL is at the same download link as before: http://www.nowsms.com/download/smppharee.zip (If the file timestamp of the DLL is before March 8, clear your cache and try the download again.) The other issues that Des mentioned still apply. -bn | |||
Hareeharan Kandavel New member Username: Haree Post Number: 10 Registered: 12-2012 |
Hi Bryce, I downloaded the dll from the link and still, I am unable to see Roman 8 conversion in the sender Id. Could you please confirm whether the attached download link is for the Roman 8 character set conversion in sender Id? Thanks, Haree | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4331 Registered: 08-2008 |
Hi Haree, If you look at an SMPP trace, do you see the expected encoding? As a test, I focused on the first character in your list: LATIN SMALL LETTER I WITH GRAVE ì From the NowSMS web form, I submitted a message with a sender address of ìtestì If the SMPP connection is configured to use the GSM character set, I see 0x07 test 0x07 in the sender address, which is the correct GSM encoding. If the SMPP connection is configured to use the Roman-8 character set, I see 0xD9 test 0xD9 in the sender address, which I believe to be the correct Roman-8 encoding. If you do not see that same encoding, verify that the file date stamp on smssmpp.dll is from March 8. -- Des NowSMS Support | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4352 Registered: 08-2008 |
Hi Haree, Just a follow-up. In case the problems you were experiencing are related to difficulties applying the single DLL update, we have posted a complete updated install at http://www.nowsms.com/download/nowsms20130401.zip. This update does include the change to encode alphanumeric senders according to the character set configured for the connection (including support for Roman-8). -- Des NowSMS Support | |||
Hareeharan Kandavel New member Username: Haree Post Number: 11 Registered: 12-2012 |
Hi Des, Apologies for the delay in responding back. It now encodes the characters appropriately. Thank you very much for your help. Thanks, Haree |