Confused about message encodings and "data_coding" in NowSMS Could ... | Search |
NowSMS Support Forums ⬆ NowSMS Cloud Edition Support ⬆ |
◄ ► |
Author | Message | |||
Yury S New member Username: Yskanavy Post Number: 3 Registered: 12-2018 |
Hi My SMPP client is sending out messages with "data_coding" set to 8, which according to the specification is UCS2 (ISO/IEC-10646), which is similar to UTF-16. I send the message containing national chars. Android SMS app on my phone shows sent and received messages correctly and with no data loss. I also see the correct national chars displayed in NowSMS log files (SMSIN-....LOG and SMSIN-....LOG). However, when my SMPP listener receives the message, "data_coding" is set to 0 (SMSC Default Alphabet) and the data has been converted to something like Ascii (GSM7 perhaps), the national chars were converted to question marks and all other chars are basically single-byte ascii. If my SMPP client sends a message with "data_coding" 0 and UTF-8 encoded data, then the entire message is seen as garbage on my phone and and SmsNow log files. So, my understanding that to ensure a specific message coding, one needs to use non-zero "data_coding" to indicate how exactly the text is coded. Seems to hold true except for when such message is sent from NowSMS SMPP to a client. On the other hand leaving "data_coding" as zero does not provide a good result at all...Not sure I understand how NowSMS SMPP operates and to receive messages correctly. Thanks. | |||
Yury S New member Username: Yskanavy Post Number: 4 Registered: 12-2018 |
"If my SMPP client sends a message with "data_coding" 0 and UTF-8 encoded data" I meant to say "UTF-16 encoded data"} | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 6087 Registered: 08-2008 |
Hi, I'm sorry, but I don't understand the overall logic flow through which the message is processed. I understand that you can send SMS OK using data_coding=8, which is expected. And I understand that when you receive SMS from NowSMS as an SMPP client, the encoding is not what is expected. But I don't understand the overall path/flow. What is nowSMS receiving the message from? Does the SMSIN log show the correct characters and DCS=8? If the SMSIN log shows the correct encoding...I don't see why or how NowSMS would change the data_coding to 0. Unless maybe you're doing some processing with a 2-way command? What you describe does not sound normal...but basically I need to see more details ... relevant log entries and an explanation of the path/flow of processing for the received message. -- Des NowSMS Support | |||
Yury S New member Username: Yskanavy Post Number: 5 Registered: 12-2018 |
I am sending a message to my own number using my program which acts an SMPP client. I set "data_coding" to 8, and encode my message as UTF-16. My program also receives messages through SMPP. When I get deliver_sm with the message I have send to myself, "data_coding" is 0 and the "short_message" has been converted to a single byte ascii (i,e. non-ascii data I have sent is damaged). Both SMSIN and SMSOUT show the correct data with all non-ascii chars present, it's just the deliver_sm I get looks incorrect. | |||
Yury S New member Username: Yskanavy Post Number: 6 Registered: 12-2018 |
For example, I am sending this message text: русский english This is the logging from my code: Send message via "short_message", data coding => 8 byte len => 32 bytes => FE FF 04 40 04 43 04 41 04 41 04 3A 04 38 04 39 00 20 00 65 00 6E 00 67 00 6C 00 69 00 73 00 68 Receive "short_message" message: data coding => 0 byte len => 15 byte data => 3F 3F 3F 3F 3F 3F 3F 20 65 6E 67 6C 69 73 68 The byte sequence I have sent out is a correct UTF-16 representation of my text. The received data is Ascii data for ??????? english i.e. the russian word at the beginning is now question marks. SMSOUT: *** SubmitUser=**** ;SMSCRouteName=***;Sender=***;DCS=8;Text="русский english" 2020-01-23 18:21:08:790,5E27478D,,**** ,OK -- LocalUser:****,SubmitUser=NowSMSModem - ****;Sender=***;Text="русский english" SMSIN: 2020-01-23 18:21:08:789,****,NowSMSModem - **** e,Text="русский english";SMSCRouteName=****;Recip=**** (I replaced sensitive info with *) I see the same issue when sending the same text from my phone or through NowSMS Send Message page. My Android SMS App also displays the sent and received messages properly. To sum up, the messages are seen correctly in the logs and phone app, but the deliver_sm sent to my program has data re-coded to ascii and data_coding reset to 0 | |||
Yury S New member Username: Yskanavy Post Number: 7 Registered: 12-2018 |
I also tried my program with SMPPSim with LOOPBACK=TRUE, so it would echo the message back. Got the expected result: Receive "short_message" message: data coding => 8 byte len => 32 byte data => FE FF 04 40 04 43 04 41 04 41 04 3A 04 38 04 39 00 20 00 65 00 6E 00 67 00 6C 00 69 00 73 00 68 | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 6089 Registered: 08-2008 |
Hi Yury, Sorry for the delay in response. We were able to recreate this problem. Unicode text messages were not being encoded correctly when routed from the Android modem interface to an SMPP client. An updated build of NowSMS fixes this problem: https://www.nowsms.com/download/nowsms20200131.zip I notice that you are have posted in the support area for the cloud edition...we have updated the cloud servers to address this problem. — Des NowSMS Support | |||
Yury S New member Username: Yskanavy Post Number: 8 Registered: 12-2018 |
Thank you, I will try it out. |