| Content is trimmed? |  Search
|
|
NowSMS Support Forums ⬆ NowSMS Support - SMS Issues ⬆ Archive through November 14, 2013 ⬆ |
◄ ► |
| Author | Message | |||
| Rik Clews New member Username: Rikclews Post Number: 37 Registered: 08-2006 |
Hi, We've had a small issue with a disparity in our accounting, where our count of segments is higher than the count NowSMS holds. Turns out that NowSMS is trimming whitespace from the end (but not the beginning) of the SMS message contents. This seems like a good thing, but we're uncomfortable with the content being updated rather than taken as submitted. Is there a way we can turn off the trimming behaviour? Regards, Rik | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4261 Registered: 08-2008 |
Hi Rik, I'd probably need a full scenario (protocols and path with example) to investigate and determine whether or not it is a problem. I do know that for HTTP submissions, we definitely trim/truncate trailing spaces. If memory serves me correctly, at some point we were seeing issues with some web browsers and HTML forms adding trailing spaces. As a result we strip trailing spaces for all HTTP variables. There is also a scenario when we add a trailing space. This is when 7-bit encoding is used via SMPP and there is an ambiguity with regard to messages that fall on particular boundaries. I'd need to know more about the scenario you are seeing to investigate or explain it. -- Des NowSMS Support | |||
| Rik Clews New member Username: Rikclews Post Number: 38 Registered: 08-2006 |
Hi Des, Thanks for the reply, it will be the fact we use it mainly with http submissions with a smaller proportion of SMPP clients. Would it be possible to have a config switch for the trimming? It's because we don't want to alter the customers submission and leave ourselves open to complaints that we'd not submitted the content we'd been given (although I admit you'd need to be slightly odd to complain about correcting a useless empty segment - but it's our policy) Cheers, Rik | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4276 Registered: 08-2008 |
Hi Rik, We are investigating. As I mentioned, our HTTP variable parsing removes trailing spaces from all HTTP input variables. Disabling that could possibly cause unexpected problems. If recipient addresses have trailing whitespace (unlikely, but like your issue who would expect a complaint about removing trailing white space in a text message), submissions that worked before might no longer work. Or the extra whitespace could be the result of error, but tolerated at present, and not tolerated after a change. We're just very concerned about making changes that could have unintended consequences, even if its a quirk. So I'm checking to see if an option could be made to only affect message text and not other variables. That will take a little longer to investigate. -- Des NowSMS Support | |||
| Rik Clews New member Username: Rikclews Post Number: 39 Registered: 08-2006 |
Hi Des, Any news on this? We're finding its happening more frequently now. Regards, Rik | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4304 Registered: 08-2008 |
Hi Rik, I apologise for not following up like I should have. As I mentioned earlier, the removal of trailing spaces is done by the code that parses the HTTP variables. Changing this behaviour raises concerns that we may encounter unexpected issues in different customer installations. That, combined with the analysis that most would consider removing trailing spaces in the scenario you describe to be beneficial, has made it difficult to get any engineering time devoted to further analysis of the issue. However, I had some time with one of our engineers today working through some other issues. And I brought this issue up so that we could look at in more detail. The first thing we noticed was that in order to embed a trailing space in an HTTP request, you have to be deliberate about it. It is not like e-mail parsing. To encode a space character in an HTTP request using URL parameters, it needs to be encoded as "+" or "%20". So including it is a very deliberate action. With that understanding, our engineering team agreed to add a configuration parameter to prevent this removal. (We are hesitant to change the default behaviour because most would consider it beneficial, and because we do not want to cause problems for customer installations that might be sending requests with trailing spaces that they are unaware of.) Unfortunately, we are in the midst of quite a few updates at the moment, so it will be several weeks before a stable update release completes testing. In the meantime, to make sure we're not missing in understanding the root issue, can you verify that my understanding of the scenario is correct. HTTP submissions going out via SMPP ... is 7-bit encoding used for long messages (this presents its own issues that may or may not be a factor). -- Des NowSMS Support | |||
| Rik Clews New member Username: Rikclews Post Number: 40 Registered: 08-2006 |
Thanks Des sounds like what we're after. To confirm it is HTTP submissions going out via SMPP, but without the 7-bit encoding. Cheers, Rik | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4307 Registered: 08-2008 |
Hi Rik, Ok, I have definitely confirmed that in this scenario the update with this setting will accept messages with trailing spaces and include the trailing spaces when output via SMPP. I tested particularly with 161 character and 307 character messages where the last character was a space. In the case of the 307 character message, the 3rd message had only UDH and a single space. For grins I also tried 307 spaces. Unfortunately I cannot yet predict when this update will be available. The target is early April, but an interim update might be available sooner. -- Des NowSMS Support | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4336 Registered: 08-2008 |
Hi Rik, I just wanted to post another follow-up. We are expecting to post an interim update the week of April 1. It will include support for a configuration parameter RemoveTrailingSpaces=No under the [SMSGW] header of SMSGW.INI that can be used to disable the behaviour that you are finding to be problematic. -- Des NowSMS Support | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4350 Registered: 08-2008 |
Hi Rik, This has been addressed in the updated posted at http://www.nowsms.com/download/nowsms20130401.zip. -- Des NowSMS Support | |||
| Hareeharan Kandavel New member Username: Haree Post Number: 12 Registered: 12-2012 |
Hi Des, Thank you for resolving this issue. Although, NowSms doesn't trim off the white spaces after introducing the mentioned parameter, our http submissions suggest that it now trims of a unicode character whose url encoding is %C2%A0. Could you look into it please? Also, could you please ensure that NowSms doesn't alter the message content with any other unicode characters as well? Thanks, Haree | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4496 Registered: 08-2008 |
Hi Haree, NowSMS converts %C2%A0 (Unicode 0x00A0 - no-break space) to a regular space (%20). We perform similar conversions for some Greek uppercase letters, converting them to visually equivalent characters. This is done because if a message contains any characters outside the GSM7 character set, the entire message must be encoded in Unicode. The message is then limited to only 70 characters to fit in a single SMS instead of the normal 160 characters, which is a very significant restriction. I've just done a test web submission adding this character to the end of the text, and it is converted to a standard space (%20), and is not trimmed. If you are seeing different behaviour, I would need more detail on recreating the issue. -- Des NowSMS Support | |||
| Hareeharan Kandavel New member Username: Haree Post Number: 13 Registered: 12-2012 |
Hi Des, Thank you for your prompt response. When I said it trims off, I actually meant that the unicode is converted to a regular space. This results in our segment count to be higher than the NowSms segment count which is causing disparity in our accounting. This also means that the submitted content is altered which is uncomfortable to us. Could you please suggest me a way to stop NowSms from altering any submitted content? Thanks, Haree | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4497 Registered: 08-2008 |
Hi Haree, If we have to add a parameter to disable Greek and non-breaking space conversion to make you comfortable, we will. I strongly believe that performing these character translations are in the customer's best interest. In the case of the non-breaking space, it generally ends up in a text message submission by accident, because text is being copied from a word processor. (Or an MMS to SMS conversion where some handsets use this character after "RE:" in a subject header.) If a 160 character message includes one non-breaking space character, and that character is not converted to a regular space, it will require 3 Unicode text messages to send the message and preserve the character as the original non-breaking space. That said, if it is causing you problems, we will add a parameter to disable these conversions. -- Des NowSMS Support | |||
| Hareeharan Kandavel New member Username: Haree Post Number: 14 Registered: 12-2012 |
Hi Des, I see what you mean. But, we just don't want to alter the customer's submitted content which is against our policy and also expose ourselves to complaints. So, when is the earliest I can expect a patch release to add the parameter? Looking forward to hearing from you. Thank you for your understanding. Thanks, Haree | |||
| Bryce Norwood - NowSMS Support Board Administrator Username: Bryce Post Number: 8068 Registered: 10-2002 |
Hi Haree, We fast-tracked support for this change as one of the final modifications before we reclassify this version as the current stable release. Our best estimate is 1-2 weeks. This is a more detailed description of the change being implemented: * Web Interface/HTTP SMS Submission: Add configuration parameter to disable conversion of some Greek uppercase letters (0x391, 0x392, 0x395, 0x396, 0x397, 0x399, 0x39A, 0x39C, 0x39D, 0x39F, 0x3A1, 0x3A4, 0x3A5, 0x3A7) and non-breaking space (0xA0) to visually equivalent characters (A, B, E, Z, H, I, K, M, N, O, P, T, Y, X and space). When this conversion is disabled, a message that includes any of these characters will require Unicode encoding for the entire message, resulting in a limit of 70 characters for a single message or 67 characters per message for a multipart message (instead of the normal 160 and 153 character limits). To disable the automatic conversion, add TranslateGreekCharacters=No to the [SMSGW] section of SMSGW.INI. Do not add this setting without understanding the effect. -bn | |||
| Hareeharan Kandavel New member Username: Haree Post Number: 15 Registered: 12-2012 |
Hi Bryce, Can I have an update on this please? Thanks, Haree | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 4554 Registered: 08-2008 |
Hi Haree, I apologise for not following up in this thread. We did post an update that includes the referenced setting at http://www.nowsms.com/download/nowsms20130531.zip. -- Des NowSMS Support | |||
