NowSMS Lite v2011.07.05 Bug: URL Space character (%20) in http api ...

NowSMS Lite v2011.07.05 Bug: URL Space character (%20) in http api ... SearchSearch
Author Message
Melvin Chia
New member
Username: Melvinchia

Post Number: 1
Registered: 01-2012
Posted on Friday, January 06, 2012 - 08:29 pm:   

Hi, I recently upgraded to the latest version of NowSMS Lite (v2011.07.05) from the previous version (v2010.11.04) and discovered that the http api has been broken.

The following is the config entry which responds to incoming SMS which start with #TEST (taken from SMSGW.INI):
ReceiveSMSCommand8=#TEST http://127.0.0.1/?User=user&Password=password&PhoneNumber=TEST&Text=[TEST]+@@SMS@@+[DO+NOT+REPLY] No

The following is the SMSIN log entry:
2012-01-05 23:42:49,+6596906168,HUAWEI Mobile Connect - 3G Modem,Text="#TEST testing again 123"

The following is the SMSOUT log entry:
2012-01-05 23:42:52,4D820AAC.req,127.0.0.1,+6596906168,OK -- HUAWEI Mobile Connect - 3G Modem,SubmitUser=melvin;Text="[TEST] testing%20again%20123 [DO NOT REPLY]"

And when I received the message on my phone, I got:
[TEST] testing%20again%20123 [DO NOT REPLY]

It seems that in the 2011 version, the URL space character (%20) was not parsed and replaced with a space before sending out the SMS. This was working in the 2010 version.

Can you verify the bug, and post an update? I need this up and working as soon as possible as this is currently an active system.
Melvin Chia
New member
Username: Melvinchia

Post Number: 2
Registered: 01-2012
Posted on Friday, January 06, 2012 - 08:32 pm:   

The bug here is that @@SMS@@ passes the actual message including spaces to the url, and when the url is parsed, the spaces are replaced by the browser with %20. The previous version caught this and replaced the %20 with spaces, but the current version didn't, hence a message with many spaces would become unreadable as all the spaces would be replaced with %20.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3707
Registered: 08-2008
Posted on Monday, January 09, 2012 - 06:48 pm:   

Hi Melvin,

I have tried multiple tests to try to recreate this problem with various versions of NowSMS (including the version you mention), but I cannot.

I would be curious to see your SMSDEBUG.LOG showing this message being received and processed as there must be some other factor which I am not considering.

The SMSDEBUG.LOG should show a sequence something like this:

13:32:35:332 [11] ThreadProcessInboundSMS: Processing 4D4A28C4.in...
13:32:35:333 [11] GetProgramToExecute: http://127.0.0.1/?user=user&password=password&phonenumber=@@SENDER@@&text=[TEST] +@@SMS@@+[DO+NOT+REPLY]

Above entry shows the matching 2-way command, next we insert the text with spaces URL encoded as %20.

13:32:35:333 [11] GetProgramToExecute: http://127.0.0.1/?user=user&password=password&phonenumber=%2B12345678901&text=[T EST]+123%20abc+[DO+NOT+REPLY]
13:32:35:333 [11] ThreadProcessInboundSMS: Executing http://127.0.0.1/?user=user&password=password&phonenumber=%2B12345678901&text=[T EST]+123%20abc+[DO+NOT+REPLY]
13:32:35:333 [11] RetrieveURL: Retrieving http://127.0.0.1/?user=user&password=password&phonenumber=%2B12345678901&text=[T EST]+123%20abc+[DO+NOT+REPLY]
13:32:35:334 [11] RetrieveURL: Retrieving ?user=user&password=password&phonenumber=%2B12345678901&text=[TEST]+123%20abc+[D O+NOT+REPLY]

In this scenario, NowSMS is connecting to itself, so we see the URL request being received, still with the space URL encoded as %20. But the resulting message has a space as it should.

13:32:35:337 [62] ThreadProcessConnection: Processing request /?user=user&password=password&phonenumber=%2B12345678901&text=[TEST]+123%20abc+[ DO+NOT+REPLY]

I can't imagine why you are seeing different results, so I would like to see what is different in your SMSDEBUG.LOG.

Either post the log here, or email it to nowsms@nowsms.com with Attention: Des in the subject line.

There is also a more recent build of NowSMS Lite that you can download at http://www.nowsms.com/download/lite20111115.zip. You may want to try that version.

--
Des
NowSMS Support
Melvin Chia
New member
Username: Melvinchia

Post Number: 3
Registered: 01-2012
Posted on Tuesday, January 10, 2012 - 07:04 am:   

I just noticed that the windows control panel says v2011.07.07, the version I quoted above is actually the version I noticed in the log file.

Anyway, here's the log fragment from my SMSDEBUG.LOG:


14:49:23:040 [5] WaitForSocketClose: WinSock reported ioctlsocket complete
14:49:49:038 [13] ModemTest: HUAWEI Mobile Connect - 3G Modem
14:49:49:112 [13] ModemTest: OK
14:52:51:088 [13] ModemTest: HUAWEI Mobile Connect - 3G Modem
14:52:51:160 [13] ModemTest: OK
14:55:53:153 [13] ModemTest: HUAWEI Mobile Connect - 3G Modem
14:55:53:227 [13] ModemTest: OK
14:58:25:093 [13] ModemReceiveMessages: AT+CPMS?
14:58:25:093 [13] ModemReceiveMessages:
+CPMS: "ME",0,23,"SM",1,30,"SM",1,30

OK

14:58:25:181 [13] ModemReceiveMessages:
+CMGL: 0,0,,39
06915669489599240A91566909168600002110014185222317236A714A05D1CB737ADA7D0685CFE1B41B1493CD00

OK

14:58:25:181 [13] ModemReceiveMessages: Decoding received message index 0 from message storage SM
14:58:25:181 [13] ModemReceiveMessages: 06915669489599240A91566909168600002110014185222317236A714A05D1CB737ADA7D0685CFE1B41B1493CD00
14:58:25:181 [13] ModemReceiveMessages: SMSC address len = 6
14:58:25:181 [13] ModemReceiveMessages: SMSC Address = +6596845999
14:58:25:181 [13] ModemReceiveMessages: SMS Message Type = SMS-DELIVER
14:58:25:181 [13] ModemReceiveMessages: Sender address len = 10
14:58:25:182 [13] ModemReceiveMessages: Sender Address = +6596906168
14:58:25:182 [13] ModemReceiveMessages: PID = 00
14:58:25:182 [13] ModemReceiveMessages: DCS = 00
14:58:25:182 [13] ModemReceiveMessages: Anticipated user data length = 23
14:58:25:182 [13] ModemReceiveMessages: Message = #TEST testing again 123
14:58:25:183 [13] ModemReceiveMessages: AT+CMGD=0

14:58:25:184 [5] ThreadProcessInboundSMS: Processing 4D820E9B.in...
14:58:25:184 [5] GetProgramToExecute: http://127.0.0.1/?User=user&Password=password&PhoneNumber=TEST&Text=[TEST]+@@SMS@@+[DO+NOT+REPLY]
14:58:25:184 [5] GetProgramToExecute: http://127.0.0.1/?User=user&Password=password&PhoneNumber=TEST&Text=[TEST]+testing%20again%20123+[DO+NOT+REPLY]
14:58:25:184 [5] ThreadProcessInboundSMS: Executing http://127.0.0.1/?User=user&Password=password&PhoneNumber=TEST&Text=[TEST]+testing%20again%20123+[DO+NOT+REPLY]
14:58:25:184 [5] RetrieveURL: Retrieving http://127.0.0.1/?User=user&Password=password&PhoneNumber=TEST&Text=[TEST]+testing%20again%20123+[DO+NOT+REPLY]
14:58:25:185 [5] RetrieveURL: Looking up 127.0.0.1
14:58:25:185 [5] RetrieveURL: Retrieving ?User=user&Password=password&PhoneNumber=TEST&Text=[TEST]+testing%20again%20123+[DO+NOT+REPLY]
14:58:25:185 [5] RetrieveURL: GET /?User=user&Password=password&PhoneNumber=TEST&Text=[TEST]+testing%20again%20123+[DO+NOT+REPLY] HTTP/1.1
User-Agent: Now SMS/MMS Gateway v2011.07.05
Accept: */*
Host: 127.0.0.1


14:58:25:186 [5] HttpResponseWait: Ok
14:58:25:186 [5] RetrieveURL: HTTP/1.1 302 Found
Date: Tue, 10 Jan 2012 06:58:25 GMT
Server: Apache/2.2.14 (Win32) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l mod_autoindex_color PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1
Location: http://127.0.0.1:8800/?User=user&Password=password&PhoneNumber=TEST&Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d
Content-Length: 534
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://127.0.0.1:8800/?User=user&amp;Password=password&amp;PhoneNumber=TEST&amp;Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d">here</a>.</p>
<hr>
<address>Apache/2.2.14 (Win32) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l mod_autoindex_color PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1 Server at 127.0.0.1 Port 80</address>
</body></html>

14:58:25:187 [5] RetrieveURL: Saving keep-alive socket
14:58:25:187 [5] RetrieveURL: got redirection response
14:58:25:187 [5] RetrieveURL: HTTP/1.1 302 Found
Date: Tue, 10 Jan 2012 06:58:25 GMT
Server: Apache/2.2.14 (Win32) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l mod_autoindex_color PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1
Location: http://127.0.0.1:8800/?User=user&Password=password&PhoneNumber=TEST&Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d
Content-Length: 534
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://127.0.0.1:8800/?User=user&amp;Password=password&amp;PhoneNumber=TEST&amp;Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d">here</a>.</p>
<hr>
<address>Apache/2.2.14 (Win32) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l mod_autoindex_color PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1 Server at 127.0.0.1 Port 80</address>
</body></html>

14:58:25:188 [5] RetrieveURL: http://127.0.0.1:8800/?User=user&Password=password&PhoneNumber=TEST&Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d
14:58:25:188 [5] RetrieveURL: Retrieving http://127.0.0.1:8800/?User=user&Password=password&PhoneNumber=TEST&Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d
14:58:25:188 [5] RetrieveURL: Looking up 127.0.0.1
14:58:25:189 [5] RetrieveURL: Retrieving ?User=user&Password=password&PhoneNumber=TEST&Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d
14:58:25:189 [1] ThreadListenForConnections: Before accept
14:58:25:189 [5] RetrieveURL: GET /?User=user&Password=password&PhoneNumber=TEST&Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d HTTP/1.1
User-Agent: Now SMS/MMS Gateway v2011.07.05
Accept: */*
Connection: close
Host: 127.0.0.1:8800


14:58:25:189 [1] ThreadListenForConnections: After accept
14:58:25:190 [17] ThreadProcessConnection: Processing connection from 127.0.0.1:32485...
14:58:25:190 [17] TheadProcessConnection: Updating cached user info
14:58:25:190 [17] ThreadProcessConnection: Processing request /?User=user&Password=password&PhoneNumber=TEST&Text=%5bTEST%5d+testing%2520again%2520123+%5bDO+NOT+REPLY%5d
14:58:25:191 [17] Debug: 1 recipient entries
14:58:25:193 [5] HttpResponseWait: Ok
14:58:25:193 [5] RetrieveURL: HTTP/1.1 200 OK
Connection: close
Content-Type: text/html
Expires: Tue, 01 Jan 1980 1:00:00 GMT
Cache-Control: no-cache
Content-Length: 263

<HTML>
<HEAD><TITLE>Message Submitted</TITLE></HEAD>
<BODY bgcolor=#cce8ff>
<p>
Message Submitted
<p>
<a href="javascript:history.back()">Continue</a>
<p>
<pre><small>
MessageID=4D820E9C.req, Recipient=+6596906168
</small></pre>
<p>
</BODY>
</HTML>

14:58:25:194 [5] RetrieveURL: got success response
14:58:25:194 [5] RetrieveURL: text/html
14:58:25:194 [17] WaitForSocketClose: WinSock reported ioctlsocket complete
14:58:25:194 [5] ThreadProcessInboundSMS: HTTP/1.1 200 OK
Connection: close
Content-Type: text/html
Expires: Tue, 01 Jan 1980 1:00:00 GMT
Cache-Control: no-cache
Content-Length: 263

<HTML>
<HEAD><TITLE>Message Submitted</TITLE></HEAD>
<BODY bgcolor=#cce8ff>
<p>
Message Submitted
<p>
<a href="javascript:history.back()">Continue</a>
<p>
<pre><small>
MessageID=4D820E9C.req, Recipient=+6596906168
</small></pre>
<p>
</BODY>
</HTML>

14:58:25:194 [17] ThreadProcessConnection: Request processing complete
14:58:25:195 [5] ThreadProcessInboundSMS: Processing complete 4D820E9B.in...
14:58:25:448 [13] ModemReceiveMessages:
OK

14:58:25:459 [13] ThreadProcessModem: Processing 4D820E9C.req...
14:58:25:476 [13] ThreadProcessModem: OUT: AT+CMGS=54

14:58:25:492 [13] ThreadProcessModem: IN:
>
14:58:25:492 [13] ThreadProcessModem: OUT: 0001030A91566909168600002F1B1EB538A56E7C207A794E4FBBCF25192C7C0EA7DD25192C269B81363CE213E47C5241D2229499DDF800
14:58:27:390 [13] ThreadProcessModem: +CMGS: 106

OK

14:58:35:196 [5] WaitForSocketClose: WinSock reported ioctlsocket complete
14:58:55:032 [13] ModemTest: HUAWEI Mobile Connect - 3G Modem
14:58:55:105 [13] ModemTest: OK

application/octet-streamSMSDEBUG.LOG fragment
SMSDEBUG.LOG (7.9 k)
Melvin Chia
New member
Username: Melvinchia

Post Number: 4
Registered: 01-2012
Posted on Tuesday, January 10, 2012 - 07:29 am:   

Update: I just upgraded to v2011.11.15 and re-did the test, no joy, the SMS sent to my phone was still padded with %20 instead of spaces..
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 3710
Registered: 08-2008
Posted on Tuesday, January 10, 2012 - 07:00 pm:   

Hi Melvin,

The SMSDEBUG.LOG clarifies what is happening.

There is an Apache web server in the mix, and that is where the extra URL encoding is taking place.

I can't speculate what the purpose of that Apache web server is in your overall setup. However, you can easily fix the problem by bypassing it. Simply change http://127.0.0.1/ in the 2-way command URL to http://127.0.0.1:8800/

--
Des
NowSMS Support