Quick Question!

Quick Question! SearchSearch
Author Message
Mike
New member
Username: Messageguy1

Post Number: 1
Registered: 01-2009
Posted on Monday, January 26, 2009 - 04:46 pm:   

Hello all,

could someone offer some general advice?
Suddenly now (for the past 2 weeks) my messages either come in right away, or take roughly 3 minutes to come in (it seems to be exactly the same). Occasionally they wont work at all.

essentially someone sends a message to the gateway, which picks up the message, hands it off to some PHP code, and the PHP code sends the result back to the phone.

From my logging to the database, it seems the php code executes very quickly and hands it back off to the gateway.

Details:
[4] HttpResponseWait: Expected 0 bytes, got 0 bytes
09:31:29:999 [4] HttpResponseWait:
09:31:29:999 [4] RetrieveURL: timeout waiting for response
09:31:31:001 [4] ThreadProcessInboundSMS: Processing 497AB8A4.in...
09:31:31:001 [4] GetProgramToExecute:
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 385
Registered: 08-2008
Posted on Monday, January 26, 2009 - 08:49 pm:   

Hi Mike,

I'd be curious to see the log entries right before the part that you posted. Are the previous entries about 3 minutes earlier?

It looks like your PHP script is not sending back an HTTP response. The web server doesn't close the connection, and NowSMS waits until it times out.

By default, only one thread is allocated to 2-way callbacks, so the next one isn't processed until this timeout occurs. (There is a way to allocate more threads, but it doesn't sound like you have a volume problem here that would require that.)

Just do something in your PHP script to return some type of a response. An "echo" command is probably enough.

(I suspect you have NOT checked the option "Command returns response text". If you're wondering why NowSMS is still waiting for a response, it may not care about the text of the response, but it does want to see that there is a valid HTTP response.)

If, on the other hand, this is a sporadic issue, where this error is reported only on some command attempts ... then I'd like to see a debug log that shows some more detail.

What version of NowSMS are you running? If its 2008.11.05 or later, we did add support for keep-alive sockets in 2-way command processing, and maybe your web server does not like this. You could try adding 2WayKeepAlive=No under the [SMSGW] header of SMSGW.INI to see if this makes a difference.

--
Des
NowSMS Support
Mike
New member
Username: Messageguy1

Post Number: 2
Registered: 01-2009
Posted on Tuesday, January 27, 2009 - 02:42 pm:   

Hmm it seems like every night after I reboot, i can send as many messages as i want and there is never a problem..When I wake up the next morning, and send something, it takes 17 minutes sometimes to send me back. The SMS IN log shows it receives it, and the SMSOUT log shows it sends it out 17 minutes later. the PHP code has it logged that it was immediately received / sent out to gateway within seconds.

What do you think is going on? I'm always echoing a result. I'm using a webservice from google, and i have no reason to believe i am being throttled by them or that the service is hanging more than 5-10 seconds..Is that time too long for the gateway? If so, what could I do to tell it to wait for google?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 397
Registered: 08-2008
Posted on Tuesday, January 27, 2009 - 03:42 pm:   

Hi Mike,

Let's take a look at the SMSDEBUG.LOG for clues.

I'd like to see a complete log if possible.

Something strange is happening, but it's not clear to me what it could be.

You can either upload it in reply here, or send it via e-mail to nowsms@nowsms.com with "Attention: Des" in the subject line of your message.

If you do e-mail it to me, drop a quick post here to tell me. Usually this type of message will get routed to me directly, but sometimes I have to dig around to locate it.

--
Des
NowSMS Support
Mike
New member
Username: Messageguy1

Post Number: 3
Registered: 01-2009
Posted on Tuesday, January 27, 2009 - 04:12 pm:   

I have sent you the e-mail and i am very anxious to see what you have to say.

If we resolve this i'll post the answer here so others may benefit.

Thanks so much!
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 398
Registered: 08-2008
Posted on Tuesday, January 27, 2009 - 07:21 pm:   

Hi Mike,

Something is definitely going wrong with the HTTP connection.

Not all of the time. Occasionally. But when there is a problem, it usually lasts for several attempts before correcting itself.

From the NowSMS perspective, what is happening is that we issue the HTTP request, but we never get any data at all back in the response. After 2 minutes, we timeout. So we try again.

Sometimes there are multiple timeouts in a row, sometimes the second attempt is ok.

This is odd. If the web server was having a problem, normally you'd still see some sort of data in the response (often an HTTP 500 series error). Or, you'd see the connection terminated after a few seconds.

But we're not seeing any errors ... or any data in the response for 2 minutes. For an error of a sporadic nature, that's a puzzle.

Let me ask you one question ... I'm assuming that you edited the SMSDEBUG.LOG slightly before sending it to me ... correct?

The log is showing the following as the 2-way command URL:

http:///E2/Handler.php?sender=@@SENDER@@&text=@@FULLSMS@@

I'm assuming that host name (which looks like an IP address on your local network based upon other information) was edited out of the URL. Correct? I mention this because if it was not edited out of the log by you, I'm surprised that it would be working at all.

It's definitely an unusual pattern. In the past, when I've seen similar types of problems, they have been caused by a script choking on a particular message, where that particular message is being retried repeatedly by NowSMS, delaying other messages from being processed. But that is not happening here ... as when the problem occurs, the messages are reprocessed later without incident.

So ... I don't have any clues why this would be failing.

The only thing I could think of doing, to try to troubleshoot this further, would be to run a trace with Wireshark/Ethereal of traffic between NowSMS and the web server. We could see if there is anything unusual revealed in that trace.

Is this web server on your local network? I'm confused because the log shows a connection to a local 192.168.1 series IP address, yet you mention a Google web service above. Is there an HTTP proxy involved by chance?

--
Des
NowSMS Support
Mike
New member
Username: Messageguy1

Post Number: 4
Registered: 01-2009
Posted on Tuesday, January 27, 2009 - 07:44 pm:   

Des,

Just to give you some more clues:

-The url has been changed (edited out) so yes you are correct it wouldn't work as it is now :-)

-The webserver/gateway is on same machine, and the google webservice is outside obviously. There is no proxy.

Let me ask you, if the web service took some time, say 5 seconds, or let's say even 30 seconds, would the gateway try again much later, or would it just wait and the second it came back with a result it would process it?

Based on my logging with the code, the gateway receives the sms message, and records the timestamp to the database. Then it processes this using a webservice, and sends the message back to the gateway (and then timestamps THAT). In the end, i've never seen more than a few seconds between the two timestamps

We never had this problem before and it just kinda suddenly happened without any code changes.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 401
Registered: 08-2008
Posted on Tuesday, January 27, 2009 - 09:10 pm:   

Hi Mike,

We'll wait up to 2 minutes (120 seconds) for a response. So waiting 5 seconds, or 30 seconds, is no problem.

As soon as the HTTP response is complete, it processes the result.

If it times out after 120 seconds, then there is a retry without delay. However, if there are other messages pending, we might try them first before retrying the one that just timed out.

Keep in mind that only one message is being processed at a time. If you want to enable more, you can find the information here:

http://blog.nowsms.com/2008/10/2-way-sms-command-speed-and-performance.html

I suspect that you might be running into some timeout problems connecting to your web service from your script. But you'd probably want to do some more logging to verify this. You say that you log the timestamp you receive the message in a database ... is that the first step of your script? Is there any chance there could be connection problems to the database? For debugging, could you debug log to a text file?

I'd suggest comparing that info against the SMSDEBUG.LOG. In the SMSDEBUG.LOG, you can see when NowSMS sends the HTTP GET command to the server.

For times when NowSMS logs "RetrieveURL: timeout waiting for response", did your script see this command at all?

For looking at today's SMSDEBUG.LOG, can you check your web server log ... maybe that will show the missing requests that your web service did not see?

Wireshark would be helpful. But unfortunately, I don't think it can intercept TCP/IP traffic that is local to a single machine ... you'd have to separate NowSMS and your web server on different machines to get a Wireshark trace.

I definitely expect that it is some sort of connectivity issue. But for NowSMS to wait 2 minutes and see no response at all is very strange ... especially with it all on the same machine.

It might also be worth changing the URL to use the loopback address 127.0.0.1 to see if that makes any difference. I wouldn't expect it to, but I don't know what else to suggest.

--
Des
NowSMS Support
Mike
New member
Username: Messageguy1

Post Number: 5
Registered: 01-2009
Posted on Tuesday, January 27, 2009 - 09:13 pm:   

I will think @ this and try some things now. I'll let you know. thanks!
Mike
New member
Username: Messageguy1

Post Number: 6
Registered: 01-2009
Posted on Monday, February 02, 2009 - 01:37 pm:   

OK so... I've done some more verifying but I am still trying to track this down.

I'm using a small programmatic delay (sleep statement in PHP) as to not abuse the web service. Whenever I cache the results in the DB, the gateway returns results right away to the phone. If it has to get new results (that were not in the cache) it generally takes anywhere from 5-20 minutes now, although this is still sporadic (about 75% of the time)

Does using any kind of sleep statement (on a windows machine) potentially screw up the script or the gateway waiting for a response possibly?
Mike
New member
Username: Messageguy1

Post Number: 7
Registered: 01-2009
Posted on Monday, February 02, 2009 - 03:27 pm:   

Another question - let's assume something is not working from the script end, I was thinking of disabling the 2-way feature and making my php script use the gateway direction.

It will send the text to be returned to the gateway directly when done. I know how to do all of this except i'm not entirely sure on the whole 1-way sms thing..

That's something I can achieve, correct? Make it so that when a user sends a msg, my script picks it up, but the gatway wont expect any output from it (and thus stop trying to query over and over and hanging up my system due to my script)?

..This way, when my script is ready, it can use the programmatic way of talking to gateway
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 415
Registered: 08-2008
Posted on Tuesday, February 03, 2009 - 06:16 pm:   

Hi Mike,

That makes more sense ... a 5 to 20 minute delay would definitely cause the 2-way command to time out, because we only wait a maximum of 2 minutes.

Unfortunately, that 2 minute delay is hardcoded.

Is there any way that you script could return some data while it is waiting? Even if you just sent a space character, that should reset the 2 minute timeout.

Alternatively, maybe you could create a simple Windows script to call your URL. Configure NowSMS to call the Windows script instead of calling your URL directly.

NowSMS waits up to 5 minutes for a command line script to complete ... but it doesn't retry the message if there is a timeout.

If you're not familiar with writing these types of scripts, it's actually pretty simple. Here's an example one that we put together that builds a URL based upon command line parameters and connects to NowSMS via that URL. You'd do something similar, building a URL for connecting to your PHP script.

--
Des
NowSMS Support


--
Cross-reference related discussion here: http://support.nowsms.com/discus/messages/12/24740.html
Mike
New member
Username: Messageguy1

Post Number: 9
Registered: 01-2009
Posted on Tuesday, February 03, 2009 - 07:14 pm:   

oh great.. so you think if i call it from a script the gateway wont even wait for a reply from the webserver, since no server is involved?

i don't see any script example though in your post, should I try to look for it on forums or you would be able to provide one?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 420
Registered: 08-2008
Posted on Tuesday, February 03, 2009 - 07:23 pm:   

If you call it from a script it will wait 5 minutes for the script to complete ... but it won't care if it times out after 5 minutes. The message will be considered processed and will not be retried.

If you're having situations where occasionally it is taking this long, you may want to allocate more threads for the 2-way command processing, so that multiple commands can be processed at the same time (instead of a message waiting 5 minutes for the previous one to timeout). Info on doing this can be found here: http://blog.nowsms.com/2008/10/2-way-sms-command-speed-and-performance.html

Regarding the missing link ... this is the script that I meant to include a link for: http://www.nowsms.com/support/bulletins/tb-nowsms-008.htm

You'd just need to modify it so that when the script is run from the command line it builds a URL for connecting to your PHP script instead of connecting to NowSMS like this script does.

One other piece of information that might be helpful to you ... here's a simple example of how you execute a Windows script from a 2-way command:

http://support.nowsms.com/discus/messages/1/333.html

(Note that I usually recommend using cscript.exe instead of the wscript.exe referenced in that particular link.)

--
Des
NowSMS Support
Mike
New member
Username: Messageguy1

Post Number: 10
Registered: 01-2009
Posted on Tuesday, February 03, 2009 - 07:43 pm:   

hmm i can also just use cmd.exe can't I and execute a batch file directly that uses php.exe to call my script?
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 421
Registered: 08-2008
Posted on Tuesday, February 03, 2009 - 07:48 pm:   

Or spawn PHP.EXE directly ... whatever will work from the command line.
Mike
New member
Username: Messageguy1

Post Number: 11
Registered: 01-2009
Posted on Wednesday, February 04, 2009 - 04:56 pm:   

I just want to let you know this resolved all problems I am having. Thank you.