NowSMS Lite causes server lockup

NowSMS Lite causes server lockup SearchSearch
Author Message
Mike Glancy
New member
Username: Web9

Post Number: 16
Registered: 05-2013
Posted on Wednesday, April 06, 2016 - 04:59 pm:   

We've been using the NowSMS Lite with an Android Phone for six months and it has been solid until two weeks ago. Suddenly the windows 2012 server hosting the app would stop responding. We reimaged the server in case of some corruption and it still continued to hang daily. We then set it up on a Windows 2008 server in case some windows update was causing a compatibility issue. Unfortunately, it is still hanging within 24 hours of a reboot. This server is dedicated to running NowSMS. The only error we see in the log is the following:

NowSMS Error Report. Error initializing SMSC Interface '10.100.2.62' (NowSMSModem - 10.100.2.62). Interface is not available.

The phone is always powered on and nothing has changed there. Any help would be appreciated. We are running version 2015.10.09.

Thanks. Mike
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5618
Registered: 08-2008
Posted on Wednesday, April 06, 2016 - 06:58 pm:   

Hi Mike,

That's very odd.

We have seen server lockups with some USB modems. But all communication with an Android device happens over IP, which is pretty straightforward.

Even if the phone drops off line, that will not cause the server to crash.

We have done a lot of research on problems that cause a device to go off-line, but all of the issues we have found have been device side. 2015.08.07 on the app/device side incorporates various recovery techniques to bring the device back online.

We are about to release a new version, 2016.03.28. The Android app is currently at http://www.nowsms.com/apkdev It is only a slight update to 2015.08.07 and addresses an issue where the app did not automatically load own a device restart on Android 6 (Marshmallow) devices and some HTC devices.

On the server side, 2016.03.28, by default, checks on the device more frequently when idle. I'd like to see you try this update. I'm not expecting it to fix the problem, but I'm wondering if this change in device checks has any effect on how long it takes your server to fail. The update for Lite is at http://www.nowsms.com/download/lite20160328.zip

I'll also follow up shortly with some troubleshooting suggestions after I consult with some colleagues.

--
Des
NowSMS Support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5619
Registered: 08-2008
Posted on Wednesday, April 06, 2016 - 10:41 pm:   

Troubleshooting follow-up...

Is there a virus scanner doing real time monitoring? If so, try disabling.

Are you running 2-way commands? What type? (Local programs, local PHP, remote HTTP?)

Open a command prompt Window and CD to the NowSMS directory. Do a DIR /s .... are there a lot of files, say more than 10,000?

Open Windows task manager ... add columns for handle count and thread count. Every hour or so, check the memory usage and handles/threads for smsgws.exe. Also look for any other processes where these numbers are growing.

It sounds like something is using excessive memory. Assuming this is a VM of some sort, try allocating more memory to the VM.

--
Des
NowSMS Support
Mike Glancy
New member
Username: Web9

Post Number: 17
Registered: 05-2013
Posted on Thursday, April 07, 2016 - 04:52 pm:   

Hi Des,
I will try the new version shortly.

Answers to your last post -

1. yes - ESET NOD32. We've put exceptions for the NowSMS folders. We also tried disabling it entirely with no change.
2. No - MMS outbound only.
3. only 268 files in the NowSMS structure
4. we're currently rebooting the server every 2 hours. Current smsgws.exe handles=209, threads=21 (up for almost 2 hours)
5. Memory usage has consistently been very low. We have 4G to the guest, physical usage is at 967M. Even when it was "hung", neither memory nor CPU have ever been a cause to point to. CPU typically runs at 1%.

The original 2012 server was a 1U dedicated unit. The next 2012 server was VM and the current 2008 server is VM as well. So it's happened on both a dedicated hardware box and VM - FYI.

Thanks.

Mike
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5620
Registered: 08-2008
Posted on Thursday, April 07, 2016 - 07:03 pm:   

Hi Mike,

Since you are primarily sending outbound MMS, look at mmsc.exe as well.

If memory usage is not excessive, what you describe sounds like it could be a handle leak somewhere. We have seen this before with some virus scanners on extremely busy servers, so I wouldn't expect this on the type of config you describe. But for Windows to lock up, there has to be some sort of resource issue.

--
Des
NowSMS Support
Mike Glancy
New member
Username: Web9

Post Number: 18
Registered: 05-2013
Posted on Thursday, April 07, 2016 - 09:11 pm:   

Mmsc running for 90 minutes, Mem=4.2M, Handles=180, Threads=15.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 5621
Registered: 08-2008
Posted on Thursday, April 07, 2016 - 09:53 pm:   

That's normal...very low, as expected.

It would take tens of thousands of handles to cause problems. So I'd be very surprised to find a handle leak of that magnitude in NowSMS.

I've been doing some searches on troubleshooting Windows hanging...

This article talks about a few things to check and monitor: https://blogs.technet.microsoft.com/askperf/2007/09/25/troubleshooting-server-ha ngs-part-one/

Then I stumbled on to this: https://forum.eset.com/topic/4724-server-2008r2-hangs-randomly-after-install-efs -6/ ... regular server hangs that appear to be directly related to ESET. One comment I noticed was that disabling ESET did not resolve the problem, but uninstalling did.

There is also reference to an MS Hotfix that might fix the ESET problem: https://support.microsoft.com/en-us/kb/2664888

I think it's worth trying to completely uninstall ESET to see what effect it has on the problem.

--
Des
NowSMS Support
Mike Glancy
New member
Username: Web9

Post Number: 19
Registered: 05-2013
Posted on Thursday, April 14, 2016 - 06:55 pm:   

Thanks Des. After running for almost a week on two servers with ESET removed, the lockups have stopped. That appears to have been the issue.

Mike

Add Your Message Here, or click here to start a new topic.
Post:
Bold text Italics Underline Create a hyperlink Insert a clipart image
Options: Automatically activate URLs in message
Action: