High CPU usage

High CPU usage SearchSearch
Author Message
Todd Matherne
New member
Username: Tmmatherne

Post Number: 9
Registered: 08-2020
Posted on Wednesday, June 09, 2021 - 07:16 am:   

Support,

Back in April I had a issue connecting to MMSC when using 44+ android devices.
Here is the support thread. You guys updated NowSMS and the problem has been resolved.
https://support.nowsms.com/discus/messages/485/113854.html

However, that was a MMS issue. At the time I was only sending MMS and no SMS. Now when sending SMS my CPU resources are maxed at 100% causing everything to slow down. I’m certain this is due to the amount of android devices connected. I scaled down to only 20 phones and CPU was back to normal range < 20%. When I add more devices CPU usage spikes. I can only assume this is being caused by the amount of android devices connected to NowSMS. (My license includes 100 devices)

Note... when sending only MMS I have no issues with high CPU usage.

Can you guys look into this?

Appreciate your help - Todd
Todd Matherne
New member
Username: Tmmatherne

Post Number: 10
Registered: 08-2020
Posted on Wednesday, June 09, 2021 - 08:25 am:   

Problem resolved..... I was throttling the SMS sending limit to 1/30 which seem to be the issue. Once I eliminated the throttle CPU returned to normal range.
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 6291
Registered: 08-2008
Posted on Wednesday, June 09, 2021 - 06:58 pm:   

Hi Todd,

I think I see what you mean.

You are referring to the SMSCSendLimit setting, correct?

Regards,

Des
NowSMS Support
Todd Matherne
New member
Username: Tmmatherne

Post Number: 11
Registered: 08-2020
Posted on Wednesday, June 09, 2021 - 07:56 pm:   

Hey Des

That's correct. My SMGW.in looks like this:

[NowSMSModem - Device1]
RouteName=Device1
ReceiveSMS=Yes
Remote=Yes
Password=*****
NowsmsModemMMSURL=http://mms.msg.eng.t-mobile.com/mms/wapenc
ReceiveMMS=Yes
DeviceName=Device1
DeviceID=1**********
SMSCSendLimit=1/30


When I throttle the sending limit it spikes the servers resources to 100%. (16GB Ram, Intel 2.5 GHz)
Is there a work around for this? It's important that I throttle the speed in which sms are sent.

Thanks - Todd
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 6292
Registered: 08-2008
Posted on Wednesday, June 09, 2021 - 10:36 pm:   

Hi Todd,

It looks like an update will be required to change this. I have referred this to our engineering team for further analysis.

Regards,

Des
NowSMS Support
Todd Matherne
New member
Username: Tmmatherne

Post Number: 12
Registered: 08-2020
Posted on Thursday, June 10, 2021 - 06:20 pm:   

Thanks Des, I'll be standing by for update
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 6293
Registered: 08-2008
Posted on Thursday, June 10, 2021 - 11:14 pm:   

Follow-up...

We are testing an update that is expected to be made available tomorrow, or possibly over the weekend.

Regards,

Des
NowSMS Support
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 6294
Registered: 08-2008
Posted on Friday, June 11, 2021 - 06:15 pm:   

Hi Todd,

We've posted an update at https://nowsms.com/download/nowsms20210609.zip.

We observed that the Linux version did not seem to suffer similar CPU spikes with these low throttling speeds, and applied this same logic.

Out of curiosity, are you running in a virtual machine or a native Windows server?

Regards,

Des
NowSMS Support
Todd Matherne
New member
Username: Tmmatherne

Post Number: 13
Registered: 08-2020
Posted on Friday, June 11, 2021 - 06:57 pm:   

Des

I’m running native windows server through Amazon AWS.
Todd Matherne
New member
Username: Tmmatherne

Post Number: 14
Registered: 08-2020
Posted on Friday, June 11, 2021 - 07:51 pm:   

Hi Des,

I just tried the update but having the same issues. See screenshot
CPU Spike
Des - NowSMS Support
Board Administrator
Username: Desosms

Post Number: 6296
Registered: 08-2008
Posted on Friday, June 11, 2021 - 09:28 pm:   

Hi Todd,

One more attempt before we have to gather more diagnostics: https://nowsms.com/download/nowsms20210611.zip

Regards,

Des
NowSMS Support
Todd Matherne
New member
Username: Tmmatherne

Post Number: 15
Registered: 08-2020
Posted on Saturday, June 19, 2021 - 08:28 pm:   

Hi Des

Sorry for the delay. That didn’t seem to work either.

Todd
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 8446
Registered: 10-2002
Posted on Monday, June 21, 2021 - 08:37 pm:   

Hi Todd,

We can definitely see an issue where a slow throttling speed setting (SMSCSendLimit=1/30) causes high CPU usage, when this setting is applied to many connections. We used 50 Android modems for this particular test case, and can easily recreate the issue.

The 2021.06.09 version drops CPU usage with this setting dramatically, and the 2021.06.11 version offers an additional minor reduction in CPU usage.

So, I'm not sure what we are missing in trying to recreate the problem that you are seeing.

Do you have any other SMSC connections defined other than the Android modems?

Regards,

Bryce Norwood
NowSMS Support
Todd Matherne
New member
Username: Tmmatherne

Post Number: 16
Registered: 08-2020
Posted on Monday, June 21, 2021 - 09:41 pm:   

Bryce,

I should have mentioned earlier that I'm sending SMS that includes emojis. (unicode messages)
Due to the length of the message, 3 messages are being queued for every message sent.
Perhaps this is contributing to the high CPU usage.



Todd
Bryce Norwood - NowSMS Support
Board Administrator
Username: Bryce

Post Number: 8447
Registered: 10-2002
Posted on Tuesday, June 22, 2021 - 08:47 pm:   

Hi Todd,

As you are running on AWS, I decided to try to replicate the problem on a resource-constrained AWS instance (free tier, about as limited as you can get).

With version 2021.04.28, the SMSCSendLimit=1/30 setting with 50 modems does drive CPU usage to 99%-100%. (This is consistent with what I saw in non-AWS environments with only a single CPU allocated.)

With version 2021.06.11, CPU usage for the smsgws process is low. Sending 3-part Unicode/emoji messages does not impact CPU usage significantly.

Disabling Windows Defender scanning of the smsgws.exe process resulted in even lower CPU load, with the CPU often sitting idle while waiting for these 30 second timers between message submission.

The only extended period of 90+% CPU usage that I observed with version 2021.06.11 was when restarting the service with a large queue, which was understandable as the server was scanning all of the queued messages to determine routing.

I'm not sure what else we could be overlooking.

The next step would probably be for us to review an SMSDEBUG.LOG to see if we can identify what is different in your scenario that could be having such a drastic effect on CPU usage. If you want us to review an SMSDEBUG.LOG, send it to nowsmstech@nowsms.com with "Attention: Bryce" in the subject line. (It will probably be too large for e-mail and require use of a dropbox-type service. Send any of those details via e-mail.)

Regards,

Bryce Norwood
NowSMS Support

Add Your Message Here, or click here to start a new topic.
Post:
Bold text Italics Underline Create a hyperlink Insert a clipart image
Options: Automatically activate URLs in message
Action: