Speed tests | Search |
NowSMS Support Forums ⬆ NowSMS Support - SMS Issues ⬆ Archive through December 14, 2010 ⬆ |
◄ ► |
Author | Message | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 124 Registered: 07-2006 |
We have spent some tests for speed of sending of messages through NowSMS http request and through SMPP. Sent about 200 000 test messages (without SMSC). Http request (Account callback it is switched on): speed of 150 messages in a second. Http request (Account callback it is switched off): speed of 500 messages in a second. SMPP sending (Account callback it is switched on): speed of 570 messages in a second. Question: why such difference at sending through HTTP request and SMPP with switched on Account callbacks? All options Account callback for smpp and http requests are identical. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 125 Registered: 07-2006 |
I apologize, I meant reсeive of messages instead of sending. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2479 Registered: 08-2008 |
Do you mean the speed at which NowSMS accepts messages from clients? | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 126 Registered: 07-2006 |
yeap. via web and smpp interfaces | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 127 Registered: 07-2006 |
Hi Des! Do you have any updates for that issue? Or you need any additional information? Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2497 Registered: 08-2008 |
Hi Alex, I suppose we do need more information about the HTTP submitter. It is not surprising that HTTP is less efficient. Keep in mind that HTTP must reauthenticate with ever submission, whereas SMPP authenticates once, and then establishes a session. If the HTTP submission is not multi-threaded, it may be slowed down by the fact that the NowSMS web interface does not support HTTP keep-alive sockets. Is the HTTP submission single threaded? -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 128 Registered: 07-2006 |
Hi Des! HTTP submission is multi-threaded. We have tried various configurations of HTTP submission: 1 thread, speed 100 per second; 4 threads, speed 150 per second; 8 threads, speed 150 per second; 20 threads, speed 150 per second. I think that there is no problem with HTTP reauthentication. Is there any limit for number of web threads? Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 129 Registered: 07-2006 |
Hi Des, We made some more test in order to confirm our suspicions about HTTP interface. Single submission(1 recip per request) dynamic callbaks: 130 SMS/sec callbacks disabled: 250 SMS/sec Bulk submission (250 recip per request) dynamic callbaks: 230 SMS/sec static callbacks: 800 SMS/sec It's impossible to use bulk submission with our custom parameters which is being forwared to PreAuth callback. And performance looks very poor in compare to SMPP. Will it be a field of any improvements? Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2503 Registered: 08-2008 |
Hi Alex, I agree that bulk HTTP submission (submission to a large number of recipients) is not practical when you are using the SMSAccountingPreAuthPerRecip=Yes setting. This was one of the major concerns that we had about implementing this setting (for you), and in many ways I regret that we did this, because a lot of your support issues seem to come back to this. That said, single recipient submissions should not be that much slower using HTTP compared to SMPP. It will be slower because authentication, user account lookup, needs to be performed for each HTTP submission whereas this is performed only once per session with SMPP (and not with each message submission). It just should not be that much slower. We are investigating this further. Does the CPU max out during your tests, or do things seem to be disk i/o bound? -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 130 Registered: 07-2006 |
Hi Des, We running tests on 4 CPU and fast SSD disks, performance monitor looks like usual. But if disable authentication for web interface will it helps? Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2504 Registered: 08-2008 |
Hi Alex, If you disable authentication for the web interface, it disables the accounting callbacks. A good solution for performance <g>, but not practical. So the CPU did not appear maxed out during the tests? Can you also tell me the exact version? (We did optimise the user authentication earlier this year, but I need to double check when that was.) -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 131 Registered: 07-2006 |
hehe, 2 sides coin :-) 2010.08.23 version. CPU seems pretty fine, load near 40%. Submission speed is very stable, doesn't look like a bottleneck. Do you think, that issue could be solved soon? Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2507 Registered: 08-2008 |
Hi Alex, The accounting callback is the "wildcard". We can't test with your accounting callback, so we generally test with static callbacks. As long as the submission is multi-threaded/multi-connection (HTTP or SMPP), it should be able to overcome most of the performance drag of an accounting callback. In my testing on one of our test systems, with a static callback, 8 HTTP threads submitting, I saw about a 10% reduction in message submission speed with accounting callbacks enabled. We studied our HTTP server implementation and the re-authentication issue, and were able to almost double the submit performance. On my test system, we could average around 400 mps submission speed before the change. After the change, it was around 775 mps. This is with a static accounting callback defined (simply requests a text file from a local server). What I can't explain is why a dynamic accounting callback would have such a severe impact on your performance (but doesn't have as much impact when used with SMPP submissions). I think you need to do some more tests to compare things with static callback vs. dynamic callback, and different numbers of submitting HTTP threads. Or maybe your dynamic callback is blocking on a database update, and that is causing the delay? When we can upload this performance update, it should offer some performance improvement, but I am still concerned that throughput drops so dramatically in your tests. I'd encourage you to do some comparisons with static callbacks to try to narrow this down further. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 133 Registered: 07-2006 |
Hi Des, Do you submit single message per request or multiple? I'm curious about your results but with multiple recipients we getting similar speed. Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2516 Registered: 08-2008 |
Hi Alex, The tests we've been running are focused on single recipient submissions. (The SMSAccountingPreAuthPerRecip=Yes setting that you need to be able to provide custom route charging to SMS User accounts complicates multiple recipient submissions because accounting callbacks are processed serially for each recipient.) I wish I could release the code, but it is based upon the code in NowSMS that performs the HTTP request for an accounting callbacks. It took me awhile to test this, because I had to ask our engineers to give me a tool to test HTTP performance. The test program simply loops repeatedly sending an HTTP request to submit a message. In multithread mode, it spawns multiple threads that perform this looping logic. Maybe I can convince them to clean it up a little more and release a version that lets you customise the URL and specify the number of messages and number of threads in an INI file. For the moment, we're still working to verify the update doesn't introduce any new problems. So far, so good. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 134 Registered: 07-2006 |
Hi Des, I see, seems that we have similar test conditions. Let's wait for that great update. Or maybe it's better to create COM-API instead of HTTP callbacks and replace funny folders with real SQL engine for storing, sending and tracking messages. I believe it would be 10x speed increase. Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2523 Registered: 08-2008 |
Update posted at http://www.nowsms.com/download/nowsms20100921.zip. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 135 Registered: 07-2006 |
Hi Des, Seems that we found a clue to the situation, CPU isn't normal, it's near a peak 90-97%. I don't know why we hadn't found that before. Vs. SMPP submission - 20-30% CPU load. Now CPU is a bottleneck with HTTP request, similar to shared user balances, finally we removed it. Please try to issue updated release ASAP or we have to do stupid job, re-making our www message-pusher from HTTP to SMPP API. It could be a waste of efforts. Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 136 Registered: 07-2006 |
Great support! Just pushed the button! We're on the same wave :-)))))))))))))))) | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 137 Registered: 07-2006 |
Hi Des, Just fresh test. Dynamic callbacks: 183 SMS/sec. Not so big improvement. But CPU dropped to 15% :-) Maybe there is any important parameters to add into config? Tomorrow we will check it with static callbacks and re-make HTTP message pusher, but it isn't practical, getting dynamic route and cost our key for success. Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 138 Registered: 07-2006 |
Could you please make tests with 30 random responses and move HTTP handler to remote host(in LAN). I guess, you'll see the difference... Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2529 Registered: 08-2008 |
Hi Alex, Indeed there is something to this. I changed the callback on my test server to call a PHP script instead, and I see similar results. We don't have any explanation for it at this time. As you've noted, CPU utilisation is reasonable, so NowSMS is not CPU bound. We are going to implement keep-alive sockets to see if this makes a difference. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 139 Registered: 07-2006 |
Hi Des, Seems new release has a bug, it's raised CPU to 100% without any reasons. We rolled back. Please revise the code. Crazy friday... Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 141 Registered: 07-2006 |
Hi Des, Any updates regarding that issue? Btw, http acception speed with shared user balances around 70mps but it was tested with 1gb channel. Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2536 Registered: 08-2008 |
Hi Alex, We are continuing to investigate and experiment. Several days ago, HTTP keep-alive support was implemented for the HTTP request handler. This replaces the experimental logic from the last update. (We may revisit the other logic in the future, as we believe we found one race condition flaw in it. But keep-alive support ultimately provides even better performance.) But keep-alive support didn't noticeably improve performance for dynamic callbacks. So we've been running a variety of tests and experiments, including benchmarking the static and dynamic callbacks that we were using directly, so that we understand their performance limitations. What we found was that the multithread test submitting to NowSMS never performed any faster than a single thread test against the callback directly, which suggested that something in NowSMS had a critical section that was serializing the processing of the callbacks. We ran a lot of internal timing tests, and never found such a problem, directly. However, the analysis did show us some internal synchronisation issues that we could improve. It's preliminary, but I'm now seeing 400mps with a dynamic callback, compared to around 150mps before. At present, this does require that the client implement HTTP keep-alive sockets. (If we re-enable the other code, which we may leave in an INI setting to do so, then similar performance could be achieved without keep-alive sockets.) More time is needed for testing and verification. -- Des NowSMS Support | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2537 Registered: 08-2008 |
Update with the keep-alive socket support and improved performance for dynamic callbacks: http://www.nowsms.com/download/nowsms20100930.zip If your client doesn't handle keep-alive sockets, the logic added in the previous release has been cleaned up and can be enabled with the setting EnableThreadPool=Yes under [SMSGW] in SMSGW.INI. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 142 Registered: 07-2006 |
Hi Des, Found the bug in new release. You have broken administrative API. We can send messages over HTTP but actions like http://192.168.0.55:8800/provision?Type=SMS&AdminAction=CreditCheck&Name=user77 fall by timeout. Please advise. Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 143 Registered: 07-2006 |
One more bug, messages don't appear in nowsms's logs. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2543 Registered: 08-2008 |
Hi Alex,
Are you sure this is not because you have used configuration parameters to turn these logs off? I see no indication of this problem.
There is indeed a problem with "/provision" requests unless EnableKeepAlive=No is set in the [SMSGW] section of SMSGW.INI. Of course, setting that parameter negates most of the performance updates. We will fix that. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 144 Registered: 07-2006 |
Log’s always enabled for sure but we'll double-check. And path's SMSGW.INI defined. Performance’s still looks the same, nothing revolutionary. Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 145 Registered: 07-2006 |
Hi Des, Logs are fine, we didn’t' see them because we tested just inbound messages. But there is no perfromance improvments, speed is always around 200mps. We tried all those parameters: EnableKeepAlive=No, EnableThreadPool=Yes and our messages pusher has keep-alive sockets enabled. Please advise. Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2550 Registered: 08-2008 |
EnableKeepAlive=No will have some performance impact (however, it was necessary to work-around the problem with the "/provision" interface). A version that fixes this problem (so EnableKeepAlive=No can be removed from your configuration) is at http://www.nowsms.com/download/nowsms20101004.zip Have you tried running HTTP load tests against your accounting callback using parameters similar to what NowSMS uses to confirm that this is not the limiting factor? Is it possible to use different user accounts on the submissions? (In my tests, I have not done this, and all submissions are from a single user account. However, I am suspicious that deadlocking over the updating of the user statistic counters and quota is a timing factor.) -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 146 Registered: 07-2006 |
Hi Des, we tried new release but got similar speed results. Could you please provide your http testing tool? Maybe we have problem with our http pusher. We also submit messages over one account. Regards, Alex K | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2554 Registered: 08-2008 |
Hi Alex, The engineer who built the tool for me is out of the office this week. There are some small changes necessary before I can send it out. He thinks he can get to it early next week. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 147 Registered: 07-2006 |
Hi Des, Any updates? Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2571 Registered: 08-2008 |
Simple HTTP tester available at http://www.nowsms.com/download/httptest.zip. Create HTTPTest.INI in the same directory with the following format to control the tester: [HttpTest] Threads=## Requests=######## URL=http://xxxxxxxxxxxxx/xxxxxxx/xxxxx The tester will create the number of "Threads" specified, each repeatedly issuing HTTP requests for "URL" until the number of "Requests" have been completed. Note that the tester stops counting when the number of completed requests is complete. Several additional requests may be completed before the tester is shutdown. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 152 Registered: 07-2006 |
Hi Des, New test results: 0921 vs. 1004 (408 vs. 160 mps) I'm little bit confused, we have same configuration file and test environment, we simply re-installed 1004 over 0921 and restarted the server. I don't know what can decrease performance in such scale. here is the smsgw.ini: [SMSGW] MessageIDPrefix=GW01 WebAuth=Yes WebMenu=Yes WebPort=8800 SMPPPort=2775 SMPPPortSSL=3550 ReceiveSMS=Yes ReceiveMMS=No ReceiveSMSCharset=utf-8 DisableUserLog=Yes DisableWebLog=Yes QDir=D:\NowSMS.Queue LogDirectory=D:\NowSMS.Logs UsersDir=D:\Users SMSInDir=D:\NowSMS.IN MessageIDTrackingDir=D:\NowSMS.DR SeparateUserQueues=Yes UserQueueThreshold=10 SMPPRejectErrorCodes=8 SMPPThrottleErrorDelay=30 UseRouteQueues=Yes TrackSMPPReceipts=Yes ReceiptRequested=Yes MaxSMPPShortCodeLen=3 RetryMaxAttempts=10 SMSAccountingPreAuthPerRecip=Yes SMSAccountingURL=http://192.168.0.75/QueryManager/ProcessSMS.aspx Please advise what decrease performance and many thanks in advance. Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 153 Registered: 07-2006 |
Hi Des, Any new updates? Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 154 Registered: 07-2006 |
Hi Des, Any updates? Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2611 Registered: 08-2008 |
Hi Alex, I'm sorry, but my product manager is concerned that you do not have a license for a product that operates at that speed. I am continuing to investigate (my question to you would be if you go back to 0921 does the speed go back up), but I think you may also want to follow up with Keith. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 155 Registered: 07-2006 |
Hi Des, We already discussed that issue with Keith several months ago. We had those VDS with trial licenses and found a lot of bugs in NowSMS with them. But we’re not using them anymore. But I don’t how to prove Keith that we don’t use them anymore? Can we ask for another test key and show you our environment? We also make similar tests with Kannel. If we're talking about inbound speed, 0921 definitely has 2 times higher inbound speed. Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2625 Registered: 08-2008 |
Hi Alex, FYI. The 0921 to 1004 change that brought back down performance has been undone in the update at http://www.nowsms.com/download/nowsms20101104.zip. -- Des NowSMS Support | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 156 Registered: 07-2006 |
Hi Des, Something is wrong with a path, getting an error and can't download from provided link. Regards, Alex K. | |||
Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2632 Registered: 08-2008 |
Looks like a problem with the web site update. It should be fixed now. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 157 Registered: 07-2006 |
Hi Des, Seems that update's performance still broken. 0921 350 sms/sec 1104 140-169 sms/sec On the same test server. Regards, Alex K. | |||
Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 158 Registered: 07-2006 |
let me double-check everything |