| User queue size |  Search
|
|
NowSMS Support Forums ⬆ NowSMS Support - SMS Issues ⬆ Archive through April 11, 2011 ⬆ |
◄ ► |
| Author | Message | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 170 Registered: 07-2006 |
Hi, I think, that would be great idea to limiting maximum queue size of particular user. There's always a chance to overflow system with junk messages (invalid numbers with price=0), if user’s queue exceed specified or default messages amount simply NowSMS returns 0x14 or 0x58. Another issue, limiting user’s queue length will provide more balanced performance, in current scenario more you send = more throughput you get (new queue folders being created) and pushing mechanism processes them in straight order, that can cause delays when you have mixed up bulk and p2p traffic on the same gateway. That’s what I’m thinking. Regards, Alex K. | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2767 Registered: 08-2008 |
Hi Alex, I think would be a good idea. I cannot make any promises as to when or if such a feature may be implemented. I would also mention that the "User Queue Size Threshold" is a way of limiting how many messages are sent from any one queue in one pass. That said, we did realize that this setting could cause problems for multipart messages, because NowSMS might process one part, hit the threshold and move on to another queue. When controls were implemented to improve throttling error handling in async mode, controls were also put in place not to switch queues because this threshold is hit in the middle of processing a multipart message. As noted in other postings, that version is still a bit rough around the edges. -- Des NowSMS Support | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 181 Registered: 07-2006 |
Hi Des, Any forecasts to implement that feature? Sometimes rising queue leads to chaotic delivery and annoying when customer wants messages to be delivered in-time. Regards, Alex K. | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2975 Registered: 08-2008 |
Hi Alex, We don't really have a good of way of doing this. It is a very interesting idea. It just would involve more tracking and a lot more tracking overhead. I'm just not optimistic about getting this into the development schedule compared to other items. So I've been trying to think of other possibilities. In this scenario you mention junk messages with invalid parameter values. I take it that when you submit those messages upstream, whatever error code comes back should be considered a permanent failure so that the message does not get retried. Do you have this configured? (I'm thinking about SMPPRejectErrorCodes as in http://www.nowsms.com/smpp-error-code-handling-in-nowsms) -- Des NowSMS Support | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 182 Registered: 07-2006 |
Hi Des, We've already implemented analisys for junk messages by dest or src addresses. We noticed the following potential problems with NowSMS queue handling: 1. Users can't control messages delays which sent over our gateway and resend them via other providers, if message is accepted it's impossible to predict delivery time without special monitoring system. As far as i know most providers return 0x58 error in order to avoid system overloading. “User Queue Size Threshold” can be implemented in other way, every 30 seconds NowSMS scans amount of req files of each user and if it exceeds limit then NowSMS puts special file(mark) into <UserFolder>\<Login>. When user submits a message NowSMS detects that file returns 0x14 or 0x58. When user’s queue decreased NowSMS deletes mark file. Problem with multi-part messages can be easily avoided if NowSMS will check that mark file only for the 1 st part of the multi-part message. Example: User queue – 8 SMS Limit – 10 SMS User is sending 3-part message, NowSMS allows to submit it and queue become 11 then next messages will only be accepted when queue size become 9. 2. Shifted load balancing - if user submits via SMPP NowSMS creates many folders #0001,#0002 with few req files but with HTTP requests NowSMS fills one folder with req files. In other words, SMPP users has more priority against HTTP users (more folders you have - more queue handling and speed) and order of delivery is random. I suggest you to fix that problem. Hope that you find recommendations reasonable  Regards, Alex K. | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 183 Registered: 07-2006 |
Hi Des, What do you think, is it possible to implement these features? Regards, Alex K. | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 2997 Registered: 08-2008 |
Hi Alex, You raise a good point. The SMPP user creating a large number of directories was a side effect of a quick fix to deal with an issue where different parts of multipart messages were split between different directories. That quick fix needs to be rethought. There is a related issue that every 10,000 submissions, an SMPP account moves on to a new Q directory. That is a minor issue, which doesn't create as many directories as the multipart issue. I've referred this back to engineering for redesign so that multipart messages don't generate so many directories. I am also recommending that the user queue threshold implementation be user based, not directory based. If a user has multiple directories, they should not get more priority. The user queue size limiting is a bit more challenging. It is still under investigation. -- Des NowSMS Support | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 184 Registered: 07-2006 |
Hi Des, Another issue, many ".LCK" files collecting in those folders. With months there're thousands... Regards, Alex K. | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3004 Registered: 08-2008 |
Hi Alex, The improvements are being actively worked on. Probably about a week and a half to two weeks. Summary of planned changes: Improved handling of "User Queue Size Threshold" to prevent problems where SMPP user accounts could be allocated multiple queue directories, giving extra priority to that account. (Also a reduction in the number of directories created when receiving multipart messages from SMPP clients.) User Outbound Message Queue Limits - When enabled, the number of messages that any individual user account can have in the outbound message queue will be limited. Limits can be applied with a system default setting, and user account specific overrides. The queue size limits will not quite be real time, but will have a 30 to 60 second delay in updating, similar to what you described. Subsequent parts from multipart messages already accepted will be allowed even if the queue size is exceeded. I'm somewhat baffled by the lock files. Especially because you say "months". As a failsafe, there is a regular clean-up for .LCK files older than 24 hours, so there should never be months worth. I have asked our engineers to investigate this further and look for reasons that these files might be left behind. -- Des NowSMS Support | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 185 Registered: 07-2006 |
Hi Des, Another question, why do you need these ".LCK" files with strict routing? As far as I understand it's needed to control flow of segmented messages among connections. But with strict routing it’s seems useless... Regards, Alex K. | |||
| ashot shahbazian Frequent Contributor Username: Animatele Post Number: 99 Registered: 06-2004 |
I'm somewhat baffled by the lock files. Especially because you say "months". As a failsafe, there is a regular clean-up for .LCK files older than 24 hours, so there should never be months worth. I have asked our engineers to investigate this further and look for reasons that these files might be left behind. Hi Des, Alex is right: here's a scenario where we've also observed lots of .lck files: An SMSC outage occurred upstream from NowSMS, while one of the users was submitting a large number of segmented messages. For an hour or so the segments would accumulate in the queue, which as some point split into sub-directories under \q\ (we do not use SeparateUserQueues.) Once the SMSC came back online, stuck messages resubmitted, but: tens of thousands of .lck files and hundreds of sub-folders under \q\ have remained, which was causing delays and high CPU load before the folders and .lck files were manually removed. NowSMS version used is 2010.11.04. Looks like the .lck clean-up routine isn't working well in this particular case. Please also note that many SMSC-s have a time limit to wait for all segments (typically 15-30 minutes) and if not all were received the entire message might fail. It may be useful to enforce a shorter cleanup interval for .lck files, by means of an uplink-specific setting. Also it could be that splitting files among sub-directories isn't helping with performance. We have a rather large configuration on one of the servers and I recall when we tried enabling SeparateUserQueues it simply brought the server down. It was on an older (2008 or 2009) version though Kind regards, Ashot | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 186 Registered: 07-2006 |
Hi Des, Another idea about queue size limit, making it build-in would be impractical again, I strongly recommend you make it mandatory parameter for Pre-Auth callback. Flexible and easy to implement :-) Regards, Alex K. | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3022 Registered: 08-2008 |
Hi Alex, I'm not sure I'd call the inbuilt functionality impractical. However, the key benefit that I can see for making the user queue size part of the preauth callback is that it can also alert you to the fact that a problem may be building with one of your customers. If the controls are inbuilt only, you are not alerted that there may be a problem. That said, it is probably too late to add this to the preauth callback for now. There is a maintenance update that is in final testing and planned to be made available for download next weekend. That update will only have the user queue size limits set in the configuration. Adding to the preauth is a good idea, that will just take longer. -- Des NowSMS Support | |||
| Alex Kaiser Frequent Contributor Username: Alex_k Post Number: 193 Registered: 07-2006 |
Hi Des, Could you please tell where queue overflow indicator is placed? For some reason we need to detect it. Regards Alex K. | |||
| Des - NowSMS Support Board Administrator Username: Desosms Post Number: 3085 Registered: 08-2008 |
Hi Alex, There is no file. The queue size is tracked in memory. -- Des NowSMS Support | |||
